A warm welcome to Mac-Forums.
To be honest with you, Julie, I don't know the reason, but I can suggest a couple of things to consider.
First off, reboot your Mac. Know it sounds daft, but is the first step I usually advise before anything else.
As a temporary workaround and as an experiment - Open Finder > Go to Downloads Folder > find the file and move it to the Desktop. You should be able to import to email very quickly.
BTW, is it only when you try to import from Downloads to email that the Download Folder takes so long to populate - or does this also happen if you go to Downloads via Finder?
Nice to know which Mac you have: Model/Year/Operating System etc.
Ian