If you dragged it from the Finder Sidebar, then it is still on your Mac. Open Finder, then Navigate to the Home Folder, in the Finder Sidebar, denoted by a House. Then in the corresponding Window in Finder, will be your Downloads Folder. Grab that and drag it to your Dock, and/or your Finder Sidebar again.
If the Home Folder isn't in the Sidebar, open Finder Preferences or ⌘, and then checkmark it and Downloads under Sidebar . . . . Hope this helps.
Oh, to explain, what went 'poof' was just a alias, and not your Downloads folder.