Generally speaking, I go from vague to specific and this seems to work well.
For example, I have a folder called "taxes," in which is all my tax information and returns. Inside "Taxes" are folders by year, and inside those folders are usually the actual return file or PDF (depending on whether I did my own taxes using software or if my CPA did them), and a folder of all the documents that were needed to prepare the taxes (W-2s, 1099s, that sort of thing).
This applies to other stuff too. My "arts" bookmark folder contains subfolders titled "galleries," "cinemas" "theatres" "venues" and "other"
I try not to have but a few items in each level before breaking it down further.