Some programs add preferences in the library a simple search will help you on that, and there are some times when you best use the actual uninstall program (Micrsoft office 2004, etc...) as they leave behind alot of excess luggage you don't care to keep. The "icon" that you are dragging to the trash is much, much, more than that. It is a package, beleive it or not that icon has many different files, images, and a whole folder structure in it. Apple had a great idea with this, one of the best accomplishments if you ask me, along with disk images, very easy way to manage data!!
Make sure you are not just dragging the alias from the dock to the trash, you must go to your applications folder, or whatever folder you installed to and drag the real deal!