How to delete user fonts?

I have a whole bunch of fonts in the user>library>fonts folder and I wanted to get rid of some of them because I never use them. When I drag them to the trash can and try to empty trash can, it says I cant delete some of them because they are in use.

I know its a big no no to mess around with system fonts, but how do I delete user fonts that I have installed myself without the dialog box popping up that says they cant be deleted because they are in use? Is there a way to boot into OSX in a safe mode or something to get rid of this stuff in my trash can?

Drag the fonts to the trash, log out and back in. You should be able to delete them then.

Is logging out the same as restarting the computer? Because I have restart the computer and try to delete them and it still says I can't because they are in use. Oh well I'll just drag them back into the fonts folder I guess if they can't be deleted.

Have you tried deleting the fonts via Font Book?