Try force quitting and then restarting your computer, then trying photobooth again.

That's happened and worked for me in the past.