If you don't want the pan and zoom effect applied to an image, make sure you deselect the Ken Burns Effect checkbox in the Photo Settings window before you add the photo to your movie.
If you add motion to a photograph in your movie and then change your mind, you can remove the pan and zoom from it.
To remove the pan and zoom effect from a photo in your movie:
Select the image in the clip viewer or timeline viewer.
Click the Media button, click Photos at the top of the Media pane, and then click Show Photo Settings.
In the Photo Settings window, deselect the Ken Burns Effect checkbox.
Click Apply.
So to answer your question, no - you shouldn't to start over from scratch.