You should definitely remove any standalone Flash components. If you absolutely need to visit a site that has Flash content, use Chrome which has a Flash player built in. As far as Java goes, again if you are using it for any applications, you'll want to keep it and update it to the latest version, otherwise it's safe to remove.
The bulk of the "regular" users won't need to be running Java on their machines.
You can grab a copy of Chrome and keep it handy when you need it.