How To Take Sound Out Of A Video With Out Taking A Voice Out?

Okay. I have a video that I want to take the sound out of, but there is a catch. There is a man talking in the background. I want the sound for the video of the guy talking. And NO music in the background. How can I do that with using GarageBand Or iMovie? If I can't with those, How can I do it without using Audacity?

Please and Thanks /

One of our audio geniuses mat have a better idea but I don't think that is possible. If the music and voices were recorded on separate tracks it could be done by removing the music track and keeping the other track.

Most videos though are saved with sound where the voice has been spread over both tracks. Applying filters to remove sound will almost by definition remove some of the voice.