I've got an Aluminum Unibody with 4Gb in it. I do some graphics type stuff with SnapFish and Pixen, some audio with Audacity, the intermittent game of Star Wars Battlefront, I'll use Celestia or Stellarium from time to time, plus Safari, Adium, iTunes, and MS Excel are open 99% of the time. Even with this, I'm yet to experience any lag due to a lack of RAM.
There are people who do far more than we do on our machines who have "only" 4Gb of RAM. I don't know if you'd see as significant of an improvement as you're expecting, although that machine shouldn't be slow by any means.