I usually hit the ATM in $40 increments, though I have no problem carrying around large sums of money. I'm a pretty intimidating looking guy and my wallet is probably the only thing I've never lost in my life. Plus, I'm fiercely protective of it in any crowds.
But I don't really like carrying a wallet simply because I hate sitting on it. That and all the other junk in my pockets (keys, phone, change, camera, anything else) is a real pain. I kind of miss when I lived in China. There it's perfectly acceptable and normal for men to carry a small leather bag or satchel for all that junk. It was usually something very plain, black leather and no strap. I tried carrying one around when I got back to the states, but got tired of the sideways glances. I think men should unite and make it acceptable to men to carry around things like this again. Just stuff a bunch of tech toys in it and call it a gadget bag.