If you're using MPlayer OSX Extended, in the "Video" tab of the Preferences there's a Subtitles section. In there you can adjust the Scale to be higher than 100% if need be. While you're in there, I'd recommend making sure both "A S S subtitles" and "Use embedded fonts" are enabled. If you;re using regular MPlayer OSX, in the "Miscelaneous" section under "Additional parameters," type -***-font-scale X where X is the scale number (note: unlike in the fancy preference option in Extended, 1.0 means 100%, 1.2 would mean 120%, etc.)
Erm, the bit that got bleeped out is a s s without the spaces.
And if your file is in the same folder as where the subtitle files are, it will detect those subtitle files automatically if they have the same file name (not including extension). I'd recommend testing your new video file in a different folder. Remember that the J key is the hotkey for scrolling through subtitle tracks in MPlayer (or you could stick with the fancy GUI dropdown list, your choice). Also, if you;re specifying the font size in the subtitle file, keep in mind that you often need a pretty large font size for the subtitles to show up well. VLC may ignore this because ignoring what subtitle files actually say and messing them up is VLC's speciality (though supposedly Version 0.9 has some improvements in this area).