Mail gets it's contacts info from Apple's Contacts app. You need to put all your contacts in there. You can import into Contacts as CSV files. This is from the Contacts Help file:
Import contacts from other file types
Import contact information saved or exported from other apps. The files can be in LDAP Interchange Format (LDIF) or text format (tab-delimited or comma-separated value (CSV)).
If you’re importing a tab-delimited or CSV file, make sure it’s formatted correctly using a text editor such as TextEdit.
Remove any line breaks within a contact’s information.
Make sure that fields in a tab-delimited file are separated by a tab, instead of by another character. Don’t include spaces before or after tabs.
Make sure that fields in a CSV file are separated by a comma, instead of by another character. Don’t include spaces before or after commas.
Make sure all addresses have the same number of fields. Add empty fields as needed.
In Contacts, choose File > Import, select the file, change the encoding if necessary, then click Open.
If you’re importing a text file, review the field labels.
If the first card contains headers, make sure the headers are correctly labeled or marked “Do not import.” Any changes you make to this card are made to all cards in the file. To not import the headers card, select “Ignore first card.”
To change a label, click the arrows next to the label and choose a new label. If you don’t want to import a field, choose “Do not import.”