Can't remove contacts from address book

That is not your Address book, if your Address Book or Contacts is not on the dock go to Finder and open Your Address Book or Contacts, which ever you have. Hold down command and select all the ones you wanted deleted and then hiy delete.

Your address book has phone number, street addresses, birthdays, notes and more. none of that is in what you are calling the address book. It get's the email addresses from the address book, but name and email is all.

If it didn't come on the dock by default it will be in Applications in Finder. It may be called Address Book or Contacts depending on your operation system. It interacts with a lot of things on your Mac, like the Birthdays you add with show up in the Calendar.