Automatic address book entries conflict with mail

So,

For some reason, the names displayed next to email address in the mail.app program seem to get confused pretty easily. Something like this:

1. Normal operation
From: Joe Blow <joe@blow.com>

2. Then I do something in mail
Like, reply all to the above message. Or some action in mail that seems to combine names.

3. Address book messed up.
Due to #2, the entry for Joe Blow includes joe@blow.com and my email address.

4. Bad mapping in mail.
Now, a message from Joe says:
From: Boberonicus Huldichinsky <joe@blow.com>

I think that's roughly what's happening. Any ideas on how to prevent this?

Robert