Apple is super-sensitive to password security. The App store was used to gain access to a news reporter's ipad and iPhone last year, the saga of which he later published widely. Apple responded by only allowing two failed attempts at the password before forcing a reset, which is what you ran into. You may have thought you entered it properly, but it apparently was not exactly right (Caps Lock key, double key hits, etc, can cause problems). When you reset, you tried the same password, which Apple also blocks as a result of that same incident. That time you typed it correctly!
The Onyx application is looking for the password you use to log into your account, assuming you set your own account up as an administrator. Look in System Preferences, Users & Groups to see if your account has admin rights. If not, you'll need to either log in to an administrator account to run Onyx, or make your account an administrator. Onyx needs that authority to do what it does, and to protect you from the possibility that someone could use Onyx on your machine to do malicious changes. Onyx is pretty powerful, so I'm ok with that caution on their part. If you set up your account as administrator with no password (not a very secure way to operate) Onyx should accept a null password.
Your iTunes account should be the same loginID and password as at the App Store, if you set it up that way. If you created a new account at the App Store, then they will be different. iTunes App store and the OSX App Store are two different stores but can share an AppleID if you want to.
On the original problem, just drag the old Onyx to the trash to delete it and install the new. Any files it leaves behind are so small that you don't need to worry about them taking up space and they won't impact the new installation at all.