One more thing...
If I delete a mail in my inbox then it is still there in my All Mail (Gmail) and if I delete a mail in my All Mail (Gmail) folder then that mail still remains in the inbox. Is there any way through which I can simultaneously delete mails from both the folders or do I have to delete in them separately every time?
If you delete one from the gmail server (Site), and then right click on the mailbox in the Mail app, and then Synchronize, it will mirror exactly what you have on the gmail server.
If however, you delete an email from the inbox (or any other folder for that matter) in the Mail app (move it to trash), it will delete it on the gmail server as well, and move it to the trash.
The use of the mail app with the gmail server is really intended to keep it so that you're using the mail app, and not switching back and forth.