Pronunciation is the first, and most significant, drift, as you say. But consider "Ye Old Two Brewers," a pub in Shaftesbury, England. The "Y" in ye, is pronounced "th," or should be. I posit that most of us would say "Yee" rather than "The" on seeing the name. But the original purpose was to represent an English character known as the thorn, þ. Printers did not have a slug for the thorn and substituted the letter "y" in its place. You can see more at the second definition of ye here:
Ye | Definition of Ye at Dictionary.com
So your assertion that by not pronouncing the word as written is incorrect is, in itself, not totally correct. As further consideration, which is the "correct" pronunciation, though, thought, cough, enough, bough, rough, plough, through, hiccough?
English gets tricky. It is so flexible that it can absorb words from other languages and dialects with ease, which at the same time makes it difficult for foreign learners to understand. But English tolerates the error of the new student and moves on to understanding, as Ratsima pointed out in post #24. But flexibility leads to drift and English drifts faster than just about any language on the planet. I suspect that if one were to time-travel 100 years into the future the English would be practically unreadable. It probably will morph into something closer to texting, with "classic" English reserved for stodgy old literature and maybe legal documents where precision of language is more important. Even legal documents of today are filled with archaic terms and usage. Just read any of those "agreements" we sign so quickly when we want to get on with the website!