Jonathan, you made my point exactly. Thanks! The original sentence needed to be restructured to remove the ambiguity of the split infinitive. And that is why one should not split infinitives.
Respectfully, I'd assert his interpretation is correct based on the structure of the sentence, and if that's not the meaning you intended to convey, then you did not formulate the sentence properly. You should have said, "I'm quickly going to the store to buy some milk" or "I'm going to the store quickly to buy some milk."