OK, I did a quick test. I opened the "Photos browser" icon, it found my picture files that are in Photos, then I selected one and dragged it to the email window and dropped it. It showed up as in line. I then right clicked on it and selected "show as icon" and it did. Then I moved the cursor to just past the image and hit Delete (it deletes backwards) and it deleted the image. Clicking the image and Delete didn't work, but clicking to the Right of the image (putting the cursor past the image) and Delete did back up over the image and it disappeared.
I also tried with Classic view unchecked, i.e., what is now the "new" appearance. Created new message. Used paperclip to add image, showed up inline. Right-clicked the image, selected "View as icon" and then moved past the image and used the delete key. Gone. Used Photos browser, added an image, viewed as icon, moved past and Deleted over and it disappeared.
Am I the ONLY one for whom it works that way?