A good read on APFS, particularly the sparse files

[MacInWin]

From Howard Oakley, a good tutorial on APFS. The section on sparse files is particularly interesting. Could explain strange behavior in systems with mixed format drives.

Inside APFS: from containers to clones

How APFS containers and volumes work. What hard links, clones and sparse files are, and when they break down.

eclecticlight.co

 

[IWT]

Very interesting, Jake. Many thanks.

There are likely to be a few, like me, who find parts of this rather challenging. Nevertheless, this article, like so many from "The Eclectic Company", is a tremendous asset for those more senior members who need this kind of information.

Ian

 

[MacInWin]

One of the eye-openers for me was that by default APFS stores files as sparse, which saves a ton of space. But if you then copy that file to a file structure that doesn't support sparse, it expands. He showed one example of a new, empty database that had taken up 90Kb (yes, Kilo Bytes) on the APFS drive, but which was 55 GB by design and which would expand to that fully 55GB if it was moved to HFS+, or a NAS, or NTFS, or FAT drive. What is key in that little operation is that a novice (like me) who didn't know about that might wonder why a 90KB file was taking so long to copy to an HFS+ drive. Sounds like a reason to format externals APFS, maybe even rotating drives. But that is a discussion for another day...

 

[IWT]

Sounds like a reason to format externals APFS, maybe even rotating drives. But that is a discussion for another day...

That's a really interesting comment.

Without the background knowledge you stated, I find that I have followed that process - not because I was clever, but because when APFS appeared, it was suggested that EHDs should be APFS if that Drive was only being used with Macs running APFS as part of the OS; but Mac OS Extended (Journaled) if a drive was likely to be used on Non-APFS Macs or used to transfer data elsewhere.

Now, you have provided the background info and reason why this is so.

Ian

 

[MacInWin]

Well, Ian, there is a downside to using APFS on rotating drives. The way APFS manages changes to files, particularly large files, can lead to severe fragmentation on rotating drives. Greatly simplified, if you have a file of 100 blocks of data and there is a change to what is in, let's say, block 57, then what APFS does is to save just block 57 and point from block 56 to the new location and from block 57 back to where block 58 was (and is). Now the file is in three parts. If another change is made in block 34, the same thing happens, so now the file is is 5 parts (1-33, 34, 35-56, 57, 57-100). It doesn't take long for that file to be severely fragmented, making the drive have to work harder to read in the entire file. So, "conventional" wisdom has been to keep rotating drives HFS+ and only use APFS on SSDs, where that fragmentation doesn't make any speed difference because any block in an SSD is accessible at basically the same speed. APFS was designed for SSDs and is optimized for that ability to access anywhere quickly.

The likelihood of an extreme example as Howard gave in the article is very low. But there could be some files in APFS that DO have significant effects from being sparse. A random, very small, sample on my Mac didn't show any with large differences between size and space taken. (You can see that by using Get Info on a file. Two sizes are shown, the allocated size and the space taken on the drive.)

Another way to see how much sparse is saving on your drive would be to copy your entire user space from the APFS internal to an external HFS+ drive and compare how much space your User space takes up. I'm not curious enough to do that.