Time Machine backup too small compared to used space

[webrider]

Hi everyone,

I'm not an Apple user but I contact you because I sold a Synology NAS to a customer to make Time Machine backups to it from an iMac.

Before using a NAS the customer made his backups to a local USB hard drive.

Connecting and configuring Time Machine is not a problem as the NAS folder appears correctly in the possible destinations and there is not so much options. I choose it and TM asks me if I want to use it as a new destination (replace) or use both (NAS + local USB HDD). I choose "replace".

When the backup is finished, the size of data transfered by Time Machine seems to be something like 250 GB and it is confirmed on the NAS (folder is about 250 GB in size).

The problem is that this iMac has a 1TB SDD which is almost full. The size of the backup is too small (the photos alone weight 500 GB).

Please note that the "exclusions list" is empty.

What do you think of that ? Any idea to help my customer ?

Thanks a lot,

webrider

 

[MacInWin]

Time Machine on a NAS is a compressed file in the format of a sparsebundle (or backupbundle if it is coming from Catalina). The reason it is compressed is that Apple wanted to minimize network traffic (and bottlenecks) that an uncompressed backup might cause or trigger. You can enter Time Machine (Either click on the icon on the dock, or on the top bar. The Dock icon goes direct into Time Machine, the top bar allows you to enter or open the preferences.) Poke around in the backup to see if the images are there. You can use Quick View (spacebar) to view the images to see if they got there safely. It will be a pain to look for EVERY file, but if you try enough you should gain confidence in the backup (or find issues).

 

[webrider]

Thank you for your reply.

Do you mean that data is compressed on a NAS but NOT on a local USB HDD ? This could explain the difference between them.

I will do what you suggest and we will see if all files are included in the backup.

Regards,

webrider

 

[webrider]

My customer has crawled the backup and yes a folder of about 500 GB is missing.

Here is the backup from this morning (in French sorry it's the system language) :



And the actual files on the Mac built-in drive :



The top most library of 517,22 GB is missing in the backup from this morning.

What I think is that there is a database somewhere which says "hey this huge folder has been backup up on a USB hard drive, don't do it again unless there is some modifications". If I'm right, my question is : how to reset this database to tell TM to backup everything from scratch ?

Thanks for your help,

webrider

 

[MacInWin]

What I think is that there is a database somewhere which says "hey this huge folder has been backup up on a USB hard drive, don't do it again unless there is some modifications". If I'm right, my question is : how to reset this database to tell TM to backup everything from scratch ?

Nope, not a database. That's not how TM works. But to the second question, the way to backup everything from scratch is to get a different drive, point TM to that drive as the destination and let it run. That will create a new backup and should copy everything.

 

[MacInWin]

BTW, both of your images are from TM, so it looks like the big photos library is in a TM backup somewhere.

 

[webrider]

That's exactly what I did when I switched from the USB local HDD to the NAS folder, isn't it considered as a different drive ? Just switching made me think it will do it from scratch but it seems not.

The second screen cap is the content of the user folder directly from the SSD, not from TM (see the bottom of the picture, Utilisateurs is Users in French and Images is Pictures).

The photo library contains JPEG images which is an already compressed type of file so hard to lower the weight of these files by compression. This 250 GB backup is definitely too small considering these 500 GB of JPEG files. It's really something annoying because I don't know what to do. There is not plenty of options for TM and I have already tested all of them.

If you encountered a similar problem and have any idea, I will gladly take it.

Thanks,

webrider

 

[MacInWin]

The second screen cap is the content of the user folder directly from the SSD, not from TM (see the bottom of the picture, Utilisateurs is Users in French and Images is Pictures).

That cannot be. They both show the TM interface screen with multiple backups showing behind the one in front. So what you are really seeing is the TM contents from two different TM backups. Ordinary Finder doesn't have the "stack" of folders behind it like that.

In any event, to your first question, yes, starting a new backup on a new drive should have triggered a full backup to that drive as the initial one. A 500 GB database will take a VERY long time to transfer, due to the compression. Yes, I know jpgs don't compress, but the system will still squeeze out all the space it can, including the unused space within a block at the end of a file. It's very aggressive in doing that. What you end up with on the target network drive is either a sparsebundle or a backupbundle file that has been stored in such a way that there is no waste space. TM does not do that for a direct-attached drive as it is not needed, but for network storage, it does that compression to get the network traffic to an absolute minimum.

 

[webrider]

The problem is that I'm not a Mac user, i have never used MacOS before and of course TM even less. I have to trust my customer.

I understand that graphically it seems to come from Time Machine because of the "stack kind" of display. But what is intriguing is that, on the first screenshot I'm reading at the bottom of the window that the root of the tree is "Copy of Time Machine backup" (translated from French maybe snapshot in English version), while at the bottom of the second screenshot I'm reading SSD as the root of the tree. What makes me believe the second screenshot is not from a TM backup but live from the SSD.

What do you think of that ?

Regards,

webrider

 

[MacInWin]

Nope, that path at the bottom shows two different backups of two locations. From your post, I gather you don't actually have the machine? Here are two pictures from my system (I did remove my username). The first is of Finder, over the website:



The second is TM, again with my username blocked:


As you can see, two entirely different looks. What you have posted are two pictures of two different backups, not a picture of Finder at all. It almost looks like they did a backup of the backup drive, not the internal drive.

 

[webrider]

Yes you're right I don't have the machine. The screenshots come directly from the customer by e-mail.

I can see the difference between both displays and I understand that the Today(Now) capture is from TM. I suppose that TM is working with snapshots and each window of the stack is a snapshot, right ? Does it mean that the "Today(Now)" window is a compilation of all snapshots behind ?

If I continue my reasoning (don't know if it's correct to say that because English is not my mother language), my customer's backup from this morning does only show the data saved this morning. Am I still right ?

If you say yes to all these questions then it's understood on my side.

Thanks a lot,

webrider

 

[webrider]

One last question : is it possible to delete a specific snapshot ?

Thanks

 

[MacInWin]

Look at my Time Machine screenshot. On the right side is a timeline going back to July 2019. Each tick on that timeline is a backup version. Similarly, on the side of the "stack" is a set of arrows up and down that can go back and forth between the times in the stack. If, for example, I wanted a version of a file from July, 2019, I would click on that tick mark on the right side and the stack would adjust to show the contents of the same folder as it existed on that day. That is why it is called "Time Machine"--you can, in a sense, go back in time to how the system looked at that time.

In TM every backup/slice shows EVERYTHING in that time element. If the file did not change from the previous time, what is really there is a symbolic link to the previous slice, going all the way back to the last time the file changed. But it is displayed in TM as if the file itself was in the backup for that day. So when you said:

If I continue my reasoning (don't know if it's correct to say that because English is not my mother language), my customer's backup from this morning does only show the data saved this morning. Am I still right ?

that is not correct. Each date shows all of the state of the machine at that time. It is not possible to see what is changed and what is just a link. The idea is that if you need EVERYTHING from a given date, you can get it all.

I am not using the term "snapshot" because that term is very specific in the sense of Time Machine. If TM is running and it comes to be time for a backup and the destination drive is NOT attached, TM will make a "snapshot" of the system and store it on the internal drive in a hidden file. Then, when the destination drive becomes available, TM will update the backup database with the snapshots it has taken over the time the destination was not available. Hence, your use of the term could cause some confusion and I don't want to get all wrapped up in that language. What you asked was if a specific backup time image version can be deleted and the answer is "yes, but only from within Time Machine and the process is difficult." When you delete a backup instance from within TM, all of the links to unchanged files/folders have to be relinked to skip over that deleted instance. If you try to use any other tool to delete a date/time slice from TM, the links will be broken and the backup will become unreadable and unusable.

Your English is very, very good, so I hope that this technical discussion isn't confusing. TM is a very complex product that does a lot of "magic" to do what it does. It's hard to explain even for folks whose native language is English.

If you want to read the technical details of how TM really works, here is a link to the first of a series of at least a dozen articles on what goes on with TM: https://eclecticlight.co/2019/12/03/time-machine-1-how-it-works-or-fails-to/

 

[webrider]

Thank you Jake for the reading, it was really interesting.

No, this discussion isn't confusing at all because I have an IT pro background in Windows environment. I also tried some BSD distributions, when I was a student, which is the base system on which MacOS relies on if I'm right.

I now understand why you told me earlier that if the folder (I thought was missing) is in the latest backup, it means it has been backup up somewhere previously. There is a hard link pointing to this folder in one of the previous backups.

What I will do now is going to my customer to get my hands on this Mac and go back through the backups to identify which one was the first containing this folder. It should be the first one because I disabled TM a few days ago by deleting all destinations and then I enabled it again to a new folder on the NAS. It should have made a full backup no more than a few days back.

I will post my findings here if it can help some people in the future.

Have a nice day.

webrider

 

[MacInWin]

Let us know what you find. I recommend you get the owner's local TM backup drive, too. That drive seems to have the missing file on it.

 

[webrider]

That drive seems to have the missing file on it.

That's what I think, but what to do if it's the case ? The customer would like to recycle the old USB HDD for another usage and put all the backup to the NAS because it has RAID-1 hard drive failure protection and replication to a remote NAS for data security. This is why I asked how to delete specific or all snapshots because I'm not against restarting the backup from scratch.

 

[webrider]

Sorry for posting to fast, I found the answer there : https://www.macworld.com/article/3260635/how-to-delete-time-machine-snapshots-on-your-mac.html

There is a command called tmutil to manage snapshots.

Regards,

webrider

 

[ferrarr]

Sorry for posting to fast, I found the answer there : https://www.macworld.com/article/3260635/how-to-delete-time-machine-snapshots-on-your-mac.html

There is a command called tmutil to manage snapshots.

Regards,

webrider

That is how to delete the "local" snapshots on the Mac boot drive.

 

[MacInWin]

Yes, but remember the definition of "snapshot" is not the dated files in the TM backup, but the quick backups it makes when the backup target drive is not available. I don't think TMUTIL will do what you think it will.

EDIT: And be careful with it. tmutil is a very powerful (and thus dangerous) command. Mess it up and there is no recovery. Unless you really, really know what you are doing, I would strongly recommend against using it.

 

[webrider]

Ok, what I learned from your article, Jake, is that snapshots from a TM point of view is a complete state of files and folders meta data made after each backup (not the data itself). Before the next backup, TM makes a new snapshot and compares it to the previous one to find what files and folders changed. Then the new files and folders are copied to the new backup while the non-modified ones are hard linked.

Therefore if I delete all snapshots up to and including the one which contains the "missing folder", TM will compare the snapshot to a previous snapshot not containing the missing folder and should back it up. Or as I said before if there is a way to delete all the snapshots (like a TM reset) I would be really happy.

If you have another idea or way to achieve this goal, please let me know because I'm stuck. If what I think is true, the "missing folder" is in an older snapshot which points to the USB drive (not connected), it will then never back it up to the NAS and my customer will always need the USB drive if he would like to restore this data.

The goal here was to put everything on the NAS to avoid relying on an external drive which has no redundancy and replication capabilities.

I know it's dangerous but keeping data on a USB drive which can drop or be stolen at any time is as much as dangerous.

Regards,

webrider