Time Machine backup too small compared to used space

[MacInWin]

If you have another idea or way to achieve this goal, please let me know because I'm stuck. If what I think is true, the "missing folder" is in an older snapshot which points to the USB drive (not connected), it will then never back it up to the NAS and my customer will always need the USB drive if he would like to restore this data.

Again, that's not how it works. What you are thinking of as "snapshots" are, in fact, key elements of the backup database. What TM calls the local snapshots that TM does when the drive is not connected are invisible and cannot be opened by TM for the user. Think of it this way: The backups.backupdb file is actually a database wherein TM controls the data. Each time it adds a new bit of data, it labels that data with a date/time stamp. When it cannot open the database because the drive is detached, it temporarily stores the transaction in a hidden file to wait for the drive to reappear. So, when the drive is connected, TM opens the snapshot to get the transaction it needs to recreate the "skipped" backups and bring the database to be complete. So there is no way for the user to show or know what is in the snapshot. Don't mess with the snapshots. Deleting them can make the entire backup useless. As I said, TM is a very complex set of activities and functions that is NOT designed to have someone meddle in the middle of it. Just like you wouldn't use a text editor to change the internals of a relational data base, you should not use any tools to change any of the internals of a TM backup system. Frankly, it would have been better if Apple had just obscured the entire process and results so that end users would NOT see what is in the database except through TM itself, but I guess they thought that making it look "nice" would be a good thing.

What you want, I think, is to have a full and complete backup of the system on the NAS. That is reasonable, and I have one of those in my system. It, too, is raided. So what you want is certainly reasonable and achievable. You just need to find the missing file and repeat the process already used.

As to what happened on this particular Mac, there are a couple of possibilities for that missing file. One is that the file that can be seen in the local backup is no longer on the internal source drive. If that is the case, the way to get it to the NAS is to restore from the local USB backup to the internal drive, then do a new, clean Time Machine backup to the NAS. If there is an existing backup already on the NAS, you will have to get rid of it so TM doesn't see it there and try to use it. I've tried deleting TM backups before and they take FOREVER. What has worked for me is to just reformat the drive. If reformatting the NAS is not possible, maybe create a new space in the NAS for this backup--a separate folder, or user, or something where there is no backup now. Point TM to that specific space and let it do the backup. Then, once the first new TM backup is done, you can try deleting the old backups and just let it run until it is done. As I said, that is a very, very slow process for some reason.

The second possibility is that somehow the backup was interrupted before it finished and the large file was simply not finished, so it doesn't appear in the backup database on the NAS. That might happen if in the middle of the backup the network connection was interrupted for any reason. TM works at a very low priority by design. The idea is that backups can happen and you don't notice them because it gives way to everything else in the system. That's good if you have it turned on to backup every hour, but not so good when all you want is for it to finish a big backup. Once you have the machine and all the drives and start the long new backup, just let it run, and run, and run, until it's done.

On a slightly new note, I don't know what your customers needs may be, but once you get the backups sorted out, you might want to look at something called TimeMachineEditor ( https://tclementdev.com/timemachineeditor/ ). I use it to schedule my TM backups to be just twice a day at noon and midnight. I don't need hourly backups any more because I am retired and no longer working on critical things I can't lose. TME is a nice scheduler for TM. One fringe benefit is that if the Mac is detached from the NAS, in your case, it won't make hourly local snapshots taking up drive space. It will only make the snapshots on the schedule set in TME and, again, once the NAS is reconnected, those local snapshots will be used to build the backup to current status.

You are on the right track, but just be careful. And don't mess with the snapshot or any other internal parts of TM. Find the missing file, get it on the internal drive, do a new backup to the NAS and you should be good to go.

 

[webrider]

Hi Jake,

Thank you for your answer you help me so much ! What I keep in mind from your last post is that snapshots system is an internal function of TM and I should not modify it.

The famous "missing folder" is still on the SSD today and I have already tested the creation of a new folder on the NAS (as new destination) but the backup still weights the same and does not contain everything.

While reading all your posts I had an idea : when I changed TM destination from the USB drive to the NAS, the USB drive was not attached to the system and it has never been connected since. Is it the reason why the old backups are still pointing to this drive ?

What if I connect the USB drive and I format it ? Will TM see that these backups are deleted ? If yes, TM will probably start to backup the data that was on the USB drive to the NAS. Maybe the solution ?

I will try this next time I see my customer.

Regards,

webrider

 

[MacInWin]

I don't think the presence or absence of the USB drive matters. TM has to be able to function if the external drive totally dies, so it will just accept any new target, look for the presence of the backup and if it doesn't find it, make a new one.

Try this: Open TM Preferences in System Preferences. Click on the "Options" and see if that file is in the exclusion list. That is about the only reason I can think of for it not being included in the backup. Because it is so large and the backup frequency is, by default, once an hour, the user may have excluded it at some point and forgotten about it since then

To set the destination drive you use the same System Preferences/Time Machine and Select Disk for the target. Not much to it as long as TM can see it.

 

[webrider]

Unfortunately, the exclusion list is empty.

I will attach the USB drive to the Mac and format it hoping that TM will see that these backups do not exist anymore. TM should then backup what was on this drive to the new drive (the NAS in this case).

All data is still on the Mac so there is no much risk to do that I think. I really hope that the SSD will not fail between the format and the end of the backup to the NAS. I kept the USB drive backups just because of that but it didn't work so no other choice.

I will tell you the result of this.

webrider

 

[MacInWin]

I would not erase the USB drive. It won't make a difference. Just don't connect it. As I said (and the articles clearly state) the TM process does NOT store previous results anywhere, it reads what is currently in the backup it can see in the target drive. Deleting that USB will eliminate the one copy of the missing file you have, leaving only the internal drive. That's risky.

 

[webrider]

Hi again everyone,

This post to thank you all because your help was perfect.

I had time to go to my customer to check the backup and everything is fine.

The reason why the compression of the pictures folder is so high is that the content is not a compressed format such as jpeg but pictures were stored in raw. When looking at the stack of backups shown by TM, I was able to confirm that the first one was done when the NAS has been configured.

Regards,

webrider