Time Machine - Backup Runs out of space

[Slydude]

I haven't use Time Machine to back up files stored in a Dropbox folder so I am not quite sure how that is handled. Everything I have read so far has been advice on whether or not to exclude the Dropbox folder from a Time machine backup. No one one the information I've read so far is using Dropbox exactly the way you describe so I haven't found anything on point.

I am starting to wonder though if Dropbox isn't isn't the reason the Time Machine backup is growing so quickly, If it's backing up the entire Dropbox sync folder then in essence it would be treating things the way it treats virtual machine files and backing up the entire folder every time a change is made.

If you're up for a little experimentation there might be a way to find out for sure whether Dropbox is the issue. Here's what I propose:

1, Initiate a Time Machine backup. When it is done make note of the amount of drive space used on the Time Machine drive.
2. Find out the size of your Dropbox folder and make note of that.
3. Use your computer as you normally do .
4. After the next Time Machine backup check the amount of space used by both the Time Machine backup and the Dropbox folder again.

If Time Machine is being smart about how it handles DropBox the change in space used will be very similar indicating that the only thing new being backed up from DropBox are new or changed documents plus a small amount for the links to unchanged files. If the change to the Time Machine backup is equal to the entire size of the dropbox synched folders, it's a good bet that's the culprit.

 

[JML_UK]

Now that’s a GREAT idea, Slydude- I’m up for that one! I’ll report back after the next backup.

But Dropbox won’t explain the almost double space taken by the initial backup on a newly formatted drive…

 

[Slydude]

Good point. I'm not sure what's going on there.

×

 

[MacInWin]

I have Dropbox and the way TM handles it is that if a file in the folder change, it gets backed up just like any file in any folder. So I seriously doubt the doubling up is Dropbox. I think we need to step back a bit and do some research.

@JML_UK, can you look at the backup drive in Finder and report what you see? You should see a folder named Backups.backupsdb and in that folder is single folder with the name of your machine. Inside that folder are folders with dates as names, in the format "YYYY-MM-DD-HHMMSS," each of which correspond to one backup. If you use TM automatic backups there should be one of those folders for every hour, then older for every day and even older one per week. That is how TM works. Given you reformatted and started new, you won't see those older consolidated backups yet.

So, is that what you see? Just one machine and just a few dates? No other folders?

- - - Updated - - -

Here is what is on my TM backup, for example:

 

[JML_UK]

Hi Jake

Yes, that’s exactly what I saw with previous backups. It all looked very normal apart from the message I was out of space. Was tempted to manually erase the older ones as advised on some sites - but then saw the advice on another thread here and held off - and posted this thread!

- - - Updated - - -

Oh, and just to confirm the backup disk is dedicated to this - there is nothing else on it

 

[MacInWin]

Don't erase through Finder. That will destroy the integrity of the backup. One more thing. Use Finder to get to your backup drive so that what you see is just the backups.backupdb folder, preferably collapsed. Now press Shift-Command-<period key> once to show all the hidden files on the drive. The hidden files will be in grey. Here is what I see on my drive:


Yours may be different. What I'm trying to see is if there is any significant space on the drive hidden from view. Once you have a look another Cmd-Shift-. will restore the normal view and hide the hidden files again.

 

[Slydude]

I have Dropbox and the way TM handles it is that if a file in the folder change, it gets backed up just like any file in any folder. So I seriously doubt the doubling up is Dropbox. I think we need to step back a bit and do some research.

Thanks for confirming that Jake. That's what I thought would happen. Something I read led me to believe that under certain circumstances things might work differently but it was an old article (circa 2008 I think) and a little confusing.

 

[JML_UK]

Don't erase through Finder. That will destroy the integrity of the backup. One more thing. Use Finder to get to your backup drive so that what you see is just the backups.backupdb folder, preferably collapsed. Now press Shift-Command-<period key> once to show all the hidden files on the drive. The hidden files will be in grey. Here is what I see on my drive:
View attachment 31581

Yours may be different. What I'm trying to see is if there is any significant space on the drive hidden from view. Once you have a look another Cmd-Shift-. will restore the normal view and hide the hidden files again.

Much the same - I've got a couple of items less in fact. Biggest file is volumeicons.icns at 1.2mb. Non of the folders are of any significant size

Not used the computer much today... so not yet a fair test, but so far the backup disk has gone from 692.66 to 693.26 between 19.45 and 23.45. The Dropbox file stands at 1.17gb, not surprisingly hasn't changed over that time

I'll use the machine more normally tomorrow and keep an eye on the sizes.

Nearly 1.00am here, so will say goodnight all!
John

 

[JML_UK]

OK, a quick update... not got to overflow state yet, but with quite light usage every subsequent backup is adding in the region of an extra 1.5 - 2gb to the disk usage. And this compares with a dropbox folder size growth from 1.17gb to 1.19... so it might be consistent with backing up the whole folder. Jury still out, I'll give it a few more days.

Re the puzzling large size of initial backup - recall my SSD had 361GB used, 125GB free, the backup HDD had 689GB used

A friend has a 3tb version of the same external hard drive. It's partitioned into 3 more or less 1tb volumes... and she kindly lent me the disk and one of the volumes.

So I plugged it into my little 11" MacBook Air (mid 2013 vintage), also running catalina and with a dropbox sync folder, which has 92.73gb of the nominal 128gb available. I reformatted "my" partition and set time machine running. The backup was 77.97gb - LESS than the original, presumably due to some compression

I reformatted the same partition and plugged in the MacBook Pro.

It's just finished the first TM backup - the MBP with 359.85gb used (note actually a little LESS than before) has taken up 694.34gb on the backup disk

Now I know that's not a perfect test, it's a partitioned disk for one thing - but does anyone have any idea what is going on with the MBP when the MBA returned a backup size as expected?

The external disk was the same, so that suggests the issue is with the MBP doesn't it? Or have I missed something?

John

 

[MacInWin]

John, I don't think the coincidence of 1.5 GB in the backup to the size of the Dropbox folder is significant, unless somehow you have set up Dropbox to re-sync every document every hour. There are a lot of hidden and system files that change all the time that could add to that 1.5GB in the backup--logs, configurations, mailboxes, etc. My twice a day backup runs about 3-5 GB if all I do is some minor editing in Word and do some web browsing.

It's very puzzling on the backup being so large. Clearly you are backing up something more than once. May even be the entire drive! Can you open TM preferences, then click on the "Options' box and see what it says about the estimated size of a full backup just below the exclude these items box?

 

[JML_UK]

Hi Jake

The estimated size of backup is close to the size reported on the SSD. But it gets to that stage of backup and just keeps going…

Really strange.

I’ll have to check the Dropbox sync time.

John

 

[MacInWin]

The easy way to test Dropbox and eliminate it from your thinking is to add it to the Exclusion list by clicking on the "+" and adding the folder on your local drive to the list. Then do a backup and that folder will be eliminated from the backup. The backup should be reduced by the size of the folder, but subsequent backups will most likely be about the same 1-1.5GB. TM only backs up files/folders with changes. If all items in a folder are unchanged, the link to the folder is preserved, no new backup is made. If a file in a folder is changed, the file is copied to the backup and the links to the folder and file are moved to the new. It's very complex, and I don't really want or need to get into how it all works, but I really, really would be surprised if DB is the issue at all. But add it to the exclusion list and see what happens. As I said earlier, I use DB, have about 1.7GB in the DB folder and I don't see any issues with it at all. It's just another folder to the system.

 

[JML_UK]

Thanks Jake - just added it to exclusions... let's see what happens overnight

 

[BobHas]

Think you don't use big files? Consider this: if you keep email going back years, that file and the database of attachments associated with it could well be gigabytes of storage, and every new message that comes in changes the file and the entire thing has to be backed up. Ditto for texts if you use Messages. Normally these are not large files, but they CAN get that way.

Question. Is this really true regarding MacMail? I have about 14GB of mail. But it appears to be saved in many different files, almost all of which do not change.

 

[JML_UK]

Hi all - just an update on progress... or lack of... the TM backup with dropbox excluded behaved just as Jake suggested it would. So it seems it's not that. And even with that removed (and the majority of my work involves files saved to and from the dropbox folder) it's been adding 1.5 to 2gb to the time machine backup every day. At this rate it'll take a while to fill, and only then will I discover whether TM starts to manage the oldest backups by itself.

The size of the initial backup remains a mystery... I can find no unusual, unknown or hidden files on the disk - and you'll recall it behaved exactly the same way, with a backup about twice the used drive size, on another, kindly loaned, USB HDD.

I guess this is another of these unsolved mysteries to draw a line under.

I'll post again if/when the drive reaches capacity and TM still refuses to delete old files

Thanks to all for your help
John

 

[ferrarr]

I wonder if it has anything to do with "smart" folders or individual apps "smart" features? Moving files and folders, will look like a change to the system.

For myself, in Time Machine, I have my Downloads and Desktop folders excluded from TM, because they are just waypoints, until the files final destination. I also do some moving of files between user accounts, so in the Shared user account, I have an Untitled folder that is also excluded from TM.

 

[JML_UK]

Just 4 days on from my last post saying my backup was going OK so far I've got the "not enough space" message again. 709gb used, 290gb free showing on Disk Utility - should be ALMOST enough space for a whole backup, but a) reporting not enough space and b) TM not managing the old files.

Over the week I've opened, read and/or edited and closed (saving to Drop Box folder for cloud sync) probably getting on for 100 Word, Excel and .pdf files. DB folder has grown by 0.6gb, backup has grown from 692 to 709 gb in the 10 days or so since i did a fresh initial backup of a disk that TM says should be about 360gb.

I'm not convinced it's the disk, as you'll recall I got the same oversize initial backup on a similar borrowed USB HDD

Is it time to accept something strange is happening and draw a line? Anyone any suggestions for an alternative incremental backup program. I already have data backed up via DropBox - but I like having the reassurance of being able to restore the whole drive if the machine goes down or, (in the unlikely event of me striking lucky on the lottery) get a new machine.

John

 

[chscag]

Is it time to accept something strange is happening and draw a line? Anyone any suggestions for an alternative incremental backup program. I already have data backed up via DropBox - but I like having the reassurance of being able to restore the whole drive if the machine goes down or, (in the unlikely event of me striking lucky on the lottery) get a new machine.

Carbon Copy Cloner has a built in option called "Safety Net" where it will store files that have been changed, modified, or deleted when it makes a cloned bootable backup. It will continue to store them as long as there is room on the backup drive. That is similar to Time Machine.

You can also schedule CCC backups to run at a specified time. You can read all about the program on their site. They have 30 day trial period where you can test the program to see if it meets your needs.

Mac Backup Software | Carbon Copy Cloner | Bombich Software

 

[Slydude]

that was going to be my suggestion as well. I also recommend that once you make the clone you test to make sure that it is bootable.

You can choose to clone the drive and not make the result bootable but I prefer to have a bootable clone. In the event of a disaster I can boot from the backup and be up and running agin in the time it takes to reboot rather than having to reload everything first. I can save the reload for when I have time.

 

[JML_UK]

That’s really helpful, thanks guys