Snapshots.db is huge

[MacInWin]

Investigating why my CCC backups have become huge (over 12G daily, even on days I don't do much), I discovered a file at /private/var/db/systemstats/snapshots.db that is over 12G just by itself. I found at article at Apple that says this file is gathering daily system stats about energy usage and although it shares a similar name, it is not associated with the snapshots of Time Machine. According to this article the file first appeared in Mavericks, had a growth problem but it was allegedly fixed in Yosemite. However, I'm running Yosemite 10.10.3 and my snapshots.db is over 12GB.

Does anybody have any idea how to stop this from happening?

If it helps, the snapshot.db file was created October 16, 2014 and has apparently been growing ever since. It was last updated today, one minute ago, so it's still growing.

EDIT: ALso, SQLite won't open the file. It reports the file is improperly formed.

 

[MacInWin]

Ok, this seemed to work. I found it at Apple. Instead of the mkdir and mv commands, I just mv'd the file to an external drive. I didn't want it on my internal SSD at all. The "sudo reboot" isn't needed, either, just reboot normally. My MBP is now much faster than it was, the huge file is gone and my SSD recovered 12GB. The daemons have restarted, so I'll probably have to repeat this process in a while, but for now, it seems sorted. I can see the snapshots.db recreated and already at 721 KB after 10 minutes, so it's an ongoing problem. I'm still looking for a permanent solution.

sudo launchctl stop com.apple.systemstatsd
sudo launchctl stop com.apple.systemstats.periodic
sudo launchctl stop com.apple.systemstats.analysis
mkdir /Users/Shared/oldsysstats
sudo mv /private/var/db/systemstats/* /Users/Shared/oldsysstats
sudo reboot

 

[MacInWin]

Ok, booted two hours ago. snapshots.db is now 11.8MB, so it's growing about 6 mb/hour, or 144 MB/day. Sounds like a weekly task to get rid of this junk.

Anybody else seeing it? Is it just me?

 

[cradom]

First I've heard of it. Just checked and mine is 639mb.
Created 4/3/15

 

[MacInWin]

@cradom, it's supposed to self-limit to about that size, but has a bug in which it occasionally fails to do that and grows endlessly. I've found a bunch of complaints about it on the net, and Apple is supposed to be aware of the problem. I'm going to keep an eye on it, now that I know it's there. My whole system perked up when I whacked that huge file, so it's worth the effort. If you keep an eye on it, you'll see that in that directory .stats files appear every 10 minutes and the .db itself is updated and grows every 10 minutes as well. Mine is now back up to 15.5MB after 2.5 hours.

 

[MacInWin]

Been doing some research and analysis. Here is a chart of my backups from 1/1/15 to yesterday. As you can see, it was hovering around 1GB until about the 7th of January, then started to grow linearly until yesterday. The spikes are from when I must have done something to update a major application or the OS. There were two days with no backup, which show as holes in the data. But clearly, the snapshots.db file started to run away in early January. I'm only posting this as a cautionary tale for anyone else who might want to look at that file every once in a while just to make sure it isn't running away on them.

I did talk to Mike at Bombich and he is aware of the file and plans to exclude it from backups in the future. He sees no need for it in a system backup as it gets rebuilt from scratch within a few minutes of removal.

Also, I noted that after I got rid of the db, Activity Monitor didn't have data on the Energy page to show battery history over the past 12 hours. As of right now, the green bar showing charge level is just over 2/3s over, reflecting that I killed the db and booted about 7 hours ago. So that db is where Activity Monitor gets that data.

Moral of the story? Watch that file! I plan to check it daily until I see that it's under control, then may move to weekly. I may craft a script to run the stuff I need to stop the daemons, delete the file and reboot if I find I have to manage this manually.

 

[bobtomay]

Just for info - Created 2/28/15 - it is 210 MB.

 

[MacInWin]

What is even more frustrating is that over the roughly 100 days the average size is close to 7 GB, which means my one-a-day Time Machine backups have something like 700 GB of that file! No wonder my TM drive is filling faster than I thought it should.

 

[MacInWin]

Update: 26 hours after deleting the file it is now at 160MB, and the folder containing it is at 172MB (the number of .stats files apparently increases by 3 every 10 minutes. There are now 252 files in that folder, most of which are .stats files and 51KB each.

 

[vansmith]

454 MB here, modified 4 minutes ago, created the day after Yosemite was released. It's grown 0.3 MB in the last five minutes and all I did was remote into my Mac (I'm not at home with it right now).

I'll take a more in depth look tomorrow.

 

[MacInWin]

As I said earlier, it's apparently supposed to self-limit to about 600 MB, so growing up to that size is not inappropriate. Don't know why my copy took off. It could be related to the fact the db was somehow damaged, maybe it couldn't be vacuumed, but right now it's at 188.7 MB, with the folder sitting at 204MB. From watching it, every 10 minutes or so new data is created in new .stats files that add up to about 300KB, then a minute later the .db is updated and grows by .3MB. I'm assuming that the new files are being copied into the .db as records. However the .stats files aren't being erased. There are now 308 of them. Given that the .db is 188.7 and the folder is 204 MB, some of the files are being overwritten, other are being created new. Looking at the metadata, most of the files are created, then modified 10 minutes later and never again. I do have a few that were created when I forced the process to quit and it restarted yesterday that are being updated, but there are only 4 that got updated today, and the last one was over an hour ago. I haven't changed my open apps much, so I can't tie the .stats files to any apps. The test will come in two days, when I predict the .db will exceed 600mb. If it doesn't then stop growing, I'll kill it again and let it restart from zero.

What in the world was Apple thinking?

 

[vansmith]

I've set up a cron task to track the size of the snapshots.db file over the course of my day. It's growing pretty slowly here. I'll report back to see what kind of growth we have over the coming hours.

 

[MacInWin]

My update is that with about 45 minutes to go to the 48 hour since restart point, the file is now 285MB, and the folder with all the other files is now at 475 files and 308MB.

If, in fact, this data is needed to provide the battery status over the last 12 hours, why does it keep data beyond that? I may set up a daily cron job to get rid of this crap. I frankly don't care about how the battery was over the past 12 hours, or 2 days or month!

 

[cradom]

Mine is down to 332.5mb. If this thing is related to energy usage, I keep this thing on almost all the time except when to and from work.

 

[MacInWin]

@cradom, do you have a bunch of .stats files in that folder also? In my case I have over 400 of them, adding 3-5 every 10 minutes and none of them have been deleted in the 48 hours since I whacked the whole thing and it restarted.

 

[MacInWin]

I found this discussion on the Apple Community pages:

https://discussions.apple.com/thread/5510715?start=0&tstart=0

Buried in the thread is a command to unload the daemon that drives this, for anyone that is interested. For now, I'm letting it run to see if it stops growing at any point short of 1GB.

 

[cradom]

@cradom, do you have a bunch of .stats files in that folder also? In my case I have over 400 of them, adding 3-5 every 10 minutes and none of them have been deleted in the 48 hours since I whacked the whole thing and it restarted.

Got 245 of them.

 

[vansmith]

I just had 100MB trimmed off of it while I was away from my Mac for lunch. I'm not sure what caused this - perhaps there was a purge in there somewhere? Regardless, mine doesn't seem to be skyrocketing in size like yours. I wonder if some errant process is preventing a purge of the data.

 

[MacInWin]

Yeah, it's supposed to prune itself at some point. Right now it's 323MB, there are 518 .stats files and the folder is just a tick under 350MB overall. No sign of slowing down. I did find out that the files are NOT in the TM backups, which is good, but that they are, for now, in CCC clones. Not a worry right now, but if it doesn't start managing size soon, I'm going to figure a way to automate getting it shutdown, purged and by that force it to restart.

 

[bobtomay]

Mine has shrunk over the last 2.5 days to 163 MB.

Did find this where someone disabled the launchdaemon that spawns those snapshots - link. Go down to the post that begins with "Alternatively,".

Which has me thinking, have any sort of 3rd party monitoring app installed?