Best software for backing up

C

chas_m

Guest
If you're finding SuperDuper complicated ... maybe you should stick to Time Machine and just keep a bootable USB thumb drive handy. I would have called it one of the easiest programs to clone a hard drive ever created, short of using the terminal.
 
OP
M
Joined
May 24, 2012
Messages
25
Reaction score
0
Points
1
chas_m:
Sorry but you're missing the point of my exercise completely, :). What I want to do is to keep all my datafiles in an external harddrive. I've been carefully keeping my previous years files into folders named by years, going all the way back to 1997. I've been able to download these files into an external harddrive which I then take wherever I go. So, if I buy a new computer or change jobs, I just upload all my files into the new computer and I am ready to go. Depending on where I am, I may use a Mac or a PC (I have yet to come across a Linux machine). So, I need this external harddrive to be compatible across both systems. As I am also accessing computers in hotels and other public places, the harddrive really needs to be cross-platform compatible.

Since the memory in new computers are always much bigger than the older versions, there hasn't been a problem, even if in the process, I copied over system files or program files which become unusable. I don't mind as long as my datafiles are good to go. I don't even mind if the files get bigger when I copy them over, as long as my overall memory needs are manageable.

This was relatively easy in the past, as files were much smaller then and the Windows disk format works across the two platforms. But now, I've hit a problem:

mrplow & chas_m:
So OK, mrplow and chas_m (and anybody else out there who can help), as I understand it, new disk formats have evolved to meet growing needs (bigger files and deeper directories). BTW, thanks, mrplow, for explaining the 255 length limitation. That was very helpful. My worry is: will the new disk format make the drive unreadable if I chance upon an old computer (I travel to all kinds of places and there's no telling if they have a computer let alone a later version).

Over the years, I've accumulated about 400 GB of data. I'm pretty sure everything I need is in there somewhere. If I were to go in and clean it up as mrplow suggests, it'd probably reduce to somewhere around 250GB, or even less. Given that I haven't been very good at housekeeping, this will take me easily a week or so to do (I work fulltime and am married, so I cannot spare spendings days on end on this). So, I am not keen to do this.

What I have done so far:
1. I tried to backup by simply copying the User folder (where most of my datafiles I think are kept) into an external harddrive. No good, I got hit by the 255 limitation.
2. So, I tried downloading free software to do this for me. Also, no good. They are similarly hit and worse, are difficult to use since they are essentially backup programs rather than what I had in mind. I actually think that what I want is a folder sync program which I can run everytime I need to - say once a day before I leave the office or when I am going on a trip. The only other thing I'd like from this folder sync program is that files that I have deleted from the main folder are not deleted in the backup. That way (for example, one of my old computers that I still use at home runs on Windows XP and only has a 50GB internal drive), I can have say, a 2 TB external harddrive, and still use a smaller capacity home computer.

My current alternatives:
1. Reformat my external harddrive as mrplow suggests. But what about older computers like my Windows 7?
2. Organize my folders so that the directories are not too deep and the filenames are similarly short. Can, but this will take quite some time.

Any better solution? The more I think about it, the more I think this will be a trend in the future - keep all your datafiles in an external harddrive, which you then take wherever you go. Like a "hard" version of Dropbox. Many people I know already do this - instead of bringing me a USB drive, they bring their harddrive. Maybe some may even have several harddrives for different uses - e.g personal, office, family, etc. I'd still want a full backup in my office computer - in case anything goes wrong. But I do see myself carrying around a harddrive with all my files in it instead of a heavy computer or laptop. Or even as well as.

BTW:
OK, there is this 255 filelength limitation. However, when I manually copy the files over, it works and the backup drive is useable (in the sense that I can go in and access the files). Am I right in saying then that the problem lies in the copy command? If the problem is in the disk format, i.e. for some reason they cannot accommodate too many layers, then the backup drive won't be working. So, why can't someone write a software which solves this length problem - i.e. open the folders in the backup drive before copying over rather than copying the entire path? That is, do what I do manually.
 
Joined
Oct 1, 2007
Messages
7,163
Reaction score
275
Points
83
Location
UK
Your Mac's Specs
Mac Mini i5 (2014 High Sierra), iPhone X, Apple Watch, iPad Pro 12.9, AppleTV (4)
I know you must find this frustrating but you're not reading what I posted.

I haven't asked you to permanently change the format you use. Merely try it to see if that's definitely the issue.
Please go back and read what I've written. Only once we know that's definitely the issue can we move forward with a solution.

If its the disk format limitation no tool can get around that. The disk format sets the rules if which all disk activity have to follow.

The last part of your post states
Also, if I copy them manually one by one (i.e. one folder at a time - I find the biggest problems when I do this is with folders with too many layers of subfolders. When I go in deeper and copy at lower levels, they copy ok

But in your earlier post you say it only works if you go further down the folder structure.
Which is it?

It's good that you've fully stated a use case now. That helps determine a solution.
But busy or not I suspect that you'll need to reorganise your files and folders to make them more portable.
I too have a very busy life and faced a similar admin task that couldn't be tackled in one go.
The way to tackle it is to do it it but by bit over time. Select a method, a pattern, a workflow and gradually move your data into a better organised structure over time.
 
OP
M
Joined
May 24, 2012
Messages
25
Reaction score
0
Points
1
Ya, mrplow, I think you got me right the first time. I do mean when I manually go down deeper so that when I do the copy, it's a shorter path. I may have to do what you suggest, :(. I'm just postponing the pain and hoping that there is a shorter way. What I don't understand is why can't someone write a program to do that? That is, the software opens the folder, go deeper, and then copy the file with a short path and then come back up again when done. Bypass the 255 restriction with some clever coding.

I am planning to reorganise my files eventually. What I planned to do was:
1. Backup everything to an external drive.
2. Delete all my datafiles on my Mac.
3. Redesign my folder structure.
4. Copy the files back into my Mac.
I expect this to take a long time, maybe months. But I can take my time because all my files are in the external drive and I can access it anytime I like. In any case, I also have my backup in my TimeMachine so I've got two backups (I will stop using my TimeMachine at this point, or begin a new TimeMachine backup to another folder in the TimeMachine drive).

Is this clear enough? It's rather complicated, I know. The problem is that I don't have lots of space to play with. My iMac is nearly full (the internal harddrive is only 500GB, let's call it Drive A). The TimeMachine is also nearly full (as it has been backing up since early last year so it has taken up nearly all the 2 TB in the drive, Drive B). I bought a new external harddrive (1 TB, Drive C) so that I can do this current backup that we have been talking about.

So, my plan is to backup to drive C. Then delete the datafiles in Drive A, which ought to free up at least 100 to 200 GB on my Mac. And then copy the files I need as I use the Mac over the rest of this year. By the end of this year, I should have a good set of my stuff on my Mac (the current one has lots of duplicates and unusable files).

The thing is, I've given myself a whole year to do what you suggest (i.e. reorganize). In the meantime, I'm looking for a quick and easy way to do step 1.
 
OP
M
Joined
May 24, 2012
Messages
25
Reaction score
0
Points
1
Oh, BTW, mrplow, I do think you're right. The problem is, I think, and as you say, the 255 restriction linked to the old disk format standards. Just too lazy to test it out... My past experiences kinda point to that, anyway.

The way I look at it, I will have to create a test folder with some files of some depth and layers. Then try to backup using copy. Keep on creating layers until it fails, yes? Let's assume you are right, :). If not, we're back to square one - i.e. not even knowing what's the problem.

I'm about 99% sure you're right (and 1% not willing to find out if you're wrong).
 
Joined
Sep 5, 2007
Messages
452
Reaction score
9
Points
18
Location
Canada
Your Mac's Specs
Macbook Pro, 2.4GHz Core2 Duo 10.6.8, MacMini
Carbon Copy Cloner will do exactly what you want. It isn't pay unless you want to but as stated on it's pages you are free to use it. As with Time Machine, you can pick and choose what folders you want to omit. You can schedule when. Yes at the beginning I found the way CCC explained things to be complicated but once I did it, I realized it wasn't as complicated as they mentioned, bundled with the idea that I still had a lot of 'windows mind set', I realized there really wasn't much complication other than my own fussiness...

You could even tell TM to backup only the certain file folders you want to your other external... Since you have given yourself a year, you probably should stick with what came with your mac and then once you have gotten yourself out of your own way, you might better be able to ask for what you want, find it or decide you already have what you need...

"You can't always get what you want
But if you try sometimes well you might find
You get what you need" Rolling Stones

Good Luck
 
Joined
Oct 1, 2007
Messages
7,163
Reaction score
275
Points
83
Location
UK
Your Mac's Specs
Mac Mini i5 (2014 High Sierra), iPhone X, Apple Watch, iPad Pro 12.9, AppleTV (4)
Aqua:
While both competent products - CCC and TM doesn't give the platform independence required.

mingteck:
While you could have an application to transform the folder structure for you at time of copy. How would it know which folders in the path you weren't interested in? Not to mention that you'd end up with a different folder structure on the external which would make future backups and the platform independence required very tricky.

Why not discard the whole home/user folder idea - those folder structures differ platform to platform anyway.

Just target your data; docs, pictures, video, audio etc. Time Machine already has a backup of everything. Limit your external drive to to you're own, quality data.
 
M

MacInWin

Guest
Also, for mingteck, the problem is never going to go away about older machines. No matter the solution, if you have to have the lowest common denominator format for old equipment, the 255 path limit is going to bite you in the end. You may have to just face the fact that you can get what you want for MOST of what you need, but there is no "miracle" solution that will enable you to carry the HD with you and never fail. The lowest common denominator, FAT32, has two limitations: the path length (255) and file size (4GB). There is no way around those limits, they are built into the format itself.

So, make some decisions about your priorities and pick the best available solution for those priorities. Want to be able to read/write from modern Windows machines? Only OSX? Want to use CCC or some similar product to make the cloning easy? Willing to pay for that solution? Willing to clone the "hard" way? What files do you NEED/WANT? (Those are two different things!)

Good luck with it!
 
OP
M
Joined
May 24, 2012
Messages
25
Reaction score
0
Points
1
OK, everyone, heads up. Good news. After 2 weeks of trying every program I can find and even trying to write my own Applescript for this, I found the program that does exactly what I need. This magical program is called Backuplist+. You can get it here: Backuplist page

It does exactly what I need: back up to an FAT32 harddrive with no problems at all. The 4GB file size is unsurmountable, and I don't expect to be able to do that. I get it. But surely, one can bypass the 255 pathname problem? I don't have files with such long names and the only reason why the pathname is so long is because the folders are added to the pathname. Anyway, who cares. This program does the job:
- it was easy to use
- it copied the User folder over (400K+ files, 500+ GB) in less than two hours
- it wasn't afraid of the FAT32 file format
- it had no problems with pathlength
- it wasn't hung up by metadata or anything else
- it kept the same folder and file structure (the backup folder looks exactly the same as the original)
- I was able to open the backup files and folders on an old Windows XP computer
- the author replied to my queries within a day
- and it's FREE!

Wahoo!!! Spread the word. All I did was set it up, go home, came back the next day, and it's done! The author expressed my sentiments exactly:
"I created backuplist+ because I got tired of other more complicated backup software, though some are certainly efficient and offer complex archiving and other details.
...
Backuplist+ is just a simple a list of source files you want copied to a certain destination, whether it is an external drive, flash drive or a network."

Thanks everyone. I really appreciate the help and suggestions you've given me. Keep It Simple, Soldiers.
 
Joined
Jan 23, 2013
Messages
2
Reaction score
0
Points
1
cross platform compatibility

Just my 2 pence – are you aware that there’s a file format out there called exFat which is compatible with both MAC OS X and Windows?
 
OP
M
Joined
May 24, 2012
Messages
25
Reaction score
0
Points
1
Nope, pehac, I didn't know about this. But (a) is it supported by the various backup software? This is the first time I heard of it. (b) what are it's advantages? Can it do big files (>4GB) and handle long pathlengths? (c) why go down that road when Backlist+ solves it all and does a great job with FAT32?

Thanks for your input, though.
 
Joined
Jan 23, 2013
Messages
2
Reaction score
0
Points
1
I have formatted my external HDD with the exFAT (using MAC OS X 10.6.8) a couple of weeks ago and the disc has been read / written to on both operating systems no problems. As it's limitations you have to do your own googling pal sorry :)
 

Shop Amazon


Shop for your Apple, Mac, iPhone and other computer products on Amazon.
We are a participant in the Amazon Services LLC Associates Program, an affiliate program designed to provide a means for us to earn fees by linking to Amazon and affiliated sites.
Top