App to Automate Copying Files/Folders to External Drive

Hi. I'm trying to find an app that will enable me to copy and paste multiple files and folders from my mac to an external hard drive which has the same named folders and files but I only want to copy across and update new files and folders and not have files and folders which have changed on my mac copy across to the external hard drive (I've altered some of the original files in the folders and I don't want to copy those across, just keep the originals). I've been searching for an app to automate this but only found sync programs for the mac and I'm not sure how to use them. Can anyone suggest some apps to try or a particular app that would do the job please? It would need to be compatible with macOS Sierra.

Just to be clear: What I mean is that some of the same named files in the same named folders on the external drive have been experimented with by me and altered but overwritten to the same named folder on my iMac hard drive. So when I go to copy and paste the entire folder to the external drive I'm worried that the changed files on my hard drive will overwrite the originals with the same name on the external drive, which is definetly not what I want to do. I want to just copy across any additional content I've added to that same named folder (and others in exactly the same situation) because the same named folder on my hard drive also contains additional purchased music and so on. The same named files in the folder on my hard drive are not files I want to have copied across to the same named folder on my external drive.

My problem is I have a really lot of these such folders containing GB's of files so I'm trying to automate the whole process rather than trying to do it manually. Plus, I've lost track of what files I've both added to the folders and do want to copy across to the external hard drive with the same folder name/s for the files and which ones I've experimented with and overwritten on my hard drive and do not want to copy across to the external hard drive that contain the same folder names as those files' folder names on my internal iMac hard drive.

I hope that better explains my situation and hope you can see my predicament - where talking a very great amount of files and in total a huge transfer of GB's of files.


Thanks in advance. Any help much appreciated.


John

I believe I understand what you want to do. However, it's the doing that's not going to be easy since any mistake will cause those files you don't want overwritten to be overwritten and thereby lose the original content.

I'm sure some clever person who is adept at writing a script (Apple script) could write it so that the process would be automated for you. I do not know of an application that does what you want. Perhaps someone else may have a suggestion for you or know of one. One of our moderators is also a very experienced programmer so perhaps he may be able to suggest something that may work.

Thanks for your reply. I hope you're right about others knowing application/s to use and the possibility of programming something to do this.

chscag said:

I believe I understand what you want to do. However, it's the doing that's not going to be easy since any mistake will cause those files you don't want overwritten to be overwritten and thereby lose the original content.

I'm sure some clever person who is adept at writing a script (Apple script) could write it so that the process would be automated for you. I do not know of an application that does what you want. Perhaps someone else may have a suggestion for you or know of one. One of our moderators is also a very experienced programmer so perhaps he may be able to suggest something that may work.

Click to expand...

Let me see if I understand this correctly. Tell me if the following assumptions are correct:

1. You have two drives lets call them A and B for convenience.
2. The file / folder structure is the same on both drives.
3. Some files on drive A have the same name/location as those on drive B but have in fact been modified.
4. You want to copy/backup only the files that have not been modified.

@Charlie If the assumptions above are correct shouldn't a file copy utility work if it compares the creation/modification dates and skips files whose dates differ.

If the assumptions above are correct shouldn't a file copy utility work if it compares the creation/modification dates and skips files whose dates differ.

Click to expand...

Yes it should but my understanding is that he has a large amount to do. Here's what he said about that:

I hope that better explains my situation and hope you can see my predicament - where talking a very great amount of files and in total a huge transfer of GB's of files.

Click to expand...

That's why my thoughts are to automate the process with a script or maybe a workflow.

I was wondering about a workflow as well but it's probably a bit above my scripting. If one of our members knows of a sync utility that is reliable and reads the modification data correctly it should be able to handle the large amount of data almost as quickly as a script.

I've got a coupe of sync utilities installed that let you set some parameters. I'll check to see if any of them claim to be able to read modification dates.

I think what you want is a clone of your original files. If you use cloning software such as Carbon Copy Cloner you will get exact copies of the original files/folders on the external HD which will update at whatever schedule you set and only update the changes.
Obviously CCC can do much more than that. It can create a complete bootable duplicate of your Macintosh HD but it does give you options to simply sync selected files.
Given the flexibility of CCC that would be my choice for your needs and a bootable clone of the OS as well is a very good idea for anyone.

Firstly, thanks to everyone who has replied to this thread. @Slydude is absolutely correct about what I need to do and what sort of app I need. I'd greatly appreciate it if he or anyone else can see if they have any apps that I can buy on the net that allow you to copy/backup only the files that have not been modified. @chscag is also correct in saying that I have a large amount to do. And they are large files/folders - GB's actually. So that would be need to be taken into consideration. I've researched this endlessly on the web, but there seems limited, if any, apps for this sort of specific thing for the mac. One I came across is ChronoSync - https://www.econtechnologies.com/chronosync/overview.html - but I don't know if it would do the job. Just thought I'd let you know what I had found already.

Thanks again everyone for all your help.

Slydude said:

I was wondering about a workflow as well but it's probably a bit above my scripting. If one of our members knows of a sync utility that is reliable and reads the modification data correctly it should be able to handle the large amount of data almost as quickly as a script.

I've got a coupe of sync utilities installed that let you set some parameters. I'll check to see if any of them claim to be able to read modification dates.

Click to expand...

Just to give us a bit of an idea what we are up against a few questions occur to me:
1. Assuming that no backup has been made initially, roughly how much data is involved (number of files/total amount of data)?
2. Roughly what percentage of the data would remain unchanged and thus need to be copied on the next backup?
3. Do you need to keep different versions of the data around? In other words suppose the first backup is made and before the second backup is made 10 files are changed. On the next backup all files except those 10 are copied. When the next backup is made is it OK to overwrite this backup with the next batch of unchanged files.

@Rod Sprague to do what the OP wants CCC would have to be configured to do somewhat the opposite of an incremental backup. In CCC's incremental backups files that are changed while the OP wants to copy files that are unchanged.

@Slydude Not sure how helpful I can be since it's been a long time since I've done a backup so there's a lot of new files and altered files all over the place that I'm afraid I've been very disorgansied about keeping track of them. But, from what I can tell:

1. About 1.5 TB of data is involved, though not all of this will be data I do not want to copy over, but a lot will be.

2. I could only guess, but I'd say about 70% or 60% is new data that would need to be copied across, but I haven't kept track of what I've changed so I'm not sure - it could just be 50% new data to be copied across and 50% not to be copied across overwriting existing data that is correct and unaltered, or 60%/40%, and so on,.

3. No, I don't need to keep the different versions of the data around on my internal hard drive, in fact I intend to delete all altered data from my hard drive by simply deleting en masse the folders I've copied across to my internal drive from my external drive and also deleting the now backed up new files. (I've decided that it's best to keep the files as I bought them and archive them that way, since I could have made mistakes when fooling around with the files I'd copied off the external drive - that's not the same for all the new files, which are of different types of data, mostly software setup files I've bought along with such things as some ripped vinyl music from my vinyl record collection, but the altered ones are either music files I've created by ripping my vinyl collection or bought music files in high definition (high resolution - greater than CD quality) which is why they're so big (sampling rates of 24bit/192 kHz for instance for an entire album containing maybe 12 or even more songs which all add up at about 100 MB each song, and which I simply want to add to the right folders on the external drive which need updating with the new files leaving the ones already on the external drive but altered on my internal drive unchanged on the external drive). Once the folders on my hard drive are copied across to the external drive with this updating the same named folders on that drive with these new files which belong in those folders to update the folders and keep them up to date, I don't want the new files to be changed and the not copied altered files added by a further copy over to the external drive - I just want to update the existing folders on the external hard drive with the new files that need to be added and haven't been altered but are just new, and leave out copying across any files on my hard drive that are in folders with the same name as the folders on my external drive that have in fact been changed on my internal drive and are not new, unaltered files. If that makes any sense. I'm sorry to be so confusing. It's so difficult to explain. And I've been quite negligent in updating these external drive folders with the new content I've bought or ripped etc and have been very careless in keeping track of what files I experimented with changing their meta-data etc. I've left it far too long to update these folders I've created on my external drive containing all the stuff I've bought, created etc. - plus I've created in those folders on my external drive sub-folders to identify where I've bought the stuff or what artist/band they belong to or what genre they belong to and so on and used those sub-folders to contain the data. So it gets complex. Maybe it's not possible to do this. I don't know. I do now that the files have added up over time because I've neglected to update them on the external drive (that is, the folders that contain files of the type I do want to copy over and want the ones I've avoided copying over to not be copied over the existing same named files). My own fault, together with being very sick recently and being unable to to all this updating for months.

I wish I could be clearer and of more help.

I should add that the reason for the huge file transfer is that although I have a 3TB internal Hard Drive, I've only got about 200 GB of space left available on it.

ChronoSync is really a more complex form of the cloning software I originally mentioned (CCC) which i still think could be tailored to suit your needs. CS provides a lot more options though backing up to multiple sources, selective file backup, cross platform and remote backup being just some of them. None the less given the amount of data you have I imagine it would be a tragedy if your device crashed and you did not have a backup plan of some sort.
I strongly suggest that you invest in some sort of backup and I think the members of this forum would all agree that a bootable clone is the best alternative.
I might add that it is advisable to have around 20% of your primary drive free for virtual memory and OS process' which would be 600Gb and you state that you only have 200Gb which is less than 10%.

I should add that the reason for the huge file transfer is that although I have a 3TB internal Hard Drive, I've only got about 200 GB of space left available on it.

Click to expand...

Why not add an external hard drive, partitioned or not, and just use it for all those files you need.

Then they'd all be in one place and also save you all your transfer time, and extra wear and tear.




- Patrick
======