OSX Move Files Behaviour Problem (Moves then Deletes, not Moves and Deletes)

[XionicFire]

Im trying to move and merge folders with large ammounts of files from one location to another as a backup, nothing exotic, i do this in windows all the time and its basic functionality there, but this time I have to do it on a mac, and aparently its not that simple.

After constant searching of the internet ive found that theres just no way to do this with the finder (FTFF), so i checked third party tools and command line functions.

some tools promise to do this but they all have the same fundamental move flaw that ive seen so far in mac.

In Windows when you move something, file that gets moved, file that gets immediately deleted from the source, which is actually good behaviour, since if for some reason the move operation gets stopped, cannceled or aborted you can pick up right where you left off. you know if its on the new folder its copied and if its deleted from the source, its moved, its a good way to keep track of move progress across sessions.

However on mac ive noticed this is fundamentally different, and a major pain....

When you try to move something, both in the finder and third party apps (and dont get me started on the replacing /data loss thing...) mac tries to copy the ENTIRE bulk of the files first... and THEN deletes the source, however since the files are so many and the sizes so big, the total time to do this would take many days.

So we said ok screw it, lets do it, and left it running,

Problem is after 2 days running there was a slight power hickup and the disk we were moving the info to disconnected accidentally

While most of the data got copied fine... we now have no way to continue where we left off, and here lies the dilema, we cant just keep copying the files hoping than during a span of 2 weeks everything will be rosy and the copy will succeed. this is crazy.

There has to be a better way to do this.

Does anyone know a way to MOVE and MERGE folders but that operates in such a way that every file moved gets deleted from the source as it goes? I cant be the only one running into this issue....

I mean its basic Widows File Explorer behaviour 101 and Linux GUIs since the 90s, how is this still not in mac OS by default?

I hope anyone else has an idea or has run into a similar situation and found a solution, Maybe a way better Finder replacement entirely? Mac is basically unix, theres got to be something like that....

Software ive tried:
Path Finder (moves then deletes, not moves and deletes)
Ultracopier
mv (terminal)
Holding COMMAND
Read at least 40 different forums with similar issues but not this exact issue

OS X El Capitan

 

[Dysfunction]

The fastest way to do this would be rsync.

Let's set this up. I've got a couple directories I wish to sync, then leave the first one empty. We'll call these testDir and testDir2. When done, testDir2 should contain everything testDir did, plus and testDir should be empty

mike@beer Documents $ find testDir -type f |wc -l;diff testDir testDir2
      10
Binary files testDir/.DS_Store and testDir2/.DS_Store differ
Only in testDir: AZBR_Event_Results_2015-12-13.xls
Only in testDir: AZBR_Event_Results_2016-01-24.xls
Only in testDir: AZBR_Event_Results_2016-02-28.xls
Only in testDir: AZBR_Event_Results_2016-03-13.xls
Only in testDir2: GoPro
Only in testDir2: Linux-101-Hacks.pdf
Only in testDir2: MFL67731037_141028.pdf
Only in testDir2: Manual_TX-NR717_English.pdf
Only in testDir: SCCAminorwaiver.pdf
Only in testDir2: ScreenFlow-5-User-Guide.pdf
Only in testDir2: UserManual.pdf
Only in testDir: rules
Only in testDir2: screenflow-shortcuts.pdf
Only in testDir: w:titles
mike@beer Documents $

We'll use rsync to do this. But we NEVER want to run a delete without a dry run so we're going to use:

rsync --dry-run --remove-source-files -azv testDir testDir2

mike@beer Documents $ rsync --dry-run --remove-source-files -azv testDir testDir2
building file list ... done
testDir/
testDir/.DS_Store
testDir/AZBR_Event_Results_2015-12-13.xls
testDir/AZBR_Event_Results_2016-01-24.xls
testDir/AZBR_Event_Results_2016-02-28.xls
testDir/AZBR_Event_Results_2016-03-13.xls
testDir/SCCAminorwaiver.pdf
testDir/rules/
testDir/rules/2015-6-23_section_13_street_category_online.pdf
testDir/rules/2016_Solo_Rules_DRAFT_c_1-8-2016_reduced-2.pdf
testDir/w:titles/
testDir/w:titles/Dec13MaranaToFast.mp4
testDir/w:titles/Dec13MaranaToFastA.mov

sent 540 bytes  received 98 bytes  425.33 bytes/sec
total size is 1262240428  speedup is 1978433.27
mike@beer Documents $

Now that we see what it's going to do, we'll remove --dry-run and run it again

mike@beer Documents $ rsync --remove-source-files -azv testDir testDir2
building file list ... done
testDir/
testDir/.DS_Store
testDir/AZBR_Event_Results_2015-12-13.xls
testDir/AZBR_Event_Results_2016-01-24.xls
testDir/AZBR_Event_Results_2016-02-28.xls
testDir/AZBR_Event_Results_2016-03-13.xls
testDir/SCCAminorwaiver.pdf
testDir/rules/
testDir/rules/2015-6-23_section_13_street_category_online.pdf
testDir/rules/2016_Solo_Rules_DRAFT_c_1-8-2016_reduced-2.pdf
testDir/w:titles/
testDir/w:titles/Dec13MaranaToFast.mp4
testDir/w:titles/Dec13MaranaToFastA.mov

sent 1245355393 bytes  received 258 bytes  24181663.13 bytes/sec
total size is 1262240428  speedup is 1.01
mike@beer Documents $

Now we'll double check that we got what we want.

mike@beer Documents $ find testDir -type f |wc -l;diff testDir testDir2
       0
Only in testDir2: .DS_Store
Only in testDir2: GoPro
Only in testDir2: Linux-101-Hacks.pdf
Only in testDir2: MFL67731037_141028.pdf
Only in testDir2: Manual_TX-NR717_English.pdf
Only in testDir2: ScreenFlow-5-User-Guide.pdf
Only in testDir2: UserManual.pdf
Only in testDir: rules
Only in testDir2: screenflow-shortcuts.pdf
Only in testDir2: testDir
Only in testDir: w:titles
mike@beer Documents $

You should do a man rsync and ensure you've got all the options you wish. These are just habitual switches for me, and may not be exactly what YOU want.

 

[chscag]

I use Path Finder for all my file actions. The fastest way may be to copy to the new location and then delete from the old location. Of course much depends on what you're using: USB 2, USB 3, or Thunderbolt. By the way, OS X uses the safer method = move first then delete. Moving files is always more dangerous than copying and then deleting. The same dangers exist in Windows using the File Explorer only more so.

 

[TattooedMac]

I have used rsync on many occasions, copying anywhere up to 25GB of data at a time.

Here is the rsync man from Apples Developer Portal

 

[Dysfunction]

I have used rsync on many occasions, copying anywhere up to 25GB of data at a time.

Here is the rsync man from Apples Developer Portal

Yea, it's also the fastest method of deleting thousand of files... I've used it (recently) to delete 1.4TB of data... that would have taken days using find/rm and took maybe 2 hours.

 

[XionicFire]

it seems incredibly complicated to do something as simple as move/merge but since were so desperate we will try anything

thanks for your assistance mike, ill post here what ever success i have, your method sounds good.

As far as path finder goes we tried it but the fact that when a copy fails it leaves duplicates and theres no way to resume makes it unfeasable, with that ammount of time it is almost certain some external factor will come into play and disrupt the transfer, we need something that whever that happens you can just pick up where it left off.

We understand the windows way sounds unsafe but in practice it isnt, whenever were moving things from one place to another and we have an issue with data safety we just make another copy,then move the copy but usually this is unecesary, most if not all disruptions will never cause data loss, and the way windows copies the files is first it ensures the file is copied to the new location, then deletes it, so the chance that the file can get deleted at the source before its finished at the destination is basically non existant.