Any way to copy data and skipping corrupt data instead of stopping?

[krs]

I have run into this a few times now, recently with my 8 TB Seagate drive that is dying.

When I copy data from a drive with some corrupt files, the copying just stops when it gets to the corrupt file or folder.
I then have to into that folder which may have sub-folders as well, and copy each file individually until I come to the one that can't be copied - skip that one and then continue with the rest.

I'm looking for a method, or even different software, that will copy all non-corrupt files and either ignore the corrupt ones, or even better, quarantine the corrupt ones so that I can investigate them later and maybe recover something - or retrieve the file from somewhere else.

 

[honestone33]

Not sure if this suggestion will completely solve your issue, but I use ClamXAv to check for Adware/Malware/Virus "issues", and if it finds any, such files/folders will be quarantined to another folder. Malwarebytes is somewhat similar, but ClamXAv tends to go "deeper".

Of course, this will depend on the type of corruption that exists on the files/folders in question.

Also, here are the results of a google search for "Software to ignore corrupted files when copying information on a Mac":

Software to ignore corrupted files when copying information on a Mac - Google Search

 

[pm-r]

Along the same sort of suggestions as honestone33 made, I found when using Carbon Copy Cloner (CCC) and it came across a corrupted file, it would stop and give the name of the file.

I don't recall if it did any more than that, but if it still does that, it would give you the name of the corrupted file so you can deal with it, and delete it or whatever.

I don't know if ClamXAv.app has a free trial period since it went commercial, but you may still have an old version hiding out on your hard drive.


- Patrick
======

 

[krs]

Just to emphasize what I wrote:

When I copy data from a drive with some corrupt files, the copying just stops when it gets to the corrupt file or folder.
I'm looking for a method, or even different software, that will copy all non-corrupt files and either ignore the corrupt ones, or even better, quarantine the corrupt ones so that I can investigate them later and maybe recover something - or retrieve the file from somewhere else.

I am not talking about adware/malware/virus issues so malwarebytes or ClamXav will do nothing.
I am also not talking about creating a clone or bootable back-up.

I am simply copying data from a drive where the odd file has been corrupted for some reason (in the current case the reason being a failing drive) and where I want to copy all non-corrupted data to another drive.

All I found so far are some suggested terminal commands that supposedly do that - I prefer not to use the terminal.

 

[pm-r]

Just to emphasize what I wrote:

OK, but to save us flogging the horse unnecessarily, what have you tried using so far???

What about using the Finder copy or move and/or what about creating a Disk Copy.app image and move it???

Or what about trying a file recovery application???

But you could still use the software that flags and stops, and then deal with the corrupted file(s) to get them out of your way.

I don't think you are going to be able to escape and take a shortcut without dealing with the corrupted files first, but maybe have a look at the suggestions that honestone33 provided.



- Patrick
======

 

[krs]

So far I just used the straight way of copying to an external drive - grab the folder and drag it to the external drive icon.
Folder is big - takes about 3 hrs to copy.
There is no indication that there is a corrupt file somewhere within that folder.
So basically what happens is that you start the copy process, go out for three hours, come back and expect copying to be completed, but no
10 minutes in, a corrupt file is encountered and all copying comes to a grinding halt.

So
a. The job didn't get done
b. I now start to compare the files and folders that did get copied to the source.
Last time that happened, only 5 MB of a 20MB folder within the 1 GB part of the drive I was copying was actually copied when the copying stopped.
So then I had to go into the 20 MB folder which of course had a bunch of other nested files and folders in it and copied those one by one until I came across a video file that couldn't be copied because it was corrupt.
I deleted that and then started the rest of the copying process.

Luckily there was no other file somewhere in the rest that was corrupt or I would have had to start that time consuming process all over again.

I also copied other 1 GB and larger folders where nothing was corrupt and the copy was done - so I don't know ahead of time if something is corrupt to stop the copying or the copy process will continue to the end.
I was just looking fpr a copy method that bypasses any corrupt files without the whole copying process stopping.

 

[MacInWin]

krs, I know exactly what you are talking about and I've never found a solution that was really automatic. Every time the copy process hits a corrupted file, it stops. And then I go searching, just like you did. I haven't tried the CCC approach, but if it will give the name of the file, at least, then it could be used to get the name, address the issue, then restart the copy process. CCC will ignore what it has already copied to the destination and get to the file after the last it copied pretty quickly. But it's still going to be a manual process.

And because the corruption is not a virus or malware, just lost sector pointers, etc., no antivirus software will fix or quarantine it.

One possibility is to run one of the disk "fixer" tools on the drive, but the risk there is that the "fix" is worse than the copy process. IF the fix deletes something incorrectly because of the file corruption, you could lose even more data.

So, from my experience, the bottom line is that you are going to have to fix the problem manually, but CCC may speed it up if it does identify the messed up file(s) for you.

 

[krs]

Jake,

If you are comfortable using the terminal - I'm not, I only use it if there is no other option - you might try this which I picked up on the net:

Terminal command:
cp -Rfv sourcefile destinationfile

where:
cp = copy
R = maintains file hierarchies
f = if an existing destination file cannot be opened, remove it and try again
v = verbose mode, displays files transferred as it progresses
sourcefile = data you want to copy
destinationfile = directory/drive you want to copy to



This command doesn't "ignore" errors but the "f" flag forces through them, only copying intact data. As you watch the progress (thanks to the verbose flag) you will see output along the lines of "data not copied due to I/O error" when it encounters bad data. Once the error is displayed it moves on to the next file without any user input.


 

[MacInWin]

You had previously said you didn't want to do Terminal commands. I use them occasionally, but I prefer to work through the GUI interface. I've only had to recover corrupted data a few times in the 38 years I've had a personal machine, so I've just put up with the hassle and done it the "slow" way.

 

[pm-r]

As you watch the progress (thanks to the verbose flag) you will see output along the lines of "data not copied due to I/O error" when it encounters bad data. Once the error is displayed it moves on to the next file without any user input.


That sounds like it's being pretty **** "automatic" and what you were after for a solution to me.


- Patrick
======

 

[honestone33]

Jake,

If you are comfortable using the terminal - I'm not, I only use it if there is no other option - you might try this which I picked up on the net:

Terminal command:
cp -Rfv sourcefile destinationfile

where:
cp = copy
R = maintains file hierarchies
f = if an existing destination file cannot be opened, remove it and try again
v = verbose mode, displays files transferred as it progresses
sourcefile = data you want to copy
destinationfile = directory/drive you want to copy to



This command doesn't "ignore" errors but the "f" flag forces through them, only copying intact data. As you watch the progress (thanks to the verbose flag) you will see output along the lines of "data not copied due to I/O error" when it encounters bad data. Once the error is displayed it moves on to the next file without any user input.


That command was the first "hit" i the link I provided above (I also saw it). Was there anything else helpful in those other "hits"?

 

[honestone33]

I don't know if ClamXAv.app has a free trial period since it went commercial, but you may still have an old version hiding out on your hard drive.


- Patrick
======

It does still have a free Trial period, for 30 days.

 

[krs]

You had previously said you didn't want to do Terminal commands. I use them occasionally, but I prefer to work through the GUI interface. I've only had to recover corrupted data a few times in the 38 years I've had a personal machine, so I've just put up with the hassle and done it the "slow" way.

I still don't want to use terminal commands and I mentioned that right at the beginning of the post.

But since it seems others, you for one, and others as well probably, have come across the same issue, I thought I would post this "solution" for those who are comfortable using the terminal.
I have no clue if that works or if there is a risk to do this.

Last time I tried to use the terminal was just a few days ago when I wanted to check the health of my external drives using DriveDX.
They provided two terminal commands to provide some interface drivers so that DriveDX could check external drives when running ElCapitan.
I copied and pasted the first one (to make sure I get all the spaces right) and the first thing I get is a huge warning telling me how I can really screw up the OS unless I really know what I'm doing and then I needed to enter a password to continue.
I tried my admin password which was just ignored, maybe there is a special way to enter it or maybe they wanted some different password.
Anyway - that was enough to keep my fingers from typing command in terminal.

 

[krs]

That sounds like it's being pretty **** "automatic" and what you were after for a solution to me.

....which makes one wonder why that is not included as an option when copying.

It's using the terminal and the potential risk involved I don't like.
The very few times I used the terminal there always seems to be the message - double check and triple check that everything is entered 100% correctly and then one doesn't even know if the person providing the terminal command knows what he/she is doing.

 

[MacInWin]

krs, when you enter a password in Terminal nothing shows on the screen. It's totally blind so that no one can peer over your shoulder to see it. Back-in-the-day, that was considered a security "feature."

 

[krs]

That command was the first "hit" i the link I provided above (I also saw it). Was there anything else helpful in those other "hits"?

To be honest, I didn't look through the hits.
I don't really need anyone to just post the link to a google search - I can google myself and I do before I post a question here.

Since you saw that potential solution, why did you not post it in the forum or at least the link to the discussion where that came up?

 

[krs]

krs, when you enter a password in Terminal nothing shows on the screen. It's totally blind so that no one can peer over your shoulder to see it. Back-in-the-day, that was considered a security "feature."

See...I didn't know that.
I expected x's or dots or something to come up.
So I should have just typed in the second line of the command...but I decided to quit while I was ahead (and hadn't screwed up anything)

 

[MacInWin]

Terminal, also called the Command Line Interface, or CLI, is a very powerful tool, and one not to be used by a novice carelessly. Some commands that look "simple" can do major damage. The command you showed was to copy, which is relatively safe to use and from the look of it, should work. If it does ANY damage, it will be to the destination drive, not the source, so that's good. The one challenge you still will have to address is that the source drive will still have lots of defective files on it, so it will probably require a full format to get to a usable state.

 

[krs]

The one challenge you still will have to address is that the source drive will still have lots of defective files on it, so it will probably require a full format to get to a usable state.

I was actually thinking of getting it replaced under warranty.
Seems things are getting worse - left is the DriveDX report two days ago, right is the one right now (after the drive was unplugged for a day so the temp only shows 25C
Do you think I should try to use it again?
I have all the files copied, so I can do a complete reformat

 

[Raz0rEdge]

Back to the original topic of discussion, virtually every software is going to come down to the basic OS call to do the copy and that call is going to throw an error when it comes across a bad block or unreadable chunk. There isn't really a way of getting past that error since you don't know where to go when an error is thrown. The error could be one of many things, the most simple one is inode corruption. A file isn't stored linearly on the file system, it is scattered around in bits and pieces and the inodes help keep track of those pieces. If the inode list is corrupted, you can't figure out where to go next and you are notified of failure.

I can't imagine any application getting around that.

However, you could like do this work with a script whereby you would catch that an error was thrown, skip that file and move onto the next file.