Obtaining the directory of a hard drive from Spotlight

Crashed external HD. Can I did into Spotlight data to find exactly what was there when it was last used?

Sorry, meant to say "dig?" I have some backups of an external four-partition 160 GB HD that crashed about six months ago. The cost to repair is so great, that I wanted to see first what, if anything, I had there that I needed and was ONLY there. Since my external HD is only used plugged into one of my macs and since spotlight is always indexing it, I figured I ought to be able to use the data in Spotlight to recreate a directory listing of my files and see what I had there that I might really find worth trying to recover. I searched the web and came up empty. Does anyone here know how to get at the data in Spotlight in order to do something like this?

Thanks.

The Directory containing the Spotlight index is kept on the root of the volume it indexes. Therefore, the Spotlight index of the dead drive would be at the root of the volume (or volumes, depending on how the partitions were created) on the dead drive.

Maybe you are correct and I cannot recall that well because it has been a few months now since it died, but I really thought that I could see in Spotlight files on external HDs after I had had disconnected them. So, no info is stored in the system of the indexing HD, only in each individual indexed HDs? I feel like the files from other (no longer attached) HDs were listed there but greyed out. Are you sure?

The index for Spotlight is housed at /.Spotlight-V100/ which is at the root of your boot drive. It's organized in a non-easily readable fashion and I'm not sure it would be parseable outside of Spotlight itself (unless a third party dev has cracked the code so to speak).

That is what I had thought. You are sure that it houses the info about HDs that are no longer connected or only the boot drive? Do you know how to open that invisible folder? Do I try getting to it from FileBuddy and then duplicating it elsewhere?

Spotlight keeps a separate /.Spotlight-V100/ Directory at the root of each individual volume it indexes. It does not keep a single /.Spotlight-V100/ Directory at the root of the startup volume that contains the index of all volumes.

You can test this yourself by:

a) Opening an external drive that has been indexed and after you enable viewing hidden files - you will find a /.Spotlight-V100/ Directory at the root of that external drive

b) Do a Spotlight search for an item that is only located on an external drive that has been indexed, both while it is connected and while it is disconnected. You will find 2 different search results.

You can apparently use the mdfind tool to get a listing from a Spotlight index. If you open up Terminal with the drive plugged in, you should be able to execute the following command to get a list of the files indexed: An easy way to get the path is to simply enter the mdfind part, press space and then drag the drive icon on to the Terminal window.

Thanks for the info on mdfind. It is very nice and gives you a complete directory listing of your files. It would be one step nicer if you could get some more info on each, such as creation date, modification date, and size. However, if bobtomay is right (and I have no reason to disbelieve him), then in my situation, I am still stuck, right? I was hoping that since Spotlight stored finder info on each of my hard drives, that if one crashed and I did not have a fully up-to-date backup, I could use spotlight data and my last backup to determine what was on the hard drive that was not backed up. Thus, I could make an more-informed decision as to whether I wanted to try to spend a large sum of money to try to recover files from the hard drive. If this data is only stored on the individual drive, it is no use when that drive crashes. Wouldn't a product that continuously backed up the spotlight index onto the boot drive serve a useful purpose?

I was hoping that since Spotlight stored finder info on each of my hard drives, that if one crashed and I did not have a fully up-to-date backup, I could use spotlight data and my last backup to determine what was on the hard drive that was not backed up.

Click to expand...

Spotlight doesn't store Finder information, so that whole premise is wrong. Making the assumption that it does do that is unfounded. Basically, what is good practice is to have backups, not try to use a search tool as some sort of database for what was.

Wouldn't a product that continuously backed up the spotlight index onto the boot drive serve a useful purpose?

Click to expand...

No.

I know of course that there is no substitute for thorough regular backups. Period. However, companies that provide hard drive recovery would have no business whatsoever if everyone backed up as they should all the time. And though Spotlight may not store all the info on files that Finder does, am I wrong to assume that it does store file names and locations? This is what you see using the mdfind command in terminal, as suggested a few posts above. Is mdfind not really reading some data from Spotlight but actually just generating this list of files and locations independently? Can you educate me on what Spotlight is doing when it indexes your hard drive? I appreciate your knowledge of the subject.

Spotlight is a GUI for the utility mds (metadata server). mds and mdworker are utilities that inventory hard drives for Spotlight. They are separate from Finder. Finder is a file manager, Spotlight (and mds and mdworker and mdfind) are all associated with the hidden index you have already discovered.

There is another command that returns the information you seek: mdls

To see how to use it, open Terminal and type

man mdls

Click to expand...

That will give you the syntax for the command. You can try to see if there is a way to take the output through some filter to get more than one file at a time, but it's going to be ugly. You might use some combination of mdfind and mdls. mdfind will give you names you might be interested in the rest of the data, then mdls can give you the information on that file. Again, ugly but works.

As for disaster recovery utilities, you are right that if everybody backed up the developers and companies would suffer. But personally, I've had a double drive failure of both the drive and the backup, so I know that even with a backup discipline, you can lose data and be forced to a recovery company or product. Using your logic, however, I could make the argument that if everybody just obeyed the law in driving then there would be no need for body shops, or tow trucks, or accident insurance. Won't happen, so those things are needed. But I'm not going to buy insurance and then drive crazy, and I'm not going to rely on data recovery to protect me when I can do a backup and be reasonably safe. (As an aside, I now do TWO backups to two separate backup drives with two separate backup products as a lesson learned from the dual failure I mentioned.)

Good luck with it, not having a backup sucks...

Jake, thanks for the explanations (and being patient with me). I do have a backup, but at the time it was about 2 wks old. At least two of my partitions I know I hadn't touched, but I was probably doing work on the other two. Since the recovery people (after I had tried running Data Rescue 3 and others) wanted about a grand, I just decided to wait. Now, I wish I knew if there was really anything there that I did not have elsewhere and really did need (or want). So, once again, do you know if it is right that this data stored by Spotlight (if I understand it right, it must be making an index all the time) is on the boot HD and contains info on all the other HDs (internal and external) or is the Spotlight data only stored on the individual HD itself? In other words, is any Spotlight data still retained on the boot drive when an external HD is disconnected? If not, then I am really stuck and have to decide if wanting to know what I may have had on my dead drive is worth the grand. Argh. As for backups of stuff that I really know is important, I like your idea. I had lost some files when my backup was to a RAID drive set up in parallel (I forgot what you call it, but essentially backing up to two HDs simultaneously.). The RAID controller somehow wouldn't let me access the second drive when the first one went bad. For now, I burn DVDs via two methods and try to feel confident that this data is safe.

Anyway, thanks again.

bobtomay said:

The Directory containing the Spotlight index is kept on the root of the volume it indexes. Therefore, the Spotlight index of the dead drive would be at the root of the volume (or volumes, depending on how the partitions were created) on the dead drive.

Click to expand...

I think Bobtomay answered this for you in post #3, and repeated in post #7.

Ok, thanks. I guess I was just hoping that it were not true. To paraphrase those ads I see for healthcare these days, "Hope is not a good backup system."

After 6 months you haven't remembered even a single piece of data that you can't find and you know was on that drive?

Personally, I would not waste another 30 seconds of thought nor effort, and definitely no money, until the above condition is met.

I'm with bobtomay, if you haven't missed it in six months, you don't really need it. If you want to be super cautious, hold onto the drive for another six months and if in that full year you don't know of anything you can't find just reformat/test and get on with it.