Shell scripts run slow on my Macs

I have a shell script that determines the status of a job directory.

When I run this script on my linux boxes for 100 directories, it takes about 2 minutes.

When I run it on my macs (G5 2.0 Ghz machines) and 100 directories, it takes about 11 minutes.

The linux boxes are similar type machines, dual processor 2.2 Ghz Opterons, so I wouldn't expect a big discrepency.

The real problem is that I have something like 30000 directories to go though, so right now it takes a bit of time.

Just wondering if anyone has any ideas. I am thinking about coding it up in perl and running it that way. Of course I would have to learn perl to do that. :headphone

Thanks,
Brett

Assuming the two directories contain the exact same thing, then perhaps it has something to do with the priority that the system gives to your scripts or the terminal process.

Perhaps one or more external commands are slower on the Mac?

Why would anyone have 30000 job directories?

If the script is complicated enough, I'd put some statements in to echo the time or time differences and try to figure out if it is one location of the code being slower on the Mac compared to the Linux box.

I can't see Perl giving you any advantages over an already working shell script.

If you figure out the real answer, please give us an explanation.

I rewrote the script in perl, very ugly perl, but performance is now about the same on all of my systems. Very odd. I don't really understand that, but it gives me my end result.

The perl script is ~ 400 loc, while the shell was ~100.

Maybe later, I'll try and figure out the differences.

Thanks,
Brett

xstep said:

...

Why would anyone have 30000 job directories?

...

Click to expand...

Its the nature of the simulation work and initial conditions. One set of initial conditions produces a 2d slice of information in a 6 dimensional space, so we simulate a lot. In order to do this we run using gridware (macs are our newest machines - we started out with gridware, but are considering xgrid) and if we run multiple initial conditions in a directory it causes some file writing issues.

BTW - thats 30000 on one mac network. I have another mac network that is smaller that has something like 5000 directories, and a sun network, and a network of dual-processor opterons, and a network of quad-processor opterons, and a network of dual-xeon boxes. Over 100k directories total. Each directory has between 10 and 50k files, although we can clean up the directories in a postprocessing step. This eats up the inodes fast. :black:

Brett