3dClustSim does not actually
read data files (at least, the way you are using it) -- the program just creates simulated data in memory and processes it. So it can use the CPUs at nearly 100%. In the jargon of computing science, it is "compute bound".
3dFWHMx, on the other hand, does pretty simple calculations after reading the dataset in (the
-acf computations are a little more complicated). It is "I/O bound" -- and in your case, this is showing up painfully.
I don't know how to help you in your situation, short of telling you to copy the input file for 3dFHWMx to a local disk, use it there, then throw it away. But if you are running it via afni_proc.py, that isn't much help.
The NIH compute cluster has a similar problem, with fast Linux nodes and a slow networked filesystem. The solution they offer is that each Linux node has a SSD (solid state drive) local to it, that can be used only by the jobs running on that node -- and when a job is done, its space on the SSD is erased. The way this speeds things up is to (1) copy all inputs to the SSD, (2) process them there, and (3) copy the outputs back to permanent storage. I am doing something like this right now on a series of jobs, and they run about 30-50% faster than they did before I discovered this "trick". Perhaps something like this is available on your server?
Edited 1 time(s). Last edit at 10/27/2016 02:24PM by Bob Cox.
