Let me start off by saying that Xgrid has been discontinued and when you upgrade your Mac Pros to 10.8 (Mountain Lion) or later, you won't be able to use it anymore. Apple has stated that they will not open source the technology so ultimately you may end up migrating to something like Hadoop (http://hortonworks.com), Condor (http://research.cs.wisc.edu/condor/), or GNU Parallel (http://www.gnu.org/software/parallel/). Any of these will manage multiple computers and multiple instances on each computer.

That said, it is possible to run AFNI programs (such as afni_proc.py) on Xgrid. If you are using Mac OS X Server to manage all of your Mac Pro nodes, then you could implement a kerberos setup that would allow for each computer to access a shared file system, reducing the amount of time to copy things to the Xgrid controller and then copying the results back at the end - more details on how to setup such a system can be found on TenGrid (http://tengrid.com).

If you don't want to go through that setup but have a Mac OS X Server, then the first step is simply to turn on the XGrid Controller. If you don't have Mac OS X Server, then you will need to download Xgrid Lite (http://edbaskerville.com/software/xgridlite/). Once you enable Xgrid Lite or the XGrid Controller via OS X Server, you will need to turn on Xgrid on all of your client nodes and point them to use your controller. Having a domain name or a static IP is going to be helpful for this. Finally, You will need to install AFNI on all of your nodes or have it on a shared file system that they all automatically mount.

Submitting jobs to an xgrid cluster can work like this:

xgrid -h <your controller name, or ip> \
-p <password for controller if you use one> \
-job submit \
-in <directory to copy to the cluster> \
<command to run on cluster> \
<single argument for command if necessary>.

The basic example:

xgrid -h myController -p myPassword -job submit -in Subj001 ls -al *

This code will run ls for a listing of the directory that is copied to the Xgrid cluster. To translate this to running AFNI, it's helpful to write a bash script that will correctly set the path and issue the series of commands that you will use. Here's an example bash script, you could put your entire afni_proc.py script in here, just remember to give it the -execute flag:

#!/bin/bash
#save as run3dvolreg.sh

export PATH=$PATH:/usr/local/abin #modify for your path to AFNI
export OMP_NUM_THREADS=2 #need to set this to a number as you may be running multiple instances simultaneously

3dvolreg -base $1[0] -source $1 -prefix run1_volreg -quintic -1Dfile $1.1D
#could do -prefix ${1%.*}_volreg to remove .nii or .HEAD

And then you would run this on the Xgrid using the following commands (note that the command is run inside the "-in" directory):

xgrid -h myController -p myPassword -job submit -in Subj001 run3dvolreg.sh run1.nii.gz

At this point the cluster will give you a job ID number. You can then get information on the job by issuing the command:

xgrid -h myController -p myPassword -job attributes -id <jobID>

with output:

OUTPUT (Job In Progress):
   jobAttributes =     {
        activeCPUPower = 2400;
        applicationIdentifier = "com.apple.xgrid.cli";
        dateNow = "2011-05-17 15:15:36 -0400";
        dateStarted = "2011-05-17 15:15:32 -0400";
        dateSubmitted = "2011-05-17 15:15:17 -0400";
        jobStatus = Running;
        name = "run3dvolreg.sh";
        percentDone = 0;
        taskCount = 1;
        undoneTaskCount = 1;
    };

You can also check the status of a job using the Xgrid Admin tool provided by Apple in the Mac OS X Server Admin Tools package. You'll need whatever version of the OS that you are running (e.g. 10.6 or 10.7).

To retrieve the results from the cluster use this command:

xgrid -h myController -p myPassword -job results -id <jobID> -out outputDirectory

This will copy the results from the script to whatever you listed as your output directory. It can be helpful to use your submission script to tar or gzip your data so that the retrieval from the server is faster.

Running AFNI on the xgrid can be a somewhat involved process. You also need to make sure that each AFNI instance (via OpenMP) won't try to use the maximum number of processors available as this will cause things to grind to a halt. In combination with setting your OMP_NUM_THREADS, you can tell each Xgrid agent how many processes it can run simultaneously using the following (change 8 to the number of simultaneous jobs you want to run:

defaults write /Library/Preferences/com.apple.xgrid.agent ProcessorCount 8

---

If you wish to use Xgrid to run your freesurfer segmentations, I have contributed this information to the FSWiki (http://surfer.nmr.mgh.harvard.edu/fswiki/XgridNotes).