History of AFNI updates  

|
September 01, 2009 09:51AM
Generically, AFNI programs might use mmap() to load datasets into memory from disk; but as you say, you can disable this functionality with the environment variable AFNI_NOMMAP.

However, 3dDeconvolve and 3dNLfim also use mmap() to create shared memory segments to hold the workspace common between the different processes fork()-ed off by the -jobs option.

There is an older version of the code which uses the shm facility (e.g., shmget) to create the shared memory required. You could try to re-enable that code, recompile 3dDeconvolve, and see if it works for you. To do so, insert the following line of code immediately after the current line 4682 in the source (you must be sure you have the latest AFNI source code, by which I mean download afni_src.tgz again today):

#undef MAP_ANON

The 4 lines just before where this should go are:

4679 #include <sys/mman.h>
4680 #if !defined(MAP_ANON) && defined(MAP_ANONYMOUS)
4681 # define MAP_ANON MAP_ANONYMOUS
4682 #endif

A limitation of using shm is that it supports at most 2 GB of shared memory, whereas the mmap() interface allows for larger blocks of memory when programs are compiled in 64-bit mode. If this works for you, then I can modify 3dDeconvolve.c to enable the shm mode of memory sharing via a command line option.

An alternative for you would be to use 3dREMLfit, which can do most things 3dDeconvolve does with its '-O' (for Ordinary Least SQuares) options, and can be compiled to use OpenMP parallelization, which is automatically shared memory all the way (unlike the fork() method that 3dDeconvolve uses, which requires explicit management of shared memory via mmap or shm). This option requires compiling with gcc 4.2 or above, or some other C compiler that supports OpenMP. At present, we only provide OpenMP-ized binaries for Mac OS X 10.5 (soon for 10.6, I hope).

[afni.nimh.nih.gov]
[afni.nimh.nih.gov]

[afni.nimh.nih.gov]
[afni.nimh.nih.gov]

However, my brief reading of the Mosix pages indicates that the shared memory approach to parallelization that I use in AFNI is not well supported. Mosix seems to want processes to communicate via sockets (or something similar), but I made the decision long ago not to go that way because it would involve very extensive changes to the source code -- using OpenMP is much simpler, and is the way future AFNI parallelization efforts are going, for as long as I can foresee:

[afni.nimh.nih.gov]
[afni.nimh.nih.gov]

Subject Author Posted

Parallelize AFNI

Giacom September 01, 2009 06:45AM

Re: Parallelize AFNI

bob cox September 01, 2009 09:51AM