AFNI Message Board

Dear AFNI users-

We are very pleased to announce that the new AFNI Message Board framework is up! Please join us at:

https://discuss.afni.nimh.nih.gov

Existing user accounts have been migrated, so returning users can login by requesting a password reset. New users can create accounts, as well, through a standard account creation process. Please note that these setup emails might initially go to spam folders (esp. for NIH users!), so please check those locations in the beginning.

The current Message Board discussion threads have been migrated to the new framework. The current Message Board will remain visible, but read-only, for a little while.

Sincerely, AFNI HQ

History of AFNI updates  

|
September 01, 2009 09:51AM
Generically, AFNI programs might use mmap() to load datasets into memory from disk; but as you say, you can disable this functionality with the environment variable AFNI_NOMMAP.

However, 3dDeconvolve and 3dNLfim also use mmap() to create shared memory segments to hold the workspace common between the different processes fork()-ed off by the -jobs option.

There is an older version of the code which uses the shm facility (e.g., shmget) to create the shared memory required. You could try to re-enable that code, recompile 3dDeconvolve, and see if it works for you. To do so, insert the following line of code immediately after the current line 4682 in the source (you must be sure you have the latest AFNI source code, by which I mean download afni_src.tgz again today):

#undef MAP_ANON

The 4 lines just before where this should go are:

4679 #include <sys/mman.h>
4680 #if !defined(MAP_ANON) && defined(MAP_ANONYMOUS)
4681 # define MAP_ANON MAP_ANONYMOUS
4682 #endif

A limitation of using shm is that it supports at most 2 GB of shared memory, whereas the mmap() interface allows for larger blocks of memory when programs are compiled in 64-bit mode. If this works for you, then I can modify 3dDeconvolve.c to enable the shm mode of memory sharing via a command line option.

An alternative for you would be to use 3dREMLfit, which can do most things 3dDeconvolve does with its '-O' (for Ordinary Least SQuares) options, and can be compiled to use OpenMP parallelization, which is automatically shared memory all the way (unlike the fork() method that 3dDeconvolve uses, which requires explicit management of shared memory via mmap or shm). This option requires compiling with gcc 4.2 or above, or some other C compiler that supports OpenMP. At present, we only provide OpenMP-ized binaries for Mac OS X 10.5 (soon for 10.6, I hope).

[afni.nimh.nih.gov]
[afni.nimh.nih.gov]

[afni.nimh.nih.gov]
[afni.nimh.nih.gov]

However, my brief reading of the Mosix pages indicates that the shared memory approach to parallelization that I use in AFNI is not well supported. Mosix seems to want processes to communicate via sockets (or something similar), but I made the decision long ago not to go that way because it would involve very extensive changes to the source code -- using OpenMP is much simpler, and is the way future AFNI parallelization efforts are going, for as long as I can foresee:

[afni.nimh.nih.gov]
[afni.nimh.nih.gov]

Subject Author Posted

Parallelize AFNI

Giacom September 01, 2009 06:45AM

Re: Parallelize AFNI

bob cox September 01, 2009 09:51AM