16.1.4. Communication: NIML and Live Data¶

AFNI programs rarely live in isolation. They exchange datasets on disk, but they also talk to each other and to the GUIs (AFNI and SUMA) while running – the real-time image feed, InstaCorr seed-to-map updates, and GroupInCorr group correlation all send structured data over a socket as the user clicks. The common substrate for all of this is NIML.

NIML is AFNI’s data-interchange layer – think “AFNI’s XML plus a binary wire format.” It defines self-describing data elements and groups, and a stream abstraction that carries them over sockets, shared memory, files, or strings with the same API. A surprising amount of AFNI is NIML underneath: .niml.dset files, surface datasets (Surfaces), the +orig header attributes, and every GUI-to-engine conversation.

The NIML data model ¶

`NI_element` – a table of typed columns ¶

An NI_element (in niml/niml.h) is a named object holding some attributes (name = value pairs) and some data columns (“vectors”). All columns share the same length; each has its own type.

typedef struct {
   int    type ;        /* NI_ELEMENT_TYPE */
   char  *name ;        /* element name, e.g. "3dGroupInCorr" */
   int    attr_num ;    /* number of attributes */
   char **attr_lhs, **attr_rhs ;   /* attribute name / value strings */
   int    vec_num ;     /* number of data columns */
   int    vec_len ;     /* length of each column (number of rows) */
   int   *vec_typ ;     /* type code of each column (NI_FLOAT, NI_INT, ...) */
   void **vec ;         /* the column data pointers */
   ...
} NI_element ;

Build one by naming it and declaring the row count, then adding columns:

NI_element *nel = NI_new_data_element( "my_table", nrows ) ;
NI_add_column( nel, NI_FLOAT, xvals ) ;      /* column 0 */
NI_add_column( nel, NI_FLOAT, yvals ) ;      /* column 1 */
NI_set_column_label( nel, 0, "x" ) ;
NI_set_attribute( nel, "source", "3dProgName" ) ;

Column type codes are NI_BYTE, NI_SHORT, NI_INT, NI_FLOAT, NI_DOUBLE, NI_COMPLEX, NI_RGB, NI_STRING (and a “rowtype” mechanism for structs). Read values back with NI_extract_float_value(nel, row, col) and friends, or index nel->vec[col] directly after casting. Free with NI_free_element(nel).

`NI_group` – a bag of elements ¶

An NI_group collects elements (and nested groups) plus its own attributes – exactly how a dataset header (many attribute elements) or a surface SUMA_DSET (data element + node-index element) is represented.

NI_group *grp = NI_new_group_element() ;
NI_set_attribute( grp, "self_prefix", "thing" ) ;
NI_add_to_group( grp, nel ) ;         /* add an element (or another group) */

Streams: the same API over any transport ¶

An NI_stream is opened from a name string that encodes the transport, so the same read/write code works whether you are talking to a socket or a file:

Stream name	Transport
`"tcp:host:port"`	TCP socket (2-way); one end opens `"w"`, one `"r"`
`"shm:key:size1+size2"`	shared memory (2-way, same machine, fast)
`"file:path"`	a file (1-way)
`"str:"`	an in-memory string buffer (great for testing)
`"fd:0"` / `"fd:1"`	an existing file descriptor (stdin/stdout)

NI_stream ns = NI_stream_open( "tcp:localhost:53212", "w" ) ;
/* a socket connect is not instantaneous -- wait for it to be good: */
if( NI_stream_goodcheck(ns, 5000) < 1 )     /* up to 5000 ms */
   ERROR_exit("could not connect") ;

NI_write_element( ns, nel, NI_BINARY_MODE ) ;   /* or NI_TEXT_MODE */
...
NI_stream_close(ns) ;

On the receiving side:

NI_stream ns = NI_stream_open( "tcp:*:53212", "r" ) ;  /* listen */
while( NI_stream_goodcheck(ns,100) < 1 ) { /* wait for a connection */ }
void *nini = NI_read_element( ns, 1000 ) ;             /* blocks up to 1000 ms */
int typ = NI_element_type(nini) ;   /* NI_ELEMENT_TYPE or NI_GROUP_TYPE */

NI_write_element serializes in text or binary; NI_read_element returns whatever comes next (element or group – check NI_element_type). For a whole file at once there are NI_read_element_fromfile and NI_read_element_fromstring. Programs that speak NIML over sockets include 3dGroupInCorr, 3dAFNItoNIML, im2niml, and the AFNI GUI’s afni_niml.c.

Talking to the AFNI/SUMA GUIs ¶

Plugouts and the `-niml` port ¶

AFNI can be driven from an external process (“plugout”). Start the GUI with afni -niml (or -yesplugouts) and it listens on well-known ports obtained through get_port_named(...) – e.g. get_port_named("AFNI_PLUGOUT_TCP_0"). Your process opens a matching tcp: stream and sends NIML command elements. The shipped plugout programs are the templates to copy:

src/plugout_drive.c – send driver commands (open a window, jump to an xyz, set a threshold, take a snapshot). This is the C sibling of the plugout_drive / DriveAFNI command line and of DriveSuma.
src/plugout_ijk.c, src/plugout_tt.c – exchange the current crosshair voxel with AFNI.
src/plugout_surf.c – surface node exchange.

The real-time plugin ¶

For streaming volumes into AFNI as they are acquired (from a scanner or a simulator), the receiver is src/plug_realtime.c and a minimal sender is src/realtime_callback_example.c. The wire format is again NIML: a control element describes the image geometry, then image data is pushed brick by brick. README.3dsvm.realtime and the real-time example are the places to start.

InstaCorr: live seed-based correlation ¶

InstaCorr computes a whole-brain correlation map from a seed voxel the instant you click. The engine is src/thd_instacorr.c and it is a good study in “prepare once, query many times”:

THD_instacorr_prepare(ICOR_setup *iset) does the expensive one-time work – it extracts every in-mask voxel time series into an MRI_vectim (via THD_dset_to_vectim_stend), detrends, band-passes, blurs, and normalizes them so that a later correlation is just a dot product.
THD_instacorr(ICOR_setup *iset, int ijk) is the fast per-click call: it grabs the seed time series for voxel ijk and returns an MRI_IMAGE of correlations against all the prepared voxels.

The GUI side (afni_pplug_instacorr.c) wires a click to that call and displays the result. The design pattern – convert once to a normalized time-first MRI_vectim, then answer each query with a dot product – recurs throughout AFNI’s correlation tools; see The Function Cookbook for the vectim and correlation primitives it is built on.

GroupInCorr: correlation across a group, over a socket ¶

GroupInCorr extends the same idea across a group of subjects and splits it across two processes that talk NIML:

src/3dGroupInCorr.c is the compute engine. It loads the pre-processed per-subject data (prepared by 3dSetupGroupInCorr into a .grpincorr.niml header plus a big .data file) and then listens on a socket for seed commands.
The AFNI or SUMA GUI is the other end. You start afni -niml; when the user picks a seed, the GUI sends the seed voxel/node to 3dGroupInCorr, which correlates every subject, runs the group statistic (t-test, etc.), and sends a result dataset back for display.

The connection is a NIML tcp: stream (SUMA’s default port is SUMA_GICORR_PORT = 53224, resolved via get_port_named("SUMA_GroupInCorr_NIML")). The messages themselves are NIML elements: the setup element is named "3dGroupInCorr"; status/error messages travel as "3dGroupInCorr_message" elements. Reading 3dGroupInCorr.c end-to-end is the best single tour of “AFNI program as a live NIML server” – it shows the listen loop, the element protocol, the per-seed compute, and the send-back.

Note

For most new programs you will not implement a socket server – you will use NIML to read/write .niml files and surface datasets. But knowing that the same NI_element / NI_stream API scales all the way up to the live GUI protocols means you rarely need a second serialization mechanism.