:orphan: .. _ahelp_3dSRM: ***** 3dSRM ***** .. contents:: :local: | Usage: 3dSRM [options] ============================================================== .. code-block:: none Fit a Shared Response Model (SRM; Chen et al., NIPS 2015) to time-locked data from several subjects, and measure how well it generalizes. Each subject i contributes a matrix X_i (features x time). SRM learns one shared response S (k x time) and, for each subject, a mapping W_i (features x k, orthonormal columns) so that X_i ~= W_i S + noise The subjects need not have the same number or arrangement of features. Subject i's voxels (or EEG channels) are read out into the same k-dimensional space, so no voxel-to-voxel correspondence across subjects is required. Only the columns must correspond. The columns can be time points of a stimulus every subject experienced at the same time (a movie, a story), or items (trial or condition betas) matched across subjects by a label, which is the right choice when trial order was randomized. See -axis. SRM is meaningless for resting-state data. Typical uses: * Subject-wise k x time series in a common space (the 'projected' output) for 3dRSA, 3dInSync, decoding, or a group t-test of a held-out score. * Dimension reduction that emphasizes shared responses. Noise can remain, and task-specific or individual differences can be lost. * Placing new subjects into an existing space, now or months later, from a saved model and without the original subjects' data (see -model_in). Required input: =========================================================== .. code-block:: none -dataTable @TABLE -dataTableFile TABLE -dataTable Subj Group InputFile s01 control s01_movie+tlrc ... TABLE needs Subj and InputFile columns, plus optional columns: Run : with one row per Subj x Run, each subject's runs are concatenated in time in the order the runs first appear. Every subject must have the same set of runs, each of the same length. See -run_column. Item : item label, with -axis item. See -item_column. Xmat : the subject's X.xmat.1D (from afni_proc.py or 3dDeconvolve), when one InputFile holds all of the subject's runs, such as afni_proc.py's errts. Its RunStart splits the input into runs, and its NRowFull must equal the input's length. See -concat. With -axis item it gives the items instead: see 'Items from afni_proc.py output' below. Either way each subject's Xmat is summarized in numeric columns that travel to scores.txt as covariates: Xmat.TR, Xmat.nrun, Xmat.ntime, Xmat.ncensored, Xmat.frac_censored, Xmat.nstim. Censor : the subject's censor file (1 keep, 0 censored), covering all of that subject's runs. See -cenmode. Mask : a mask dataset for that subject, on that subject's own grid. Subjects may then have different voxel counts and different grids. A subject's Mask must be the same on all their rows. Group : subject group, used to label and compare scores. See -group_column. others : any other column can be named in -score_by. Numeric columns are carried to the output tables, so covariates such as motion travel with the scores. InputFile is an AFNI dataset, or a .1D file. A .1D file is read as time x features (one row per time point, one column per channel or region), the usual AFNI orientation. Datasets and .1D files can be mixed in one table, which is how non-voxel data (EEG band power at TR resolution, parcel means) enter the same model. Every input must have the same number of time points. -nfeatures K[,K2,...] Dimension k of the shared space. Required, except with -model_in, which takes it from the model. K must not exceed the smallest feature count or the number of time points. Several values require -cv and produce a comparison table (PREFIX.nfeat.txt) and no model outputs; rerun with the chosen K for those. Model options: =========================================================== .. code-block:: none -prefix PREFIX Output prefix [SRM]. -algorithm ALG prob : probabilistic SRM, fitted by EM [default]. Estimates a noise variance for every subject, so noisier subjects (or noisier modalities) count for less in S. det : deterministic SRM, alternating Procrustes steps. Subjects are weighted by their feature count and variance only. -niter N Maximum iterations [500]. -tol T Stop when the relative change in the objective falls below T [1e-7]. SRM converges slowly on real data, which is why the defaults are generous. For 22 subjects hearing a story (1000 voxels each, 15 components), the shared response after 10 iterations, BrainIAK's default, differed from the converged one by 38%, after 20 by 30%, after 100 by 15%, and after 200 by 4%. Full convergence (1e-13) took about 550 iterations. -tol 1e-7 stops near 170 and -tol 1e-8 near 250. A fit stopped earlier is not wrong, only unconverged, and it depends on the seed. Whether a converged or an earlier fit predicts held-out data better is an empirical question: -cv answers it. For a quick look, -niter 20 is enough. -seed S Seed for the random starting W_i [1234567]. Until a fit has converged, its subspaces depend on the seed. Compare fits with rotation-invariant quantities, not component by component. -nstart N Fit from N different random starts and keep the one with the best objective [1]. Starts run one after another, so the cost is N times as much. The objective of every start is in PREFIX.fit.txt: if they agree to the tolerance you ran at, they found the same optimum. If they differ by more, run longer first. Only then is a difference evidence of a local optimum. In tests, both models reached the same optimum from every start on real data. One hard synthetic problem (noise differing 6-fold between subjects) stalled the deterministic model at a slightly worse stationary point from one start, so N of 3 to 5 is cheap insurance when the data are that uneven. -run_coupling L shared : one W per subject for all of their runs [default] L >= 0 : a separate W for each subject and run, each tied to a per-subject anchor W. The tie is a prior worth L time points of data: L=0 fits every run on its own, L equal to a run's length weighs the anchor as much as that run's data, and L much larger than the runs is nearly 'shared'. Try values from about 0.2 to 2 times a typical run length. Requires runs, and -axis time. The fit starts from the shared-W fit (so it takes about twice as long), and a very large L stays close to it: use 'shared' instead. Runs not in the fit (held-out runs, new data) use the anchor. See -cv_adapt. In-sample, a smaller L always fits better, because a W per run has more freedom; whether it generalizes is what -cv shows. With L=0 the runs can fall into different orientations of S. Data options: =========================================================== .. code-block:: none -mask MASK A mask for every subject lacking a Mask column. All such subjects then must share MASK's grid. Datasets need a mask from one source or the other; .1D inputs never do. Voxels with zero variance in any run (over all samples, with -zscore all) are dropped for that subject and counted in the log. -run_column NAME Run column in the table [Run]. If absent, each subject is a single run, unless -concat or an Xmat column says where runs start inside each input. -concat SPEC Run starts within each subject's one input, as in 3dDeconvolve and 3dTproject: the time-point index at which each run begins, starting with 0, as a 1D file or '1D: 0 300 600'. The same for every subject. For one input per subject that holds several runs (afni_proc.py's errts.SUBJ+tlrc, say); without it such an input is one long run, so z-scoring and -cv would ignore the boundaries. An Xmat column does the same per subject, from RunStart (every subject must still have the same runs). Not with a Run column. -group_column NAME Group column in the table [Group]. Only used to label the output tables. -zscore MODE run : z-score every feature separately within each run before fitting [default with -axis time] all : z-score every feature over all of the subject's samples together [default with -axis item: betas from one GLM share a baseline] none : use the data as given (it should already be centered and scaled) -censor FILE One common 0/1 vector with one value per input time point. Retained time points are identical for all subjects. Applied before z-scoring. -axis time only. -zcensor Treat a subject's volumes that are all zero in the mask as censored for that subject (afni_proc.py's errts zeroes its censored volumes). -axis time only (an all-zero item is always an error). Each subject's own censoring comes from any of: a Censor column (one 0/1 file per subject, 1 = keep, covering all of the subject's runs in order, such as afni_proc.py's censor_SUBJ_combined_2.1D), the GoodList of an Xmat column, and -zcensor. What a censored time point becomes: -cenmode MODE interp : filled in, for that subject only, by linear interpolation between the nearest kept time points of the same run (the nearest one at a run's edge), as 3dTproject -cenmode NTRP. Everyone keeps every time point [default: every subject's projected series stays complete, ready for 3dInSync or 3dRSA]. drop : the time point is dropped for EVERYONE. Each subject's censoring is then lost by all: 30 subjects with 5% censored at random keep only about a fifth of the time points. missing : that subject is left out at that time point only. The fit uses each subject where it was observed (deterministic: S at each time point is the mean of the subjects observed there; probabilistic: the posterior uses their precision only, and each subject's noise variance its own observed samples). Z-scoring is over observed samples. Scores use the samples where the subject, and someone else for the reference, were observed; segment matching skips windows that hold a missing sample. The more exact model, but PROJECTED series are 0 at a subject's missing time points. Each run needs at least 3 observed time points per subject (and, with -run_coupling 0, k). With -run_coupling, each run's W uses its observed samples (the anchor fills in the rest). A saved model keeps how many subjects were observed at each time point, so -update_S adds new subjects (with or without missing points) correctly. -apply_W z-scores (or centers) the new data over observed samples and writes 0 at missing ones. -censor (common to everyone) always drops. With -model_in the model's timeline is fixed, so drop is an error there. -zcensor_warn PCT Warn for each subject with more than PCT percent of its time points censored [20]. 3dSRM does not regress nuisance signals, filter, or change temporal alignment. Inputs should already be preprocessed and aligned to the stimulus. For EEG-derived inputs, the HRF and any lag must be handled before the data reach this program. Time-locked or item-indexed: =========================================================== .. code-block:: none -axis time Columns are the time points of a stimulus presented identically to everyone [default]. With one row per Subj x Run, runs are concatenated. -axis item Columns are items. The table has one row per Subj x Item, whose InputFile is a single volume (or a sub-brick selector such as betas+tlrc'[7]' or betas+tlrc'[face#0_Coef]'), or a .1D file with one row. Every subject must have every item exactly once, in any row order; items are matched by label, not position. Missing, repeated, or extra items are errors that name the Subj and Item. A dataset that holds many items is read once per subject, however many rows select from it. An item that is zero everywhere in the mask (usually a failed or missing estimate) is an error. -item_column NAME Item-label column [Item]. Labels may not contain ':'. Items from afni_proc.py output. A table with one row per subject, an Xmat column (the X.xmat.1D of the regression) and InputFile the stats bucket (stats.SUBJ+tlrc, or the REML one), and no Item column, is expanded to one item per stimulus column of the Xmat: a condition with one column is one item named by its label (house), and one with several columns gives an item per column, named as 3dDeconvolve names it (face#0, face#1, ...: the trials of -stim_times_IM, or the lags of a TENT basis). Each reads the bucket's LABEL#n_Coef volume. In a table file '#' starts a comment only at the start of a word, so face#0 and stats+tlrc[face#0_Coef] can be written as they are, or quoted as a whole ("face#0"). The new rows carry the subject's other columns (Group, Mask, covariates), a Condition column (the stimulus label, for -fold_column Condition: leave one condition out), and, when every item's regressor lies inside one run, a Run column, so -cv run works on trials. -show_table prints the expanded table. The Xmat knows each trial's condition and order, not which stimulus it was: trial items match across subjects only if everyone had the same order; otherwise list the items in the table yourself. LSS trial betas and explicit stimulus identities: -lss With -axis item, InputFile is an existing 3dLSS output dataset, Xmat is the design used by 3dLSS, and Events names an event-to-item table. This reads LSS betas; it does not run 3dLSS or fit a GLM to raw BOLD data. The subject table has Subj Xmat Events InputFile, plus optional subject metadata (Group, Mask, covariates). Each Events file has a header with: Trial Item [Condition Repeat Run other_metadata ...] Trial is the ZERO-BASED LSS volume index, not a TR, onset, original Xmat column, or one-based trial number. Map every volume exactly once. Rows and columns may be reordered; all Events files must have the same columns. Item is the actual shared stimulus identity, not its presentation order. -item_column and -repeat_column select alternative metadata column names. Repeated items need matching repeats across subjects, as described below. Condition, if wanted, comes from Events: one IM regressor family can hold several experimental categories. Additional columns can define folds or score groups. Subject and event column names must not overlap. All paths are relative to the working directory, not the table's directory. The Xmat must describe exactly one -stim_times_IM family, as for 3dLSS. Its IM column count and the dataset's ordered LABEL#000, LABEL#001, ... labels must agree. Use the original, unselected LSS dataset. Renamed or selected datasets can instead use the explicit item-table method below. -xmat_stims is not used with -lss. Xmat summary covariates are retained. If neither table supplies Run (or -run_column's name), it is inferred as 1, 2, ... only when every IM regressor lies wholly in one run, accounting for GoodList censoring. Otherwise supply Run explicitly for run CV. For separate per-run LSS datasets, supply the actual Run on each subject row (or in Events); each separate Xmat otherwise starts its runs at 1. Use -zscore run for separate run fits. No censor vector is applied to beta volumes. Trial mappings must be complete; use an explicit item table to analyze a common subset, keeping the same item/repeat keys for everyone. New way (s01.events.txt and s02.events.txt can have different trial orders): # lss_subjects.txt Subj Xmat Events InputFile s01 s01.X.xmat.1D s01.events.txt s01.LSS+tlrc s02 s02.X.xmat.1D s02.events.txt s02.LSS+tlrc # s01.events.txt Trial Item Condition 0 face_017 face 1 house_042 house # s02.events.txt Trial Item Condition 0 house_042 house 1 face_017 face # ...one row per remaining trial in EACH file... 3dSRM -axis item -lss -dataTableFile lss_subjects.txt -show_table 3dSRM -prefix task -axis item -lss -nfeatures 5 -cv run \ -mask roi_mask+tlrc -dataTableFile lss_subjects.txt Old way (still supported; no -lss, Xmat, or Events columns): # explicit_items.txt Subj Item Condition Run InputFile s01 face_017 face 1 s01.LSS+tlrc[0] s01 house_042 house 1 s01.LSS+tlrc[1] s02 house_042 house 1 s02.LSS+tlrc[0] s02 face_017 face 1 s02.LSS+tlrc[1] # ...one row per remaining item/repeat for every subject... 3dSRM -prefix task -axis item -nfeatures 5 -cv run \ -mask roi_mask+tlrc -dataTableFile explicit_items.txt These commands assume a full table with multiple runs and enough training items/voxels for 5 features; the two displayed trials only illustrate the mapping. Use held-out runs/items to evaluate alignment. LSS estimates may mix neighboring trial responses, so run-wise validation is a useful start. Identical Item labels must mean the same stimulus across people; neither the Xmat nor matching LSS volume numbers establishes stimulus identity. -xmat_stims LIST Comma list of the stimulus labels to use [all]. -repeat_column NAME Repeat-label column [Repeat], used if present: it names an item's repeats (1, 2, ... or early, late). Normalization. Betas from one GLM over all runs (the usual afni_proc.py analysis, including trial-wise betas from -stim_times_IM) share one baseline, so the default, -zscore all, z-scores each feature over all of the subject's items together. With betas from a separate GLM per run, use -zscore run, which works over the items of each of the subject's own runs (and needs a Run column). Each run then needs at least 3 items, and z-scores over fewer than about 10 are noisy (a warning says so). Runs. A Run column orders repeats and can define the -cv folds. Subjects need not share runs: when trial order was randomized, an item may be in run 2 for one subject and run 3 for another (the folds then come from -fold_column). Repeats. An item measured more than once (separate betas: trial-wise, or one per run) gives a column per repeat, keyed Item:1, Item:2, ... in run order (runs ordered as they first appear in the table, as with -axis time) and then table order, so two repeats in the same run are numbered by their rows. A Repeat column names them instead (Item:R; see -repeat_column), which does not depend on row order. The model is free to use the repeats, and they are what within-subject reliability is computed from (see -reliability within). Every subject must have the same repeats of each item. What changes with -axis item: runs are not concatenated in time, there is no temporal adjacency, so 'match' becomes item identification (the held-out item must be the best correlate of its own counterpart among all items; the correlation is across the k components, so it is weak when k is a handful: compare k values with -nfeatures a,b,c), and the folds of -cv are runs, or the levels of -fold_column. When those differ between subjects for an item (randomized run order), each subject's folds are its own: -cv loso fits the group without the subject on every item; -cv run refits everyone on every item with each subject's own held-out items missing for that subject only (see -cenmode missing). Options about time are errors with -axis item, each saying what to do instead: -censor, -zcensor, -cv_blocks, -match_window, -time_labels, -repeated_stimulus, and -run_coupling. The outputs list the item keys: in the header of PREFIX.S.1D (one row per item) and as the volume labels of PREFIX.proj.SUBJ. -fold_column NAME The table column whose levels are the folds of -cv run [Run]. Naming a category column gives leave-one-category-out: the model must then generalize to a kind of item it never saw, a stricter test. -time_labels NAME FILE With -axis time, one numeric label per retained time point (0 = unlabelled), to score by stimulus content such as speech/no speech or scene type. NAME is the factor name for -score_by. May be given repeatedly. One label per input time point also works (then censored ones are skipped). Condition from the Xmat. With an Xmat column on the time axis, -score_by Condition labels each time point with the stimulus whose regressor, scaled to its own peak, is largest there, if it reaches -xmat_label_frac of that peak [0.5]; other time points, and censored ones, are unlabelled. The regressors are already convolved, so the labels follow the BOLD response. -xmat_stims chooses the stimuli here too. -xmat_label_frac F See above [0.5]. Which one? If every subject saw the same stimuli in the same order, time mode uses the full time series. If trial order differed, only item mode is valid. Time mode cannot align subjects whose events are not simultaneous. Mapping new subjects into a saved model: ========================================================== .. code-block:: none Every fit writes PREFIX.srm. It holds the shared response S, the noise variances, and exactly how the data were put on the model's timeline (the censored time points, the run lengths, and the z-scoring). It does not hold any subject's W_i or data, so it is small and can be passed between sites. It is plain text (about 1.2 MB for 30 components and 1310 time points), written to a temporary name and renamed, so an interrupted run never leaves half a model. The numbers are stored with enough digits to be read back exactly, and a model that is read and then updated gives exactly the result that updating the original would have. The format is described in thd_srm.h; keys and sections this version does not know are skipped, and a file from a newer version is refused with a message, not misread. A damaged or truncated file is reported as such. -model_in MODEL.srm Do not fit S. Instead estimate a W for every subject in the table, against the S in MODEL.srm. Each subject needs the same stimulus and the same number of time points and runs the model was built from; this is checked first, and the data are censored and z-scored as the model's were. Their feature counts, masks, and grids can be anything with at least k features. The model defines -nfeatures, -algorithm, -seed, -run_coupling, -censor, -zcensor, and -zscore; giving any of them with -model_in is an error, and so is -niter or -tol unless -update_S is given (they then control that update). -cv loso is also an error, since there is no group fit to leave anyone out of. -cv none or -cv run are allowed, and mean something different here: see below. A subject who is already in the model is a warning (their scores are optimistic: S contains them), and an error with -update_S. A model made with -run_coupling places new subjects with one W for all their runs, unless -update_S, which gives them a W per run. A model fit with -axis item needs -axis item here too (and a time-axis model needs it absent). Each new subject must have exactly the model's item keys, in any table order (Item, or Item:n for repeats, as the fit made them; a mismatch names the first missing or extra key). -cv run then holds out the levels of -fold_column (or Run) and scores item identification. Those levels may differ between subjects (randomized runs): S is fixed, so each subject's own held-out items are simply left out of their own W. A new subject's W is fitted to their own data, so its fit to that data is flattering by construction. Use -cv run with -model_in: W is then estimated from all of the subject's runs but one (or blocks but one), and the subject is scored on the part left out. The scores are analogous to -cv loso on the original subjects, but use the full reference group rather than a reference group with one subject removed. -update_S ** EXPERIMENTAL ** Let S absorb the new subjects, and write a new PREFIX.srm that includes them. Without this option the saved model is untouched and S does not move. It is an approximation to a refit (see below), and its behavior and the model file's sums may change. The model keeps running sums of its subjects' projected data (weighted by their noise variances), so S can be updated from the new subjects alone, with no access to the original data. This is the point of the model file: a site can add its subjects to a shared model without receiving or sending any subject's data. What it is, and is not. The old subjects' W stay as they were fitted; only S and the new subjects' W move. That is an approximation to refitting everyone. The subjects added together in one run are treated jointly, so their order within the table cannot matter; subjects added in different runs can matter, slightly (S differed by about 3% between arrival orders in tests). What it costs, measured on 22 subjects hearing a story (30 components, fit on two runs, scored on a third by the correlation of each subject's projected data with the shared response from everyone else). Adding 1, 3, and 6 subjects to 21, 19, and 16 gave a drift of 0.003, 0.015, and 0.038. Against refitting everyone from scratch, the EXISTING subjects' scores were lower by 1.6%, 1.2%, and 5.0% (their W had not been refitted); the added subjects' scores were no lower. Updating S brought it 16-25% closer to the refit S than leaving it frozen, but did not raise held-out scores above those of simply mapping the new subjects into the frozen S. So the update keeps the model's S current and the model growing; it is not a substitute for a refit. The loss was 1-2% up to adding about a sixth of the original subjects and 5% at a third, so refit from the data when drift reaches a few percent. These are one data set and one split: differences of 0.001 score of 0.13 are within noise. The gauge of S is never changed by an update, so the components and the projected series of earlier analyses stay valid. Scores for the new subjects are always computed BEFORE the update, from the model they were not part of (-cv with -model_in). The model file contains sums over subjects of projected data. With only a few subjects those sums are not anonymous, so treat a model built from fewer than about five as if it contained their data. An updated model is written to PREFIX.srm; it will not overwrite the -model_in file unless it is the same name and -overwrite is given. Without -update_S, adding subjects never changes old results. Applying a subject's W to other data from the same subject: =========================================================== .. code-block:: none To keep task labels out of alignment fitting and model selection, learn W on independent data, then apply each subject's W to their task data. The retained dimensions must capture information useful for the new task; success on the alignment stimulus alone does not establish that transfer. -apply_W PREFIX Use the W datasets from an earlier -save_W run (the PREFIX.W.SUBJ datasets and PREFIX.srm). For each table subject, find their W, take the new InputFile data on W's own grid and voxels (no mask needed or accepted), z-score as the model did, and write the data in the shared space (PREFIX.proj.SUBJ). The new data may have any length, order, and -axis; nothing is fitted. A subject in the table but not in the model is an error. Subjects may differ in length from one another. A feature with no variance in a run of the new data is zeroed. With -zscore none in the earlier run, each feature is centered on the new data, since the means of the old data are not saved. -mask, -cv and -save_W are errors here. With -axis item, each subject is read from their own rows, so subjects may have different items (a dropped trial, randomized stimuli); each needs at least 3. The projection's volumes (or the 1D header) carry the item keys. If the model z-scored within runs and the table has no Run column, the items are z-scored over all of the subject's items instead, and this is said. The projected series of different subjects can then go to 3dRSA, 3dInSync, or a classifier. Independent fitting avoids fitting the tested task labels, but normalization, dimension reduction and the alignment sample still affect the result. Test transfer on held-out task data. Requires that the earlier run used -save_W. Validation: =========================================================== .. code-block:: none An SRM fits every subject's data with nfeatures x voxels free parameters. Anything measured on the data it was fit to is optimistic. These options evaluate on data the fit never saw. -cv MODE none : fit once to everything. Reported scores are in-sample and are labelled as such [default]. run : leave one run out. For each run, fit on the other runs, then score every subject on the held-out run (or -cv_blocks block). With -model_in, S is fixed, and only each subject's own W is refitted without the held-out run. loso : leave one subject out. For each subject, fit S on all OTHER subjects and every time point. Then estimate that subject's W from their own training runs and score them on a held-out run. With cold starts, their held-out data touch neither S nor any fitted W used for scoring, so the score is a clean estimate of how well a new person maps into an existing space. Folds within the subject are leave-one-run-out, or -cv_blocks contiguous blocks if there is one run. Uses independent cold starts by default; costs one full group fit per subject (-niter, -tol, -nstart). Written projections are aligned to the pooled fit AFTER scoring; this output alignment itself depends on all subjects. -cv_blocks B Contiguous time blocks used as folds in place of runs when the data have a single run [2]. Time points near a block edge are autocorrelated with the other side, so scores are a little optimistic; separate runs are better. -loso_start MODE cold [default]: fit each leave-out model independently on the other subjects, using -niter, -tol and -nstart. Score in each leave-out model's own canonical basis. legacy: reproduce the old full-model warm start, with -loso_iter steps. WARNING: that start already contains the held-out subject's influence and can inflate scores. Requires -cv loso in fit mode (time or item axis). -loso_iter M Iterations for -loso_start legacy only [5]. Ignored for cold starts. More legacy iterations do not guarantee independence from the held-out subject. -cv_metrics LIST Comma list from: isc, match, r2 [all three]. isc : correlation of the subject's projected held-out data with the shared response estimated from the other subjects, per component, and their mean in Fisher-z space (returned as r). match : time-segment matching accuracy (Chen et al.), as in BrainIAK's SRM example. Each window of -match_window TRs of the subject's projected data is matched to the best-correlated window of the others' shared response, among its own counterpart and every window that does not overlap it. With -axis item it becomes item identification: each held-out item must be the best correlate of its own counterpart among all items. Chance is reported alongside. r2 : fraction of the subject's held-out variance reconstructed from W_i and that shared response. On voxel data this is usually tiny or slightly negative, because variance that is specific to a subject dominates (about -0.001 for 5000 voxels of story data, where segment matching reached 12% against 0.3% chance). Use it to compare methods or settings on the same data, not as a measure of fit; isc and match are the informative scores. -match_window N Segment length in TRs for 'match' [9]. -cv_adapt FRAC With -run_coupling L and -cv: the held-out run has no W of its own, so it uses the anchor. FRAC in (0,1) instead estimates that run's W from its first FRAC of samples, tied to the anchor with the same L, and scores only the rest. The samples used to estimate W are never scored. isc, match, and r2 are then the run W's scores, and isc_anchor, match_anchor, and r2_anchor beside them are the anchor's on the same samples, so the comparison shows whether a run-specific W helps. The S of those first samples: with -cv loso, the group fit's (it saw the run); with -cv run, the mean of every subject's anchor projection of them, and every subject gets a run W, so each is scored against the others' run-W projections (and the anchor scores against the others' anchors). PREFIX.proj stays the anchor's. With -cv_fit and -balance both columns are averaged over the fits that scored the subject, and under -cv_fit reference the other groups' subjects (placed in the reference fit's S) adapt as -cv loso does [0: off]. Reliability, and comparing groups or conditions: =========================================================== .. code-block:: none A low score for a group or a condition has three possible causes: its signal is unreliable, its signal is reliable but not shared with the others, or the model was built mostly from someone else. These options separate them. -score_by NAMES Comma list of table columns (or -time_labels names) whose levels define subsets of the columns. Every -cv metric is computed separately for each level, for every subject, so you get faces vs houses within each subject and then within each group. The model is still fitted to all samples. Needs -cv. Where the levels come from: -axis item: a table column (Category, or Condition from an Xmat table), read on each subject's rows. -axis time: a -time_labels NAME; Condition, from each subject's Xmat (below); or a table column, its value on each run's row (Run itself, say). Within a level, isc and r2 use the subject's held-out samples of that level, and matching is among them only: windows that lie wholly inside the level, or the level's items. Chance is reported for each level. Per-subject columns (Group, covariates) are not levels: scores.txt carries them already. -cv_fit MODE How the -cv models are fit (the written model, S and the maps, is always the fit to everyone): pooled : one model from everyone [default] within : one model per Group, fit only to that group, with -cv run or loso inside it. Asks: does this group have a shared response at all? reference:LABEL : fit on group LABEL only, then place the other groups' subjects in that model (with their W estimated from their own training folds). Asks: do they use the reference group's space? Reference subjects are scored leave-one-subject-out, the others as new subjects. Reference members use N-1 training subjects; outside members use N. Both exclude the scored subject with cold starts, but the training sample sizes still differ. Needs -cv loso. Held-out projections are rotated into the gauge of the fit to everyone, so all subjects' stay comparable. -balance B Groups of unequal size bias scores: a model built from 40 people maps its members better than one built from 12. With -balance, every group fit is repeated on B random subsets the size of the smallest group (with -cv_fit pooled: B subsets with that many from every group), and each subject's scores are averaged over the fits that scored it [off]. A group that is already the smallest is fitted once. -seed sets the draws, which are listed in PREFIX.balance.txt. A subject no subset drew gets NA (a warning says so: raise B). With -balance the projected outputs are the full fit's, in-sample: each subject was scored by several models. -reliability LIST Comma list from: within, split. within : a subject's own reliability, computed from repeats of the same stimulus (the same Item in different runs; or with -axis time and -repeated_stimulus, the runs). It is the correlation of the two halves' patterns (r_pattern across voxels, r_series across time or items), with the Spearman-Brown correction, in the subject's own voxel space with no model involved. It bounds what the shared-space score can reach. The halves are the odd and the even repeats (or runs), averaged; r_pattern needs 2 repeated items (or time points), r_series 3. Per -score_by level too (an item counts in its first repeat's level). Needs -cv: it goes into scores.txt (rel_pattern, rel_series, and rel_ratio = isc / rel_pattern). split : group-level. Randomly split each group's subjects in two, fit S to each half, and compare them with a rotation-invariant subspace similarity. A group with no stable shared response gives halves that disagree. The similarity is the mean cos^2 of the principal angles between the two S (1 = the same space). Halves are half the group (rounded down) against the rest; a group needs 4 subjects. PREFIX.reliability.txt summarizes, PREFIX.reliability_splits.txt lists each split. Does not need -cv. -nsplit N Number of random splits for 'split' [100]. Each is two group fits, warm-started from the full fit. -repeated_stimulus With -axis time: every run shows the same stimulus. The runs are averaged for the fit (after z-scoring each), and their agreement (odd runs against even runs) gives 'within'. The runs must keep the same time points. The fit then has one run, so -cv uses -cv_blocks, and the saved model describes the averaged run. With -cenmode missing, each time point is averaged over the runs where the subject was observed (missing only if missing in every run), and 'within' uses the time points both halves observed. With -score_by, the levels are read on the runs before averaging and must agree in every repeat (so Run itself is an error); -time_labels may then also give one label per time point of the stimulus. Not with -run_coupling (there is no run left to have its own W). How to read them together (all subjects scored on held-out data): low within, low shared : the group or condition is simply noisy. high within, low shared : each person is reliable but they do not share a response, or the model does not speak their language. Compare -cv_fit within against reference. high within, high shared: SRM works for them. 'Within' needs real repeats. Without them it is not estimated, and a low score cannot be attributed. -nperm P Test group and condition differences in the scores by permutation, written to PREFIX.tests.txt [0 = none]: two groups : group labels permuted over subjects two levels of a -score_by factor : level labels swapped within each subject both : the group x level interaction, by permuting the within-subject differences between groups P must be at least 20. -seed applies. Subjects are the exchangeable units; the folds of one subject are never split. Every score is tested (isc, match or ident, r2; overall and per level for the group test), with Welch's t for groups and a one-sample t for differences; p is two-sided, and p_fwe is family-wise over the rows of one test (max-statistic). When the relabelings number P or fewer, all of them are used (exact). For anything more elaborate (covariates, three groups) take PREFIX.scores.txt to R. Outputs: =========================================================== .. code-block:: none PREFIX.srm The saved model (see above). With -model_in no new model is written. PREFIX.S.1D Shared response, time x k (one column per component). Components are ordered by variance and sign-fixed, so repeated fits are comparable up to noise. PREFIX.rho2.1D Noise variance per subject (-algorithm prob). PREFIX.objective.1D Objective per iteration. For prob it is the marginal log-likelihood and rises; for det it is the summed squared error and falls. PREFIX.fit.txt One row per subject: features used, features dropped, rho2, and the log of any censoring. With -update_S it also gives the model's drift and number of updates. PREFIX.coupling.txt With -run_coupling L: per subject and run, how far that run's W is from the anchor (0 = identical, 2 = orthogonal, as |W_ir - W_i|^2 / nfeatures) and the gain in reconstruction R^2 over the anchor in the fit data. Large deviations show where the subject's mapping really moved. Unless -no_projected: PREFIX.proj.SUBJ Each subject's data in the shared space, W_i' X_i. With -cv run or loso these are the held-out projections concatenated across folds; with -cv none they are in-sample. -proj_format FMT dset : a dataset with k x 1 x 1 voxels and one volume per time point, directly usable as input by 3dInSync (per-component ISC) and 3dRSA [default] 1D : time x k text file With -cv run or loso: PREFIX.scores.txt One row per Subj x level of each -score_by factor (and 'all'), averaged over folds. Columns: Subj, Group, Factor, Level, nsamp, isc, match (or ident), r2, and with -reliability within: rel_pattern, rel_series, and the ratio of the shared score to rel_pattern. Numeric table columns are carried along. One score per subject per condition is the right input for a group comparison. PREFIX.scores_folds.txt The same, one row per Subj x fold (x -score_by level, with Factor and Level columns as in scores.txt). PREFIX.balance.txt With -balance: the subjects of every fit. PREFIX.reliability.txt With -reliability split: per Group, the subspace similarity of the two halves over the splits (mean, and 2.5 and 97.5 percentiles). PREFIX.reliability_splits.txt With -reliability split: each split's two halves and their similarity. PREFIX.tests.txt With -nperm: the tests above, one row per score: Test, Factor, Score, Contrast, N1, N2, effect, t, p, p_fwe, nperm, exact. Optional, since they are large: -save_W PREFIX.W.SUBJ: k volumes per subject, on that subject's grid. These are the spatial maps of the components. A .1D subject gets one row per column of the input; the rows of dropped (constant) features are zero. -save_recon PREFIX.recon.SUBJ: W_i S, the denoised reconstruction of the subject's time series. -save_r2 PREFIX.r2.SUBJ: voxelwise reconstruction R^2 for each subject (held-out under -cv). Maps on a common grid can go to 3dttest++. Resources: =========================================================== .. code-block:: none -memory_limit G Refuse an estimated peak above G GiB. Without this option the limit is 80% of detected physical RAM. -memory_override Continue after the memory estimate exceeds its limit. -no_compress Hold every subject's data whole (voxels x T), as before -compress became the default. The results are the same up to rounding; it needs far more memory. -compress [the default when every subject has more features than time points (or items), outside -model_in and -apply_W; given explicitly, it is used regardless] Hold each subject as an exact T x T (at most) factor R of its prepared data X, with R'R = X'X (its Gram matrix: see below), instead of voxels x T. Everything the fit and -cv do with X goes through X'X, and W always lies in the span of X, so S, the scores, the projections, rho2 and the model are those of -no_compress, up to rounding (about 1e-7; the fit starts where the uncompressed one would). Subjects are read, censored, z-scored and compressed one at a time, so the data are never all in memory: for whole-brain naturalistic data this is the difference between tens of GB and a few. -save_W and -save_recon read each subject again at the end (W on the voxels is X M, M small). With -nstart > 1 only the first start is the uncompressed one's, so the best of several starts can differ from -no_compress. -save_r2 notes each scored fold's W and reference and makes the voxelwise R^2 when the subject is read again. Item reliability (-reliability within) is made from each subject's voxels before they are compressed. -cenmode drop with -zcensor reads every subject once more first, to find the zero volumes. Not with -model_in or -apply_W (they hold only the new subjects). Saves memory when subjects have more features than time points (or items). -progress M auto, bar, line, or off [auto]. -quiet Suppress progress and informational diagnostics. -show_table Print the parsed table and exit. Memory, compressed (the default when subjects have more features than time points): one subject's data while it is read, plus one input on its full grid as floats, plus each subject's factor (at most T x T x 4 bytes). With -no_compress: the sum over subjects of voxels x time x 4 bytes, plus voxels x k x 4 bytes per subject for W, plus the larger of the copies -cv run makes of the largest fold (about (T - shortest run + longest run) / T of the data; -cv loso and -model_in copy one subject at a time) and one input being read. The estimate is printed when it is over half of physical memory, or whenever -memory_limit is given. For example, 22 subjects, 161k voxels, three runs (526, 452, 332 TRs), k = 50, -cv run: 3.7 GiB and 3.9 min by default; 38.0 GiB and 6.6 min with -no_compress. Most of the time goes to reading, and then to matrix products, run subject-parallel under OpenMP. A Gram matrix is X'X for data X (voxels x time points): entry (s,t) is the dot product of the brain patterns at time points s and t, summed over voxels, so it is T x T however many voxels there are. When every subject has at least T+k voxels (T time points), the fit forms each subject's (T+k) x (T+k) Gram matrix once and iterates on that, so an iteration costs about (T+k) x T x k per subject instead of voxels x T x k, and W is formed from the data at the end. This adds (T+k)^2 x 8 bytes per subject. The fit is the same either way, up to rounding. Setting AFNI_SRM_KERNEL to VOXEL or GRAM forces one path (for testing). Examples: =========================================================== .. code-block:: none Replace the sample dataset names below with your own inputs. Inline -dataTable entries go last in the command; keep the trailing backslashes on every line except the last row. For -dataTableFile, save the displayed header and rows as plain text, without backslashes or shell quotes. Time-series examples assume synchronized movie runs, with enough time points and retained voxels for the requested number of features, including in each training fold. These illustrate commands, not guaranteed biological results. Numerical checks and held-out scores on the alignment stimulus do not establish cross-task transfer or anatomical localization. Example 1: Fit, with held-out run scores 3dSRM -prefix movie_srm -nfeatures 50 -cv run \ -mask brain+tlrc \ -dataTable Subj Group Run InputFile \ s01 control r1 s01_r1+tlrc \ s01 control r2 s01_r2+tlrc \ s02 control r1 s02_r1+tlrc \ s02 control r2 s02_r2+tlrc This writes movie_srm.srm for Examples 3 and 6. Example 8 shows how to choose the number of features before fitting a final model. Example 2: Native-space responses projected into shared features 3dSRM -prefix native_srm -nfeatures 30 -cv loso \ -dataTableFile native.txt Choose a mask for each subject that includes the system you want to study, independently of the held-out effects. Grids and voxel counts may differ across people. Within a person, their runs and mask must already be aligned to the same anatomy and grid; the +orig suffix alone does not ensure this. Contents of native.txt (at least 3 subjects for leave-one-subject-out): Subj Run Mask InputFile s01 r1 s01_mask+orig s01_r1+orig s01 r2 s01_mask+orig s01_r2+orig s02 r1 s02_mask+orig s02_r1+orig s02 r2 s02_mask+orig s02_r2+orig s03 r1 s03_mask+orig s03_r1+orig s03 r2 s03_mask+orig s03_r2+orig SRM learns weighted combinations of these voxels from corresponding movie responses. It does not label FFA, estimate a spatial warp, or move activity to matching anatomical coordinates. Output locations index shared components; they are not a brain grid for voxelwise cluster inference. Example 3: Place new subjects in the model from Example 1 and score them The new subjects must have experienced the same movie runs, with the same time-point correspondence as in Example 1. 3dSRM -prefix new_subj -model_in movie_srm.srm -cv run \ -mask brain+tlrc \ -dataTable Subj Run InputFile \ s03 r1 s03_r1+tlrc \ s03 r2 s03_r2+tlrc \ s04 r1 s04_r1+tlrc \ s04 r2 s04_r2+tlrc Example 4: Item-indexed betas, scored by category, controls as reference 3dSRM -prefix faces -axis item -nfeatures 20 -cv loso \ -mask brain+tlrc -score_by Category -cv_fit reference:control \ -balance 50 -nperm 5000 \ -dataTableFile betas.txt Sample rows from betas.txt: Subj Group Item Category Run InputFile s01 control face_017 face r1 s01_betas_r1+tlrc[0] s01 control face_023 face r1 s01_betas_r1+tlrc[1] s01 control house_042 house r1 s01_betas_r1+tlrc[2] s01 control face_017 face r2 s01_betas_r2+tlrc[0] s01 control face_023 face r2 s01_betas_r2+tlrc[1] s01 control house_042 house r2 s01_betas_r2+tlrc[2] s05 patient house_042 house r1 s05_betas_r1+tlrc[0] s05 patient face_017 face r1 s05_betas_r1+tlrc[1] s05 patient face_023 face r1 s05_betas_r1+tlrc[2] s05 patient face_023 face r2 s05_betas_r2+tlrc[0] s05 patient house_042 house r2 s05_betas_r2+tlrc[1] s05 patient face_017 face r2 s05_betas_r2+tlrc[2] These are excerpts: include all subjects and items in the actual file. Use at least 3 control subjects for LOSO, enough subjects for the group comparison, and enough training items and voxels for 20 features. Item labels match stimuli across subjects even when volume order differs. The reference-group comparison requires -cv loso. These are comparisons of held-out alignment scores, not voxelwise face-minus-house activation tests. Controls use N-1 reference subjects, patients use all N. Balancing group sizes does not remove this training-size difference. For separate GLMs per run, add -zscore run (at least 3 items in each run). Example 5: Test whether movie-derived features transfer to a visual task Use independent visual/movie data from the same subjects and masks that cover the visual system of interest. The common ventral_temporal mask below assumes anatomically aligned inputs; for native-space inputs use per-subject masks as in Example 2. A story fit in an auditory mask does not establish a useful alignment for a face/house task. Fit the movie runs with -save_W, using a new prefix: 3dSRM -prefix movie -nfeatures 50 -save_W -mask ventral_temporal+tlrc \ -dataTable Subj Run InputFile \ s01 r1 s01_r1+tlrc \ s01 r2 s01_r2+tlrc \ s02 r1 s02_r1+tlrc \ s02 r2 s02_r2+tlrc Register each subject's task data to their movie data before estimating or resampling the betas. Matching grid headers is necessary but does not establish anatomical alignment. Apply W to the corresponding voxels. Quote selectors in an inline table so the shell passes the brackets. The rows below are excerpts: add the remaining trials and subjects. 3dSRM -prefix task -axis item -apply_W movie \ -dataTable Subj Item InputFile \ s01 face_017 's01_task+tlrc[0]' \ s01 face_023 's01_task+tlrc[1]' \ s01 house_042 's01_task+tlrc[2]' \ s02 house_042 's02_task+tlrc[0]' \ s02 face_017 's02_task+tlrc[1]' \ s02 face_023 's02_task+tlrc[2]' Each output trial is a K-component vector, W_i' times that subject's prepared voxel pattern. No task mapping is fitted and no anatomical activation map is relocated. The saved normalization is applied to the NEW betas: here, with no Run column, each voxel is z-scored across items. These features do not retain the original beta amplitude units. With -zscore none in the saved fit, application still centers the new data. Evaluate cross-subject classification or RSA on independent task data, keeping test labels out of alignment/mask/K selection. Compare against anatomical alignment and a dimension-matched baseline. Good movie scores alone do not show that face/house information transfers. For claims about FFA location or activation amplitude, use an independent localizer and evaluate that anatomical/amplitude question explicitly. Example 6: Add another site's subjects to the model from Example 1 As in Example 3, these subjects saw the same movie runs. -update_S updates the shared response using the new subjects; the original data stay local. This is an approximate update with the old W fixed. Here -cv run scores new subjects against the pre-update S; it does not validate the updated S. 3dSRM -prefix movie_v2 -model_in movie_srm.srm -update_S -cv run \ -mask brain+tlrc \ -dataTable Subj Run InputFile \ s05 r1 s05_r1+tlrc \ s05 r2 s05_r2+tlrc \ s06 r1 s06_r1+tlrc \ s06 r2 s06_r2+tlrc Example 7: Allow each run its own W, tied to the subject's anchor Supply at least THREE synchronized runs per subject: each run-CV fit needs two remaining runs for run-specific W. Compare with a shared-W fit on the same three-run table. Adaptation uses the first quarter of the held-out run; adapted and unadapted scores both use the remaining samples. With K=50 and -cv_adapt 0.25, each run needs at least 200 retained samples before any subject-specific missing samples in the adaptation portion. 3dSRM -prefix movie_rc -nfeatures 50 -run_coupling 200 -cv run \ -cv_adapt 0.25 -mask brain+tlrc \ -dataTable Subj Group Run InputFile \ s01 control r1 s01_r1+tlrc \ s01 control r2 s01_r2+tlrc \ s01 control r3 s01_r3+tlrc \ s02 control r1 s02_r1+tlrc \ s02 control r2 s02_r2+tlrc \ s02 control r3 s02_r3+tlrc Example 8: Choose k using held-out scores Save the table from Example 1 as subjects.txt to reuse it across commands: Subj Group Run InputFile s01 control r1 s01_r1+tlrc s01 control r2 s01_r2+tlrc s02 control r1 s02_r1+tlrc s02 control r2 s02_r2+tlrc 3dSRM -prefix scan -nfeatures 10,20,30,50,80 -cv run \ -mask brain+tlrc -dataTableFile subjects.txt Inspect scan.nfeat.txt, then rerun Example 1 with the chosen -nfeatures value and a fresh prefix to write the model and projected data. Those scores were used for model selection; evaluate the chosen setting on an additional independent split for a final predictive claim. Reading the results: =========================================================== .. code-block:: none * Components are coordinates in a learned representation, not named brain regions. Do not match component numbers across independently fitted models or interpret component locations as anatomical coordinates. * Reconstruction R^2 is invariant to a consistent orthogonal rotation. Component-wise ISC is not; do not assume all scores are rotation-invariant. The default cold LOSO fits use their own canonical basis for scoring. * Under default -cv loso, the group reference excludes the scored subject. -loso_start legacy does not provide that independence. Training sample size, noise and convergence still matter for group comparisons. -cv run evaluates held-out runs of subjects included in training. * To compare groups, use -cv_fit and -balance. A pooled model is built more from the larger or less noisy group and maps that group better, with nothing to do with the groups differing in function. * The model fits samples in proportion to their variance, so a condition with a large response can dominate and a weak one be fitted poorly even in everyone. Score by condition, and check that a weak condition is weak in 'within' too before reading it as a failure of SRM. * If you will RSA the aligned task data, align on other data (-apply_W), or score with held-out folds. SRM maximizes cross-subject agreement of its own columns, which inflates their shared structure. * Larger k always fits the training data better. Choose k by held-out score, not by training fit. Scope boundary: =========================================================== .. code-block:: none Version 1 fits and validates SRM on time-locked or item-indexed columns, maps new subjects into a saved model and optionally updates the model's S from them, carries a fitted W to other data, allows a separate run-wise W tied to a subject anchor, and scores reliability by subject, group, and condition. Not yet provided: a prior on S that depends on condition, exact (data-based) refitting from a model alone, searchlight or atlas-parcel looping, robust SRM, and time-lagged or convolved fusion of EEG with fMRI. The fitting library (thd_srm) is written so that these can be added without changing the options above. * This binary version of 3dSRM is compiled using OpenMP, a semi- automatic parallelizer software toolkit, which splits the work across multiple CPUs/cores on the same shared memory computer. * OpenMP is NOT like MPI -- it does not work with CPUs connected only by a network (e.g., OpenMP doesn't work across cluster nodes). * For some implementation and compilation details, please see https://afni.nimh.nih.gov/pub/dist/doc/misc/OpenMP.html * The number of CPU threads used will default to the maximum number on your system. You can control this value by setting environment variable OMP_NUM_THREADS to some smaller value (including 1). * Un-setting OMP_NUM_THREADS resets OpenMP back to its default state of using all CPUs available. ++ However, on some systems, it seems to be necessary to set variable OMP_NUM_THREADS explicitly, or you only get one CPU. ++ On other systems with many CPUS, you probably want to limit the CPU count, since using more than (say) 16 threads is probably useless. * You must set OMP_NUM_THREADS in the shell BEFORE running the program, since OpenMP queries this variable BEFORE the program actually starts. ++ You can't usefully set this variable in your ~/.afnirc file or on the command line with the '-D' option. * How many threads are useful? That varies with the program, and how well it was coded. You'll have to experiment on your own systems! * The number of CPUs on this particular computer system is ...... 1. * The maximum number of CPUs that will be used is now set to .... 1. ++ Compile date = Nov 17 2026 {AFNI_26.3.12:linux_ubuntu_24_64}