:orphan: .. _ahelp_1dXmatTool: ********** 1dXmatTool ********** .. contents:: :local: | .. code-block:: none 1dXmatTool: manipulate AFNI X.xmat.1D design matrices Split a multi-run 3dDeconvolve design into run-local matrices: 1dXmatTool -infile X.xmat.1D -uncensored X.nocensor.xmat.1D \ -split_runs -prefix split/X Operations (one at a time; -describe and -validate may be combined): -describe / -show Full-matrix and per-run rank, DF, censor/stimulus summary. -validate As above; exit 2 for failed checks, 0 for pass. -for lss|reml With validation: check per-run LSS trial-model ranks, or full/per-run GLM rank and residual degrees of freedom. -rank_tol T Relative singular-value cutoff on unit-norm columns (default 1e-7). Duplicate pairs use 1-abs(cosine)<=T. -json FILE Optional JSON report with describe/validate. -force_inferred_metadata With describe/validate only: allow support reports using reader defaults for missing RunStart, GoodList or NRowFull. Warns and labels diagnostics as inferred. Does not repair metadata or bypass extraction/validation errors. -warn_support_frac F With describe/validate, warn when the retained IM support-row fraction is below F (0..1). Opt-in only; never changes validity or exit status. Needs full support. -select_columns LIST Zero-based column indices, e.g. 0..3,8,10. -select_labels LIST Exact comma-separated ColumnLabels. -select_stimuli LIST Exact comma-separated StimLabels. -select_groups LIST baseline (-1), nuisance (0), stimuli (>0), or group IDs. Selections combine by union; original column order stays. Writes PREFIX.xmat.1D, .rows.1D and .columns.1D. -drop_glts Required for selection when input contains GLTs. -compare FILE Match unique column labels/full TR indices; distinguish reorder, censor/run/stimulus changes and changed values. Exit 2 for differences, 0 for equivalence (reordering OK). -atol A -rtol R Comparison tolerance A+R*max(abs(a),abs(b)); default 1e-6. TR selector query (stdout only; no output files): -show_trs Print one comma-separated AFNI sub-brick selector. Requires -runs N (one one-based run), one selection option, and exactly one explicit threshold. Diagnostics use stderr. -select_stimuli LABEL / -select_labels LABEL / -select_columns INDEX Must resolve to exactly ONE design column. Multi-basis or IM stimulus families require an explicit column; no sum, absolute magnitude, winner selection or HRF shift is added. -tr_above VALUE Select retained TRs with ideal value strictly > VALUE. -tr_above_frac F Select ideal > F * positive peak, with 0 <= F < 1. Peak is within the selected run/window BEFORE censoring. Censored matrices require a matching -uncensored reference for fractional thresholds. Zero/negative-only peaks fail. -tr_window A..B Optional zero-based, inclusive run-local TR window. Use explicit hemodynamically aligned block windows; block identities are not inferred from a condition regressor. Without a window, all qualifying blocks in the run combine. Output indices refer to the original full-length run, never compressed censored rows. Censored TRs are always excluded. Empty, ambiguous or invalid queries fail with empty stdout. Cannot combine -show_trs with reports or file-writing modes. Example (Bash; check selection before invoking 3dTstat): trs=$(1dXmatTool -infile X.xmat.1D -uncensored X.nocensor.xmat.1D \ -show_trs -runs 1 -select_stimuli face \ -tr_above_frac 0.5 -tr_window 8..15) || exit 1 3dTstat -mean -prefix block01.nii.gz "cleaned.r01.nii.gz[$trs]" -export_manifest Write PREFIX.trials.1D from an IM design plus explicit events. -events FILE Header/table: Run Volume Condition [Item]; complete unique keys; Run one-based, Volume zero-based within each run. -subject ID Subject label for the manifest. -beta FILE Single-run beta bucket, or use -beta_pattern for many runs. -beta_pattern P One {run} placeholder, replaced by 01,02,... (quote it). -manifest_format manifest|selected Default manifest: whole InputFile plus Volume for 3dMVPA -trial_manifest. selected: InputFile[n] for direct 3dMVPA/3dRSA/3dSRM tables. Files need not yet exist. Conditions are supplied by you, never guessed from IM labels. Beta-sidecar input (Picard or an existing LSS trial manifest): -beta_manifest FILE Alternative to -infile; use with -export_manifest. -subject ID -prefix PREFIX Required. Uses actual beta Volume, not XColumn. -manifest_run_base 0|1 Input Run convention; default 0 for Picard. Use 1 for a previous 1dXmatTool LSS manifest. -input_base DIR Base for relative paths; default: manifest directory. For Picard prefixes with directories, pass its original cwd. -events FILE Optional annotations: Volume Condition [Run Trial Item Basis]. Volume is the actual whole-bucket beta index; Run is ONE-based. Complete unique Volume keys required; single bucket only. Annotations fill missing identities; contradictions fail. -runs LIST Select ONE-based runs, e.g. 1 or 1,3..5. No renumbering. -manifest_format manifest|selected|rsa|rsa_runwise manifest: PREFIX.trials.1D, whole InputFile + Volume for 3dMVPA. selected: PREFIX.trials.1D, InputFile[Volume] for direct tables. rsa: PREFIX.rsa.1D, Subj Condition InputFile; one beta per condition. rsa_runwise: PREFIX.runwise.1D and PREFIX.rNN.trials.1D Trial/Condition files for 3dRSA -runwiseTable. Requires >=2 runs, explicit unique Trial IDs, one coefficient/trial and one bucket/run. Repeated conditions are averaged within runs by RSA, not here. Both RSA formats also write PREFIX.identities.1D with full provenance. Exported Run is ONE-based; SourceRun retains the input value (-1 if absent). Missing Run/Trial/Basis remain -1; conditions are never inferred from labels. CandidateManifest and DesignIDFile/Volume are retained, without a fictitious single Xmat. Dataset headers/volume bounds and fixed XColumn bounds are checked. Runs must be <=1000000. Paths are made absolute; outputs never overwrite files. No beta images, residuals, fits, HRF choices or averages are computed here. Runwise export does not establish independent runs or prevent selection leakage. It exports unwhitened crossnobis inputs; ResidFile/whitening is not inferred. Example after 3dPicard -write_manifest (from Picard's working directory): 1dXmatTool -beta_manifest fitted.betas.tsv -export_manifest \ -subject s01 -input_base . -manifest_format manifest -prefix mvpa 3dMVPA -mode classify -trial_manifest mvpa.trials.1D \ -target_column Condition -cv_column Run -mask mask.nii -prefix decoded 1dXmatTool -beta_manifest fitted.betas.tsv -export_manifest \ -subject s01 -input_base . -manifest_format rsa_runwise -prefix rsa Add -events annotations.1D when a fixed-design sidecar lacks identities. Inputs/splitting: -infile FILE NIML-comment matrix from 3dDeconvolve. -split_runs Extract runs, preserving values and column order. -prefix PREFIX Output prefix; its directory must already exist. Optional: -uncensored FILE Matching full-row design, recommended with censoring. Determines column support before censoring. Dimensions, labels, stimuli, run starts and retained values must match. -runs LIST One-based run numbers, e.g. '1,3..5' (default: all). -zero_tol T Relative support threshold, 0 <= T < 1 (default: 1e-6). A column's peak within a run must exceed T times its global peak. Use 0 for exact nonzero support. -help This help. Outputs (r01 denotes the original, one-based run number): PREFIX.r01.xmat.1D Run matrix with local GoodList, NRowFull, RunStart=0, ColumnGroups and remapped stimulus/Basis metadata. PREFIX.r01.rows.1D Row, SourceRow, RunTR, SourceTR (all zero-based). PREFIX.r01.columns.1D Every input column, including dropped columns; numeric columns only; labels in # ColumnLabel comments. local Column/Stim/IMVolume are -1 when unavailable. IMVolume is within the corresponding IM stimulus, and is the LSS volume when there is one IM stimulus. PREFIX.runs.1D Original run number, start TR, full length, retained rows and columns; output paths are in # RunFiles comments. Comment text escapes backslash, tab, newline and CR as \\, \t, \n and \r. All mappings refer to the immediate input file. Matrix headers also record XmatSource* indices and the support file/tolerance. No files are overwritten. All selected runs are checked before writing. Existing outputs cause failure. Dangers and diagnostics: * Use -uncensored X.nocensor.xmat.1D for censored designs, including -validate -for lss and -validate -for reml. Quote paths containing spaces or parentheses. Without a reference, pre-censor fractions are unknown. * Fully censored IM: error 'loses its support after censoring'. Without a reference: 'trial may be fully censored'. Zero in the full design: check onsets/basis. Cross-run support is a separate ownership error. * Partial loss: -describe -warn_support_frac 0.5 can warn 'support fraction 0.25, peak fraction 0.015'. This is a heuristic, not a power/quality test. Support rows have abs(value)>zero_tol*full_peak; peak fraction uses abs peaks. Zero full peak gives unknown ratios. No default warning threshold. * IMCounts reports assigned IM columns per run (trial counts for one-column IM bases). Original counts can differ legitimately; censoring losses warn, e.g. 'run 4: 11 of 12 assigned IM columns retained after censoring'. * Event mismatch: 'missing event for run R volume V' or 'events contain unmatched rows'. Check run/volume indexing and event/design correspondence; censoring alone does not establish the cause. Lost IMs fail extraction. * Failed extraction preserves NKept; unavailable derived diagnostics use -1 (condition: nan). Unknown support ratios use unknown / JSON null. Missing timeline metadata suppresses support/count diagnostics and input run rows by default (JSON null support/counts; empty runs). Explicit force permits inferred reports only: one run, consecutive rows, stored row count are the respective defaults. These assumptions cannot recover censoring. -describe warns but exits 0; -validate exits 2 for failed checks. Policy: * Requires explicit RunStart, NRowFull and GoodList. No timing is guessed. * Drops columns inactive in the selected run; retains shared nuisance columns. Does not refit, normalize or demean. Values retain thd_xmat float precision; original double-precision text is not preserved beyond that precision. * IM columns must belong to exactly one run. Zero/cross-run IM support, or loss of an IM column's support after censoring, is an error. Fully censored runs also fail. Without -uncensored, support is known only in retained rows. * Rebuilds stimulus ranges; preserves column labels and per-stimulus Basis attributes. Input stimulus IDs may be sparse; output IDs are compact. Drops GLTs and ColumnMasks (and records their removal). * Extraction does not establish full rank or satisfy every downstream tool. 3dLSS still requires exactly one IM stimulus and sufficient observations. Example downstream use (run 1, full-length per-run scaled EPI): 3dLSS -matrix split/X.r01.xmat.1D \ -input pb03.SUBJ.r01.scale+orig -prefix LSS.r01 Use the matching run's FULL time series: 3dLSS applies the local GoodList. Do not concatenate censored rows or feed the entire multi-run dataset. This tool splits an existing design; it does not create new regressors.