AFNI Message Board

Dear AFNI users-

We are very pleased to announce that the new AFNI Message Board framework is up! Please join us at:

https://discuss.afni.nimh.nih.gov

Existing user accounts have been migrated, so returning users can login by requesting a password reset. New users can create accounts, as well, through a standard account creation process. Please note that these setup emails might initially go to spam folders (esp. for NIH users!), so please check those locations in the beginning.

The current Message Board discussion threads have been migrated to the new framework. The current Message Board will remain visible, but read-only, for a little while.

Sincerely, AFNI HQ

History of AFNI updates  

|
September 22, 2009 02:44PM
Hi AFNI Users,
This is our first attempt at AM regression and we are coming up with "strong collinearity" error messages - here's a sampling:

Largest singular value =4.54934
1 singular value is less than cutoff=4.54934-e07
Implies strong collinearity in the matrix columns!.....

For the full output, please see below. I am suspecting the event files. Is it possible that this is something we can push past with a larger GOFORIT?

thanks in advance
Brenda

Output:
++ '-stim_times_AM1 1 stimuli/20617-1fraf1.txt' has 1 auxiliary values per time point
++ '-stim_times_AM1 1': basis function model 'dmBLOCK' uses 1 parameters,
out of the 1 found in timing file 'stimuli/20617-1fraf1.txt'
++ '-stim_times_AM1 2 stimuli/20617-1fraf2.txt' has 1 auxiliary values per time point
++ '-stim_times_AM1 2': basis function model 'dmBLOCK' uses 1 parameters,
out of the 1 found in timing file 'stimuli/20617-1fraf2.txt'
++ '-stim_times_AM1 3 stimuli/20617-1frbf1.txt' has 1 auxiliary values per time point
++ '-stim_times_AM1 3': basis function model 'dmBLOCK' uses 1 parameters,
out of the 1 found in timing file 'stimuli/20617-1frbf1.txt'
++ '-stim_times_AM1 4 stimuli/20617-1frbf2.txt' has 1 auxiliary values per time point
++ '-stim_times_AM1 4': basis function model 'dmBLOCK' uses 1 parameters,
out of the 1 found in timing file 'stimuli/20617-1frbf2.txt'
++ '-stim_times_AM1 5 stimuli/20617-1vwait.txt' has 1 auxiliary values per time point
++ '-stim_times_AM1 5': basis function model 'dmBLOCK' uses 1 parameters,
out of the 1 found in timing file 'stimuli/20617-1vwait.txt'
++ 3dDeconvolve: AFNI version=AFNI_2008_07_18_1710 (Sep 16 2009) [32-bit]
++ Authored by: B. Douglas Ward, et al.
++ loading dataset pb04.20617.r01.scale+orig.HEAD pb04.20617.r02.scale+orig.HEAD
++ Auto-catenated datasets start at: 0 362
++ misfit/FDR automask has 219148 voxels (out of 331776 = 66.1%)
++ Input polort=6; Longest run=832.6 s; Recommended minimum polort=6 ++ OK ++
++ -stim_times using TR=2.3 s for stimulus timing conversion
++ -stim_times using TR=2.3 s for any -iresp output datasets
++ [you can alter the -iresp TR via the -TR_times option]
++ -stim_times_AM1 1 using LOCAL times
++ -stim_times_AM1 2 using LOCAL times
++ -stim_times_AM1 3 using LOCAL times
++ -stim_times_AM1 4 using LOCAL times
++ -stim_times_AM1 5 using LOCAL times
------------------------------------------------------------
GLT matrix from 'SYM: +rrfr -rnfr':
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 -1 0 0 0 0 0 0 0

------------------------------------------------------------
GLT matrix from 'SYM: +rnfr -rrfr':
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -1 0 1 0 0 0 0 0 0 0

------------------------------------------------------------
GLT matrix from 'SYM: +rrfr -rrfn':
0 0 0 0 0 0 0 0 0 0 0 0 0 0 -1 1 0 0 0 0 0 0 0 0 0

------------------------------------------------------------
GLT matrix from 'SYM: +rrfn -rrfr':
0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 -1 0 0 0 0 0 0 0 0 0

------------------------------------------------------------
GLT matrix from 'SYM: +rnfr -rnfn':
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -1 1 0 0 0 0 0 0 0

------------------------------------------------------------
GLT matrix from 'SYM: +rnfn -rnfr':
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 -1 0 0 0 0 0 0 0

------------------------------------------------------------
GLT matrix from 'SYM: +rrfr +rnfr -rrfn -rnfn':
0 0 0 0 0 0 0 0 0 0 0 0 0 0 -1 1 -1 1 0 0 0 0 0 0 0

------------------------------------------------------------
GLT matrix from 'SYM: +rrfr +rrfn -rnfr -rnfn':
0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 -1 -1 0 0 0 0 0 0 0

------------------------------------------------------------
GLT matrix from 'SYM: +rrfr -baseline':
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 -1 0 0 0 0 0 0

------------------------------------------------------------
GLT matrix from 'SYM: +rrfn -baseline':
0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 -1 0 0 0 0 0 0

------------------------------------------------------------
GLT matrix from 'SYM: +rrfr +rnfr -2*baseline':
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 -2 0 0 0 0 0 0

------------------------------------------------------------
GLT matrix from 'SYM: +rrfn +rnfn -2*baseline':
0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 -2 0 0 0 0 0 0

*+ WARNING: -stim_file 6: file length is 1051, longer than expected 703 (from dataset)
*+ WARNING: -stim_file 7: file length is 1051, longer than expected 703 (from dataset)
*+ WARNING: -stim_file 8: file length is 1051, longer than expected 703 (from dataset)
*+ WARNING: -stim_file 9: file length is 1051, longer than expected 703 (from dataset)
*+ WARNING: -stim_file 10: file length is 1051, longer than expected 703 (from dataset)
*+ WARNING: -stim_file 11: file length is 1051, longer than expected 703 (from dataset)
++ Memory required for output bricks = 1934917632 bytes (about 1.9 billion)
++ Wrote matrix image to file X.jpg
++ Wrote matrix values to file X.xmat.1D
++ ========= Things you can do with the matrix file =========
++ (a) Linear regression with ARMA(1,1) modeling of serial correlation:

3dREMLfit -matrix X.xmat.1D \
-input "pb04.20617.r01.scale+orig.HEAD pb04.20617.r02.scale+orig.HEAD" \
-fout -tout -Rbuck stats.20617_REML -Rvar stats.20617_REMLvar \
-Rfitts fitts.20617_REML -Rerrts errts.20617_REML -verb

++ N.B.: 3dREMLfit command above written to file stats.REML_cmd
++ (b) Visualization/analysis of the matrix via ExamineXmat.R
++ (c) Synthesis of sub-model datasets using 3dSynthesize
++ ==========================================================
++ ----- Signal+Baseline matrix condition [X] (495x25): 1623.54 ++ OK ++
*+ WARNING: !! in Signal+Baseline matrix:
* Largest singular value=4.54934
* 1 singular value is less than cutoff=4.54934e-07
* Implies strong collinearity in the matrix columns!
++ Signal+Baseline matrix singular values:
2.36909e-10 1.72592e-06 0.00145009 0.00246522 0.00535354
0.0196994 0.0257479 0.0964718 0.191477 0.314207
0.382698 0.796788 0.806621 0.930177 0.955658
1.04239 1.09121 1.15697 1.24835 1.59853
1.95337 2.33152 2.57335 2.92617 4.54934
++ ----- Signal-only matrix condition [X] (495x5): 1.01422 ++ VERY GOOD ++
++ ----- Baseline-only matrix condition [X] (495x20): 1551.7 ++ OK ++
*+ WARNING: !! in Baseline-only matrix:
* Largest singular value=4.26916
* 1 singular value is less than cutoff=4.26916e-07
* Implies strong collinearity in the matrix columns!
++ Baseline-only matrix singular values:
2.52515e-10 1.77307e-06 0.00147394 0.00252534 0.00562549
0.0198543 0.0277432 0.100008 0.198359 0.360187
0.989907 1.03045 1.0455 1.07183 1.42522
1.93601 2.29273 2.54082 2.68259 4.26916
++ ----- stim_base-only matrix condition [X] (495x6): 7.13277 ++ VERY GOOD ++
++ ----- polort-only matrix condition [X] (495x14): 1105.72 ++ OK ++
*+ WARNING: !! in polort-only matrix:
* Largest singular value=2.41543
* 1 singular value is less than cutoff=2.41543e-07
* Implies strong collinearity in the matrix columns!
++ polort-only matrix singular values:
2.60533e-10 1.97563e-06 0.00201808 0.300356 0.969759
0.975513 0.982974 0.988926 1 1.03556
1.04727 1.99731 2.28488 2.41543
++ +++++ Matrix inverse average error = 1.50447e-08 ++ VERY GOOD ++
++ Matrix setup time = 5.62 s
** ERROR: !! 3dDeconvolve: Can't run past 3 matrix warnings without '-GOFORIT 3'
** ERROR: !! Currently at -GOFORIT 0
** ERROR: !! See file 3dDeconvolve.err for all WARNING and ERROR messages !!
** ERROR: !! Be sure you understand what you are doing before using -GOFORIT !!
** ERROR: !! If in doubt, consult with someone or with the AFNI message board !!
** FATAL ERROR: !! 3dDeconvolve (regretfully) shuts itself down !!
** Program compile date = Sep 16 2009
---------------------------------------
** 3dDeconvolve error, failing...
(consider the file 3dDeconvolve.err)




Script:
set subj = 20617

# ================================ regress =================================
# run the regression analysis
3dDeconvolve -input pb04.$subj.r??.scale+orig.HEAD \
-local_times \
-polort 6 \
-num_stimts 11 \
-stim_times_AM1 1 stimuli/20617-1fraf1.txt 'dmBLOCK' \
-stim_label 1 rrfn \
-stim_times_AM1 2 stimuli/20617-1fraf2.txt 'dmBLOCK' \
-stim_label 2 rrfr \
-stim_times_AM1 3 stimuli/20617-1frbf1.txt 'dmBLOCK' \
-stim_label 3 rnfn \
-stim_times_AM1 4 stimuli/20617-1frbf2.txt 'dmBLOCK' \
-stim_label 4 rnfr \
-stim_times_AM1 5 stimuli/20617-1vwait.txt 'dmBLOCK' \
-stim_label 5 baseline \
-stim_file 6 dfile.rall.1D'[0]' -stim_base 6 -stim_label 6 roll \
-stim_file 7 dfile.rall.1D'[1]' -stim_base 7 -stim_label 7 pitch \
-stim_file 8 dfile.rall.1D'[2]' -stim_base 8 -stim_label 8 yaw \
-stim_file 9 dfile.rall.1D'[3]' -stim_base 9 -stim_label 9 dS \
-stim_file 10 dfile.rall.1D'[4]' -stim_base 10 -stim_label 10 dL \
-stim_file 11 dfile.rall.1D'[5]' -stim_base 11 -stim_label 11 dP \
-censor censor.1D \
-iresp 1 iresp_rrfn.$subj \
-iresp 2 iresp_rrfr.$subj \
-iresp 3 iresp_rnfn.$subj \
-iresp 4 iresp_rnfr.$subj \
-iresp 5 iresp_baseline.$subj \
-num_glt 12 \
-gltsym 'SYM: +rrfr -rnfr' \
-glt_label 1 rrfr_rnfr \
-gltsym 'SYM: +rnfr -rrfr' \
-glt_label 2 rnfr_rrfr \
-gltsym 'SYM: +rrfr -rrfn' \
-glt_label 3 rrfr_rrfn \
-gltsym 'SYM: +rrfn -rrfr' \
-glt_label 4 rrfn_rrfr \
-gltsym 'SYM: +rnfr -rnfn' \
-glt_label 5 rnfr_rnfn \
-gltsym 'SYM: +rnfn -rnfr' \
-glt_label 6 rnfn_rnfr \
-gltsym 'SYM: +rrfr +rnfr -rrfn -rnfn' \
-glt_label 7 rrfrrnfr_rrfnrnfn \
-gltsym 'SYM: +rrfr +rrfn -rnfr -rnfn' \
-glt_label 8 rrfrrrfn_rnfrrnfn \
-gltsym 'SYM: +rrfr -baseline' \
-glt_label 9 rrfr_baseline \
-gltsym 'SYM: +rrfn -baseline' \
-glt_label 10 rrfn_baseline \
-gltsym 'SYM: +rrfr +rnfr -2*baseline' \
-glt_label 11 fr_baseline \
-gltsym 'SYM: +rrfn +rnfn -2*baseline' \
-glt_label 12 fn_baseline \
-fout -tout -x1D X.xmat.1D -xjpeg X.jpg \
-fitts fitts.$subj \
-errts errts.$subj \
-bucket stats.$subj


# if 3dDeconvolve fails, terminate the script
if ( $status != 0 ) then
echo '---------------------------------------'
echo '** 3dDeconvolve error, failing...'
echo ' (consider the file 3dDeconvolve.err)'
exit
endif


# create an all_runs dataset to match the fitts, errts, etc.
3dTcat -prefix all_runs.$subj pb04.$subj.r??.scale+orig.HEAD

# ============================ blur estimation =============================
# compute blur estimates
touch blur_est.$subj.1D # start with empty file

# -- estimate blur for each run in errts --
touch blur.errts.1D

set b0 = 0 # first index for current run
set b1 = -1 # will be last index for current run
foreach reps ( 362 341 348 )
@ b1 += $reps # last index for current run
3dFWHMx -detrend -mask full_mask.$subj+orig \
errts.$subj+orig"[$b0..$b1]" >> blur.errts.1D
@ b0 += $reps # first index for next run
end

# compute average blur and append
set blurs = ( `3dTstat -mean -prefix - blur.errts.1D\'` )
echo average errts blurs: $blurs
echo "$blurs # errts blur estimates" >> blur_est.$subj.1D


# =========================== gen_epi_review.py ============================
# generate a review script for the unprocessed EPI data
gen_epi_review.py -script @epi_review.$subj \
-dsets pb00.$subj.r??.tcat+orig.HEAD

# ================================ cleanup =================================

# remove temporary rm.* files
\rm -f rm.*

# return to parent directory
cd ..




# ==========================================================================
# script generated by the command:
#
# afni_proc.py -subj_id 20617 -script afni_proc_py_script -dsets \
# OutBrick_r1+orig.HEAD OutBrick_r2+orig.HEAD OutBrick_r3+orig.HEAD \
# -tcat_remove_first_trs 0 -volreg_align_to last -blur_size 6 \
# -regress_basis dmBLOCK -regress_stim_times 20617-1fraf1.txt \
# 20617-1fraf2.txt 20617-1frbf1.txt 20617-1frbf2.txt 20617-1vwait.txt \
# -regress_stim_labels rrfn rrfr rnfn rnfr baseline \
# -regress_est_blur_errts -regress_errts_prefix errts -regress_opts_3dD \
# -num_glt 12 -gltsym 'SYM: +rrfr -rnfr' -glt_label 1 rrfr_rnfr -gltsym \
# 'SYM: +rnfr -rrfr' -glt_label 2 rnfr_rrfr -gltsym 'SYM: +rrfr -rrfn' \
# -glt_label 3 rrfr_rrfn -gltsym 'SYM: +rrfn -rrfr' -glt_label 4 \
# rrfn_rrfr -gltsym 'SYM: +rnfr -rnfn' -glt_label 5 rnfr_rnfn -gltsym \
# 'SYM: +rnfn -rnfr' -glt_label 6 rnfn_rnfr -gltsym 'SYM: +rrfr +rnfr \
# -rrfn -rnfn' -glt_label 7 rrfrrnfr_rrfnrnfn -gltsym 'SYM: +rrfr +rrfn \
# -rnfr -rnfn' -glt_label 8 rrfrrrfn_rnfrrnfn -gltsym 'SYM: +rrfr \
# -baseline' -glt_label 9 rrfr_baseline -gltsym 'SYM: +rrfn -baseline' \
# -glt_label 10 rrfn_baseline -gltsym 'SYM: +rrfr +rnfr -2*baseline' \
# -glt_label 11 fr_baseline -gltsym 'SYM: +rrfn +rnfn -2*baseline' \
# -glt_label 12 fn_baseline
Subject Author Posted

collinearity in AM regression

Brenda Benson September 22, 2009 02:44PM

Re: collinearity in AM regression

rick reynoldr September 22, 2009 05:50PM

Re: collinearity in AM regression

rick reynoldr September 23, 2009 01:35PM