Hi,

We are making progress with afni_proc.py (with the updated version of afni). We ran into one problem though, and got this error message: THD_generic_detrend_LSQ: fit fails - no detrending.

Below are the commands we use (we are running it for all runs for Subject 1 for now):
afni_proc.py -subj_id afni_proc_Results \
-blocks despike tshift align volreg regress \
-copy_anat anat_noskull.nii \
-anat_follower_ROI Ventricles epi aseg_Ventricles+orig \
-anat_follower_ROI WM epi wm_eroded+orig \
-dsets fm_run?+orig.HEAD \
-tcat_remove_first_trs 0 \
-tshift_opts_ts -ignore 5 \
-volreg_align_e2a \
-volreg_align_to first \
-volreg_warp_dxyz 3 \
-regress_ROI_PC Ventricles 3 \
-regress_ROI WM \
-regress_anaticor_fast \
-regress_anaticor_label WM \
-regress_censor_motion 0.2 \
-regress_censor_outliers 0.1 \
-regress_skip_first_outliers 5 \
-regress_apply_mot_types demean deriv \
-regress_run_clustsim no

tcsh -xef proc.afni_proc_Results

I would like to attach a copy of the codes that was generated from the command lines, but it is not in a format that can be attached to this message. Thus I am copying the codes below (kind of long). We know that it ran into problem at Line 307, which I highlighted in red below.

It would be great to know what can be done to fix it! Thanks a lot!

Best,
Marina

#!/bin/tcsh -xef

echo "auto-generated by afni_proc.py, Tue Jul 14 11:44:45 2015"
echo "(version 4.46, June 16, 2015)"
echo "execution started: `date`"

# execute via :
# tcsh -xef proc.afni_proc_Results |& tee output.proc.afni_proc_Results

# =========================== auto block: setup ============================
# script setup

# take note of the AFNI version
afni -ver

# check that the current AFNI version is recent enough
afni_history -check_date 1 Apr 2015
if ( $status ) then
echo "** this script requires newer AFNI binaries (than 1 Apr 2015)"
echo " (consider: @update.afni.binaries -defaults)"
exit
endif

# the user may specify a single subject to run with
if ( $#argv > 0 ) then
set subj = $argv[1]
else
set subj = afni_proc_Results
endif

# assign output directory name
set output_dir = $subj.results

# verify that the results directory does not yet exist
if ( -d $output_dir ) then
echo output dir "$subj.results" already exists
exit
endif

# set list of runs
set runs = (`count -digits 2 1 6`)

# create results and stimuli directories
mkdir $output_dir
mkdir $output_dir/stimuli

# copy anatomy to results dir
3dcopy anat_noskull.nii $output_dir/anat_noskull

# copy anatomical follower datasets into the results dir
3dcopy aseg_Ventricles+orig $output_dir/copy_af_Ventricles
3dcopy wm_eroded+orig $output_dir/copy_af_WM

# ============================ auto block: tcat ============================
# apply 3dTcat to copy input dsets to results dir, while
# removing the first 0 TRs
3dTcat -prefix $output_dir/pb00.$subj.r01.tcat fm_run1+orig'[0..$]'
3dTcat -prefix $output_dir/pb00.$subj.r02.tcat fm_run2+orig'[0..$]'
3dTcat -prefix $output_dir/pb00.$subj.r03.tcat fm_run3+orig'[0..$]'
3dTcat -prefix $output_dir/pb00.$subj.r04.tcat fm_run4+orig'[0..$]'
3dTcat -prefix $output_dir/pb00.$subj.r05.tcat fm_run5+orig'[0..$]'
3dTcat -prefix $output_dir/pb00.$subj.r06.tcat fm_run6+orig'[0..$]'

# and make note of repetitions (TRs) per run
set tr_counts = ( 256 266 266 266 266 266 )

# -------------------------------------------------------
# enter the results directory (can begin processing data)
cd $output_dir


# ========================== auto block: outcount ==========================
# data check: compute outlier fraction for each volume
touch out.pre_ss_warn.txt
foreach run ( $runs )
3dToutcount -automask -fraction -polort 4 -legendre \
pb00.$subj.r$run.tcat+orig > outcount.r$run.1D

# censor outlier TRs per run, ignoring the first 5 TRs
# - censor when more than 0.1 of automask voxels are outliers
# - step() defines which TRs to remove via censoring
1deval -a outcount.r$run.1D -expr "1-step(a-0.1)*step(t-4)" > \
rm.out.cen.r$run.1D

# outliers at TR 0 might suggest pre-steady state TRs
if ( `1deval -a outcount.r$run.1D"{0}" -expr "step(a-0.4)"` ) then
echo "** TR #0 outliers: possible pre-steady state TRs in run $run" \
>> out.pre_ss_warn.txt
endif
end

# catenate outlier counts into a single time series
cat outcount.r*.1D > outcount_rall.1D

# catenate outlier censor files into a single time series
cat rm.out.cen.r*.1D > outcount_${subj}_censor.1D

# ================================ despike =================================
# apply 3dDespike to each run
foreach run ( $runs )
3dDespike -NEW -nomask -prefix pb01.$subj.r$run.despike \
pb00.$subj.r$run.tcat+orig
end

# ================================= tshift =================================
# time shift data so all slice timing is the same
foreach run ( $runs )
3dTshift -tzero 0 -quintic -prefix pb02.$subj.r$run.tshift \
-ignore 5 \
pb01.$subj.r$run.despike+orig
end

# ================================= align ==================================
# for e2a: compute anat alignment transformation to EPI registration base
# (new anat will be intermediate, stripped, anat_noskull_ns+orig)
align_epi_anat.py -anat2epi -anat anat_noskull+orig \
-save_skullstrip -suffix _al_junk \
-epi pb02.$subj.r01.tshift+orig -epi_base 0 \
-epi_strip 3dAutomask \
-volreg off -tshift off

# ================================= volreg =================================
# align each dset to base volume, align to anat

# register and warp
foreach run ( $runs )
# register each volume to the base
3dvolreg -verbose -zpad 1 -base pb02.$subj.r01.tshift+orig'[0]' \
-1Dfile dfile.r$run.1D -prefix rm.epi.volreg.r$run \
-cubic \
-1Dmatrix_save mat.r$run.vr.aff12.1D \
pb02.$subj.r$run.tshift+orig

# create an all-1 dataset to mask the extents of the warp
3dcalc -overwrite -a pb02.$subj.r$run.tshift+orig -expr 1 \
-prefix rm.epi.all1

# catenate volreg and epi2anat transformations
cat_matvec -ONELINE \
anat_noskull_al_junk_mat.aff12.1D -I \
mat.r$run.vr.aff12.1D > mat.r$run.warp.aff12.1D

# apply catenated xform : volreg and epi2anat
3dAllineate -base anat_noskull_ns+orig \
-input pb02.$subj.r$run.tshift+orig \
-1Dmatrix_apply mat.r$run.warp.aff12.1D \
-mast_dxyz 3 \
-prefix rm.epi.nomask.r$run

# warp the all-1 dataset for extents masking
3dAllineate -base anat_noskull_ns+orig \
-input rm.epi.all1+orig \
-1Dmatrix_apply mat.r$run.warp.aff12.1D \
-mast_dxyz 3 -final NN -quiet \
-prefix rm.epi.1.r$run

# make an extents intersection mask of this run
3dTstat -min -prefix rm.epi.min.r$run rm.epi.1.r$run+orig
end

# make a single file of registration params
cat dfile.r*.1D > dfile_rall.1D

# ----------------------------------------
# create the extents mask: mask_epi_extents+orig
# (this is a mask of voxels that have valid data at every TR)
3dMean -datum short -prefix rm.epi.mean rm.epi.min.r*.HEAD
3dcalc -a rm.epi.mean+orig -expr 'step(a-0.999)' -prefix mask_epi_extents

# and apply the extents mask to the EPI data
# (delete any time series with missing data)
foreach run ( $runs )
3dcalc -a rm.epi.nomask.r$run+orig -b mask_epi_extents+orig \
-expr 'a*b' -prefix pb03.$subj.r$run.volreg
end

# create an anat_final dataset, aligned with stats
3dcopy anat_noskull_ns+orig anat_final.$subj

# -----------------------------------------
# warp anat follower datasets (identity: resample)
3dAllineate -source copy_af_Ventricles+orig \
-master pb03.$subj.r01.volreg+orig \
-final NN -1Dparam_apply '1D: 12@0'\' \
-prefix follow_ROI_Ventricles
3dAllineate -source copy_af_WM+orig \
-master pb03.$subj.r01.volreg+orig \
-final NN -1Dparam_apply '1D: 12@0'\' \
-prefix follow_ROI_WM

# ================================ regress =================================

# compute de-meaned motion parameters (for use in regression)
1d_tool.py -infile dfile_rall.1D -set_run_lengths 256 266 266 266 266 266 \
-demean -write motion_demean.1D

# compute motion parameter derivatives (for use in regression)
1d_tool.py -infile dfile_rall.1D -set_run_lengths 256 266 266 266 266 266 \
-derivative -demean -write motion_deriv.1D

# create censor file motion_${subj}_censor.1D, for censoring motion
1d_tool.py -infile dfile_rall.1D -set_run_lengths 256 266 266 266 266 266 \
-show_censor_count -censor_prev_TR \
-censor_motion 0.2 motion_${subj}

# combine multiple censor files
1deval -a motion_${subj}_censor.1D -b outcount_${subj}_censor.1D \
-expr "a*b" > censor_${subj}_combined_2.1D

# create ROI regressor: WM
# (get each ROI average time series and remove resulting mean)
foreach run ( $runs )
3dmaskave -quiet -mask follow_ROI_WM+orig \
pb03.$subj.r$run.volreg+orig \
| 1d_tool.py -infile - -demean -write rm.ROI.WM.r$run.1D
end
# and catenate the demeaned ROI averages across runs
cat rm.ROI.WM.r*.1D > ROI.WM_rall.1D

# ------------------------------
# create ROI PC ort sets: Ventricles

# create a time series dataset to run 3dpc on...

# detrend, so principal components are not affected
foreach run ( $runs )
3dDetrend -polort 4 -prefix rm.det_pcin_r$run \
pb03.$subj.r$run.volreg+orig
end

# catenate runs, prepare to censor TRs
3dTcat -prefix rm.det_pcin_rall rm.det_pcin_r*+orig.HEAD
set ktrs = `1d_tool.py -infile censor_${subj}_combined_2.1D \
-show_trs_uncensored encoded`

# make ROI PCs and uncensor (zero-pad) : Ventricles
3dpc -mask follow_ROI_Ventricles+orig -pcsave 3 -prefix roi_pc_01_Ventricles \
rm.det_pcin_rall+orig"[$ktrs]"
1d_tool.py -censor_fill_parent censor_${subj}_combined_2.1D \
-infile roi_pc_01_Ventricles_vec.1D -write roi_pc_01_Ventricles_noc.1D

# ------------------------------
# run the regression analysis
3dDeconvolve -input pb03.$subj.r*.volreg+orig.HEAD \
-censor censor_${subj}_combined_2.1D \
-ortvec ROI.WM_rall.1D ROI.WM \
-ortvec roi_pc_01_Ventricles_noc.1D ROI.PC.Ventricles \
-polort 4 -float \
-num_stimts 12 \
-stim_file 1 motion_demean.1D'[0]' -stim_base 1 -stim_label 1 roll_01 \
-stim_file 2 motion_demean.1D'[1]' -stim_base 2 -stim_label 2 pitch_01 \
-stim_file 3 motion_demean.1D'[2]' -stim_base 3 -stim_label 3 yaw_01 \
-stim_file 4 motion_demean.1D'[3]' -stim_base 4 -stim_label 4 dS_01 \
-stim_file 5 motion_demean.1D'[4]' -stim_base 5 -stim_label 5 dL_01 \
-stim_file 6 motion_demean.1D'[5]' -stim_base 6 -stim_label 6 dP_01 \
-stim_file 7 motion_deriv.1D'[0]' -stim_base 7 -stim_label 7 roll_02 \
-stim_file 8 motion_deriv.1D'[1]' -stim_base 8 -stim_label 8 pitch_02 \
-stim_file 9 motion_deriv.1D'[2]' -stim_base 9 -stim_label 9 yaw_02 \
-stim_file 10 motion_deriv.1D'[3]' -stim_base 10 -stim_label 10 dS_02 \
-stim_file 11 motion_deriv.1D'[4]' -stim_base 11 -stim_label 11 dL_02 \
-stim_file 12 motion_deriv.1D'[5]' -stim_base 12 -stim_label 12 dP_02 \
-fout -tout -x1D X.xmat.1D -xjpeg X.jpg \
-x1D_uncensored X.nocensor.xmat.1D \
-fitts fitts.$subj \
-errts errts.${subj} \
-x1D_stop \
-bucket stats.$subj

# -- use 3dTproject to project out regression matrix --
3dTproject -polort 0 -input pb03.$subj.r*.volreg+orig.HEAD \
-censor censor_${subj}_combined_2.1D -cenmode ZERO \
-ort X.nocensor.xmat.1D -prefix errts.${subj}.tproject



# if 3dDeconvolve fails, terminate the script
if ( $status != 0 ) then
echo '---------------------------------------'
echo '** 3dDeconvolve error, failing...'
echo ' (consider the file 3dDeconvolve.err)'
exit
endif


# display any large pairwise correlations from the X-matrix
1d_tool.py -show_cormat_warnings -infile X.xmat.1D |& tee out.cormat_warn.txt

# create an all_runs dataset to match the fitts, errts, etc.
3dTcat -prefix all_runs.$subj pb03.$subj.r*.volreg+orig.HEAD

# --------------------------------------------------
# generate ANATICOR result: errts.$subj.fanaticor+orig

# catenate volreg dsets in case of censored sub-brick selection
3dTcat -prefix rm.all_runs.volreg pb03.$subj.r*.volreg+orig.HEAD

# mask white matter before blurring
3dcalc -a rm.all_runs.volreg+orig -b follow_ROI_WM+orig \
-expr "a*bool(b)" -datum float -prefix rm.all_runs.volreg.mask

# generate time series averaged over the closest white matter
3dmerge -1blur_fwhm 30 -doall -prefix Local_WM_rall \
rm.all_runs.volreg.mask+orig

# diagnostic volume: voxel correlation with local white matter
# (above and beyond X-matrix regressors)
3dTcorrelate -prefix WML_corr -ort X.nocensor.xmat.1D \
rm.all_runs.volreg+orig Local_WM_rall+orig

# diagnostic volume: raw correlation, no X-matrix regressors
3dTcorrelate -prefix WML_corr_raw \
rm.all_runs.volreg+orig Local_WM_rall+orig

# -- use 3dTproject to project out regression matrix --
3dTproject -polort 0 -input pb03.$subj.r*.volreg+orig.HEAD \
-censor censor_${subj}_combined_2.1D -cenmode ZERO \
-dsort Local_WM_rall+orig \
-ort X.nocensor.xmat.1D -prefix errts.$subj.fanaticor

# --------------------------------------------------
# create a temporal signal to noise ratio dataset
# signal: if 'scale' block, mean should be 100
# noise : compute standard deviation of errts
3dTstat -mean -prefix rm.signal.all all_runs.$subj+orig
3dTstat -stdev -prefix rm.noise.all errts.$subj.fanaticor+orig
3dcalc -a rm.signal.all+orig \
-b rm.noise.all+orig \
-expr 'a/b' -prefix TSNR.$subj

# --------------------------------------------------------
# compute sum of non-baseline regressors from the X-matrix
# (use 1d_tool.py to get list of regressor colums)
set reg_cols = `1d_tool.py -infile X.nocensor.xmat.1D -show_indices_interest`
3dTstat -sum -prefix sum_ideal.1D X.nocensor.xmat.1D"[$reg_cols]"

# also, create a stimulus-only X-matrix, for easy review
1dcat X.nocensor.xmat.1D"[$reg_cols]" > X.stim.xmat.1D

# ================== auto block: generate review scripts ===================

# generate a review script for the unprocessed EPI data
gen_epi_review.py -script @epi_review.$subj \
-dsets pb00.$subj.r*.tcat+orig.HEAD

# generate scripts to review single subject results
# (try with defaults, but do not allow bad exit status)
gen_ss_review_scripts.py -mot_limit 0.2 -out_limit 0.1 -exit0

# ========================== auto block: finalize ==========================

# remove temporary files
\rm -f rm.*

# if the basic subject review script is here, run it
# (want this to be the last text output)
if ( -e @ss_review_basic ) ./@ss_review_basic |& tee out.ss_review.$subj.txt

# return to parent directory
cd ..

echo "execution finished: `date`"




# ==========================================================================
# script generated by the command:
#
# afni_proc.py -subj_id afni_proc_Results -blocks despike tshift align volreg \
# regress -copy_anat anat_noskull.nii -anat_follower_ROI Ventricles epi \
# aseg_Ventricles+orig -anat_follower_ROI WM epi wm_eroded+orig -dsets \
# fm_run1+orig.HEAD fm_run2+orig.HEAD fm_run3+orig.HEAD fm_run4+orig.HEAD \
# fm_run5+orig.HEAD fm_run6+orig.HEAD -tcat_remove_first_trs 0 \
# -tshift_opts_ts -ignore 5 -volreg_align_e2a -volreg_align_to first \
# -volreg_warp_dxyz 3 -regress_ROI_PC Ventricles 3 -regress_ROI WM \
# -regress_anaticor_fast -regress_anaticor_label WM \
# -regress_censor_motion 0.2 -regress_censor_outliers 0.1 \
# -regress_skip_first_outliers 5 -regress_apply_mot_types demean deriv \
# -regress_run_clustsim no