AFNI Message Board

Dear AFNI users-

We are very pleased to announce that the new AFNI Message Board framework is up! Please join us at:

https://discuss.afni.nimh.nih.gov

Existing user accounts have been migrated, so returning users can login by requesting a password reset. New users can create accounts, as well, through a standard account creation process. Please note that these setup emails might initially go to spam folders (esp. for NIH users!), so please check those locations in the beginning.

The current Message Board discussion threads have been migrated to the new framework. The current Message Board will remain visible, but read-only, for a little while.

Sincerely, AFNI HQ

History of AFNI updates  

Previous Message Next Message
Message List New Topic Print View
|
Nicholas Balderston
March 25, 2019 12:13PM
Hi everyone,

I haven't gotten a reply, so I might ping someone from the AFNI team. In the meantime, I think I have found a workaround that works for me. At its core, I follow 3 basic steps.


  1. run afni_proc on each run with minimal preprocessing using tedana to combine the echos.
  2. extract the "bad" component timeseries using @extract_meica_ortvec
  3. run the full afni_proc command using the combine OC option, adding the "bad" component timeseries to the 3ddeconvolve command using the -ortvec flag.


Although the default tedana parameters worked for most files, I had to run several afni_proc commands with different tedana options to get tedana to process all the datasets ~383 total (96 subs x 4 runs -1 missing dataset). I ran the commands successively, so that once one afni_proc command completed successfully, the others did not run for that file. Here's a breakdown of the different tedana options and the number of files processed for each.

tedana options, number of datasets completed
  1. kdaw=10, rdaw=1 (default), n=320
  2. kdaw=5, rdaw=1, n=36
  3. kdaw=15, rdaw=1, n=2
  4. kdaw=-1, rdaw=-1, n=21
  5. kdaw=10, rdaw=1, stabilize, n=0
  6. kdaw=-1, rdaw=-1, stabilize, n=3
  7. fail and move on n = 1

I'm not sure if this is the best approach, but it worked for all but one file, so it seems pretty robust. I realize that the basic approach is similar to using the tedana_OC_tedort flag; however, I think it has 2 added benefits.

  1. It allows you to use different tedana parameters for each run if needed.
  2. Given that the final afni_proc command does not rely on tedana for the denoising, it will not fail even if tedana fails on one run. Instead, the "bad" component timeseries for the the failed run are just not passed to the second afni_proc command, which defaults to the OC combine method.


Here is the code for each of the 3 steps.

#########################
Initial afni_proc commands
######################### 
afni_proc.py \
-script proc.${subject}.${run}.me \
-scr_overwrite \
-subj_id ${subject} \
-out_dir ${deldir}/${subject}.results.${run}.me \
-blocks despike tshift volreg mask combine  \
-volreg_align_to MIN_OUTLIER \
-dsets_me_run $d \
-echo_times ${e[1]} ${e[2]} ${e[3]} \
-combine_method tedana \
-reg_echo 2 \
-remove_preproc_files \
-execute

######################################kdaw5 
if (-e ${deldir}/${subject}.results.${run}.me/tedana_r01/TED.r01/dn_ts_OC.nii ) then 
echo "######################################reg worked"
else

rm -rf ${deldir}/${subject}.results.${run}.me

afni_proc.py \
-script proc.${subject}.${run}.me \
-scr_overwrite \
-subj_id ${subject} \
-out_dir ${deldir}/${subject}.results.${run}.me \
-blocks despike tshift volreg mask combine  \
-volreg_align_to MIN_OUTLIER \
-dsets_me_run $d \
-echo_times ${e[1]} ${e[2]} ${e[3]} \
-combine_method tedana \
-combine_opts_tedana --kdaw=5 \
-reg_echo 2 \
-remove_preproc_files \
-execute

endif

######################################kdaw15 
if (-e ${deldir}/${subject}.results.${run}.me/tedana_r01/TED.r01/dn_ts_OC.nii ) then 
echo "######################################kdaw5 worked"
else

rm -rf ${deldir}/${subject}.results.${run}.me

afni_proc.py \
-script proc.${subject}.${run}.me \
-scr_overwrite \
-subj_id ${subject} \
-out_dir ${deldir}/${subject}.results.${run}.me \
-blocks despike tshift volreg mask combine  \
-volreg_align_to MIN_OUTLIER \
-dsets_me_run $d \
-echo_times ${e[1]} ${e[2]} ${e[3]} \
-combine_method tedana \
-combine_opts_tedana --kdaw=15 \
-reg_echo 2 \
-remove_preproc_files \
-execute

endif

######################################kdaw-1 
if (-e ${deldir}/${subject}.results.${run}.me/tedana_r01/TED.r01/dn_ts_OC.nii ) then 
echo "######################################kdaw15 worked"
else

rm -rf ${deldir}/${subject}.results.${run}.me

afni_proc.py \
-script proc.${subject}.${run}.me \
-scr_overwrite \
-subj_id ${subject} \
-out_dir ${deldir}/${subject}.results.${run}.me \
-blocks despike tshift volreg mask combine  \
-volreg_align_to MIN_OUTLIER \
-dsets_me_run $d \
-echo_times ${e[1]} ${e[2]} ${e[3]} \
-combine_method tedana \
-combine_opts_tedana --kdaw=-1 --rdaw=-1 \
-reg_echo 2 \
-remove_preproc_files \
-execute

endif

######################################stabilize 
if (-e ${deldir}/${subject}.results.${run}.me/tedana_r01/TED.r01/dn_ts_OC.nii ) then 
echo "######################################kdaw-1 worked"
else

rm -rf ${deldir}/${subject}.results.${run}.me

afni_proc.py \
-script proc.${subject}.${run}.me \
-scr_overwrite \
-subj_id ${subject} \
-out_dir ${deldir}/${subject}.results.${run}.me \
-blocks despike tshift volreg mask combine  \
-volreg_align_to MIN_OUTLIER \
-dsets_me_run $d \
-echo_times ${e[1]} ${e[2]} ${e[3]} \
-combine_method tedana \
-combine_opts_tedana --stabilize \
-reg_echo 2 \
-remove_preproc_files \
-execute

endif

######################################stabilize kdaw-1
if (-e ${deldir}/${subject}.results.${run}.me/tedana_r01/TED.r01/dn_ts_OC.nii ) then 
echo "######################################stabilize worked"
else

rm -rf ${deldir}/${subject}.results.${run}.me

afni_proc.py \
-script proc.${subject}.${run}.me \
-scr_overwrite \
-subj_id ${subject} \
-out_dir ${deldir}/${subject}.results.${run}.me \
-blocks despike tshift volreg mask combine  \
-volreg_align_to MIN_OUTLIER \
-dsets_me_run $d \
-echo_times ${e[1]} ${e[2]} ${e[3]} \
-combine_method tedana \
-combine_opts_tedana --kdaw=-1 --rdaw=-1 --stabilize \
-reg_echo 2 \
-remove_preproc_files \
-execute

endif



#########################
@extract_meica_ortvec step
######################### 

foreach subject (${subs})


cd ${subdir}/${subject}


set runs=`ls -d ${subject}.results.run*.me | cut -d. -f3 | uniq`
set nruns=`ls -d ${subject}.results.run*.me | cut -d. -f3 | uniq | wc -l`
set trs=`3dinfo -nt ${subject}.results.run?.me/tedana_r01/tedprep_Zcat_all.nii.gz`
#TED.r01/dn_ts_OC.nii`
set r=1

foreach run ($runs)


echo "#######################################extract "
@extract_meica_ortvec \
-meica_dir ${subject}.results.${run}.me/tedana_r01/TED.r01 \
-work_dir ${subject}.results.${run}.me/tedana_r01/regwork.r01 \
-prefix ${subject}.tedana.badorts.$run.1d \


echo "################################################################1dtool $r"
1d_tool.py \
-infile ${subject}.tedana.badorts.$run.1d \
-pad_into_many_runs ${r} ${nruns} \
-set_run_lengths ${trs} \
-write ${subject}.tedana.badorts.pad.$run.1d

@ r++

end
paste \
${subject}.tedana.badorts.pad.run?.1d \
> ${subject}.tedana.badorts.pad.1d


end




#########################
full afni_proc command
######################### 

afni_proc.py \
-script proc.${subject}.me \
-scr_overwrite \
-subj_id ${subject} \
-out_dir ${deldir}/${subject}.results.me \
-blocks despike tshift align tlrc volreg mask combine blur scale regress \
-copy_anat ${t1} \
-align_opts_aea \
-giant_move \
-cost lpc+ZZ \
-partial_axial \
-volreg_align_to MIN_OUTLIER \
-volreg_align_e2a \
-volreg_tlrc_warp \
-anat_has_skull no \
-tlrc_base MNI152_T1_2009c+tlrc \
-tlrc_NL_warp \
-blip_forward_dset $forwardepi \
-blip_reverse_dset $reversedepi \
-volreg_align_to MIN_OUTLIER \
$epi \
-echo_times ${e[1]} ${e[2]} ${e[3]} \
-combine_method OC \
-reg_echo 2 \
-blur_in_mask yes \
-regress_opts_3dD \
-ortvec ${subdir}/${subject}/${subject}.tedana.badorts.pad.1d tedana \
-regress_motion_per_run \
-regress_apply_mot_types demean deriv \
-regress_censor_motion 0.5 \
-regress_censor_outliers 0.15 \
-regress_censor_first_trs 4 \
-regress_est_blur_errts \
-regress_stim_times \
${stmdir}/${subject}.button_presses.1d \
${stmdir}/${subject}.shocks.1d \
${stmdir}/${subject}.cue.saf.hgh.hgh.l.1d \
${stmdir}/${subject}.cue.saf.hgh.hgh.r.1d \
${stmdir}/${subject}.cue.saf.hgh.low.l.1d \
${stmdir}/${subject}.cue.saf.hgh.low.r.1d \
${stmdir}/${subject}.cue.saf.low.hgh.l.1d \
${stmdir}/${subject}.cue.saf.low.hgh.r.1d \
${stmdir}/${subject}.cue.saf.low.low.l.1d \
${stmdir}/${subject}.cue.saf.low.low.r.1d \
${stmdir}/${subject}.cue.thr.hgh.hgh.l.1d \
${stmdir}/${subject}.cue.thr.hgh.hgh.r.1d \
${stmdir}/${subject}.cue.thr.hgh.low.l.1d \
${stmdir}/${subject}.cue.thr.hgh.low.r.1d \
${stmdir}/${subject}.cue.thr.low.hgh.l.1d \
${stmdir}/${subject}.cue.thr.low.hgh.r.1d \
${stmdir}/${subject}.cue.thr.low.low.l.1d \
${stmdir}/${subject}.cue.thr.low.low.r.1d \
${stmdir}/${subject}.square.saf.hgh.hgh.l.1d \
${stmdir}/${subject}.square.saf.hgh.hgh.r.1d \
${stmdir}/${subject}.square.saf.hgh.low.l.1d \
${stmdir}/${subject}.square.saf.hgh.low.r.1d \
${stmdir}/${subject}.square.saf.low.hgh.l.1d \
${stmdir}/${subject}.square.saf.low.hgh.r.1d \
${stmdir}/${subject}.square.saf.low.low.l.1d \
${stmdir}/${subject}.square.saf.low.low.r.1d \
${stmdir}/${subject}.square.thr.hgh.hgh.l.1d \
${stmdir}/${subject}.square.thr.hgh.hgh.r.1d \
${stmdir}/${subject}.square.thr.hgh.low.l.1d \
${stmdir}/${subject}.square.thr.hgh.low.r.1d \
${stmdir}/${subject}.square.thr.low.hgh.l.1d \
${stmdir}/${subject}.square.thr.low.hgh.r.1d \
${stmdir}/${subject}.square.thr.low.low.l.1d \
${stmdir}/${subject}.square.thr.low.low.r.1d \
${stmdir}/${subject}.target.saf.hgh.hgh.l.1d \
${stmdir}/${subject}.target.saf.hgh.hgh.r.1d \
${stmdir}/${subject}.target.saf.hgh.low.l.1d \
${stmdir}/${subject}.target.saf.hgh.low.r.1d \
${stmdir}/${subject}.target.saf.low.hgh.l.1d \
${stmdir}/${subject}.target.saf.low.hgh.r.1d \
${stmdir}/${subject}.target.saf.low.low.l.1d \
${stmdir}/${subject}.target.saf.low.low.r.1d \
${stmdir}/${subject}.target.thr.hgh.hgh.l.1d \
${stmdir}/${subject}.target.thr.hgh.hgh.r.1d \
${stmdir}/${subject}.target.thr.hgh.low.l.1d \
${stmdir}/${subject}.target.thr.hgh.low.r.1d \
${stmdir}/${subject}.target.thr.low.hgh.l.1d \
${stmdir}/${subject}.target.thr.low.hgh.r.1d \
${stmdir}/${subject}.target.thr.low.low.l.1d \
${stmdir}/${subject}.target.thr.low.low.r.1d \
-regress_stim_types \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
times \
-regress_basis_multi \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
'BLOCK(1,1)' \
-regress_stim_labels \
"button_presses" \
"shocks" \
"cue.saf.hgh.hgh.l" \
"cue.saf.hgh.hgh.r" \
"cue.saf.hgh.low.l" \
"cue.saf.hgh.low.r" \
"cue.saf.low.hgh.l" \
"cue.saf.low.hgh.r" \
"cue.saf.low.low.l" \
"cue.saf.low.low.r" \
"cue.thr.hgh.hgh.l" \
"cue.thr.hgh.hgh.r" \
"cue.thr.hgh.low.l" \
"cue.thr.hgh.low.r" \
"cue.thr.low.hgh.l" \
"cue.thr.low.hgh.r" \
"cue.thr.low.low.l" \
"cue.thr.low.low.r" \
"square.saf.hgh.hgh.l" \
"square.saf.hgh.hgh.r" \
"square.saf.hgh.low.l" \
"square.saf.hgh.low.r" \
"square.saf.low.hgh.l" \
"square.saf.low.hgh.r" \
"square.saf.low.low.l" \
"square.saf.low.low.r" \
"square.thr.hgh.hgh.l" \
"square.thr.hgh.hgh.r" \
"square.thr.hgh.low.l" \
"square.thr.hgh.low.r" \
"square.thr.low.hgh.l" \
"square.thr.low.hgh.r" \
"square.thr.low.low.l" \
"square.thr.low.low.r" \
"target.saf.hgh.hgh.l" \
"target.saf.hgh.hgh.r" \
"target.saf.hgh.low.l" \
"target.saf.hgh.low.r" \
"target.saf.low.hgh.l" \
"target.saf.low.hgh.r" \
"target.saf.low.low.l" \
"target.saf.low.low.r" \
"target.thr.hgh.hgh.l" \
"target.thr.hgh.hgh.r" \
"target.thr.hgh.low.l" \
"target.thr.hgh.low.r" \
"target.thr.low.hgh.l" \
"target.thr.low.hgh.r" \
"target.thr.low.low.l" \
"target.thr.low.low.r" \
-test_stim_files no \
-remove_preproc_files \
-execute
Reply Quote
RSS
Subject Author Posted

multi-echo, tedana, block order, alignment

Nicholas Balderston March 17, 2019 11:51AM

Re: multi-echo, tedana, block order, alignment

Nicholas Balderston March 25, 2019 12:13PM