AFNI Message Board

Dear AFNI users-

We are very pleased to announce that the new AFNI Message Board framework is up! Please join us at:

https://discuss.afni.nimh.nih.gov

Existing user accounts have been migrated, so returning users can login by requesting a password reset. New users can create accounts, as well, through a standard account creation process. Please note that these setup emails might initially go to spam folders (esp. for NIH users!), so please check those locations in the beginning.

The current Message Board discussion threads have been migrated to the new framework. The current Message Board will remain visible, but read-only, for a little while.

Sincerely, AFNI HQ

History of AFNI updates  

|
July 26, 2016 03:02PM
Hi,

I'm experimenting with 3dttest++ to explore teh new resid and clustsim options but have run into a problem with it not working.

I'm running 3dttest++ as follows:
cd /data/sanDiego/rsfcGraphAnalysis/data//Group.results ; mkdir tmp.resid.clustsim.L_whole_amygdala.3mm ; 3dttest++ -resid new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid \
-CLUSTSIM \
-prefix_clustsim resid.CStemp.L_whole_amygdala.3mm \
-tempdir tmp.resid.clustsim.L_whole_amygdala.3mm \
-mask mask.grey.mddAndCtrl.union.masked+tlrc.HEAD \
-prefix new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried \
-center NONE \
-setA MDD 106_A /data/sanDiego/rsfcGraphAnalysis//data/106_A/rsfc/L_whole_amygdala.3mm/L_whole_amygdala.3mm.z-score+tlrc.HEAD \
111_A /data/sanDiego/rsfcGraphAnalysis//data/111_A/rsfc/L_whole_amygdala.3mm/L_whole_amygdala.3mm.z-score+tlrc.HEAD \
112_A /data/sanDiego/rsfcGraphAnalysis//data/112_A/rsfc/L_whole_amygdala.3mm/L_whole_amygdala.3mm.z-score+tlrc.HEAD \
113_A /data/sanDiego/rsfcGraphAnalysis//data/113_A/rsfc/L_whole_amygdala.3mm/L_whole_amygdala.3mm.z-score+tlrc.HEAD \
[trimmed] \
-setB NCL 108_A /data/sanDiego/rsfcGraphAnalysis//data/108_A/rsfc/L_whole_amygdala.3mm/L_whole_amygdala.3mm.z-score+tlrc.HEAD \
109_A /data/sanDiego/rsfcGraphAnalysis//data/109_A/rsfc/L_whole_amygdala.3mm/L_whole_amygdala.3mm.z-score+tlrc.HEAD \
116_A /data/sanDiego/rsfcGraphAnalysis//data/116_A/rsfc/L_whole_amygdala.3mm/L_whole_amygdala.3mm.z-score+tlrc.HEAD \
121_A /data/sanDiego/rsfcGraphAnalysis//data/121_A/rsfc/L_whole_amygdala.3mm/L_whole_amygdala.3mm.z-score+tlrc.HEAD \
[trimmed] \
-covariates /data/sanDiego/rsfcGraphAnalysis/data//Group.data/3dttest.covariates.mddAndCtrl.L_whole_amygdala.3mm.txt

but it's failing with the following messages (for some reason it's doubling up on the tempdir in filenames:

[colmconn@kryten scripts]$ export OMP_NUM_THREADS=40
[colmconn@kryten scripts]$ bash   new-07-ttest.withCovariates.mddAndCtrl.L_whole_amygdala.3mm.sh
++ 3dttest++: AFNI version=AFNI_16.2.05 (Jul 22 2016) [64-bit]
++ Authored by: Zhark++
++ Number of -clustsim threads set to 40
++ 67116 voxels in -mask dataset
++ Covariates file: 2 columns, each with 101 rows
 + Covariate column labels: subject;Full
 + Found 1 numeric column: Full 
++ running 3dClustSim --> changed '-resid' prefix to NIFTI form: new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii
++ 2-sample test: '-AminusB' option is assumed
++ 2-sample test: results will be MDD - NCL
++ loading covariates
++ Two samples: t-testing fixed covariates for similarity between groups
 +  Full: mean of setA-setB=-9.504717 ; 2-sided p-value=0.0001
++ loading input datasets
++ Memory usage now = 60,513,316 (61 million [mega])
++ t-testing:0123456789.0123456789.0123456789.0123456789.0123456789.!
 + skipped 39 voxels completely for having constant data
++ ---------- End of analyses -- freeing workspaces ----------
++ Memory usage now = 124,757,340 (125 million [mega])
++ Creating FDR curves in output dataset
*+ WARNING: Smallest FDR q [1 MDD-NCL_Zscr] = 0.677335 ==> few true single voxel detections
*+ WARNING: Smallest FDR q [3 MDD-NCL_Full_Zscr] = 0.999898 ==> few true single voxel detections
++ Smallest FDR q [5 MDD_Zscr] = 4.14588e-14
*+ WARNING: Smallest FDR q [7 MDD_Full_Zscr] = 0.930404 ==> few true single voxel detections
++ Smallest FDR q [9 NCL_Zscr] = 4.92629e-14
*+ WARNING: Smallest FDR q [11 NCL_Full_Zscr] = 0.961098 ==> few true single voxel detections
 + Added 6 FDR curves to dataset
*+ WARNING: Over-writing dataset ./new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried+tlrc.HEAD
*+ WARNING: Over-writing dataset ./new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii.gz
++ Output dataset ./new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii.gz
 + 2-sample test: results are MDD - NCL
++ ================ Starting -clustsim calculations ================
 + ===== temporary files will have prefix resid.CStemp.L_whole_amygdala.3mm =====
 + ===== running 40 -randomsign jobs (251 iterations per job) =====
 + #0 jobs command:
   3dttest++ -DAFNI_AUTOMATIC_FDR=NO -DAFNI_DONT_LOGFILE=YES -randomsign 251 -nomeans -toz -no1sam -mask mask.grey.mddAndCtrl.union.masked+tlrc.HEAD -nocov -covariates /data/sanDiego/rsfcGraphAnalysis/data//Group.data/3dttest.covariates.mddAndCtrl.L_whole_amygdala.3mm.txt -center NONE -setA rAAA 106_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[0]' 111_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[1]' 112_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[2]' 113_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[3]' 118_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[4]' 120_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[5]' 133_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[6]' 134_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[7]' 136_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[8]' 137_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[9]' 140_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[10]' 144_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[11]' 147_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[12]' 154_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[13]' 158_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[14]' 160_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[15]' 164_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[16]' 167_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[17]' 301_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[18]' 304_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[19]' 310_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[20]' 313_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[21]' 330_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[22]' 331_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[23]' 332_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[24]' 335_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[25]' 336_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[26]' 339_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[27]' 342_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[28]' 343_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[29]' 351_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[30]' 353_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[31]' 355_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[32]' 356_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[33]' 358_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[34]' 359_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[35]' 360_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[36]' 362_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[37]' 363_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[38]' 364_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[39]' 366_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[40]' 368_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[41]' 371_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[42]' 373_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[43]' 378_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[44]' 389_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[45]' 390_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[46]' 393_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[47]' -setB rBBB 108_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[48]' 109_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[49]' 116_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[50]' 121_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[51]' 122_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[52]' 123_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[53]' 124_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[54]' 126_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[55]' 127_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[56]' 131_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[57]' 138_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[58]' 139_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[59]' 141_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[60]' 142_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[61]' 143_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[62]' 145_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[63]' 151_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[64]' 152_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[65]' 155_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[66]' 157_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[67]' 159_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[68]' 162_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[69]' 163_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[70]' 165_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[71]' 306_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[72]' 307_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[73]' 312_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[74]' 321_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[75]' 326_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[76]' 328_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[77]' 337_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[78]' 348_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[79]' 354_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[80]' 365_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[81]' 367_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[82]' 374_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[83]' 377_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[84]' 380_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[85]' 382_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[86]' 395_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[87]' 397_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[88]' 400_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[89]' 402_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[90]' 403_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[91]' 406_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[92]' 409_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[93]' 413_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[94]' 414_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[95]' 417_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[96]' 419_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[97]' 421_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[98]' 422_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[99]' 423_A new.ttest.mddAndCtrl.L_whole_amygdala.3mm.covaried.resid.nii'[100]' -prefix tmp.resid.clustsim.L_whole_amygdala.3mm/resid.CStemp.L_whole_amygdala.3mm.000.nii
++ 3dttest++: AFNI version=AFNI_16.2.05 (Jul 22 2016) [64-bit]
++ Authored by: Zhark++
++ 67116 voxels in -mask dataset
++ Covariates file: 2 columns, each with 101 rows
 + Covariate column labels: subject;Full
 + Found 1 numeric column: Full 
++ 2-sample test: '-AminusB' option is assumed
++ 2-sample test: results will be rAAA - rBBB
++ loading covariates
++ Two samples: t-testing fixed covariates for similarity between groups
 +  Full: mean of setA-setB=-9.504717 ; 2-sided p-value=0.0001
++ loading input datasets
++ Memory usage now = 32,633,595 (33 million [mega])
++ t-test randomsign:0123456789.0123456789.0123456789.0123456789.0123456789.!
++ ---------- End of analyses -- freeing workspaces ----------
++ Memory usage now = 276,352,523 (276 million [mega])
** ERROR (nifti_image_write_hdr_img2): cannot open output file 'tmp.resid.clustsim.L_whole_amygdala.3mm/tmp.resid.clustsim.L_whole_amygdala.3mm/resid.CStemp.L_whole_amygdala.3mm.000.nii.gz'
 + 2-sample test: results are rAAA - rBBB
 + ===== jobs have finished (358.92 s elapsed) =====
 + ===== starting 3dClustSim =====
   3dClustSim -DAFNI_DONT_LOGFILE=YES -inset tmp.resid.clustsim.L_whole_amygdala.3mm/resid.CStemp.L_whole_amygdala.3mm.???.nii -prefix resid.CStemp.L_whole_amygdala.3mm.CSim -LOTS -both -nodec -cmd resid.CStemp.L_whole_amygdala.3mm.CSim.cmd -mask mask.grey.mddAndCtrl.union.masked+tlrc.HEAD
++ 3dClustSim: AFNI version=AFNI_16.2.05 (Jul 22 2016) [64-bit]
++ Authored by: RW Cox and BD Ward
** ERROR: -inset 'tmp.resid.clustsim.L_whole_amygdala.3mm/resid.CStemp.L_whole_amygdala.3mm.???.nii': failure to open dataset
** FATAL ERROR: no valid datasets opened after -inset sad smiley
** Program compile date = Jul 22 2016
** FATAL ERROR: ===== 3dClustSim command failed sad smiley(( =====
** Program compile date = Jul 22 2016
[colmconn@kryten scripts]$

Is the cause of this error the command line or a bug in3dttest++.


Thanks,

Colm.



Edited 1 time(s). Last edit at 07/27/2016 01:20PM by Colm Connolly.
Subject Author Posted

3dttest CLUSTSIM and tempdir problem

Colm Connolly July 26, 2016 03:02PM

Re: 3dttest CLUSTSIM and tempdir problem

Colm Connolly July 27, 2016 02:17PM

Re: 3dttest CLUSTSIM and tempdir problem

rick reynolds July 28, 2016 04:07PM