Hi ptaylor,

First of all thank you for the highly detailed reply, especially considering the holiday! I also appreciate the extra resources for transparent thresholding and the helpful Python commands. I'll read more into the former and let you know if I have questions. As for the Python commands, unfortunately I can't install Python on our machine because it's an older iMac that requires an update to do so, and some of the people in the lab are apprehensive about updating the macOS when we are mid-analysis. So for now I suppose I'll stick with going straight to the 3dttest++ command itself! Thanks again for the suggestions.

Here are my replies in turn:

1) Great, so if the number of iterations doesn't matter much then I'll just play it safe and leave it on the default number (which I believe is 10,000). Since you asked, there are 18 subjects in the analysis and 2,640,253 voxels in the mask. I'll paste the entire 3dttest++ output at the end of this post. I also confirmed that the iMac I'm using is a quad-core CPU and the commands you offered also confirm there are 4 cores being used. The analyses I was running earlier that took multiple days seemed to be because the computer was going to sleep after some time and pausing the analysis. Once I changed some system settings to disallow this, the computer was able to run the analysis in 2-3 hours. This is still longer than expected but is more doable. To be clear, the analysis is working without any issues, it just takes some time. I'll reply to Rick's posts with more details on this front.

2) Gotcha - one simulation per contrast.

3) As far as I know we don't plan on doing ETAC, so I will try to use the smoothed data files. I'll also try and look back in our procedure to see if any smoothing was done earlier (I inherited this project part way along the analysis pipeline).




I was following a post on Andy's Brain Blog and I had a few new questions if you don't mind. I'm referring to this guide: https://andysbrainbook.readthedocs.io/en/latest/AFNI/AFNI_Short_Course/AFNI_07_GroupAnalysis.html

4) When creating a composite mask, is it best to use 'union' or 'intersect'? In the linked tutorial, the author used the '-union' command but named the resulting mask 'intersection'

Quote
3dmask_tool -input <path/to/masks/> -prefix mask_intersection+tlrc -union

5) Also I'm wondering how to use / read the output from the -ClustSim analysis. I understand the differences between NN1 vs. NN2, bisided vs. one-sided etc., but what I can't figure out is how to 'read' the table of cluster thresholds. From the blog guide linked above, it seems that you would typically want to set alpha to .05 by using the column labelled '.05000'. But then what are the row headers for? How can you use alpha = .05 and p < .001 at the same time? And further, why do smaller p-values in the row headers seem to require smaller cluster thresholds? To me, it seems at face value that you can set lower p-values which seems more robust while at the same time having it be easier to find significant clusters because the minimum cluster size gets smaller and smaller as the rows go down.

6) Finally, when I load the clusterized mask and functional data into AFNI, I presumably use the zscore as the ULAY and OLAY rather than the 'mean' option. To produce images of the corrected data based on the newly found cluster thresholds, do I then use the 'Clusterize' button and set the minimum cluster size and NN option there too, then set p = .001? Or does the zscore data loaded into the OLAY and ULAY already account for cluster thresholding naturally without having to use the Clusterize button?





Finally, if it helps you to see more details, here is the code and output I got when using 3dttest++:

v1040-wn-rt-b-34-154:anova_TEST mafernan_admin$ AFNI_TTEST_NUMCSIM=10000
v1040-wn-rt-b-34-154:anova_TEST mafernan_admin$ DAFNI_TTEST_NUMCSIM=10000
v1040-wn-rt-b-34-154:anova_TEST mafernan_admin$ cd /Users/mafernan_admin/Desktop/anova_TEST
v1040-wn-rt-b-34-154:anova_TEST mafernan_admin$ 3dttest++ -ClustSim -prefix DW-Con-Results                    \
>            -mask /Users/mafernan_admin/Desktop/anova_TEST/mask_union+tlrc.HEAD \
>            -setA DW_Con MM03 "/Users/mafernan_admin/Desktop/anova_TEST/p03_DW+tlrc" \
>                          MM04 "/Users/mafernan_admin/Desktop/anova_TEST/p04_DW+tlrc" \
>                          MM05 "/Users/mafernan_admin/Desktop/anova_TEST/p05_DW+tlrc" \
>                          MM06 "/Users/mafernan_admin/Desktop/anova_TEST/p06_DW+tlrc" \
>                          MM07 "/Users/mafernan_admin/Desktop/anova_TEST/p07_DW+tlrc" \
>                          MM08 "/Users/mafernan_admin/Desktop/anova_TEST/p08_DW+tlrc" \
>                          MM09 "/Users/mafernan_admin/Desktop/anova_TEST/p09_DW+tlrc" \
>                          MM10 "/Users/mafernan_admin/Desktop/anova_TEST/p10_DW+tlrc" \
>                          MM11 "/Users/mafernan_admin/Desktop/anova_TEST/p11_DW+tlrc" \
>                          MM14 "/Users/mafernan_admin/Desktop/anova_TEST/p14_DW+tlrc" \
>                          MM15 "/Users/mafernan_admin/Desktop/anova_TEST/p15_DW+tlrc" \
>                          MM16 "/Users/mafernan_admin/Desktop/anova_TEST/p16_DW+tlrc" \
>                          MM17 "/Users/mafernan_admin/Desktop/anova_TEST/p17_DW+tlrc" \
>                          MM18 "/Users/mafernan_admin/Desktop/anova_TEST/p18_DW+tlrc" \
>                          MM19 "/Users/mafernan_admin/Desktop/anova_TEST/p19_DW+tlrc" \
>                          MM20 "/Users/mafernan_admin/Desktop/anova_TEST/p20_DW+tlrc" \
>                          MM21 "/Users/mafernan_admin/Desktop/anova_TEST/p21_DW+tlrc" \
>                          MM101 "/Users/mafernan_admin/Desktop/anova_TEST/p101_DW+tlrc"
++ 3dttest++: AFNI version=AFNI_19.2.24 (Sep 13 2019) [64-bit]
++ Authored by: Zhark++
++ Number of -Clustsim threads set to 4
 + Default clustsim prefix set to 'TTnew'
++ 2640253 voxels in -mask dataset
++ option -setA :: processing as LONG form (label label dset label dset ...)
++ have 18 volumes corresponding to option '-setA'
++ loading -setA datasets
*+ WARNING: 3dttest++ -setA :: 7150 vectors are constant
++ t-testing:0123456789.0123456789.0123456789.0123456789.0123456789.!
 + skipped 7150 voxels completely for having constant data
++ ---------- End of analyses -- freeing workspaces ----------
++ Creating FDR curves in output dataset
*+ WARNING: Smallest FDR q [1 DW_Con_Zscr] = 0.789136 ==> few true single voxel detections
 + Added 1 FDR curve to dataset
++ Output dataset ./DW-Con-Results+tlrc.BRIK
++ Output dataset ./TTnew.resid.nii
++ ================ Starting -ClustSim calculations ================
 + === temporary files will have prefix TTnew ===
 + === running 4 -randomsign jobs (2500 iterations per job) ===
 + === creating 52,805,060,000 (53 billion) bytes of pseudo-data in .sdat files ===
 + --- 3dClustSim reads .sdat files to compute cluster-threshold statistics ---
 + --- there is 8,589,934,592 (8.6 billion) bytes of memory on your system ---
*+ WARNING: --- runs may be slow (or crash) due to memory needs :( ---
++ 3dttest++: AFNI version=AFNI_19.2.24 (Sep 13 2019) [64-bit]
++ Authored by: Zhark++
++ 2640253 voxels in -mask dataset
++ option -setA :: processing as SHORT form (all values are datasets)
++ have 18 volumes corresponding to option '-setA'
++ random seeds are 964209600 1278368865
++ opened file ./TTnew.0000.sdat for output
++ loading -setA datasets
*+ WARNING: 3dttest++ -setA :: 7150 vectors are constant
++ t-test randomsign:0123456789.0123456789.0123456789.0123456789.0123456789.!
++ saving main effect t-stat MIN/MAX values in ./TTnew.0000.minmax.1D
++ output short-ized file ./TTnew.0000.sdat
 + 3dttest++ ===== simulation jobs have finished (600351.6 s elapsed)
 + 3dttest++ ===== starting 3dClustSim A: elapsed = 600362.7 s
++ 3dClustSim: AFNI version=AFNI_19.2.24 (Sep 13 2019) [64-bit]
++ Authored by: RW Cox and BD Ward
++ Loading -insdat datasets
++ -insdat had 10000 volumes = new '-niter' value
++ Startup clock time = 0.2 s
++ Using 4 OpenMP threads
Simulating:0123456789.0123456789.0123456789.0123456789.0123456789.!
++ Clock time now = 5279.9 s
++ Command fragment to put cluster results into a dataset header;
 + (also echoed to file TTnew.CSimA.cmd for your scripting pleasure)
 + Append the name of the datasets to be patched to this command:
 3drefit -atrstring AFNI_CLUSTSIM_NN1_1sided file:TTnew.CSimA.NN1_1sided.niml -atrstring AFNI_CLUSTSIM_NN2_1sided file:TTnew.CSimA.NN2_1sided.niml -atrstring AFNI_CLUSTSIM_NN3_1sided file:TTnew.CSimA.NN3_1sided.niml -atrstring AFNI_CLUSTSIM_NN1_2sided file:TTnew.CSimA.NN1_2sided.niml -atrstring AFNI_CLUSTSIM_NN2_2sided file:TTnew.CSimA.NN2_2sided.niml -atrstring AFNI_CLUSTSIM_NN3_2sided file:TTnew.CSimA.NN3_2sided.niml -atrstring AFNI_CLUSTSIM_NN1_bisided file:TTnew.CSimA.NN1_bisided.niml -atrstring AFNI_CLUSTSIM_NN2_bisided file:TTnew.CSimA.NN2_bisided.niml -atrstring AFNI_CLUSTSIM_NN3_bisided file:TTnew.CSimA.NN3_bisided.niml 
++ Environment variable AFNI_DONT_LOGFILE already set to 'NO'. Value of 'YES' from /Users/mafernan_admin/.afnirc is ignored. 
To kill such warnings Set AFNI_ENVIRON_WARNINGS to NO
++ 3drefit: AFNI version=AFNI_19.2.24 (Sep 13 2019) [64-bit]
++ Authored by: RW Cox
++ Processing AFNI dataset ./DW-Con-Results+tlrc.HEAD
 + atrcopy
++ applying attributes
++ 3drefit processed 1 datasets
++ 3dttest++ ----- Cleaning up intermediate files:
TTnew.resid.nii
./TTnew.0000.sdat
./TTnew.0001.sdat
./TTnew.0002.sdat
./TTnew.0003.sdat
TTnew.CSimA.NN1_1sided.niml
TTnew.CSimA.NN1_2sided.niml
TTnew.CSimA.NN1_bisided.niml
TTnew.CSimA.NN2_1sided.niml
TTnew.CSimA.NN2_2sided.niml
TTnew.CSimA.NN2_bisided.niml
TTnew.CSimA.NN3_1sided.niml
TTnew.CSimA.NN3_2sided.niml
TTnew.CSimA.NN3_bisided.niml
 + 3dttest++ =============== -ClustSim work is finished :) ===============
++ ----- 3dttest++ says so long, farewell, and happy trails to you :) -----

Edited 4 time(s). Last edit at 11/24/2022 08:04PM by BradyRoberts.