Hi, I have two questions about using -ClustSim within 3dttest++:

1) I'm using 3dMEMA for the actual group statistics. For cluster size thresholds, I'm used to using 3dClustSim with -acf, but recently a colleague was told that it's fine to use the clustsim output from 3dttest++ with a MEMA analysis. As this method is newer and the 3dClustSim help file says it's the most accurate at controlling the false positive rate, I gave it a try, but I've been getting the "permute failure" error (see full output below). In this thread, Gang says it's still fine to interpret the results despite this error, but he also suggests the error is due to voxelwise covariates. I don't think that applies to my case, as all of my covariates are subject-level, and they're formatted in the same way as for 3dMEMA. Could you just confirm that I'm ok to use the clustsim output from 3dttest++ even with the permute failure?

2) I'm also confused as to whether the 3dttest++ cluster method really is preferred over 3dClustsim (with ACF parameters estimated from 3dFWHMx) when used in conjunction with 3dMEMA. I've done it both ways and the difference in cluster size is non-negligible, i.e. around 100 voxels (e.g. 142 vs. 42). My colleague running similar analyses also found this magnitude of difference between the methods. Is this normal, and which threshold should I go with?

Thanks in advance! By the way, for visualization I plan to use Gang's recommended "highlight but not hide" approach, so I'll just be using the cluster threshold to do the highlighting (and also for the coordinates tables).

Here's the output I got from 3dttest++:

++ 3dttest++: AFNI version=AFNI_21.2.04 (Jul 28 2021) [64-bit]
++ Authored by: Zhark++
++ Number of -Clustsim threads set to 72
++ option -setA :: processing as LONG form (label label dset label dset ...)
++ have 33 volumes corresponding to option '-setA'
++ option -setB :: processing as LONG form (label label dset label dset ...)
++ have 33 volumes corresponding to option '-setB'
++ 259767 voxels in -mask dataset
++ Covariates file: 5 columns, each with 66 rows
+ Covariate column labels: subj;meanFD;runs;age;avgRT
+ Found 4 numeric columns: meanFD runs age avgRT
++ 2-sample test: '-AminusB' option is assumed
++ 2-sample test: results will be AUT - TD
++ loading covariates
** ERROR: -permute failure for unfathomable reasons /:(
nx=33 ny=33 p_nxy=0 x==non-NULL y==non-NULL
Function traceback follows:
permute_arrays
TT_matrix_setup
3dttest++ main
** Command line was:
3dttest++ -prefix AUTvTD_3dttest_P-C_G -Clustsim -setA AUT REDCAT201 'REDCAT201_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT204 'REDCAT204_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT205 'REDCAT205_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT206 'REDCAT206_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT207 'REDCAT207_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT212 'REDCAT212_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT216 'REDCAT216_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT217 'REDCAT217_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT218 'REDCAT218_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT224 'REDCAT224_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT226 'REDCAT226_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT229 'REDCAT229_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT230 'REDCAT230_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT234 'REDCAT234_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT242 'REDCAT242_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT248 'REDCAT248_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT252 'REDCAT252_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT253 'REDCAT253_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT256 'REDCAT256_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT258 'REDCAT258_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT262 'REDCAT262_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT263 'REDCAT263_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT266 'REDCAT266_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT270 'REDCAT270_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT272 'REDCAT272_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT274 'REDCAT274_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT275 'REDCAT275_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT277 'REDCAT277_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT286 'REDCAT286_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT287 'REDCAT287_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT291 'REDCAT291_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT297 'REDCAT297_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT400 'REDCAT400_stats_dm_REML+tlrc[P-C_G#0_Coef]' -setB TD REDCAT107 'REDCAT107_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT108 'REDCAT108_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT109 'REDCAT109_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT110 'REDCAT110_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT111 'REDCAT111_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT115 'REDCAT115_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT116 'REDCAT116_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT117 'REDCAT117_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT119 'REDCAT119_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT125 'REDCAT125_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT132 'REDCAT132_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT133 'REDCAT133_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT145 'REDCAT145_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT148 'REDCAT148_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT152 'REDCAT152_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT157 'REDCAT157_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT167 'REDCAT167_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT168 'REDCAT168_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT170 'REDCAT170_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT173 'REDCAT173_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT174 'REDCAT174_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT175 'REDCAT175_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT177 'REDCAT177_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT182 'REDCAT182_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT191 'REDCAT191_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT193 'REDCAT193_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT195 'REDCAT195_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT196 'REDCAT196_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCAT197 'REDCAT197_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCMNT130 'REDCMNT130_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCMNT137 'REDCMNT137_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCMNT153 'REDCMNT153_stats_dm_REML+tlrc[P-C_G#0_Coef]' REDCMNT166 'REDCMNT166_stats_dm_REML+tlrc[P-C_G#0_Coef]' -mask aim2_resampledaroma-MNIPediatricAsym_cohort-4_res-2_desc-brain_mask.nii -covariates Covariates.txt
++ Two samples: t-testing fixed covariates for similarity between groups
+ meanFD: mean of setA-setB=-0.003208 ; 2-sided p-value=0.8728
+ runs: mean of setA-setB=0 ; 2-sided p-value=1.0000
+ age: mean of setA-setB=0.000605 ; 2-sided p-value=0.9989
+ avgRT: mean of setA-setB=0.012769 ; 2-sided p-value=0.8591
++ loading -setA datasets
*+ WARNING: 3dttest++ -setA :: 244 vectors are constant
++ loading -setB datasets
*+ WARNING: 3dttest++ -setB :: 244 vectors are constant
++ t-testing:0123456789.0123456789.0123456789.0123456789.0123456789.!
+ skipped 244 voxels completely for having constant data
++ ---------- End of analyses -- freeing workspaces ----------
++ Creating FDR curves in output dataset
*+ WARNING: Smallest FDR q [1 AUT-TD_Zscr] = 0.791921 ==> few true single voxel detections
*+ WARNING: Smallest FDR q [3 AUT-TD_meanFD_Zscr] = 0.999878 ==> few true single voxel detections
*+ WARNING: Smallest FDR q [5 AUT-TD_runs_Zscr] = 0.698372 ==> few true single voxel detections
*+ WARNING: Smallest FDR q [7 AUT-TD_age_Zscr] = 0.999879 ==> few true single voxel detections
*+ WARNING: Smallest FDR q [9 AUT-TD_avgRT_Zscr] = 0.118615 ==> few true single voxel detections
++ Smallest FDR q [11 AUT_Zscr] = 1.28362e-06
*+ WARNING: Smallest FDR q [13 AUT_meanFD_Zscr] = 0.239615 ==> few true single voxel detections
*+ WARNING: Smallest FDR q [15 AUT_runs_Zscr] = 0.446913 ==> few true single voxel detections
*+ WARNING: Smallest FDR q [17 AUT_age_Zscr] = 0.999869 ==> few true single voxel detections
*+ WARNING: Smallest FDR q [19 AUT_avgRT_Zscr] = 0.729422 ==> few true single voxel detections
++ Smallest FDR q [21 TD_Zscr] = 6.91128e-06
*+ WARNING: Smallest FDR q [23 TD_meanFD_Zscr] = 0.999894 ==> few true single voxel detections
*+ WARNING: Smallest FDR q [25 TD_runs_Zscr] = 0.888514 ==> few true single voxel detections
*+ WARNING: Smallest FDR q [27 TD_age_Zscr] = 0.999788 ==> few true single voxel detections
*+ WARNING: Smallest FDR q [29 TD_avgRT_Zscr] = 0.849718 ==> few true single voxel detections
+ Added 15 FDR curves to dataset
++ Output dataset ./AUTvTD_3dttest_P-C_G+tlrc.BRIK
++ Output dataset ./AUTvTD_3dttest_P-C_G.resid.nii
+ 2-sample test: results are AUT - TD
++ ================ Starting -Clustsim calculations ================
+ === temporary files will have prefix AUTvTD_3dttest_P-C_G ===
+ === running 72 -randomsign jobs (140 iterations per job) ===
+ === creating 5,236,902,720 (5.2 billion) bytes of pseudo-data in .sdat files ===
+ --- 3dClustSim reads .sdat files to compute cluster-threshold statistics ---
+ --- there is 134,921,015,296 (135 billion) bytes of memory on your system ---
++ 3dttest++: AFNI version=AFNI_21.2.04 (Jul 28 2021) [64-bit]
++ Authored by: Zhark++
++ 259767 voxels in -mask dataset
++ Covariates file: 5 columns, each with 66 rows
+ Covariate column labels: subj;meanFD;runs;age;avgRT
+ Found 4 numeric columns: meanFD runs age avgRT
++ option -setA :: processing as LONG form (label label dset label dset ...)
++ have 33 volumes corresponding to option '-setA'
++ option -setB :: processing as LONG form (label label dset label dset ...)
++ have 33 volumes corresponding to option '-setB'
++ random seeds are 1750462215 2064621480
++ 2-sample test: '-AminusB' option is assumed
++ 2-sample test: results will be rAAA - rBBB
++ loading covariates
** ERROR: -permute failure for unfathomable reasons /:(
nx=33 ny=33 p_nxy=0 x==non-NULL y==non-NULL
Function traceback follows:
permute_arrays
TT_matrix_setup
3dttest++ main
++ Two samples: t-testing fixed covariates for similarity between groups
+ meanFD: mean of setA-setB=-0.003208 ; 2-sided p-value=0.8728
+ runs: mean of setA-setB=0 ; 2-sided p-value=1.0000
+ age: mean of setA-setB=0.000605 ; 2-sided p-value=0.9989
+ avgRT: mean of setA-setB=0.012769 ; 2-sided p-value=0.8591
++ opened file ./AUTvTD_3dttest_P-C_G.0000.sdat for output
++ loading -setA datasets
*+ WARNING: 3dttest++ -setA :: 244 vectors are constant
++ loading -setB datasets
*+ WARNING: 3dttest++ -setB :: 244 vectors are constant
++ t-test randomsign/permute:0123456789.0123456789.0123456789.0123456789.0123456789.0123456789.0123456789.!
++ saving main effect t-stat MIN/MAX values in ./AUTvTD_3dttest_P-C_G.0000.minmax.1D
++ output short-ized file ./AUTvTD_3dttest_P-C_G.0000.sdat
+ 3dttest++ ===== simulation jobs have finished (707.4 s elapsed)
+ 3dttest++ ===== starting 3dClustSim A: elapsed = 730.3 s
++ 3dClustSim: AFNI version=AFNI_21.2.04 (Jul 28 2021) [64-bit]
++ Authored by: RW Cox and BD Ward
++ Loading -insdat datasets
++ -insdat had 10080 volumes = new '-niter' value
++ Startup clock time = 0.1 s
Simulating:0123456789.0123456789.0123456789.0123456789.0123456789.!
++ Clock time now = 2940.2 s
++ Command fragment to put cluster results into a dataset header;
+ (also echoed to file AUTvTD_3dttest_P-C_G.CSimA.cmd for your scripting pleasure)
+ Append the name of the datasets to be patched to this command:
3drefit -atrstring AFNI_CLUSTSIM_NN1_1sided file:AUTvTD_3dttest_P-C_G.CSimA.NN1_1sided.niml -atrstring AFNI_CLUSTSIM_NN2_1sided file:AUTvTD_3dttest_P-C_G.CSimA.NN2_1sided.niml -atrstring AFNI_CLUSTSIM_NN3_1sided file:AUTvTD_3dttest_P-C_G.CSimA.NN3_1sided.niml -atrstring AFNI_CLUSTSIM_NN1_2sided file:AUTvTD_3dttest_P-C_G.CSimA.NN1_2sided.niml -atrstring AFNI_CLUSTSIM_NN2_2sided file:AUTvTD_3dttest_P-C_G.CSimA.NN2_2sided.niml -atrstring AFNI_CLUSTSIM_NN3_2sided file:AUTvTD_3dttest_P-C_G.CSimA.NN3_2sided.niml -atrstring AFNI_CLUSTSIM_NN1_bisided file:AUTvTD_3dttest_P-C_G.CSimA.NN1_bisided.niml -atrstring AFNI_CLUSTSIM_NN2_bisided file:AUTvTD_3dttest_P-C_G.CSimA.NN2_bisided.niml -atrstring AFNI_CLUSTSIM_NN3_bisided file:AUTvTD_3dttest_P-C_G.CSimA.NN3_bisided.niml
++ 3drefit: AFNI version=AFNI_21.2.04 (Jul 28 2021) [64-bit]
++ Authored by: RW Cox
++ Processing AFNI dataset ./AUTvTD_3dttest_P-C_G+tlrc.HEAD
+ atrcopy
++ applying attributes
++ 3drefit processed 1 datasets
++ 3dttest++ ----- Cleaning up intermediate files:
removed ‘AUTvTD_3dttest_P-C_G.resid.nii’
removed ‘./AUTvTD_3dttest_P-C_G.0000.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0001.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0002.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0003.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0004.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0005.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0006.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0007.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0008.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0009.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0010.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0011.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0012.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0013.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0014.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0015.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0016.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0017.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0018.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0019.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0020.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0021.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0022.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0023.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0024.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0025.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0026.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0027.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0028.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0029.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0030.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0031.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0032.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0033.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0034.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0035.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0036.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0037.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0038.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0039.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0040.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0041.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0042.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0043.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0044.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0045.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0046.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0047.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0048.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0049.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0050.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0051.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0052.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0053.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0054.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0055.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0056.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0057.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0058.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0059.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0060.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0061.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0062.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0063.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0064.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0065.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0066.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0067.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0068.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0069.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0070.sdat’
removed ‘./AUTvTD_3dttest_P-C_G.0071.sdat’
removed ‘AUTvTD_3dttest_P-C_G.CSimA.NN1_1sided.niml’
removed ‘AUTvTD_3dttest_P-C_G.CSimA.NN1_2sided.niml’
removed ‘AUTvTD_3dttest_P-C_G.CSimA.NN1_bisided.niml’
removed ‘AUTvTD_3dttest_P-C_G.CSimA.NN2_1sided.niml’
removed ‘AUTvTD_3dttest_P-C_G.CSimA.NN2_2sided.niml’
removed ‘AUTvTD_3dttest_P-C_G.CSimA.NN2_bisided.niml’
removed ‘AUTvTD_3dttest_P-C_G.CSimA.NN3_1sided.niml’
removed ‘AUTvTD_3dttest_P-C_G.CSimA.NN3_2sided.niml’
removed ‘AUTvTD_3dttest_P-C_G.CSimA.NN3_bisided.niml’
+ 3dttest++ =============== -Clustsim work is finished :) ===============
++ ----- 3dttest++ says so long, farewell, and happy trails to you :) -----