hi all
We have a study with three types of video clip: no context (NC), low context (LC) and high context (HC). We have rerun an analysis based on criticisms from a (very savvy) reviewer and I want to see whether we are making any errors that would cause our conclusions to be invalid (for the purposes of the study I am happy if the tests are overly conservative but not if we are in any way double-dipping).
We first ran an omnibus F (via 3dANOVA2 -alevels 3 -blevels 21 -type 3) and then did post hoc tests in the regions that survive whole-brain correction. If we then examine average effects in the clusters that result can we determine:
1) that if in a significant cluster (from the ANOVA) the difference in betas averaged in the cluster between HC and LC are significantly different from 0 AND the differences between LC and NC are not significantly different from 0 AND the differences between HC and LC are significantly greater than the differences between LC and NC then it is fair for us to characterize the pattern of response in that cluster as HC > LC = NC? And likewise if the differences between LC and NC are significant then we can characterize the pattern as HC > LC > NC?
2) Imagine now that we show that one region according to the tests above shows HC > LC = NC and another shows HC > LC> NC, and we want to test the interaction. Specifically, we would like to show that not just is LC vs NC nonsignificant in one cluster and significant in another but that this difference is itself significant. Can we use a paired t-test to show that the LC > NC differences are themselves significantly different? Or just run a 2x2 (or 2x3 if there are three regions) ANOVA?
Is anything wrong with what I am describing in 1) and 2)? Also, do you have any other suggestions on how to tackle this?
Basically, I understand that when you select the voxels to test based on a given contrast you bias tests that are related to that contrast (e.g., if you test LC vs NC in a cluster that was identified by the test HC > LC then you are biasing the former test to be nonsignificant). More generally, it seems almost impossible to make any inferences about differences in patterns across ROIs identified by any analysis, although in the case of our ANOVA the significance of a cluster can arise from a significant difference between any of the three conditions.
Thanks in advance for reading a rather long message!
James
