AFNI Message Board

Dear AFNI users-

We are very pleased to announce that the new AFNI Message Board framework is up! Please join us at:

https://discuss.afni.nimh.nih.gov

Existing user accounts have been migrated, so returning users can login by requesting a password reset. New users can create accounts, as well, through a standard account creation process. Please note that these setup emails might initially go to spam folders (esp. for NIH users!), so please check those locations in the beginning.

The current Message Board discussion threads have been migrated to the new framework. The current Message Board will remain visible, but read-only, for a little while.

Sincerely, AFNI HQ

History of AFNI updates  

|
June 14, 2010 01:48PM
> if it is possible (or even advisable) to do a 2 group x 2 time points ANOVA with
> uneven sample sizes for the two groups.

Almost everybody was (and still is) taught in school that analysis of a linear system with categorical variables (factors) would be typically handled through analysis of variance (ANOVA). If an experiment falls nicely into a balanced design and if all the relevant assumptions are reasonable met, a factorial ANOVA seems nice because everything is wrapped into one setting and you can get all the post-hoc tests in the end. However, thanks to the popularity of such an approach, people tend to forget about a couple of issues involved in ANOVA:

(1) Although sophisticated ANOVA programs do have a lot of flexible room dealing with covariance structure, the straightforward ANOVA methods such as 3dANOVA* assume compound symmetry (e.g., equal correlation across levels of a factor), or homoscedasticity (equal variance across the levels of a between-subjects factor such as group) for F-tests (and for t-tests as well in the older version of 3dANOVA*), and some popular covariance structures such as serial correlation are not allowed.

(2) It might possible to deal with missing data or unequal number of subjects across groups under the ANOVA framework, but the approaches are usually pretty awkward, leading to compromised assumptions.

(3) It's quite awkward to consider covariates (continuous variables such as age, behavioral data, etc.) under the ANOVA framework.

Nowadays ANOVA can be thought of as a special case under linear mixed-effects model. However we don't even have to go that far into the linear mixed-effect modeling territory since most of the time we just need to change our perspective. Instead of focusing on the experiment design in the perspective of the ANOVA mode with factors, levels, etc., we could simply list all the tests we want to get out of the FMRI group analysis, and then run each test separately. This new perspective usually does not require us to consider factors, levels, balancedness, fancy assumptions (homoscedasticity, compound symmetry), etc., and it gives us more flexibility in terms of analysis strategy.

Suppose we have a categorical variable (factor) with three levels: A1, A2, and A3. If one subject only performed A1 and A2, and we don't have any data of A3 from this subject. It's very difficult to handle this scenario in the ANOVA framework. While a linear mixed-effects model sounds very appealing, there are some subtle issues regarding the testing statistics, especially there is some thorny point about the degrees of freedom involved in the testing statistics. However, if pair-wise comparisons or linear combinations among the three levels are of our interest, the method will be simple and straightforward: we simply run one paired or one-sample t-test a time (dropping the subject if A3 is involved since that subject does not provide any information for that specific test), easily avoiding the sticky issue of missing data. For example, for the contrast of A2 - A3, intuitively the subject with no data on A3 would not provide any useful information; and adding this subject in a model would lead to some assumptions that are hard to validate most of the time, unnecessarily complicating the modeling strategy.

Using your case as another example: there are two categorical variables, one between-subject factor (group with 2 levels), and one within-subject variable (time point with 2 levels). First of all, think about all the tests you want to run, and then for each t-test (one degree of freedom test), I would go with 3dMEMA or 3dttest.

I do realize that people would like to have F-tests for main effects and interactions as well, and such an obsession comes from the traditional textbook. I admit that an F-test provides a neat summary about the overall effect of a factor or interaction among factors. However, such a summarized result comes with cost:

(1) Sophisticated assumptions are usually involved as discussed above, and sometimes we're not really sure to what extent they are reasonable.

(2) The information from an F-test is pretty limited and actually more or less redundant. For instance, a significant main effect does not tell us where the difference is across the levels of a factor with more than two levels. Similarly a significant interaction between two factor does not reveal where and in what direction exactly such an interaction occurs. In the end we will still have to run individual t-tests to sort out the specifics and directionality. So honestly I don't really care about F-tests: why do I want those F-tests since I'll have to resort to those individual t-tests in the end anyway? More subtly there are some discrepancies in terms of significance between an F-test and the corresponding t-tests. In other words, it's not rare to see that people are puzzled by a situation where an F-test for an interaction or main effect is significant (or not significant) while the corresponding individual t-tests tell the opposite story.

Back to your situation with one between-subject variable (group with 2 levels), and one within-subject variable (time point with 2 levels), the F-tests for the two main effects can essentially be replaced with two t-tests, and are not as informative as the t-tests since the F-tests lose the directionality information. The F-test for the interaction has the same story: It basically tests against the same hypothesis as the t-test for (A1B1-A1B2)-(A2B1-A2B2) or (A1B1-A2B1)-(A1B2-A2B2), which can be obtained with 3dMEMA or 3dtest, but we can say more with the t-test than F: for example, a positive t-value shows A1B1-A1B2 > A2B1-A2B2 and A1B1-A2B1 > A1B2-A2B2.

Having said this, I'd like to point out there are two old programs in AFNI for your case along the line of batch mode fashion if you haven't been totally convinced by my arguments above: 3dLME (http://afni.nimh.nih.gov/sscc/gangc/lme.html) or GroupAna (http://afni.nimh.nih.gov/sscc/gangc/Unbalanced.html/).

Gang
Subject Author Posted

3dANOVA3 with uneven sample sizes

philippe goldin June 11, 2010 07:07PM

Re: 3dANOVA3 with uneven sample sizes

Gang Chen June 14, 2010 01:48PM

Re: 3dANOVA3 with uneven sample sizes

bob cox June 14, 2010 01:57PM

Re: 3dANOVA3 with uneven sample sizes

philippe goldin June 14, 2010 06:49PM

Re: 3dANOVA3 with uneven sample sizes

Matt Weber June 22, 2011 03:27PM

Re: 3dANOVA3 with uneven sample sizes

Gang Chen June 22, 2011 04:34PM

Re: 3dANOVA3 with uneven sample sizes

Sean Drummond July 20, 2010 12:03PM

Re: 3dANOVA3 with uneven sample sizes

Gang Chen July 20, 2010 10:09PM

Re: 3dANOVA3 with uneven sample sizes

Matt Weber September 14, 2010 05:09PM

Re: 3dANOVA3 with uneven sample sizes

Gang Chen September 14, 2010 06:39PM

Re: 3dANOVA3 with uneven sample sizes

Matt Weber September 15, 2010 11:02AM

Re: 3dANOVA3 with uneven sample sizes

Gang Chen September 15, 2010 03:52PM

Re: 3dANOVA3 with uneven sample sizes

Matt Weber September 15, 2010 04:29PM

Re: 3dANOVA3 with uneven sample sizes

yangjiepsy August 07, 2015 01:19AM

Re: 3dANOVA3 with uneven sample sizes

Christopher Bell September 22, 2010 06:44PM

Re: 3dANOVA3 with uneven sample sizes

Gang Chen September 23, 2010 11:44AM

Re: 3dANOVA3 with uneven sample sizes

Christopher Bell September 23, 2010 11:58AM

Re: 3dANOVA3 with uneven sample sizes

Jodi May 05, 2011 04:27PM

Re: 3dANOVA3 with uneven sample sizes

Gang Chen May 06, 2011 09:49AM

A question about 3dANOVA3 with uneven sample sizes

yangjiepsy August 07, 2015 10:15AM

Re: A question about 3dANOVA3 with uneven sample sizes

gang August 07, 2015 10:28AM

Re: A question about 3dANOVA3 with uneven sample sizes

yangjiepsy August 07, 2015 09:18PM