> there is between-subject variability that is confounded by the number of
> responses that differ for each subject.

Yes, that is true, but the effect is pretty much on the sensitivity (or power of activation detection) at individual subject level. In other words, such a between-subject variability would lead to different variance estimate for those corresponding conditions across subjects, leading to different t or F values even with exactly the same size effect.

However there is not much we can do about it for two reasons: (1) Currently we don't bring intra-subject variations to group analysis unless mixed-effect modeling is available; (2) The beta estimates are unbiased regardless the sample size; that is, the mean of each beta can be shown to be equal to the true beta being estimated no matter what the sample size is.

Gang