One reason I haven't answered this is that I have been working hard on another approach for cluster-thresholding group statistical maps. In this new approach, smoothness estimation would no longer be required. The reason for this effort is that the false positive rate for the ACF smoothness+cluster simulation method is still significantly too liberal in some cases. I have just today (Saturday 06 Feb 2016) finished the first test of my new method, and it was never too liberal in all the test cases (but was too conservative in a few cases). I'm next going to work on packaging this method for more general use (that is, to make it usable by people who aren't ME).

For your first result (-acf gives bigger cluster size thresholds), I'm not surprised at all.

For your second results (-acf gives smaller cluster size thresholds in a restricted mask), I'm surprised and don't have an explanation off the top of my head. However, since I'm hoping to retire the whole use of 3dFWHMx soon, I'm not highly motivated to investigate this issue.