I'm still seeing the -check option for 3dAllineate, so you might be running a very old version by mistake. Other than that, you can have 3dAllineate print out all available cost functionals with -allcost during the alignment or call it separately with -allcostX to evaluate the cost functionals, display the values and exit. If you go that route, you will have to develop some heuristics on your own for what may be reasonable values for your data.

You could check for outrageous misalignment by looking at overlap of simple masks from your original volumes and then 3dcalc or 3dOverlap.

If you look at our NeuroImage paper, there is an alternative way to look at cost functions in Figure 7 that looks at each cost in a range of rotations and translations. The script that generated the figures is not in our Matlab distribution but could be made available if you would like it.

[dx.doi.org]