I resampled the dataset so that it had 1mm resolution, and then re-did the 3dcalc, and now it works as I expected. I guess I could have done that before posting my question, but there's another question in there which is still open, so I'll justify the post with that.