Okay, that explains where the difference comes from. I can work around it for now.
However, I don't think it's very helpful of to3d to silently ignore the -xFOV options. If we assume the user knows what they're doing by overriding the geometry to3d reads from the DICOM header, then perhaps to3d should just go ahead and skip the NIFTI-related obliquity support to do what the user wants. That would be my preference. At a minimum, a warning message would be appreciated (then to3d can loudly ignore me).