Addendum: what is happening is not quite what I reported. What seems to be happening is that because I'm using abids_tool, if there is a read error from the filesystem it consequently writes out a corrupted file in the same spot (the error does not cause the program to terminate, so it continues on by writing the existing, corrupted, data back to disk in the same filename). Using pigz does seem to decrease the chances of this happening quite dramatically but it appears to be a problem nonetheless on biowulf, which I imagine is tied to network drives because I simply cannot replicate the issue using local storage.
