3dcalc's max is nothing like 3dBrickStat's -max!
3dcalc is a voxel-wise calculator -- it knows nothing about values that span a whole volume (that's 3dBrickStat's forte). So 3dcalc's max(a,b) takes as input 2 numbers from a voxel and outputs the larger of the twain.
If you want to input a constant from outside, in bash you could do something like
m=`3dBrickStat -max fred+orig`
3dcalc -a elvis+orig -expr "sqrt(a/$m)" -datum float -prefix elmer
Note the use of the backwards single quotes to capture the output of 3dBrickStat to the command line and thus to assign it to the bash variable 'm'. Then notice the use of the double quotes to contain the expression "sqrt(a/$m)", where the $m in the expression means to substitute the value of the shell variable. You must use double quote characters for this -- if you use forward single quote characters, the $m will not be interpreted and then the calc parser will be unhappy with you.