Compressed Dataset .BRIK Files
==============================
AFNI now supports the use of compressed .BRIK files.  The routines
that open and read these files detect the compression mode using
the filename suffix, and will use the correct decompression program
to read them in from disk.  The character 'z' is added to the end
of a dataset's listing in the AFNI menus if the .BRIK is compressed;
for example, "elvis [epan]z".

No other files used by AFNI can be compressed and still be readable
by the software.  This includes the .HEAD files, timeseries (.1D)
files, etc.  Note also that the programs 2swap and 4swap don't
do compression or decompression, so that if you need to do byte
swapping on a compressed .BRIK file, you must manually decompress
it, swap the bytes, and (optionally) recompress the file.

How to Compress
===============
You can compress the .BRIK files manually.  The following 3 programs
are supported:

  Name      Suffix  Compression Command  Uncompress Command
  --------  ------  -------------------  -------------------
  compress   .Z     compress -v *.BRIK   uncompress *.BRIK.Z
  gzip       .gz    gzip -1v *.BRIK      gzip -d *.BRIK.gz
  bzip2      .bz2   bzip2 -1v *.BRIK     bzip2 -d *.BRIK.bz2

  "compress" is available on almost all Unix systems.
  "gzip" is available on many Unix systems, and can also be
     ftp-ed from the AFNI distribution site.
  "bzip2" is included in the AFNI distribution.  It generally
     compresses more than the other two programs, but is much
     slower at both compression and uncompression.  (See the
     file README.bzip2 for details about this program.)

For large MR image datasets, "compress" and "gzip" have about the
same compression factor and take about the same CPU time (at least
in the samples I've tried here.)

Do NOT compress the .HEAD files!  AFNI will not be able to read them.

Automatic Compression
=====================
If you set the environment variable AFNI_COMPRESSOR to one of
the strings "COMPRESS", "GZIP", or "BZIP2", then most programs
will automatically pass .BRIK data through the appropriate
compression program as it is written to disk.  Note that this
will slow down dataset write operations.

Penalties for Using Compression
===============================
Datasets must be uncompressed when they are read into AFNI (or other
programs), which takes time.  In AFNI itself, a dataset .BRIK file
is only read into the program when its values are actually needed
-- when an image or graph window is opened.  When this happens, or
when you "Switch" to a compressed dataset, there can be a noticeable
delay.  For "compress" and "gzip", this may be a few seconds.  For
"bzip2", the delays will generally be longer.

The speed penalty means that it is probably best to keep the
datasets you are actively using in uncompressed form.  This can
be done by compressing datasets manually, and avoiding the use
of AFNI_COMPRESSOR (which will compress all .BRIKs).  Datasets
that you want to keep on disk, but don't think you will use
often, can be compressed.  They can still be viewed when the
need arises without manual decompression.

Large .BRIK files are normally directly mapped to memory.  This
technique saves system swap space, but isn't useful with compressed
files.  Compressed .BRIK files are read into "malloc" allocated
memory, which will take up swap space.  This may limit the number
of datasets that can be used at once.  AFNI will try to purge unused
datasets from memory if a problem arises, but it may not succeed.
If necessary, the "-purge" option can be used when starting AFNI.

Very large datasets (larger than the amount of RAM on your system)
should not be compressed, since it will be impossible to read such
an object into memory in its entirety.  It is better to rely on
the memory mapping facility in such cases.

Effect on Plugins and Other Programs
====================================
If you use the AFNI supplied routines to read in a dataset, then
everything should work well with compressed .BRIK files.  You can
tell if a dataset is compressed after you open it by using the
DSET_COMPRESSED(dset) macro -- it returns 1 if "dset" is compressed,
0 otherwise.

How it Works
============
Using Unix pipes.  Files are opened with COMRESS_fopen_read or
COMPRESS_fopen_write, and closed with COMPRESS_fclose.  The code
is in files thd_compress.[ch], if you want to have fun.  If you
have a better compression utility that can operate as a filter,
let me know and I can easily include it in the AFNI package.

=================================
| Robert W. Cox, PhD            |
| Biophysics Research Institute |
| Medical College of Wisconsin  |
=================================