16.3.3. run_afni_test.py command info¶

run_afni_tests.py is a wrapper script to help run tests for the AFNI suite of tools. This wrapping is an attempt to reduce the burden of executing tests and to facilitate the various usage patterns. Such usage patterns include: running the tests using dependencies installed on the local host; using a container to encapsulate most build/testing dependencies; making use of the cmake build system to make the iterative process of changing code and running tests easier; running the tests while making use of all the cores on the computer; subsetting the tests that are executed during a test run.

usage: run_afni_tests.py [--help] [--installation-help]
                         [--build-dir DIR | --abin DIR]
                         [--debug | --use-all-cores | --trace]
                         [--ignore-dirty-data] [--coverage]
                         [--extra-args PYTEST_ARGS]
                         [--verbosity {quiet,normal,summary,verbose,traceback,diarrhetic}]
                         [--log-file-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
                         [--filter-expr EXPR] [--file FILE] [--lf] [--runslow]
                         [--runveryslow] [--runall] [--create-sample-output]
                         [--diff-with-sample DIFF_WITH_SAMPLE]
                         [--marker-expression EXPR]
                         [--overwrite-args PYTEST_ARGS]
                         {container,local,examples} ...

Named Arguments¶

--help, -help, -h

show this help message and exit (-help is more verbose)

Default: False

--installation-help

Show details regarding the setup for tests execution.

Default: False

--build-dir, -b

This is convenient because it enables testing within the build tree so you don’t have to install the afni suite to test it and you don’t accidentally test the wrong programs due to incorrect PATH etc.

Note, in order to use this option you must have afnipy installed.

--abin

Provide the path to the installation directory of AFNI produced by the make build system.

--debug, -d

Do not catch exceptions and show exception traceback Drop into pdb debugger. As well as passing the –pdb flag to pytest this enables an exception hook for code executed prior to the execution of the pytest command

Default: False

--use-all-cores, -u

Make use of all cpus for tests (requires pytest-parallel).

Default: False

--trace

Immediately drop into the call stack (the pdb debugger) for each test.

Default: False

--ignore-dirty-data

Do not fail if the CI test data repository has uncommitted modifications

Default: False

--coverage

Use codecov for test coverage (needs –build-dir).

Default: False

pytest execution modifiers¶

--extra-args, -e

This should be a quoted string that is passed directly through to pytest. e.g. –extra-args=’-k gui –trace’. Passing –help through to pytest will give you a sense of all the possibilities…

--verbosity, -v

Possible choices: quiet, normal, summary, verbose, traceback, diarrhetic

Increase the verbosity of reporting. Levels other than quiet and normal conflict with –debug.

Default: “normal”

--log-file-level

Possible choices: DEBUG, INFO, WARNING, ERROR, CRITICAL

Set the verbosity of the output recorded in the log file. There is no log file by default.

--filter-expr, -k

Expression for pytest to use to filter tests. Equivalent to passing –extra-args=’-k EXPR’.

--file, -f

Relative path to test module to run. e.g. scripts/test_3dcopy.py

--lf, -l

Only run tests that failed on the last test run.

Default: False

--runslow

Run default tests and tests marked with ‘slow’.

Default: False

--runveryslow

Run default tests and tests marked with ‘slow’ or ‘veryslow’.

Default: False

--runall

Ignore all test markers and run everything.

Default: False

--create-sample-output, -c

Create sample output instead of running a diff with pre- existing output. This is a required step when initially writing a test.

Default: False

--diff-with-sample

Provide a path of pre-existing output that you wish to compare against that is not the default data saved in the datalad repository afni_ci_test_data.

--marker-expression, -m

Provide an expression for filtering tests using markers. This should be quoted if more than one word. It is a boolean expression; ‘A and B’ would run all tests that are marked with the A AND B. If you wished to run both types of tests you would use ‘A or B’. See the pytest help documentation for more information.

Manual pytest management (conflicts with pytest execution modifiers)¶

--overwrite-args: This should be a quoted string that is passed directly through to pytest

Sub Commands¶

One is required

subparser: Possible choices: container, local, examples

Sub-commands¶

container¶

Running tests in an AFNI development container. This requires a docker installation and the python docker package

run_afni_tests.py container [-h]
                            [--source-mode {host,test-code,test-data-only,test-data-volume}]
                            [--image-name IMAGE_NAME]
                            [--container-name CONTAINER_NAME] [--no-rm]
                            [--intermediate] [--only-use-local]
                            [--reuse-build] [--do-not-forward-git-credentials]

Named Arguments¶

--source-mode

Possible choices: host, test-code, test-data-only, test-data-volume

This defines how much of the source code on the host is used when running tests in the container. ‘host’ mounts the whole repo from the local filesystem during the tests. ‘test-code’ only includes the tests sub-directory but is useful as it allows a different version of the test tree to be used from code that is installed in the container. ‘test-data-only’ and ‘test-data-volume, should rarely be needed, but only mount the tests/afni_ci_test_data sub- directory into the container or volume provided by running docker container. Mounting directories into the container has implications for file permissions. If this option is not used, no issues occurs as all files are owned by the user in the container. If ‘host’ is chosen, AFNI’s source code is mounted into the container. The source directory is owned by the host user, so the container user is changed to this id and the build and home directory are appropriately chowned. If ‘test-data-only’ is used, privileged permissions will be required to transfer ownership of the test data back to the host user.

--image-name

Image used for testing container. Default is likely something like afni/afni_cmake_build

--container-name

Name used for testing container. Default is randomly generated.

--no-rm

Do not delete containers after execution of the tests.

Default: False

--intermediate

Search intermediate image layers for the image-name. This can sometimes fail if layers have been pruned etc. but can be used with untagged images.

Default: False

--only-use-local

Raise error if image name does not exist locally.

Default: False

--reuse-build

Use build dir in container (conflicts with –build-dir).

Default: False

--do-not-forward-git-credentials

Prevent git name and email being passed into the container. Instead AFNI CIRCLECI user will be used for any commits.

Default: False

local¶

Running tests on the host requires additional dependencies. See run_afni_tests.py –installation-help

run_afni_tests.py local [-h]

examples¶

Show usage examples

run_afni_tests.py examples [-h] [--verbose]

Named Arguments¶

--verbose, -v

Include a verbose explanation along with the examples

Default: False

16.3.3. run_afni_test.py command info¶

Named Arguments¶

pytest execution modifiers¶

Manual pytest management (conflicts with pytest execution modifiers)¶

Sub Commands¶

Sub-commands¶

container¶

Named Arguments¶

local¶

examples¶

Named Arguments¶

Table of Contents

This Page

Quick search