Scientific feature testing

Scientific feature testing (SFT) verifies that EMOD features are functioning as expected. That is, they are requirements-based tests of model features that are quantitatively verified. They are implemented as a Python post-processing script named dtk_post_process.py.

All of the files can be found in the Regression directory of the DtkTrunk GitHub repository.

Run on COMPS

If you have access to COMPS, do the following:

  1. Verify that you can log in by running the following command in a command prompt window:

    job submit /scheduler:bayesianhpc.internal.idm.ctr /workdir:\bayesianfil01\idm\home<user>\output\simulations\<etc> /user:internal<user> echo
    
  2. Make a file listing all of the SFTs you want to run, using on of the files in Regression<sim>_science.json as an example.

  3. From the Regression directory in a command prompt window, run the following, adding --scons if you built the Eradication.exe using SCONS:

    python regression_test.py my_sfts.json
    

    Note

    Enter python regression_test.py --help for a list of all arguments you can use with the testing script.

  4. The output of the SFT simulation runs can be found in the location indicated by “Sim root”.

Run locally

If you are running locally without access to COMPS or another HPC, do one the following depending on your operating system:

Run on Windows using regression_test.py

  1. In the Regression directory, edit the regression_test.cfg file to list the directories where you want the simulations to be run and output saved.

  2. Make a file listing all of the SFTs you want to run, using on of the files in Regression<sim>_science.json as an example.

  3. From the Regression directory in a command prompt window, run the following, adding --scons if you built the Eradication.exe using SCONS:

    python regression_test.py my_sfts.json --local
    

    Note

    Enter python regression_test.py --help for a list of all arguments you can use with the testing script.

Run on Windows using run_test.cmd

  1. Generate a single config.json file from a base file and the parameter_overrides.json that contains (see Configuration overlay files for instructions).

  2. Navigate to the directory that contains your config.json file and other test files.

    Note

    Verify in the config.json file that the paths to the demographics and other additional input files are correct.

  3. In a command prompt window, enter the following, substituting your path to the testing script as necessary:

    ../../run_test.cmd
    

Run on CentOS using run_test.sh

  1. Generate a single config.json file from a base file and the parameter_overrides.json that contains (see Configuration overlay files for instructions).

  2. Navigate to the directory that contains your config.json file and other test files.

    Note

    Verify in the config.json file that the paths to the demographics and other additional input files are correct.

  3. In a command prompt window, enter the following, substituting your path to the testing script as necessary:

    ../../run_test.sh