runTestDir {tinytest2JUnit}

Run all the test files in a directory

Description

runTestDir() is a drop in replacement for tinytest::run_test_dir() with the key difference that errors thrown from within a test file are caught and get reported with a a stacktrace in the JUnit report. In addition, some extra metrics are recored for the JUnit report, such as: timestamp, test duration, hostname and if tests are disabled (see details for more info).

Usage

runTestDir(
  dir = "inst/tinytest",
  at_home = FALSE,
  pattern = "^test.*\\.[rR]$",
  cluster = NULL,
  lc_collate = getOption("tt.collate", NA),
  ...
)

Arguments

dir

character(1) path to directory

at_home

logical(1) should local tests be run? By default FALSE. Unlike tinytest::run_test_dir() which is meant to be called in a local interactive context. This function is meant to be called in a non-interactive CI environment, where we want to mimic the behaviour of how tests would get run by R CMD Check. See also at_home documentation in tinytest package.

pattern

character(1) A regular expression that is used to find scripts in dir containing tests (by default .R or .r files starting with test).

cluster

A cluster object to run the test files on. Note, it is expected that the clusters has already been prepared. Most notable, the package to test should already been loaded. runTestDir will load the package "tinytest" for you into the clusters. See tinytest::run_test_dir() for more details.

lc_collate

See tinytest::run_test_dir().

...

Arguments passed on to tinytest::run_test_file()

Details

runTestDir() is meant as a CI-friendly alternative to the native tinytest::run_test_dir(). It catches errors that are raised in the tests files and adds them as a "failed" tinytest in the output.

tinytest::run_test_dir() would have let the error bubble up, stop the testing process and not report any failures from other tests. One is then also forced to look into the logs of the CI to see what the error was. The output of runTestDir() in combination with writeJUnit() will present you the error in the JUnit togheter with a stack trace. Next to the test results of the other files that ran without a problem.

If you prefer the behaviour of tinytest::run_test_dir() you can still use it in combination writeJUnit().

Caught errors are returned in the output as as sub-class of tinytest object. This is however considred implemenation detail and can be subject to change.

Note, function arguments explicilty listed in tinytest::run_test_dir() but not here can still still be provided via ...

Value

A tinytests2Junit object to be provided to the writeJUnit() function.

tinytests2JUnit

The returned object is a tinytests2JUnit object (note the plural). This object contains additional info compared to a tinytests object that is used in the JUnit report.

The following additional info will get reported:

• The timestamp per test file on when it got invoked.

• The test duration per test file.

• The system hostname per test file on where it got invoked. This is mainly of interests for different clusters.

• If a test file is disabled. A test file is considered disabled if no tests occur with in the file. It is then assumed that at the top of file some conditional statement made the test file exist early.

See Also

• tinytest::run_test_dir() for how the function is inteded to behave.

• writeJUnit() where it is expected that the output of this function to be provided to.

• testPackage() for an higher-level function to simply test a package.

Examples

# Run tests with `tinytest`
dirWithTests <- system.file("example_tests/multiple_files",package = "tinytest2JUnit")
testresults <- runTestDir(dirWithTests)

writeJUnit(testresults) # Writes content to stdout