svUnit - A framework for unit testing in R

Introduction

Unit testing (see https://en.wikipedia.org/wiki/Unit_test) is an approach successfully used to develop software, and to ease code refactoring for keeping bugs to the minimum. It is also the insurance that the software is doing the right calculation (quality insurance). Basically, a test just checks if the code is running and is producing the correct answer/behavior in a given situation. As such, unit tests are build in R package production because all examples in documentation files, and perhaps, test code in /tests subdirectory are run during the checking of a package (R CMD check <pkg>). However, the R approach lacks a certain number of features to allow optimal use of unit tests as in extreme programming (test first – code second):

• Tests are related to package compilation and cannot easily be run independently (for instance, for functions developed separately).
• Once a test fails, the checking process is interrupted. Thus one has to correct the bug and launch package checking again… and perhaps get caught by the next bug. It is a long and painful process.
• There is no way to choose one or several tests selectively: all are run or not (depending on command line options) during package checking.
• It is very hard, or near to impossible to program in R in a test driven development (write tests first) with the standard tools (https://en.wikipedia.org/wiki/Test-driven_development).
• Consequently, the ‘test-code-simplify’ cycle is not easily accessible yet to R programmer, because of the lack of an interactive and flexible testing mechanism providing immediate, or quasi immediate feedback about changes made.
• We would like also to emphasize that test suites are not only useful to check code, they can also be used to check data, or the pertinence of analyses.

Unit testing in R without ‘svUnit’

Besides the “regular” testing mechanism of R packages, one can find the ‘testthat’ and ‘RUnit’ packages on CRAN (https://CRAN.R-project.org/package=testthat and https://CRAN.R-project.org/package=RUnit, respectively)1. Another package used to provide an alternate implementation of test unit: ‘butler’, but it is archived because it is not maintained any more and has given up in favor of ‘RUnit’.

The ‘testthat’ package is heavily used by the UseR community (there are many more packages on CRAN that use it than any other testing system). It implements R testing in a rather original way. ‘RUnit’, on the other hand sticks with the framework initially defined by junit for Java and largely developed for many different languages. In this document, we will not discuss further about ‘testthat’. We will focus on the junit-like framework implemented in ‘RUnit’ and ‘svUnit’. ‘RUnit’ implements the following features:

• Assertions, checkEquals(), checkEqualsNumeric(), checkIdentical() and checkTrue() and negative tests (tests that check error conditions, checkException()).
• Assertions are grouped into R functions to form one test function that runs a series of related individual tests. It is easy to temporarily inactivate one or more tests by commenting lines in the function. To avoid forgetting tests that are commented out later on, there is special function, named DEACTIVATED() that tags the test with a reminder for your deactivated items (i.e., the reminder is written in the test log).
• A series of test functions (whose name typically start with test....) are collected together in a source-able R code file (name starting with runit....) on disk. This file is called a test unit.
• A test suite (object RUnitTestSuite) is a special object defining a battery of tests. It points to one or several directories containing test units. A test suite is defined by defineTestSuite().
• One or more test suites can be run by calling runTestSuite(). There is a shortcut to define and run a test suite constituted by only one test unit by using the function runTestFile(). Once the test is run, a RUnitTestData object is created that contains all the information collected from the various tests run.
• One can print a synthetic report (how many test units, test functions, number of errors, fails and deactivated item), or get a more extensive summary() of the tests with indication about which ones failed or produced errors. The function printTextProtocol() does the same, while printHTMLProtocol() produces a report in HTML format.
• ‘RUnit’ contains also functions to determine which code is run in the original function when tested, in order to detect the parts of the code not covered by the test suite (code coverage function inspect() and function tracker()).

As complete and nice as ‘RUnit’ is, there is no tools to integrate the test suite in a given development environment (IDE) or graphical user interface (GUI), as far as we know. In particular, there is no real-time reporting mechanism used to easy the test-code-simplify cycle. The way tests are implemented and run is left to the user, but the implementation suggests that the authors of ‘RUnit’ mainly target batch execution of the tests (for instance, nightly check of code in a server), rather that real-time interaction with the tests.

There is also no integration with the “regular” R CMD check mechanism of R in ‘RUnit’.

Unit testing framework for R with ‘svUnit’

Our initial goal was to implement a GUI layer on top of ‘RUnit’, and to integrate test units as smoothly as possible in a code editor, as well as, making tests easily accessible and fully compatible with R CMD check on all platforms supported by R. Ultimately, the test suite should be easy to create, to use interactively, and should be able to test functions in a complex set of R packages.

However, we encountered several difficulties while trying to enhance ‘RUnit’ mechanism. When we started to work on this project, ‘RUnit’ (version 0.4-17) did not allow to subclass its objects (version 0.4-19 solved this problem). Moreover, its RUnitTestData object is optimized for quick testing, but not at all for easy reviewing of its content: it is a list of lists of lists,… requiring embedded for loop and lapply() / sapply() procedures to extract some content. Moreover, ‘RUnit’ is implemented as S4 classes, requiring the heavy ‘methods’ package to be loaded, making it impossible to use in a lightweight environment without it, or to test R without the ‘methods’ package loaded. Finally, the concept of test units as source-able files on disk is a nice idea, but it is too rigid for quick writing test cases for objects not associated (yet) with an R packages.

We did a first implementation of the ‘RUnit’ GUI based on these objects, before realizing that it is really not designed for such an use. So, we decide to write a completely different unit testing framework in R: ‘svUnit’, but we make it test code compatible with ‘RUnit’ (i.e., the engine and objects used are totally different, but the test code run in ‘RUnit’ or ‘svUnit’ is interchangeable).

Finally, ‘svUnit’ is also designed to be integrated in the SciViews GUI on top of Komodo IDE (https://www.activestate.com/products/komodo-ide/), and to approach extreme programming practices with automatic code testing while you write it. A rather simple interface is provided to link and pilot ‘svUnit’ from any GUI/IDE, and the Komodo IDE implementation could be used as an example to program similar integration panels for other R GUIs. ‘svUnit’ also formats its report with creole wiki syntax. It is directly readable, but it can also be displayed in a much nicer way using any wiki engine compatible with the creole wiki language. It is thus rather easy to write test reports in wiki servers, possibly through nightly automatic process for your code, if you like.

This vignette is a guided tour of ‘svUnit’, showing its features and the various ways you can use it to test your R code.

Installation

The ‘svUnit’ package is available on CRAN (https://CRAN.R-project.org/package=svUnit), and its latest development version is also available on Github (https://github.com/SciViews/svUnit). You can install it with2:

install.packages("svUnit")

This package has no required dependencies other than R >= 1.9.0 and ‘utils’ (‘XML’ is also used for some export formats, but it is not required to run the tests). However, if you would like to use its interactive mode in a GUI editor, you must also install Komodo Edit or Komodo IDE and SciViews-K, or use RStudio. Its ‘pkgdown’ web site is at https://www.sciviews.org/svUnit/.

Once the ‘svUnit’ package is installed, you can check it is working correctly on your machine with the following example code:

library(svUnit)
Square <- function(x) return(x^2)
test(Square) <- function() {
checkEqualsNumeric(9, Square(3))
checkEqualsNumeric(10, Square(3))   # This intentionally fails
checkEqualsNumeric(9, SSSquare(3))  # This raises error
checkEqualsNumeric(c(1, 4, 9), Square(1:3))
checkException(Square("xx"))
}
clearLog()
(runTest(Square))
#> * : checkEqualsNumeric(10, Square(3)) run in 0.002 sec ... **FAILS**
#> Mean relative difference: 0.1
#>  num 9
#> * : checkEqualsNumeric(9, SSSquare(3)) run in less than 0.001 sec ... **ERROR**
#> Error in SSSquare(3) : could not find function "SSSquare"
#>
#> == test(Square) run in less than 0.1 sec: **ERROR**
#>
#> //Pass: 3 Fail: 1 Errors: 1//
#>
#> * : checkEqualsNumeric(10, Square(3)) run in 0.002 sec ... **FAILS**
#> Mean relative difference: 0.1
#>  num 9
#>
#> * : checkEqualsNumeric(9, SSSquare(3)) run in less than 0.001 sec ... **ERROR**
#> Error in SSSquare(3) : could not find function "SSSquare"

Although test unit code is compatible with both ‘svUnit’ and ‘RUnit’, do not load both packages in R memory at the same time, or you will badly mix incompatible code!

Overview of ‘svUnit’

You ensure that code you write in your R functions does the expected work by defining a battery of tests that will compare the output of your code with reference values. In ‘svUnit’, the simplest way to define such a battery of tests is by attaching it to functions loaded in R memory3. Of course, you can also define batteries of tests that are independent of any R object, or that check several of them together (so called, integration tests). Here is a couple of examples:

library(svUnit)
# Create two R functions that include their own test cases
Square <- function(x) return(x^2)
test(Square) <- function() {
checkEqualsNumeric(9, Square(3))
checkEqualsNumeric(c(4, 9), Square(2:3))
checkException(Square("xx"))
}

Cube <- function(x) return(x^3)
test(Cube) <- function() {
checkEqualsNumeric(27, Cube(3))
checkEqualsNumeric(c(8, 28), Cube(2:3))
checkException(Cube("xx"))
}

# Add a separate test case
test_Integrate <- svTest(function() {
checkTrue(1 < 2, "check1")
v <- c(1, 2, 3)  # The reference
w <- 1:3         # The value to compare to the reference
checkEquals(v, w)
})

When you run a test in ‘svUnit’, it logs its results in a centralized logger. The idea is to get a central repository for tests that you can manipulate as you like (print, summarize, convert, search, display in a GUI, etc.). If you want to start new tests, you should first clean this logger by clearLog(). At any time, the logger is accessible by Log(), and a summary of its content is displayed using summary(Log()). So, to run test for your Square() function as well as your test_Integrate integration test, you simply do the following:

clearLog()
runTest(Square)
runTest(test_Integrate)
Log()
#> = A svUnit test suite run in less than 0.1 sec with:
#>
#> * test_Integrate ... OK
#> * test(Square) ... OK
#>
#>
#> == test_Integrate run in less than 0.1 sec: OK
#>
#> //Pass: 2 Fail: 0 Errors: 0//
#>
#>
#> == test(Square) run in less than 0.1 sec: OK
#>
#> //Pass: 3 Fail: 0 Errors: 0//

In this report, you see that all your tests succeed. Note that ‘svUnit’ is making the distinction between a test that fails (the code is run correctly, but the result is different from what was expected) and code that raises error (it was not possible to run the test because its code is incorrect, or for some other reasons). Note also that the function checkException() is designed to explicitly test code that should stop() in R4. So, if that test does not raises an exception, it is considered to have failed. This is useful to check that your functions correctly trap wrong arguments, for instance, like in checkException(Square("xx")) here above (a character string is provided where a numerical value is expected).

Now, let’s look what happens if we test the Cube() function without clearing the logger:

runTest(Cube)
#> * : checkEqualsNumeric(c(8, 28), Cube(2:3)) run in less than 0.001 sec ... **FAILS**
#> Mean relative difference: 0.03571429
#>  num [1:2] 8 27
Log()
#> = A svUnit test suite run in less than 0.1 sec with:
#>
#> * test(Cube) ... **FAILS**
#> * test_Integrate ... OK
#> * test(Square) ... OK
#>
#>
#> == test(Cube) run in less than 0.1 sec: **FAILS**
#>
#> //Pass: 2 Fail: 1 Errors: 0//
#>
#> * : checkEqualsNumeric(c(8, 28), Cube(2:3)) run in less than 0.001 sec ... **FAILS**
#> Mean relative difference: 0.03571429
#>  num [1:2] 8 27
#>
#> == test_Integrate run in less than 0.1 sec: OK
#>
#> //Pass: 2 Fail: 0 Errors: 0//
#>
#>
#> == test(Square) run in less than 0.1 sec: OK
#>
#> //Pass: 3 Fail: 0 Errors: 0//

We note this:

1. When a test succeeds, nothing is printed by default (result is returned invisibly). But when a test fails, or raises errors, the guilty test(s) results are printed. We expected c(8, 28) (made intentionally wrong for the sake of the demonstration) from Cube(2:3) in checkEqualsNumeric(c(8, 28), Cube(2:3)), and (of course), we got c(8, 27). This test shows how ‘svUnit’ presents test failures.
2. The results of the tests on Cube() are added to the previous report. So, it is possible to build rather easily reports that summarize tests on several objects, by adding test results in the logger sequentially. ‘svUnit’ does this naturally and transparently. Starting a new report is equally simple: just use clearLog()

Assertions in ‘svUnit’

The most basic item in a test suite is an assertion represented by a checkXXX() function in ‘svUnit’/‘RUnit’. Five such functions are currently defined:

• checkEquals(current, target) determines if data in target is the same as data in current.
• checkEqualsNumeric(current, target) does the same but allows for a better comparison for numbers (variation allowed within a tolerance window).
• checkIdentical(current, target) checks whether two R objects are strictly identical.
• checkTrue(expr) only succeed if expr is TRUE. Note a difference in ‘svUnit’ and ‘RUnit’ (at least, in its version 0.4-17): the ‘RUnit’ function is not vectorized and expr must return a single atomic logical value. The corresponding ‘svUnit’ function also accepts a vector of logical values. In this case, all elements of the vector must be TRUE for the test to succeed. When you make sure that expr always returns a single logical value (for instance by using all(expr)), both functions should be compatible.
• checkException(expr) verifies that a given code raises an exception (in R, it means that a line of code with stop() is executed).
• DEACTIVATED() makes sure that all tests following this instruction (in a test function, see next paragraph) are deactivated, and inserts a reminder in the logger about the fact that some tests are deactivated in this suite.

For all these functions, you have an additional optional argument msg = where you can provide a (short) message to print in front of each text in the report. These functions return invisibly: TRUE if the test succeeds, or FALSE if it fails (code is executed correctly, but does not pass the test), and NA if there was an error (the R code of the test was not executed correctly). Moreover, these functions record the results, the context of the test and the timing in a logger (object svSuiteData inheriting from environment) called .Log and located in the user’s workspace. So, executing a series of assertions and getting a report is simply done as (in its simplest form, you can use the various checkXXX() functions directly at the command line):

clearLog()
checkEqualsNumeric(1, log(exp(1)))
checkException(log("a"))
checkTrue(1 == 2)
#> * : checkTrue(1 == 2) run in less than 0.001 sec ... **FAILS**
#>  logi FALSE
Log()
#> = A svUnit test suite run in less than 0.1 sec with:
#>
#> * eval ... **FAILS**
#>
#>
#> == eval run in less than 0.1 sec: **FAILS**
#>
#> //Pass: 2 Fail: 1 Errors: 0//
#>
#> * : checkTrue(1 == 2) run in less than 0.001 sec ... **FAILS**
#>  logi FALSE

As you can see, the checkXXX() functions work hand in hand with the test logger (the checkXXX() functions also return the result of the test invisibly, so, you can also assign it to a variable if you like). These function are mainly used for their side-effect of adding an entry to the logger.

The last command Log() prints the content of the logger. You see how a report is printed, with a first part being a short summary by categories (assertions run at the command line are placed automatically in the eval category: there is no better context known for them. Usually, those assertions should be placed in test functions, or in test units, as we will see later in this manual, and the category will reflect this organization). A detailed report on the tests that failed or raised an error is also printed at the end of the report.

Of course, the same report is much easier to manipulate from within the graphical tree in the Komodo’s R Unit tab, but the simple text report in R has the advantage of being independent from any GUI, and from Komodo. It can also be generated in batch mode. Last, but not least, it uses a general Wiki formatting called creole wiki (http://www.wikicreole.org/wiki/Creole1.0).

The figure illustrates the way the same report looks like in DokuWiki with the creole plugin (http://www.wikicreole.org/wiki/DokuWiki) installed. Note the convenient table of content that lists here a clickable list of all tests run. From this point, it is relatively easy to define nightly cron task jobs on a server to run a script that executes these tests and update a wiki page (look at your particular wiki engine documentation to determine how you can access wiki pages on the command line).

Manipulating the logger data

The ‘svUnit’ package provides a series of functions to manipulate the logger from the command line, in particular, stats(),summary(), metadata() and ls():

# Clear test exclusion list for running all test suites
options(svUnit.excludeList = NULL)
# Clear the logger
clearLog()
# Run all currently defined tests
runTest(svSuiteList(), name = "AllTests")
#> * : checkEqualsNumeric(c(8, 28), Cube(2:3)) run in less than 0.001 sec ... **FAILS**
#> Mean relative difference: 0.03571429
#>  num [1:2] 8 27
#> A svUnit test suite definition with:
#>
#> - Test function:
#> [1] "test_svSuite"
#> *
#>   runTest(bar) does not work inside test functions:  ... DEACTIVATED
#>
#> * Check a double is equal to a named double: checkEquals(c(x = 2), 2) run in less than 0.001 sec ... **FAILS**
#> names for target but not for current
#>  num 2
#> * Check if two different numbers are equal: checkEqualsNumeric(2, 3) run in less than 0.001 sec ... **FAILS**
#> Mean relative difference: 0.5
#>  num 3
#> * Check a double is exactly the expected value: checkIdentical(2, sqrt(2)^2) run in less than 0.001 sec ... **FAILS**
#>  num 2
#> * Check if FALSE is TRUE: checkTrue(FALSE) run in less than 0.001 sec ... **FAILS**
#>  logi FALSE
#> * log(10) produces and exception: checkException(log(10)) run in less than 0.001 sec ... **FAILS**
#> No exception generated!
#> * Wrong expression in checkEquals(): checkEquals(fff(2), 2) run in less than 0.001 sec ... **ERROR**
#> Error in fff(2) : could not find function "fff"
#> * Wrong expression in checkEqualsNumeric(): checkEqualsNumeric(fff(2), 2) run in less than 0.001 sec ... **ERROR**
#> Error in fff(2) : could not find function "fff"
#> * Wrong expression in checkIdentical(): checkIdentical(2, fff(2)) run in less than 0.001 sec ... **ERROR**
#> Error in fff(2) : could not find function "fff"
#> * Wrong expression in checkTrue(): checkTrue(fff()) run in less than 0.001 sec ... **ERROR**
#> Error in fff() : could not find function "fff"
# Get some statistics
stats(Log())[, 1:3]
#>                                    kind timing                time
#> testBadTests                  **ERROR**  0.000 2021-04-18 21:35:27
#> test_Integrate                       OK  0.000 2021-04-18 21:35:27
#> testis.test                          OK  0.001 2021-04-18 21:35:27
#> test_R                               OK  0.000 2021-04-18 21:35:27
#> testMyVirtualBaseClass.getX          OK  0.000 2021-04-18 21:35:27
#> testsvSuite                          OK  0.001 2021-04-18 21:35:27
#> testCube                      **FAILS**  0.000 2021-04-18 21:35:27
#> testbar                              OK  0.021 2021-04-18 21:35:27
#> testrunTest                 DEACTIVATED  0.001 2021-04-18 21:35:27
#> testmat                              OK  0.000 2021-04-18 21:35:27
#> testCreateClass                      OK  0.001 2021-04-18 21:35:27
#> testMyVirtualBaseClass.setX          OK  0.002 2021-04-18 21:35:27
#> testSquare                           OK  0.000 2021-04-18 21:35:27
#> testsvTest                           OK  0.002 2021-04-18 21:35:27
#> testsvSuiteList                      OK  0.004 2021-04-18 21:35:27
# A slightly different presentation than with print
summary(Log())
#> = A svUnit test suite run in less than 0.1 sec with:
#>
#> * test_Integrate ... OK
#> * testis.test ... OK
#> * test_R ... OK
#> * testMyVirtualBaseClass.getX ... OK
#> * testsvSuite ... OK
#> * testCube ... **FAILS**
#> * testbar ... OK
#> * testrunTest ... DEACTIVATED
#> * testmat ... OK
#> * testCreateClass ... OK
#> * testMyVirtualBaseClass.setX ... OK
#> * testSquare ... OK
#> * testsvTest ... OK
#> * testsvSuiteList ... OK
#>
#>
#> == testBadTests (in runitBadTests.R) run in less than 0.1 sec: **ERROR**
#>
#> //Pass: 0 Fail: 5 Errors: 4//
#>
#> === Failures
#> [1] Check a double is equal to a named double: checkEquals(c(x = 2), 2)
#> [2] Check if two different numbers are equal: checkEqualsNumeric(2, 3)
#> [3] Check a double is exactly the expected value: checkIdentical(2, sqrt(2)^2)
#> [4] Check if FALSE is TRUE: checkTrue(FALSE)
#> [5] log(10) produces and exception: checkException(log(10))
#>
#> === Errors
#> [6] Wrong expression in checkEquals(): checkEquals(fff(2), 2)
#> [7] Wrong expression in checkEqualsNumeric(): checkEqualsNumeric(fff(2), 2)
#> [8] Wrong expression in checkIdentical(): checkIdentical(2, fff(2))
#> [9] Wrong expression in checkTrue(): checkTrue(fff())
#>
#>
#> == test_Integrate (in runitAllTests.R) run in less than 0.1 sec: OK
#>
#> //Pass: 2 Fail: 0 Errors: 0//
#>
#>
#> == testis.test (in runitsvTest.R) run in less than 0.1 sec: OK
#>
#> //Pass: 15 Fail: 0 Errors: 0//
#>
#>
#> == test_R (in runitsvSuite.R) run in less than 0.1 sec: OK
#>
#> //Pass: 1 Fail: 0 Errors: 0//
#>
#>
#> == testMyVirtualBaseClass.getX (in runitVirtualClass.R) run in less than 0.1 sec: OK
#>
#> //Pass: 3 Fail: 0 Errors: 0//
#>
#>
#> == testsvSuite (in runitsvSuite.R) run in less than 0.1 sec: OK
#>
#> //Pass: 7 Fail: 0 Errors: 0//
#>
#>
#> == testCube (in runitAllTests.R) run in less than 0.1 sec: **FAILS**
#>
#> //Pass: 2 Fail: 1 Errors: 0//
#>
#> === Failures
#> [2] : checkEqualsNumeric(c(8, 28), Cube(2:3))
#>
#>
#> == testbar (in runitBadTests.R) run in less than 0.1 sec: OK
#>
#> //Pass: 2 Fail: 0 Errors: 0//
#>
#>
#> == testrunTest (in runitsvTest.R) run in less than 0.1 sec: DEACTIVATED
#>
#> //Pass: 1 Fail: 0 Errors: 0//
#>
#>
#> == testmat (in runitBadTests.R) run in less than 0.1 sec: OK
#>
#> //Pass: 2 Fail: 0 Errors: 0//
#>
#>
#> == testCreateClass (in runitVirtualClass.R) run in less than 0.1 sec: OK
#>
#> //Pass: 2 Fail: 0 Errors: 0//
#>
#>
#> == testMyVirtualBaseClass.setX (in runitVirtualClass.R) run in less than 0.1 sec: OK
#>
#> //Pass: 6 Fail: 0 Errors: 0//
#>
#>
#> == testSquare (in runitAllTests.R) run in less than 0.1 sec: OK
#>
#> //Pass: 3 Fail: 0 Errors: 0//
#>
#>
#> == testsvTest (in runitsvTest.R) run in less than 0.1 sec: OK
#>
#> //Pass: 14 Fail: 0 Errors: 0//
#>
#>
#> == testsvSuiteList (in runitsvSuite.R) run in less than 0.1 sec: OK
#>
#> //Pass: 6 Fail: 0 Errors: 0//
# Metadata collected on the machine where tests are run
#> $.R.version #> _ #> platform x86_64-apple-darwin15.6.0 #> arch x86_64 #> os darwin15.6.0 #> system x86_64, darwin15.6.0 #> status #> major 3 #> minor 6.3 #> year 2020 #> month 02 #> day 29 #> svn rev 77875 #> language R #> version.string R version 3.6.3 (2020-02-29) #> nickname Holding the Windsock #> #>$.sessionInfo
#> R version 3.6.3 (2020-02-29)
#> Platform: x86_64-apple-darwin15.6.0 (64-bit)
#> Running under: macOS Catalina 10.15.7
#>
#> Matrix products: default
#> BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
#> LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib
#>
#> locale:
#> [1] C/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#>
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base
#>
#> other attached packages:
#> [1] svUnit_1.0.6
#>
#> loaded via a namespace (and not attached):
#>  [1] compiler_3.6.3       magrittr_1.5         htmltools_0.5.0.9000
#>  [4] tools_3.6.3          yaml_2.2.1           stringi_1.4.6
#>  [7] rmarkdown_2.1        knitr_1.28           stringr_1.4.0
#> [10] digest_0.6.25        xfun_0.13            rlang_0.4.5
#> [13] evaluate_0.14
#>
#> $.time #> [1] "2021-04-18 21:35:27 CEST" # List content of the log ls(Log()) #> [1] "testBadTests" "testCreateClass" #> [3] "testCube" "testMyVirtualBaseClass.getX" #> [5] "testMyVirtualBaseClass.setX" "testSquare" #> [7] "test_Integrate" "test_R" #> [9] "testbar" "testis.test" #> [11] "testmat" "testrunTest" #> [13] "testsvSuite" "testsvSuiteList" #> [15] "testsvTest" As you can see, ls() lists all components recorded in the test suite. Each component is a svTestData object inheriting from data.frame, and it can be easily accessed through the $ operator. There are, of course similar methods defined for those svTestData objects, like print(), summary() and stats():

myTest <- Log()$testCube class(myTest) #> [1] "svTestData" "data.frame" myTest #> #> == testCube (in runitAllTests.R) run in less than 0.1 sec: **FAILS** #> #> //Pass: 2 Fail: 1 Errors: 0// #> #> * : checkEqualsNumeric(c(8, 28), Cube(2:3)) run in less than 0.001 sec ... **FAILS** #> Mean relative difference: 0.03571429 #> num [1:2] 8 27 summary(myTest) #> #> == testCube (in runitAllTests.R) run in less than 0.1 sec: **FAILS** #> #> //Pass: 2 Fail: 1 Errors: 0// #> #> === Failures #> [2] : checkEqualsNumeric(c(8, 28), Cube(2:3)) stats(myTest) #>$kind
#>          OK   **FAILS**   **ERROR** DEACTIVATED
#>           2           1           0           0
#>
#> $timing #> timing #> 0 As the logger inherits from environment, you can manage individual test data the same way as objects in any other environment. For instance, if you want to delete a particular test data without touching to the rest, you can use: ls(Log()) #> [1] "testBadTests" "testCreateClass" #> [3] "testCube" "testMyVirtualBaseClass.getX" #> [5] "testMyVirtualBaseClass.setX" "testSquare" #> [7] "test_Integrate" "test_R" #> [9] "testbar" "testis.test" #> [11] "testmat" "testrunTest" #> [13] "testsvSuite" "testsvSuiteList" #> [15] "testsvTest" rm(test_R, envir = Log()) ls(Log()) #> [1] "testBadTests" "testCreateClass" #> [3] "testCube" "testMyVirtualBaseClass.getX" #> [5] "testMyVirtualBaseClass.setX" "testSquare" #> [7] "test_Integrate" "testbar" #> [9] "testis.test" "testmat" #> [11] "testrunTest" "testsvSuite" #> [13] "testsvSuiteList" "testsvTest" As we will see in the following section, ‘svUnit’ proposes several means to organize individual assertions in modules: test functions, test units and test suites. This organization is inspired from ‘RUnit’, but with additional ways of using tests in interactive sessions (for instance, the ability to attach a test to the objects to be tested). Test function The first organization level for grouping assertions together is the test function. A test function is a function without arguments whose name must start with test. It typically contains a series of assertions applied to one object, method, or function to be checked (this is not obligatory, assertions are not restricted to one object, but good practices strongly suggest such a restriction). Here is an example: test_function <- function() { checkTrue(1 < 2, "check1") v <- c(1, 2, 3) # The reference w <- 1:3 # The object to compare to the reference checkEqualsNumeric(v, w) } # Turn this function into a test test_function <- as.svTest(test_function) is.svTest(test_function) #> [1] TRUE A test function should be made a special object called svTest, so that ‘svUnit’ can recognize it. This svTest object, is allowed to live on its own (for instance, in the user’s workspace, or anywhere you like). It can be defined in a R script, be saved in a .RData file, etc… Note that this is very different from ‘RUnit’ where test must always be located in a unit test file on disk). In ‘svUnit’ (not ‘RUnit’), you run such a test simply by using runTest(), which returns the results invisibly and add it to the logger: clearLog() runTest(test_function) Log() #> = A svUnit test suite run in less than 0.1 sec with: #> #> * test_function ... OK #> #> #> == test_function run in less than 0.1 sec: OK #> #> //Pass: 2 Fail: 0 Errors: 0// Now, a test function is most likely designed to test an R object. The ‘svUnit’ package also provides facilities to attach the test function to the object to be tested. Hence, the test cases and the tested object conveniently form a single entity that one can manipulate, copy, save, reload, etc. with all the usual tools in R. This association is simply made using test(myobj) <-: # A very simple function Square <- function(x) return(x^2) # A test case to associate with the Square() function test(Square) <- function() { checkEqualsNumeric(9, Square(3)) checkEqualsNumeric(c(1, 4, 9), Square(1:3)) checkException(Square("xx")) } is.test(Square) # Does this object contain tests? #> [1] TRUE One can retrieve the test associated with the object by using: test(Square) #> svUnit test function: #> { #> checkEqualsNumeric(9, Square(3)) #> checkEqualsNumeric(c(1, 4, 9), Square(1:3)) #> checkException(Square("xx")) #> } And of course, running the test associated with an object is as easy as: runTest(Square) Log() # Remember we didn't clear the log! #> = A svUnit test suite run in less than 0.1 sec with: #> #> * test_function ... OK #> * test(Square) ... OK #> #> #> == test_function run in less than 0.1 sec: OK #> #> //Pass: 2 Fail: 0 Errors: 0// #> #> #> == test(Square) run in less than 0.1 sec: OK #> #> //Pass: 3 Fail: 0 Errors: 0// Now that you master test functions, you will discover how you can group them in logical units, and associate them to R packages. Test units An unit is a coherent piece of software that can be tested separately from the rest. Typically, a R package is a structured way to compile and distribute such code units in R. Hence, we need a mean to organize tests related to this “unit” conveniently. Since a package can contain several functions, data frames, or other objects, our unit should collect together individual test functions related to each of these objects that compose our package. Also, the test unit should accommodate the well-define organization of a package, and should integrate in the already existing testing features of R, in particular, R CMD check. In both ‘RUnit’, and ‘svUnit’, one can define such test units, and they are made code compatible between the two implementations. A test unit is a source-able text file that contains one or more test functions, plus possibly .setUp() and .tearDown() functions (see the online help for further information on these special functions). In ‘RUnit’, you must write such test unit files from scratch. With ‘svUnit’, you can “promote” one or several test functions (associated to other objects, or “living” alone as separate svTest objects) by using makeUnit(). Here is how you promote the test associated with our Square() function to a simple test unit containing only one test function: # Create a test unit on disk and view its content unit <- makeUnit(Square) file.show(unit, delete.file = TRUE) You got the following file whose name must start with runit, with an .R extension (runit*.R), and located by default in the temporary directory of R. Specify another directory with the dir = argument of makeUnit() for a more permanent record of this test unit file. Note also that .setUp() and .tearDown() functions are constructed automatically for you. They specify the context of these tests. This context is used, for instance, by the GUI in Komodo Edit/IDE to locate the test function and the code being tested. ## Test unit 'Square' .setUp <- function() { # Specific actions for svUnit: prepare context if ("package:svUnit" %in% search()) { .Log <- Log() # Make sure .Log is created .Log$..Unit <- "/tmp/RtmpBoZnId/runitSquare.R"
.Log$..File <- "" .Log$..Obj <- ""
.Log$..Tag <- "" .Log$..Msg <- ""
rm(..Test, envir = .Log)
}
}

.tearDown <- function() {
# Specific actions for svUnit: clean up context
if ("package:svUnit" %in% search()) {
.Log$..Unit <- "" .Log$..File <- ""
.Log$..Obj <- "" .Log$..Tag <- ""