In computer programming, a unit test is a method of testing the correctness of a particular module of source code. The idea is to write test cases for every non-trivial function or method in the module so that each test case is separate from the others if possible. -- WikiPedia

The Sorcery tests are located in perforce, in //sgl/scripts/sorcery-tests. There is a script, a library, and test files. The script (sut.sh) is passed pairs of library/test files to run and is generaly uninteresting. The library contains the functions that are used for testing and is interesting. The library is documented, but may be lacking in explanation of how everything fits together.

What to Test

Idealy, all functions would behave with every possible input. However, it's usualy a trade off between defensive programming and speed. We are going to make an assumtion that data is checked when it enters Sorcery, thus for internal sorcery functions, it is not important to test the cases of invalid input (however empty strings should probably be tested in any case). However, API functions should probably be tested with invalid input.

When to Make a Sorcery Bug

  • If the documented behviour is not what happens in reality, then make a bug
  • If there is no documentation for what valid input to a function is, then make a bug
  • If the function fails with what is documented to be valid input, then make a bug

What to Include in a Bug Report

  • The function being tested
  • If it's a misbehaving function, include:
    • The parameters, env vars, etc... being used to test it
    • The expected behaviour
    • The observed behaviour
  • If it's a documentation bug, include:
    • What information is missing, as specifically as possible

Details of SUT

The interesting general purpose functions in misc.sh:

  • sut_isolate_functions: Sources the script passed to it, but only allows the specified function definitions out. sut_isolate_functions libfoo bar for example when completed, the function called bar from libfoo will be defined. Note: If the functions need others to work, they will not be exported automaticly, you must explicitly add them to the list of functions to isolate. This is by design.

  • function sut_ignore_functions: The inverse of sut_isolate_functions, sources all functions from the given library except those specified.

  • sut_functions_in: Lists all functions defined in the given library.

  • sut_test: Runs a single test, does the output for pass/fail, and adds to the appropriate counters.

Important constants defined in misc.sh: TEST_OK=0 TEST_FAIL=1 TEST_WARNING=2 The individual test functions should return $TEST_OK or $TEST_FAIL.

  1. sut.sh runs sut_test_module once for each library to test.

  2. sut_test_module sources the test script and looks at the TESTS variable. This variable must be defined in each test file and lists each test group to run. In this example, we are going to assume that TESTS="foo bar" which means to run the foo group and bar group.

  3. sut_test_group runs foo_gtest $src, where $src is the library to be tested.
  4. foo_gtest is defined in the test file. It runs the individual tests. The group test functions carry out all tests relateing to a single function. The individual tests should each test a single aspect, a single case.

How to do these tests is best described with an example. Take explode() from libmisc. We create a file called libmisc.test, which will have only one group test in it for not, explode_gtest. The explode function takes a string and breaks it into an pieces at specified delimiters. The normal tests we'd need to do are empty string, too few arguments, a normal call, and use of a regex delimiter.

Example Test

Here is an example gtest function: 

function explode_gtest() {
 sut_isolate_functions $1 explode
 sut_test "explode (empty)" explode_test_Empty
 sut_test "explode (too few args)" explode_test_TooFewArgs
 sut_test "explode (normal)" explode_test_Normal
 sut_test "explode (regex separator)" explode_test_RegexSeparator
}

This group test would be run if "explode" were listed in the TESTS variable. Let's take this line by line: 

function explode_gtest() {

Function definition of the explode group test. 

sut_isolate_functions $1 explode

One of a couple of handy functions defined in misc.sh. 

sut_test "explode (empty)" explode_test_Empty

Run the explode_test_Empty function as a test, the string to name this test is "explode (empty)". The other tests are similar, so I'm skipping them. 

 }

End the group test function definition.

It should be noted that if there are arguments after the test function name passed to sut_test, they will be passed on to the test function. Ex: 

...
sut_test "Whatever" some_test 1 2 3
...

function some_test() {
echo "$@"
}

Would echo 1 2 3

Continuing on, here would be the individual test functions: 

function explode_test_Empty() {
 local A=()
 explode "" " " A
 [ ${#A[*]} -eq 0 ]
}

function explode_test_TooFewArgs() {
 explode "foo bar" " " >/dev/null 2>&1 && return 1
 return 0
}

function explode_test_Normal() {
 local A=()
 explode "foo bar baz" " " A
 [[ ${A[0]} == foo ]] && [[ ${A[1]} == bar ]] && [[ ${A[2]} == baz ]]
}

function explode_test_RegexSeparator() {
 local A=()
 explode "fooMbarNbaz" '[MN]' A
 [[ ${A[0]} == foo ]] && [[ ${A[1]} == bar ]] && [[ ${A[2]} == baz ]]
}

CategorySorcery

Sorcery/UnitTests (last edited 2008-09-22 23:35:10 by localhost)