Changing unit test library

[Neil Hodgson]

Since early in 2011, Scintilla has shipped with some unit tests written in C++ for testing its basic data structures. These used the Google Test framework but Google Test has been a little too difficult to install and use on all the platforms. This has meant that other contributors were less likely to run tests or create a new test when they found a bug. I’d like to move to a simpler testing framework, preferably a light-weight one that has a license that makes it easy to include with the Scintilla distribution.

IDEs now come with built-in testing features but each IDE works with one test library and test format. Xcode uses OCUnit and Visual Studio uses its own CppUnitTestFramework. Tests built for one library don’t work with others.

Since Scintilla started using Google Test, header-only unit test frameworks have become more common. The projects I’ve been looking at include Catch and lest. Both of these are available as a single header file. Catch is 270K and lest is very small at 5K but Catch has more features than lest. Some C++11 features are used by the main download of lest but the current versions of all 3 compilers work and there is also a C++03 variant of lest. Both libraries have reasonably simple syntax for tests. Here is one test for lest:

"EnsureLength", [] {
SplitVector<int> sv;
sv.EnsureLength(4);
EXPECT(4 == sv.Length());
for (int i=0; i<sv.Length(); i++) {
EXPECT(0 == sv.ValueAt(i));
}
},

and the equivalent in Catch:

TEST_CASE("SplitVector tests", "[SplitVector]") {
SplitVector<int> sv;
REQUIRE(0 == sv.Length());
SECTION("EnsureLength") {
sv.EnsureLength(4);
REQUIRE(4 == sv.Length());
for (int i=0; i<sv.Length(); i++) {
REQUIRE(0 == sv.ValueAt(i));
}
}
}

Catch allows extracting setup code so it doesn’t need to be repeated in each test. Both libraries use the Boost license which shouldn’t cause any problems.

https://github.com/philsquared/Catch
https://github.com/martinmoene/lest

As well as these unit tests for data structures, there are also tests written in Python that work on a larger scale. I don’t plan to change the Python tests.

Neil

[Neil Hodgson]

The Catch test framework is now used for Scintilla unit tests found in the test/unit directory. This should work on all three supported operating systems by running “make test” for GNU Make or “nmake -f test.mak test” for Microsoft Visual C++.

Change set:
https://sourceforge.net/p/scintilla/code/ci/9479c1fb62ee505c0d06c3ac76f7a0de9cb3a3aa/

Neil