Automating C++ Unit Tests in MSVC

Test Driven Development using C++ can be more challenging than in other more trendy languages. But, with a bit of creativity and know-how, it really isn't too difficult.

In a recent project I wanted to follow a more formal TDD process than perhaps sometimes I would. I always write defensive code littered with assertions, etc., but always stop short of a full unit test suite for all my code.

Being such a fan of the Boost libraries, naturally I turned to boost.org and the Boost.Test library. It's very complete if a bit complex in its naming conventions, but I won't cover the library here, you can read the documentation for yourself here.

Let's talk about how to configure an MSVC solution for automated unit testing. It turns out not to be very difficult, but did take some trial-and-error.

Goals

With an automated unit test suite, I am trying to achieve a set of tests that satisfy these goals:

1. the tests should run as a part of the build process
2. if the application fails to build, don't run the tests
3. building the project, the tests should be re-run even if the application hasn't changed
4. the results of the unit test should be available in the Output Window with the compiler output
5. if any of the tests fail, the build should fail
6. if any of the tests fail, the failing line should be easily accessible in the IDE

Method

I have two projects in my Solution. The first (Project1) is the application I'm developing (in a complex app, there'll obviously be more projects for DLLs, etc.), the second (Project2) is the unit test application. (Goal 1)

In the Project Dependencies, set Project2 to be dependent on Project1. This will ensure the build order so the application builds before the unit tests. (Goal 2)

We'll use a file on disk to control the running of the tests, using a file-dependency mechanism. Before running the tests, we'll delete the file...
In Project Properties, Build Events, Pre-Build Event, set Command Line to

del "$(SolutionDir)$(Configuration)\$(ProjectName).txt"

Now, we'll use a Custom Build Step to invoke the test execution after the test suite has compiled.

In Project Properties, Custom Build Step, set Command Line to (all on one line)

"$(SolutionDir)$(Configuration)\$(ProjectName).exe" &&
echo Ok > $(SolutionDir)$(Configuration)\$(ProjectName).txt"

This trick will run the freshly built test suite application and, if it succeeds, will write "Ok" to a text file. Now, we have to tell MSVC where the file is that we've written to:
Set Outputs to

$(SolutionDir)$(Configuration)\$(ProjectName).txt

MSVC will use this file dependency to control whether the test suite succeeded or not. On success, the file contains "Ok", but if the test suite fails, the file will not be written and won't exist because of the delete in the pre-build step.(Goal 3)

Unit Tests

Using Boost.Test a very simple example test suite is shown below. Output of the running tests is send to stdout, which will be captured by MSVC and written to the Output Window. (Goal 4) If any tests fail, Boost.Test returns ERROR_FAILURE which will stop the build. (Goal 5)

#define BOOST_TEST_MODULE fileparts
#include "boost/test/included/unit_test.hpp"

BOOST_AUTO_TEST_CASE( free_test_function )
{
    BOOST_CHECK(false);
}

This test suite will always fail, producing the output below:

1>CustomBuildStep:
1>  Description: Performing Custom Build Step
1>  Running 1 test case...
1>  testsuite.cpp(13): error in "free_test_function":check false failed
1>  
1>  *** 1 failure detected in test suite "fileparts"

The error is in the correct format to enable you to double-click this line in the Output Window. (Goal 6)

Conclusion

This is a neat implementation with a few tricks and hacks that make a streamlines Unit Test environment for TDD in C++.