Garlic Software

Nov 2010

Unit testing with cppunit

I like to use various types of unit testing frameworks (mostly in the xUnit family) for testing my code to make sure it works as intended. This article is going to show how to write some tests that match the code you write using the C++ language on a Macintosh computer. You could apply these tests either in a test driven development (TDD) environment or after the code has been written which is what I’ve seen the majority of the time.

What is a unit test?

I think of a unit test as a piece of code that tests another piece of code and the test runs every time a build is done. Ideally the pieces are small so the tests and the code are easy to verify. Once you build up a large collection of test and it begins to approach 100% code coverage, you will feel confident that your code does what it intended to do.
Here are a couple examples of cppunit unit tests that we will use later:

    1: CPPUNIT_ASSERT_MESSAGE("check pointer is not null", test3 != NULL);
    2: CPPUNIT_ASSERT_EQUAL(false, test5->fNegative);
    3: CPPUNIT_ASSERT_EQUAL(static_cast<UInt16>(3), test5->fIntPart);
    4: CPPUNIT_ASSERT_DOUBLES_EQUAL(0.14, test5->fFractPart,  FixedPoint::kMarginOfError());

Where to get cppunit

You can get the latest version of cppunit here: Sourceforge
At the time of this writing, the latest stable release is 1.12.1 which will be used for all of the examples in this article. Compiling the cppunit library is the standard UNIX style:

./configure
make
sudo make install

Once you have cppunit installed we can begin to use it with Xcode.

Getting started

I’m going to show how to setup a few tests using sample C++ code. The reason this is done in C++ is that you can use this on an iPhone, Android, Macintosh, or Windows computer as it is cross platform. If your familiar with the model-view-controller design pattern, the model portion is a good candidate for writing in a cross platform fashion. The code we are going to test models a fixed point number.

The Fixed Point Problem

Design and implement a class to work with the following encoding of a fixed-point number. We often work with client hardware that uses proprietary or unusual protocols or data formats. For this exercise, assume that we are developing software that receives data values from a piece of client-provided hardware in the binary-encoded format described below. The task at hand is to create a class that can correctly encode and decode those values and make them usable by other software.
In this instance, a fixed-point number is packed into a 32-bit unsigned integer as follows:
     s i i i i i i i   i i i i i i i i   i f f f f f f f   f f f f f f f f
where:

s : sign bit (1 == negative)
i : 16 integer bits
f : 15 fraction bits

Examples:

ValueFixed Point encoding
1.00x00008000
-1.00x80008000
2.00x00010000
-2.50x80014000
3.140x000191eb
100.990x00327eb8

Your class should support at least the following operations:
  • Create Fixed Point object from floating point value
  • Create Fixed Point object from packed 32-bit value
  • Idiomatic conversion to string (as <<optional sign>><<integer part>>.<<fractional part>>) For example, an instance of your Fixed Point class initialized from an encoded value of 0x80008000 would be converted to the string "-1.0".
  • Convert value to closest floating point equivalent
This problem is derived from a 2008 programming challenge from Art & Logic. This is a good example to use as you want to show that your class not only adheres to the requirements, but it is proven correct.

Quick Xcode setup

We need to make a couple changes to a default Xcode project to get cppunit included. Create a new project in Xcode or use an existing one. I chose the command line tool template for this example but this can also be used with the iPhone or other project types. A link to download the complete project with code is at the end of this article. Once your new project is created or you opened an existing project in Xcode, select Edit Project Settings from the Project menu as shown.

XcodeScreenSnapz001

You can change the project settings to search the include path /usr/local/include and library path /usr/local/lib which are the default install locations for cppunit. I also include a debugging flag (see screenshot below) that is included with the Debug configuration only so that I can trigger the tests in the Debug build only and make sure they are not used in the Release build.

XcodeScreenSnapz002

Once you have the project settings modified, we need to add in the library file to link against. One way to do this is to right-click (control click) on one of the folders with your source code and select Add then Existing Frameworks….

XcodeScreenSnapz003

Change the popup menu at the top of the sheet to Dylibs and then select libcppunit.dylib. You don’t have to use the dylib here and you can link against the .a version. There are pros and cons to either approach which you should understand before using them. This article will not cover these issues and we will use the dylib for simplicity.

XcodeScreenSnapz004

Now that the project file is all setup with the necessary pieces to run unit tests we will look at some code.

Some code

The strategy I’m going to use it to have all of the tests run every time the Debug version of the code is run. Then once the code is ready, we compile it in Release mode so that the tests will be removed. Once we have a new project we want to modify the main.cpp to make it run the tests.

We add a bunch of headers:

    1: #if defined(qDebug)
    2: #include <cppunit/BriefTestProgressListener.h>
    3: #include <cppunit/CompilerOutputter.h>
    4: #include <cppunit/extensions/TestFactoryRegistry.h>
    5: #include <cppunit/TestResult.h>
    6: #include <cppunit/TestResultCollector.h>
    7: #include <cppunit/TestRunner.h>
    8: #include <stdexcept>
    9: #endif // qDebug

And then we add the test runner at the beginning of the main method. This is all just a template that I copy into each project.

    1: #if defined (qDebug)
    2: // Create the event manager and test controller
    3: CPPUNIT_NS::TestResult controller;
    4:
    5: // Add a listener that colllects test result
    6: CPPUNIT_NS::TestResultCollector result;
    7: controller.addListener( &result );        
    8:
    9: // Add a listener that print dots as test run.
   10: CPPUNIT_NS::BriefTestProgressListener progress;
   11: controller.addListener( &progress );      
   12:
   13: // Add the top suite to the test runner
   14: CPPUNIT_NS::TestRunner runner;
   15: runner.addTest( CPPUNIT_NS::TestFactoryRegistry::getRegistry().makeTest() );
   16: runner.run( controller );
   17:
   18: // Print test in a compiler compatible format.
   19: CPPUNIT_NS::CompilerOutputter outputter( &result, std::cerr );
   20: outputter.write(); 
   21: #endif // qDebug

Once these changes are in, you should be able to build and run the project and see the OK message in the console output. You can open the console by selecting Console from the Run menu. Then click the Build and Run button and you should see output similar to the following:

Running…
OK (0)

The 0 (zero) after the OK is the number of tests that ran and in this case we don’t have any yet so we only see that the unit testing framework is setup and ready to go. We are first going to look at a single method to understand what it is doing and what we need to do to test it. I need to give a little background on the class definition here for understanding.

    1: typedef unsigned short UInt16;
    2: typedef unsigned long int UInt32;
    3:
    4: class FixedPoint {
    5: public:
    6:         // construction, assignment and destruction...
    7:         explicit FixedPoint(double inValue);
    8:         explicit FixedPoint(UInt32 inValue);
    9:         explicit FixedPoint(const FixedPoint& inValue);
   10:         FixedPoint& operator=(const FixedPoint& rhs);
   11:         virtual ~FixedPoint(void) {;}
   12:
   13:         // return the value as a string, float, or int
   14:         std::string StringValue(void) const;
   15:         double FloatValue(void) const;
   16:         UInt32 EncodedValue(void) const;
   17:
   18: private:
   19:         bool    fNegative;      // true if the number is negative
   20:         UInt16  fIntPart;       // integer part of the number
   21:         float   fFractPart;     // fractional part of the number
   22:
   23:         // given the number of bits available for this class, it has limits
   24:         // to the size of the number it can represent. These constants are used for
   25:         // bounds checking any value assigned.
   26:         static const double kMinValue(void) {return -65535.99997;}
   27:         static const double kMaxValue(void) {return 65535.99997;}
   28:         static const float kMarginOfError(void) {return 0.00003;}
   29: };

The class uses three private member variables to track the three parts of the fixed point number: fNegative, fIntPart, and fFractPart. So the method that we are going to look at in depth deals with constructing an object with a floating point number.

    1: FixedPoint::FixedPoint(double inValue) {
    2:         if (inValue > kMaxValue())
    3:                 throw std::out_of_range("value is greater than the kMaxValue possible");
    4:         if (inValue < kMinValue())
    5:                 throw std::out_of_range("value is less than the kMinValue possible");
    6:
    7:         fNegative = inValue < 0.0;
    8:
    9:         if (fNegative)
   10:                 inValue = -inValue;
   11:
   12:         double tint = 0.0;
   13:         double tfract = modf(inValue, &tint);
   14:
   15:         fIntPart = static_cast<UInt32> (tint);
   16:         fFractPart = static_cast<float> (tfract);
   17: }

The method above has three exits that we want to test: the two throws (lines 3 and 5) and the successful return (line 17). Not only do we want to test the returns, but also the math and the way the three internal variables are setup after construction. To test the throws you can use the CPPUNIT_ASSERT_THROW test macro. An example is shown below.

    1: FixedPoint* special100 = NULL;
    2: CPPUNIT_ASSERT_THROW(special100 = new FixedPoint(65535.999971), std::out_of_range);

What this is doing is attempting to create a new object with a value (65535.999971) that is out of range for the class, in this case greater than kMaxValue (65535.99997). The other part you might want to test that it doesn’t throw when given a value within range, for this you would use CPPUNIT_ASSERT_NO_THROW. While this is not strictly needed for testing I find that it is a good idea to test both cases if you are going to use exceptions.

    1: CPPUNIT_ASSERT_NO_THROW(special100 = new FixedPoint(65535.99997));

The next type of test is to see if all of the internal variables are setup correctly after a new object is created. The problem statement above included several numbers and their encoded values for testing against. We will use all of them in the tests. While we really don’t need to include so many of the exact same types of tests, testing against all of the sample numbers provided in the problem, we will just for the sake of showing that it solves the given problem with the values provided. In real world testing you might not want to include multiple copies of the same test as you will have to maintain them all when the rest of the code changes and having multiple tests of the same thing doesn’t serve any real purpose (it’s not more accurate because there are many of the same tests).

    1: FixedPoint special5(3.14);
    2: CPPUNIT_ASSERT_EQUAL(false, special5.fNegative);
    3: CPPUNIT_ASSERT_EQUAL(static_cast<UInt16>(3), special5.fIntPart);
    4: CPPUNIT_ASSERT_DOUBLES_EQUAL(0.14, special5.fFractPart, FixedPoint::kMarginOfError());

This test above is calling the same float constructor with a value (3.14) and then testing all of the internal components to see if they were setup as expected. The macro CPPUNIT_ASSERT_EQUAL makes these tests pretty straight forward to read. The only trick you need to remember is to typecast any ambiguous types so that the comparison is using the exact same types. If you forget this, the compiler should display error messages about type mismatch. Another special note is that the macro CPPUNIT_ASSERT_DOUBLES_EQUAL takes a third parameter which is the margin of error for the floating point comparison. The reasons why you need this for floating point numbers are beyond this article, but suffice to say that you need it and it’s important.

Below is the full unit test for the float value constructor. Note that several of these tests are testing the exact same thing and are not really needed nor desired for production code. You want to have the minimum number of tests that test all of the functionality but not duplicates. All of the extra tests were included just to show the results for the numbers provided in the problem statement above. Also any special edge cases, you will want to call out as in this example at line 52.

    1: void FixedPointTest::floatConstructorTest(void) {
    2:         FixedPoint special1(1.0);
    3:         CPPUNIT_ASSERT_EQUAL(false, special1.fNegative);
    4:         CPPUNIT_ASSERT_EQUAL(static_cast<UInt16>(1), special1.fIntPart);
    5:         CPPUNIT_ASSERT_DOUBLES_EQUAL(0.0, special1.fFractPart, FixedPoint::kMarginOfError());
    6:
    7:         FixedPoint special2(-1.0);
    8:         CPPUNIT_ASSERT_EQUAL(true, special2.fNegative);
    9:         CPPUNIT_ASSERT_EQUAL(static_cast<UInt16>(1), special2.fIntPart);
   10:         CPPUNIT_ASSERT_DOUBLES_EQUAL(0.0, special2.fFractPart, FixedPoint::kMarginOfError());
   11:
   12:         FixedPoint special3(2.0);
   13:         CPPUNIT_ASSERT_EQUAL(false, special3.fNegative);
   14:         CPPUNIT_ASSERT_EQUAL(static_cast<UInt16>(2), special3.fIntPart);
   15:         CPPUNIT_ASSERT_DOUBLES_EQUAL(0.0, special3.fFractPart, FixedPoint::kMarginOfError());
   16:
   17:         FixedPoint special4(-2.5);
   18:         CPPUNIT_ASSERT_EQUAL(true, special4.fNegative);
   19:         CPPUNIT_ASSERT_EQUAL(static_cast<UInt16>(2), special4.fIntPart);
   20:         CPPUNIT_ASSERT_DOUBLES_EQUAL(0.5, special4.fFractPart, FixedPoint::kMarginOfError());
   21:
   22:         FixedPoint special5(3.14);
   23:         CPPUNIT_ASSERT_EQUAL(false, special5.fNegative);
   24:         CPPUNIT_ASSERT_EQUAL(static_cast<UInt16>(3), special5.fIntPart);
   25:         CPPUNIT_ASSERT_DOUBLES_EQUAL(0.14, special5.fFractPart, FixedPoint::kMarginOfError());
   26:
   27:         FixedPoint special6(100.99);
   28:         CPPUNIT_ASSERT_EQUAL(false, special6.fNegative);
   29:         CPPUNIT_ASSERT_EQUAL(static_cast<UInt16>(100), special6.fIntPart);
   30:         CPPUNIT_ASSERT_DOUBLES_EQUAL(0.99, special6.fFractPart, FixedPoint::kMarginOfError());
   31:
   32:         FixedPoint special7(0.0);
   33:         CPPUNIT_ASSERT_EQUAL(false, special7.fNegative);
   34:         CPPUNIT_ASSERT_EQUAL(static_cast<UInt16>(0), special7.fIntPart);
   35:         CPPUNIT_ASSERT_DOUBLES_EQUAL(0.0, special7.fFractPart, FixedPoint::kMarginOfError());
   36:
   37:         FixedPoint special8(-65535.99997);
   38:         CPPUNIT_ASSERT_EQUAL(true, special8.fNegative);
   39:         CPPUNIT_ASSERT_EQUAL(static_cast<UInt16>(65535), special8.fIntPart);
   40:         CPPUNIT_ASSERT_DOUBLES_EQUAL(0.99997, special8.fFractPart, FixedPoint::kMarginOfError());
   41:
   42:         FixedPoint special9(65535.99997);
   43:         CPPUNIT_ASSERT_EQUAL(false, special9.fNegative);
   44:         CPPUNIT_ASSERT_EQUAL(static_cast<UInt16>(65535), special9.fIntPart);
   45:         CPPUNIT_ASSERT_DOUBLES_EQUAL(0.99997, special9.fFractPart, FixedPoint::kMarginOfError());
   46:
   47:         FixedPoint special10(65535.99994);
   48:         CPPUNIT_ASSERT_EQUAL(false, special10.fNegative);
   49:         CPPUNIT_ASSERT_EQUAL(static_cast<UInt16>(65535), special10.fIntPart);
   50:         CPPUNIT_ASSERT_DOUBLES_EQUAL(0.99994, special10.fFractPart, FixedPoint::kMarginOfError());
   51:
   52:         // Note that this value is not round tripped here as -0.0 is no different from 0.0
   53:         FixedPoint special11(-0.0);
   54:         CPPUNIT_ASSERT_EQUAL(false, special11.fNegative);
   55:         CPPUNIT_ASSERT_EQUAL(static_cast<UInt16>(0), special11.fIntPart);
   56:         CPPUNIT_ASSERT_DOUBLES_EQUAL(0.0, special11.fFractPart, FixedPoint::kMarginOfError());
   57:
   58:         FixedPoint* special100 = NULL;
   59:         CPPUNIT_ASSERT_THROW(special100 = new FixedPoint(65535.999971), std::out_of_range);
   60:         delete special100;
   61:
   62:         CPPUNIT_ASSERT_THROW(special100 = new FixedPoint(-65535.999971), std::out_of_range);
   63:         delete special100;
   64:
   65:         CPPUNIT_ASSERT_THROW(special100 = new FixedPoint(65536.0), std::out_of_range);
   66:         delete special100;
   67:
   68:         CPPUNIT_ASSERT_THROW(special100 = new FixedPoint(-65536.0), std::out_of_range);
   69:         delete special100;
   70:
   71:         CPPUNIT_ASSERT_NO_THROW(special100 = new FixedPoint(65535.99997));
   72:         delete special100;
   73:
   74:         CPPUNIT_ASSERT_NO_THROW(special100 = new FixedPoint(0.999971));
   75:         delete special100;
   76: }

Creating a test suite

To get all of the tests to run, you need to add them to a test suite which is added to the test runner. There are several macros that you use to setup all of this for you and an example is shown below for adding in the two tests discussed in this article. This test class can either be in the same file as your class methods or in a new file. For this example the tests are included at the end of the class methods to keep all the code in the same place. While this does make the file longer, it also makes it easy for developers to update the tests whenever they change the code.

    1: #if defined(qDebug)
    2: #include <cppunit/extensions/HelperMacros.h>
    3:
    4: class FixedPointTest : public CPPUNIT_NS::TestFixture {
    5:         CPPUNIT_TEST_SUITE( FixedPointTest );
    6:         CPPUNIT_TEST( floatConstructorTest );
    7:         CPPUNIT_TEST( encodedValueTest );
    8:         CPPUNIT_TEST_SUITE_END();
    9:
   10:         public:
   11:                 void setUp(void);
   12:                 void tearDown(void);
   13:
   14:         protected:
   15:                 void floatConstructorTest(void);
   16:                 void encodedValueTest(void);
   17:
   18:         private:
   19:                 FixedPoint* test1;
   20: };
   21:
   22: CPPUNIT_TEST_SUITE_REGISTRATION( FixedPointTest );
   23:
   24: void FixedPointTest::setUp(void) {
   25:         test1 = new FixedPoint(static_cast<UInt32>(0x00008000));
   26: }
   27:
   28: void FixedPointTest::tearDown(void) {
   29:         delete test1;
   30: }
   31:
   32: void FixedPointTest::floatConstructorTest(void) {}
   33: void FixedPointTest::encodedValueTest(void) {}
   34: #endif // qDebug

The important macro calls of CPPUNIT_TEST_SUITE, CPPUNIT_TEST_SUITE_END, CPPUNIT_TEST, and CPPUNIT_TEST_SUITE_REGISTRATION are used to define the test suite start and stop, add tests to the suite, and add the suite to the runner. To add a new test you only need to add the line for the macro CPPUNIT_TEST (line 7), add in the method declaration (line 16), and write the test method (line 33).

Another nice feature of cppunit is that each test class can have a setUp method which will be called before every test that is run and a tearDown method that will be called after every test. This allows you to setup other objects for testing and not have to write the code in each test case. I use that strategy above to setup a test value that is then used in all of the test cases. Below I add many more test values to to the test class and setUp method.

Another Test

This next example is a little easier to test as it is only returning a value and has no extra exits from the method.

    1: // This method will return a packed 32-bit representation of the fixed point number
    2: UInt32 FixedPoint::EncodedValue(void) const {
    3:         UInt32 val = fIntPart << 15;
    4:
    5:         val |= static_cast<UInt32> (fFractPart * static_cast<float> (0x8000));
    6:
    7:         if (fNegative)
    8:                 val |= 0x80000000;
    9:
   10:         return val;
   11: }

The easiest way to test this method is to check for the round trip of the original value set. For convenience I have setup several test values within the FixedPointTest class detailed above. You’ll also note that the test values include all of the test values given in the original problem statement.

    1: void FixedPointTest::setUp(void) {
    2:         test1 = new FixedPoint(static_cast<UInt32>(0x00008000));
    3:         test2 = new FixedPoint(static_cast<UInt32>(0x80008000));
    4:         test3 = new FixedPoint(static_cast<UInt32>(0x00010000));
    5:         test4 = new FixedPoint(static_cast<UInt32>(0x80014000));
    6:         test5 = new FixedPoint(static_cast<UInt32>(0x000191eb));
    7:         test6 = new FixedPoint(static_cast<UInt32>(0x00327eb8));
    8:         test7 = new FixedPoint(static_cast<UInt32>(0x00000000));
    9:         test8 = new FixedPoint(static_cast<UInt32>(0xFFFFFFFF));
   10:         test9 = new FixedPoint(static_cast<UInt32>(0x7FFFFFFF));
   11:         test10 = new FixedPoint(static_cast<UInt32>(0x7FFFFFFE));
   12:         test11 = new FixedPoint(static_cast<UInt32>(0x80000000));
   13: }
   14:
   15: void FixedPointTest::encodedValueTest(void) {
   16:    CPPUNIT_ASSERT_EQUAL(static_cast<UInt32>(0x00008000), test1->EncodedValue());
   17:    CPPUNIT_ASSERT_EQUAL(static_cast<UInt32>(0x80008000), test2->EncodedValue());
   18:    CPPUNIT_ASSERT_EQUAL(static_cast<UInt32>(0x00010000), test3->EncodedValue());
   19:    CPPUNIT_ASSERT_EQUAL(static_cast<UInt32>(0x80014000), test4->EncodedValue());
   20:    CPPUNIT_ASSERT_EQUAL(static_cast<UInt32>(0x000191eb), test5->EncodedValue());
   21:    CPPUNIT_ASSERT_EQUAL(static_cast<UInt32>(0x00327eb8), test6->EncodedValue());
   22:    CPPUNIT_ASSERT_EQUAL(static_cast<UInt32>(0x00000000), test7->EncodedValue());
   23:    CPPUNIT_ASSERT_EQUAL(static_cast<UInt32>(0xFFFFFFFF), test8->EncodedValue());
   24:    CPPUNIT_ASSERT_EQUAL(static_cast<UInt32>(0x7FFFFFFF), test9->EncodedValue());
   25:    CPPUNIT_ASSERT_EQUAL(static_cast<UInt32>(0x7FFFFFFE), test10->EncodedValue());
   26:    CPPUNIT_ASSERT_EQUAL(static_cast<UInt32>(0x80000000), test11->EncodedValue());
   27: }

It is important to note that this test is very simple because all it is testing is the output of the method. We don’t need to test the constructors or that the internal values are setup correctly as those are all handled by the appropriate test cases elsewhere. I’d also like to note that this number of tests is overkill and should be pared down to a smaller list of necessary tests along with the edge cases. I’m showing the extra ones here just because they were all in the problem statement.

Results

Once you have added several tests to your test suite you will see them outputting their results in the console every time you run the Debug version of the app. The full test suite for this example has the following output:

Running…
FixedPointTest::uint32ConstructorTest : OK
FixedPointTest::floatConstructorTest : OK
FixedPointTest::copyConstructorTest : OK
FixedPointTest::assignmentOperatorTest : OK
FixedPointTest::floatValueTest : OK
FixedPointTest::stringValueTest : OK
FixedPointTest::encodedValueTest : OK
OK (7)

The only other case you need to be aware of is when a test breaks and it will tell you what broke and on which line it failed. Below is a contrived example of a failed test I intentionally broke to show a sample.

    1: Running…
    2: FixedPointTest::uint32ConstructorTest : OK
    3: FixedPointTest::floatConstructorTest : OK
    4: FixedPointTest::copyConstructorTest : OK
    5: FixedPointTest::assignmentOperatorTest : assertion
    6: FixedPointTest::floatValueTest : OK
    7: FixedPointTest::stringValueTest : OK
    8: FixedPointTest::encodedValueTest : OK
    9: …/fp/FixedPoint.cpp:399: Assertion
   10: Test name: FixedPointTest::assignmentOperatorTest
   11: equality assertion failed
   12: - Expected: 1
   13: - Actual  : 0
   14:
   15: Failures !!!
   16: Run: 7   Failure total: 1   Failures: 1   Errors: 0

As you can see from this broken test example the error occurred in the assignmentOperatorTest test as shown on line 5 and then on line 9 it tells you which file and line in the test caused the failure. This makes if very easy to figure out which test broke the run and allow you to find it quickly for fixing.

Conclusion

I would like to say that adding unit tests to your new and existing code is a great idea. While it is an investment of time and energy to add all of the tests, in the long run it will save you time. Even if you just start out with one class being tested, it is a good start. I encourage everyone to at least download the sample code and the test framework and to at least try running the sample code. You may not use it today but hopefully you will see the value of it and try to use it in a future project.

Sample Code

This is the Xcode project and sample code used in this article. fp

References

For more in-depth information on topics covered here, please see the following references.

cppunit site: http://sourceforge.net/apps/mediawiki/cppunit/index.php?title=Main_Page
TDD book: Test Driven Development: By Example by Kent Beck [Amazon]
xUnit book: xUnit Test Patterns: Refactoring Test Code by Gerard Meszaros [Amazon]
Unit Testing book: Unit Test Frameworks by Paul Hamill [Amazon]