Setup
Of course, it’d be rude not to use a temperature converter as the sample project in a testing blog post. The only permitted alternative is a flawed bank account model.
I’ll create a folder for my project, then inside it a folder for the tests:
$ mkdir -p TemperatureConverter/test
$ cd TemperatureConverter
The test runner, gnustep-tests, is a shell script that looks for tests in subfolders of the current folder. If I run it now, nothing will happen because there aren’t any tests. I’ll tell it that the test folder will contain tests by creating an empty marker file that the script looks for.
$ touch test/TestInfo
Of course, there still aren’t any tests, so I should give it something to build and run. The test fixture files themselves can be Objective-C or Objective-C++ source files.
$ cat > converter.m
#include "Testing.h"
int main(int argc, char **argv)
{
}
^D
Now the test runner has something to do, though not very much. Any of the invocations below will cause the runner to find this new file, compile and run it. It’ll also look for test successes and failures, but of course there aren’t any yet. Still, these invocations show how large test suites could be split up to let developers only run the parts relevant to their immediate work.
$ gnustep-tests #tests everything
$ gnustep-tests test #tests everything in the test/ folder
$ gnustep-tests test/converter.m #just the tests in the specified file
The first test
Following the standard practice of red-green-refactor, I’ll write the test that I want to be able to write and watch it fail. This is it:
#include "Testing.h"
int main(int argc, char **argv)
{
TemperatureConverter *converter = [TemperatureConverter new];
float minusFortyF = [converter convertToFahrenheit:-40.0];
PASS(minusFortyF == -40.0, "Minus forty is the same on both scales");
return 0;
}
The output from that:
$ gnustep-tests
Checking for presence of test subdirectories ...
--- Running tests in test ---
test/converter.m:
Failed build:
1 Failed build
Unfortunately we could not even compile all the test programs.
This means that the test could not be run properly, and you need
to try to figure out why and fix it or ask for help.
Please see /home/leeg/GNUstep/TemperatureConverter/tests.log for more detail.
Unsurprisingly, it doesn’t work. Perhaps I should write some code. This can go at the top of the converter.m test file for now.
#import <Foundation/Foundation.h>
@interface TemperatureConverter : NSObject
- (float)convertToFahrenheit:(float)celsius;
@end
@implementation TemperatureConverter
- (float)convertToFahrenheit:(float)celsius;
{
return -10.0; //WAT
}
@end
I’m reasonably confident that’s correct. I’ll try it.
$gnustep-tests
Checking for presence of test subdirectories ...
--- Running tests in test ---
test/converter.m:
Failed test: converter.m:19 ... Minus forty is the same on both scales
1 Failed test
One or more tests failed. None of them should have.
Please submit a patch to fix the problem or send a bug report to
the package maintainer.
Please see /home/leeg/GNUstep/TemperatureConverter/tests.log for more detail.
Oops, I seem to have a typo which should be easy enough to correct. Here’s proof that it now works:
$ gnustep-tests
Checking for presence of test subdirectories ...
--- Running tests in test ---
1 Passed test
All OK!
Second point, first set
If every temperature in Celsius were equivalent to -40F, then the two scales would not be measuring the same thing. It’s time to discover whether this class is useful for a larger range of inputs.
All of the tests I’m about to add are related to the same feature, so it makes sense to document these tests as a group. The suite calls these groups “sets”, and it works like this:
int main(int argc, char **argv)
{
TemperatureConverter *converter = [TemperatureConverter new];
START_SET("celsius to fahrenheit");
float minusFortyF = [converter convertToFahrenheit:-40.0];
PASS(minusFortyF == -40.0, "Minus forty is the same on both scales");
float freezingPoint = [converter convertToFahrenheit:0.0];
PASS(freezingPoint == 32.0, "Water freezes at 32F");
float boilingPoint = [converter convertToFahrenheit:100.0];
PASS(boilingPoint == 212.0, "Water boils at 212F");
END_SET("celsius to fahrenheit");
return 0;
}
Now at this point I could build a look-up table to map inputs onto outputs in my converter method, or I could choose a linear equation.
- (float)convertToFahrenheit:(float)celsius;
{
return (9.0/5.0)*celsius + 32.0;
}
Even tests have aspirations
Aside from documentation, test sets have some useful properties. Imagine I’m going to add a feature to the app: the ability to convert from Fahrenheit to Celsius. This is the killer feature, clearly, but I still need to tread carefully.
While I’m developing this feature, I want to integrate it with everything that’s in production so that I know I’m not breaking everything else. I want to make sure my existing tests don’t start failing as a result of this work. However, I’m not exposing it for public use until it’s ready, so I don’t mind so much if tests for the new feature fail: I’d like them to pass, but it’s not going to break the world for anyone else if they don’t.
Test sets in the GNUstep test suite can be hopeful, which represents this middle ground. Failures of tests in hopeful sets are still reported, but as “dashed hopes” rather than failures. You can easily separate out the case “everything that should work does work” from broken code under development.
START_SET("fahrenheit to celsius");
testHopeful = YES;
float minusFortyC = [converter convertToCelsius:-40.0];
PASS(minusFortyC == -40.0, "Minus forty is the same on both scales");
END_SET("fahrenheit to celsius");
The report of dashed hopes looks like this:
$ gnustep-tests
Checking for presence of test subdirectories ...
--- Running tests in test ---
3 Passed tests
1 Dashed hope
All OK!
But we were hoping that even more tests might have passed if
someone had added support for them to the package. If you
would like to help, please contact the package maintainer.
Promotion to production
OK, well one feature in my temperature converter is working so it’s time to integrate it into my app. How do I tell the gnustep-tests script where to find my class if I remove it from the test file?
I move the classes under test not into an application target, but a library target (a shared library, static library or framework). Then I arrange for the tests to link that library and use its headers. How you do that depends on your build system and the arrangement of your source code. In the GNUstep world it’s conventional to define a target called “check” so developers can write make check to run the tests. I also add an optional argument to choose a subset of tests, so the three examples of running the suite at the beginning of this post become:
$ make check
$ make check suite=test
$ make check suite=test/converter.m
I also arrange for the app to link the same library and use its headers, so the tests and the application use the same logic compiled with the same tools and settings.
Here’s how I arranged for the TemperatureConverter to be in its own library, using gnustep-make. Firstly, I broke the class out of test/converter.m and into a pair of files at the top level, TemperatureConverter.[hm]. Then I created this GNUmakefile at the same level:
include $(GNUSTEP_MAKEFILES)/common.make
LIBRARY_NAME=TemperatureConverter
TemperatureConverter_OBJC_FILES=TemperatureConverter.m
TemperatureConverter_HEADER_FILES=TemperatureConverter.h
-include GNUmakefile.preamble
include $(GNUSTEP_MAKEFILES)/library.make
-include GNUmakefile.postamble
Now my tests can’t find the headers or the library, so it doesn’t build again. In GNUmakefile.postamble I’ll create the “check” target described above to run the test suite in the correct argument. GNUmakefile.postamble is included (if present) after all of GNUstep-make’s rules, so it’s a good place to define custom targets while ensuring that your main target (the library in this case) is still the default.
TOP_DIR := $(CURDIR)
check::
@(\
ADDITIONAL_INCLUDE_DIRS="-I$(TOP_DIR)";\
ADDITIONAL_LIB_DIRS="-L$(TOP_DIR)/$(GNUSTEP_OBJ_DIR)";\
ADDITIONAL_OBJC_LIBS=-lTemperatureConverter;\
LD_LIBRARY_PATH="$(TOP_DIR)/$(GNUSTEP_OBJ_DIR):${LD_LIBRARY_PATH}";\
export ADDITIONAL_INCLUDE_DIRS;\
export ADDITIONAL_LIB_DIRS;\
export ADDITIONAL_OBJC_LIBS;\
export LD_LIBRARY_PATH;\
gnustep-tests $(suite);\
grep -q “Failed test” tests.sum; if [ $$? -eq 0 ]; then exit 1; fi\
)
The change to LD_LIBRARY_PATH is required to ensure that the tests can load the build version of the library. This must come first in the library path so that the tests are definitely investigating the code in the latest version of the library, not some other version that might be installed elsewhere in the system. The last line fails the build if any tests failed (meaning we can use this check as part of a continuous integration system).
More information
The GNUstep test framework is part of gnustep-make, and documentation can be found in its README. Nicola Pero has some useful tutorials about the rest of the make system, having written most of it himself.
