SignatureTests

From APIDesign

Since the rise of XP and other Agile methodologies testing become more and more popular. Not only among dedicated quality departments, but also among developers. It is not surprising that these days a successful projects is not just code, but also set of tests to verify that the release is good enough and that the project moves in the right direction.

The API Design is not exceptional in this aspect. Proper API Design requires testing and verification. The goal when designing an API is to prevent the amoeba shaking effect - e.g. prevent each new release to completely destabilize existing usages. This requires standard unit testing as well, as something special - testing for binary compatibility.

Signature Testing Tools

Java

The NetBeans project is using its slight extension to sigtest open source testing tool. The sigtest is the official OpenJDK tool and since version 2.1 (which has been release in Mar 2009) it supports also BackwardCompatibility tests. We run the tests daily to ensure that our API are still binary compatible and whether our APIs are changing intentionally and not accidentally. This is example of one of sigtest reports:

Base version: 1.2
Tested version: 1.3
 
Added Methods
-------------
 
org.nb.cnd.api.lexer.CndLexerUtilities: 
method public final static boolean CndLexerUtilities.isKeyword(java.lang.String)

Whenever you make a release of your API, you can use the sigtest Ant task to generate a golden file and store it in your version control next to your API sources. Since then, you can always verify that the APIs that you are creating are still compatible with the golden file by running the signature check against new version of your library.

Simple and useful. I cannot imagine API design without such verification tool. Thanks the sigtest team for releasing such mature 2.1 version!

Existing alternatives to sigtest:

• Clirr
• Java API Compliance Checker

Sample usage:

   japi-compliance-checker -lib NAME -old OLD.jar -new NEW.jar

C/C++

There is a backward binary compatibility checker for shared libraries in C/C++. An anonymous coward just provided a link to API compliance checker.

It is good that C world gets such a useful tool, just realize the difference with Java. Adding new field into a struct deserves a warning(!) as it may be incompatible for everyone who embeds such structure inside own struct. Adding new virtual method in backward compatible way is almost impossible, as far as I can see.

Don't get me wrong, I am glad the need for BackwardCompatibility is realized among C programmers. Just the self control to achieve such compatibility needs to be much higher than among Java programmers (and even those need a lot of self control). We still need a language that will give use evolution capabilities while letting us act in almost cluelessness mode.

--JaroslavTulach 19:08, 9 August 2009 (UTC)