ContinuousIntegration
From APIDesign
There is nothing wrong on having your own ContinuousIntegration server like Jenkins or Hudson set up. In fact having it can only help quality of your project. As Chapter 9 of TheAPIBook explains, Chapter 9 is the sole of good frameworks and the most important part of good projects.
Can't Test Everything!
On the other hand, people get so used to this comfortable safety net provided by the by ContinuousIntegration systems that they tend to forget there is also something behind the horizon. The quality departments are well aware of the fact that regardless of the amount of CodeCoverage, there can always be errors visible to end users. The situation is even more complicated when it comes to API Design.
A common advice would be to increase your CodeCoverage or add integration tests to verify the real workflow. The biggest problem when designing an API of a framework however is: you can't run all the users code! You don't have it, you don't even know about it. When your framework is successful, then running all projects that use your API wouldn't even scale.
Control the Observables
As such one has to do something else. Rather than testing all the depending code, one has to make sure the externally observable aspects of your framework as stable - e.g. they don't share like the amoeba in the Amoeba Model. What does that mean?
- Don't rename exported symbols
- Don't change method signatures
- Don't change the exposed behavior
Item #1 is important for linkage - e. g. to make sure that various modules (JARs or .so or .js files) built on top of your API find the symbol they expect. Proper linkage is a per-requisite for any other communication. If the users of your API cannot connect to it, the rest is completely useless.
Keeping unchanged method signatures is important in compiled languages. This is often important for linkage as well. In case of Java one can use Sigtest to prevent accidental changes of number or types of parameters. In dynamic languages like JavaScript this check has to be done dynamically and thus it shifts the check closer to the Runtime_Aspects_of_APIs.
It is easy to say to keep existing behavior, but what if it buggy? Well, sometimes even buggy (from one perspective) behavior can be useful (see the Arithmetica example). What can you do when you really need to change it? In such case support AlternativeBehaviors! There are few ways to do it applicable to any language (for example XML does that by encoding its own version in the document header), but the best is when the language or underlaying runtime container provides some support to it.
Follow
