ContinuousIntegration

From APIDesign

Revision as of 13:53, 4 April 2018

There is nothing wrong on having your own ContinuousIntegration server like Jenkins or Hudson set up. Having it contributes to quality of your project. As Chapter 9 of TheAPIBook explains, tests are the soul of good frameworks and the most important part of good projects.

U Can't Test This!

People get so used to this comfortable safety net provided by ContinuousIntegration systems that they tend to forget there is also something behind the horizon. On the other hand the quality departments are well aware of that (regardless of the amount of CodeCoverage) there are always remaining errors visible to end users.

The situation is even more complicated when it comes to API Design. A common advice to get better quality is to increase your CodeCoverage or add integration tests to verify the real workflow. Yet the biggest problem when designing an API of a framework is: you can't run the users code! Either you don't have it, or you don't even know about it. When your framework gets really successful, then running all projects that use your API wouldn't even scale.

APIs are like stars: you never know who's watching!

Control the Observables

As such one has to substitute the continuous execution of every piece of code by something else. Rather than testing all the depending code, one has to make sure the externally observable aspects of your framework are stable - e.g. they don't shake like an amoeba in the Amoeba Model. What does that mean?

1. Don't rename externally visible symbols
2. Don't change method signatures
3. Don't change the exposed behavior

Item #1 is important for linkage - e. g. to make sure that various modules (JARs or .so or .js files) built on top of your API find the symbol they expect. Proper linkage is a prerequisite for any other communication. If the users of your API cannot connect to it, the rest is completely useless.

Keeping stable method signatures is important in compiled languages. This is often part of the linkage as well. In case of Java one can use Sigtest to prevent accidental changes of number or types of parameters. In dynamic languages like JavaScript this check has to be done dynamically and thus it shifts the check closer to the Runtime_Aspects_of_APIs.

As far as runtime compatibility goes, it is easy to say: keep existing behavior. But what if such behavior is buggy? Well, sometimes even buggy (from one perspective) behavior can be useful (see the Arithmetica example) and should be preserved. What can you do when you really need to change it?

In such case choose to provide AlternativeBehaviors! There are few ways to do it applicable to any language, but the best is when the language or its underlaying runtime container provides some support to do it smoothly.

For example XML does that by encoding its own version in the document header and than it is clear which parser to use to provide the right AlternativeBehavior. On the other hand Python is famous for not identifying its language version and changing its syntax in a slightly incompatible way that started years of the Python 2.x vs. 3.0 battle.

What's the Goal?

The goal is to make sure that people can write applications, libraries and plugins in a way that continues to work with multiple versions of your API. Nobody wants to publish update as soon as you provide new version of the API. Things shall just continue to work. This is called BackwardCompatibility.

NetBeans IDE supported plugins since 2000 and the most important feature people asked for was ability to write a plugin and make it work with last few versions of NetBeans simultaneously. Their core business wasn't to write plugins, so they wanted to minimize the time spent doing that. The last thing they wanted was to watch us to release new version of NetBeans and at that time immediately re-publish an update of their plugin suitable to run with that version.

The same situation applies to transitive dependencies between libraries. If somebody builds their API on top of yours then there has to be a combination of versions of these two APIs working together. By doing non-BackwardCompatible changes one complicates finding such version configuration a lot (remember LibraryReExportIsNPComplete). This can be witnessed by everyone who tried to assemble five libraries using few shared pieces into a coherent and working system in an environment which pays little attention to BackwardCompatibility.

U Can't Change It Whole!

Traditional in-house development scheme suggests refactorings to clean up the code and improve its quality. In such model it is fine to do incompatible changes in the code base. If they break some tests, one fixes the tests as well. Ones your ContinuousIntegration is green again, you can integrate.

There is however a hidden assumption: one needs an instant access to all the code and perform the whole change at once. This is far from reality when it comes to APIDesign and maintaining frameworks. APIs are like Stars and building them feels like building Universe, not a house. You never know who's watching (e.g. using) your API and you can't fix all the usages at once.

As a result of that API maintainers need to invest in defining and following proper lifecycle of their API. On one side, there is a strict request to keep BackwardCompatibility, on the other side there is a deprecation (as a notice) followed by some period of time to give downstream projects a chance to adopt to the API change. Both approaches satisfy the major goal and are certainly better than Big Bang rewrites.

It makes no sense to claim: Everything is OK, my ContinuousIntegration server is green! when you had to fix your tests. If your API is any good, there are tons of code similar to your tests out there and there is nobody to fix them as part of your refactoring. Better to learn and investing in keeping a bit of BackwardCompatibility.

Conclusion

Having ContinuousIntegration is a must for any project (including ones that provide an API). However in contrast to building final applications one cannot fit the whole code into the ContinuousIntegration check. As a result of that one needs to apply also other techniques when judging correctness of a change. Keeping BackwardCompatibility is a useful initial step that greatly pays off in a long run.

We have just found another paradox of APIDesign: relying only on ContinuousIntegration is bad!