'. '

ContinuousIntegration

From APIDesign

(Difference between revisions)
Jump to: navigation, search
(What's the Goal?)
Current revision (07:13, 5 April 2018) (edit) (undo)
(U Can't Change It Whole!)
 
(17 intermediate revisions not shown.)
Line 3: Line 3:
== [[wikipedia:U_Can't_Touch_This|U Can't Test This]]! ==
== [[wikipedia:U_Can't_Touch_This|U Can't Test This]]! ==
-
People get so used to this comfortable safety net provided by [[ContinuousIntegration]] systems that they tend to forget there is also something behind the [[horizon]]. On the other hand the quality departments are well aware of that (regardless of the amount of [[CodeCoverage]]) there can always be errors visible to end users.
+
People get so used to this comfortable safety net provided by [[ContinuousIntegration]] systems that they tend to forget there is also something behind the [[horizon]]. On the other hand the quality departments are well aware of that (regardless of the amount of [[CodeCoverage]]) there are always remaining errors visible to end users.
The situation is even more complicated when it comes to [[API Design]]. A common advice to get better quality is to increase your [[CodeCoverage]] or add integration tests to verify the real workflow. Yet the biggest problem when designing an [[API]] of a framework is: you can't run the users code! Either you don't have it, or you don't even know about it. When your framework gets really successful, then running all projects that use your [[API]] wouldn't even scale.
The situation is even more complicated when it comes to [[API Design]]. A common advice to get better quality is to increase your [[CodeCoverage]] or add integration tests to verify the real workflow. Yet the biggest problem when designing an [[API]] of a framework is: you can't run the users code! Either you don't have it, or you don't even know about it. When your framework gets really successful, then running all projects that use your [[API]] wouldn't even scale.
 +
 +
[[API]]s are like [[star]]s: you never know who's watching!
== Control the Observables ==
== Control the Observables ==
Line 21: Line 23:
As far as runtime compatibility goes, it is easy to say: ''keep existing behavior''. But what if such behavior is buggy? Well, sometimes even buggy (from one perspective) behavior can be useful (see the [[Arithmetica]] example) and should be preserved. What can you do when you really need to change it?
As far as runtime compatibility goes, it is easy to say: ''keep existing behavior''. But what if such behavior is buggy? Well, sometimes even buggy (from one perspective) behavior can be useful (see the [[Arithmetica]] example) and should be preserved. What can you do when you really need to change it?
-
In such case choose to provide [[AlternativeBehavior]]s! There are few [[AlternativeBehavior|ways to do it]] applicable to any [[language]], but the best is when the language underlaying [[NetBeans Runtime Container|runtime container]] provides some support to it.
+
In such case choose to provide [[AlternativeBehavior]]s! There are few [[AlternativeBehavior|ways to do it]] applicable to any [[language]], but the best is when the language or its underlaying [[NetBeans Runtime Container|runtime container]] provides some support to do it smoothly.
-
For example [[XML]] does that by encoding its own version in the document header and than it is clear which parser to use to provide the right [[AlternativeBehavior]]. On the other hand [[Python]] is famous for not identifying its [[language]] version and changing its syntax in a slight incompatible way that started years of the ''2.x'' vs. ''3.0'' battle.
+
For example [[XML]] does that by encoding its own version in the document header and than it is clear which parser to use to provide the right [[AlternativeBehavior|alternative interpretation]]. On the other hand [[Python]] is famous for not identifying its [[language]] version and changing its syntax in a slightly incompatible way that started years of the [[Python]] ''2.x'' vs. ''3.0'' battle.
== What's the Goal? ==
== What's the Goal? ==
-
The goal is to make sure that people can write applications, libraries and plugins in a way that continues to work with multiple [[version]]s of your [[API]]. Nobody wants to publish update as soon as you provide new version of the [[API]]. Things shall just continue to work. This is called [[BackwardCompatibility]].
+
The goal is to make sure that people can write applications, libraries and plugins in a way that continues to work with multiple [[version]]s of your [[API]]. Nobody wants to publish updates as soon as you provide new version of your [[API]]. Downstream projects shall just continue to work. This is called [[BackwardCompatibility]].
[[NetBeans]] IDE supported plugins since 2000 and the most important feature people asked for was ability to write a plugin and make it work with last few versions of [[NetBeans]] simultaneously. Their core business wasn't to write plugins, so they wanted to minimize the time spent doing that. The last thing they wanted was to watch us to release new version of [[NetBeans]] and at that time immediately re-publish an update of their plugin suitable to run with that version.
[[NetBeans]] IDE supported plugins since 2000 and the most important feature people asked for was ability to write a plugin and make it work with last few versions of [[NetBeans]] simultaneously. Their core business wasn't to write plugins, so they wanted to minimize the time spent doing that. The last thing they wanted was to watch us to release new version of [[NetBeans]] and at that time immediately re-publish an update of their plugin suitable to run with that version.
Line 33: Line 35:
The same situation applies to transitive [[dependencies]] between [[libraries]]. If somebody builds their [[API]] on top of yours then there has to be a combination of versions of these two [[API]]s working together. By doing non-[[BackwardCompatible]] changes one complicates finding such [[version]] configuration a lot (remember [[LibraryReExportIsNPComplete]]). This can be witnessed by everyone who tried to assemble five [[libraries]] using few shared pieces into a coherent and working system in an [[environment]] which pays little attention to [[BackwardCompatibility]].
The same situation applies to transitive [[dependencies]] between [[libraries]]. If somebody builds their [[API]] on top of yours then there has to be a combination of versions of these two [[API]]s working together. By doing non-[[BackwardCompatible]] changes one complicates finding such [[version]] configuration a lot (remember [[LibraryReExportIsNPComplete]]). This can be witnessed by everyone who tried to assemble five [[libraries]] using few shared pieces into a coherent and working system in an [[environment]] which pays little attention to [[BackwardCompatibility]].
-
== U Can't Change It All Right Now! ==
+
== U Can't Change It Whole! ==
-
Traditional in-house development scheme suggests [[refactoring]]s to clean up the code and improve its quality. In such model it is fine to do incompatible changes in the code base. If they break some tests, one fixes the tests as well. Ones your [[ContinuousIntegration]] is green again, you can integrate.
+
Traditional in-house development scheme suggests [[refactoring]]s to clean up the code and improve its quality. In such model it is fine to do incompatible changes in the code base. If they break some tests, one fixes the tests as well. Once your [[ContinuousIntegration]] is green again, you can integrate.
-
There is however a hidden assumption: one needs an instant access to all the code and perform the whole change at once. This is far from reality when it comes to [[APIDesign]] and maintaining frameworks. [[API]]s are like [[Star]]s and building them feels like building Universe, not a house. You never know who's watching (e.g. using) your [[API]] and you can't fix all the usages at once.
+
There is however a hidden assumption: one needs an ''instant access to all the code'' and perform the whole change at once. This is far from reality when it comes to [[APIDesign]] and maintaining frameworks. [[API]]s are like [[Star]]s and building them feels like building Universe, not a house. You never know who's watching (e.g. using) your [[API]] and you can't fix all the usages at once.
 +
It makes no sense to claim: ''Everything is OK, my [[ContinuousIntegration]] server is green''! It may be, when you had to fix your tests. If your [[API]] is any popular, there are tons of code similar to your tests out there and there is nobody to fix them as part of your [[refactoring]]. Better to learn and invest in keeping a bit of [[BackwardCompatibility]].
-
As a result of that [[API]] maintainers need to invest in defining and following proper [[lifecycle]] of their [[API]]. On one side, there is a strict request to keep [[BackwardCompatibility]], on the other side there is a [[deprecation]] (as a notice) followed by some period of time to give downstream projects a chance to adopt to the [[API]] change. Both approaches satisfy the major goal and are certainly better than [[Big Bang]] rewrites.
+
As a result of that [[API]] maintainers need to invest in defining and following [[API]]-like [[lifecycle]] of their framework:
 +
* they can either strictly keep [[BackwardCompatibility]] - [[I|my]] favorite, of course
 +
* or at least use [[deprecation]]s as a notice to give downstream projects a chance to adopt to the [[API]] changes.
 +
Both approaches satisfy the major goal of avoiding complete [[Big Bang]] rewrites and provide some interval when downstream code works with multiple [[version]]s. In any case, if you have to modify your tests, there is something fishy with your change!
== Conclusion ==
== Conclusion ==
-
Having [[ContinuousIntegration]] is a must for any project (including ones that provide an [[API]]). However in contrast to building final applications one cannot fit the whole code into the [[ContinuousIntegration]] check. As a result of that one needs to apply also other techniques when judging correctness of a change. Keeping [[BackwardCompatibility]] is a useful initial step that greatly pays off in a long run.
+
Having [[ContinuousIntegration]] is a must for any project (including ones that provide an [[API]]). However in contrast to building final applications one cannot fit the whole code into the [[ContinuousIntegration]] check. As a result of that one needs to apply also other techniques when judging correctness of a change. Keeping [[BackwardCompatibility]] (at least temporarily) is a useful initial step that greatly pays off in a long run.
We have just found another [[paradox]] of [[APIDesign]]: relying only on [[ContinuousIntegration]] is bad!
We have just found another [[paradox]] of [[APIDesign]]: relying only on [[ContinuousIntegration]] is bad!
[[Category:APIDesignPatterns:Anti]]
[[Category:APIDesignPatterns:Anti]]

Current revision

There is nothing wrong on having your own ContinuousIntegration server like Jenkins or Hudson set up. Having it contributes to quality of your project. As Chapter 9 of TheAPIBook explains, tests are the soul of good frameworks and the most important part of good projects.

Contents

U Can't Test This!

People get so used to this comfortable safety net provided by ContinuousIntegration systems that they tend to forget there is also something behind the horizon. On the other hand the quality departments are well aware of that (regardless of the amount of CodeCoverage) there are always remaining errors visible to end users.

The situation is even more complicated when it comes to API Design. A common advice to get better quality is to increase your CodeCoverage or add integration tests to verify the real workflow. Yet the biggest problem when designing an API of a framework is: you can't run the users code! Either you don't have it, or you don't even know about it. When your framework gets really successful, then running all projects that use your API wouldn't even scale.

APIs are like stars: you never know who's watching!

Control the Observables

As such one has to substitute the continuous execution of every piece of code by something else. Rather than testing all the depending code, one has to make sure the externally observable aspects of your framework are stable - e.g. they don't shake like an amoeba in the Amoeba Model. What does that mean?

  1. Don't rename externally visible symbols
  2. Don't change method signatures
  3. Don't change the exposed behavior

Item #1 is important for linkage - e. g. to make sure that various modules (JARs or .so or .js files) built on top of your API find the symbol they expect. Proper linkage is a prerequisite for any other communication. If the users of your API cannot connect to it, the rest is completely useless.

Keeping stable method signatures is important in compiled languages. This is often part of the linkage as well. In case of Java one can use Sigtest to prevent accidental changes of number or types of parameters. In dynamic languages like JavaScript this check has to be done dynamically and thus it shifts the check closer to the Runtime_Aspects_of_APIs.

As far as runtime compatibility goes, it is easy to say: keep existing behavior. But what if such behavior is buggy? Well, sometimes even buggy (from one perspective) behavior can be useful (see the Arithmetica example) and should be preserved. What can you do when you really need to change it?

In such case choose to provide AlternativeBehaviors! There are few ways to do it applicable to any language, but the best is when the language or its underlaying runtime container provides some support to do it smoothly.

For example XML does that by encoding its own version in the document header and than it is clear which parser to use to provide the right alternative interpretation. On the other hand Python is famous for not identifying its language version and changing its syntax in a slightly incompatible way that started years of the Python 2.x vs. 3.0 battle.

What's the Goal?

The goal is to make sure that people can write applications, libraries and plugins in a way that continues to work with multiple versions of your API. Nobody wants to publish updates as soon as you provide new version of your API. Downstream projects shall just continue to work. This is called BackwardCompatibility.

NetBeans IDE supported plugins since 2000 and the most important feature people asked for was ability to write a plugin and make it work with last few versions of NetBeans simultaneously. Their core business wasn't to write plugins, so they wanted to minimize the time spent doing that. The last thing they wanted was to watch us to release new version of NetBeans and at that time immediately re-publish an update of their plugin suitable to run with that version.

The same situation applies to transitive dependencies between libraries. If somebody builds their API on top of yours then there has to be a combination of versions of these two APIs working together. By doing non-BackwardCompatible changes one complicates finding such version configuration a lot (remember LibraryReExportIsNPComplete). This can be witnessed by everyone who tried to assemble five libraries using few shared pieces into a coherent and working system in an environment which pays little attention to BackwardCompatibility.

U Can't Change It Whole!

Traditional in-house development scheme suggests refactorings to clean up the code and improve its quality. In such model it is fine to do incompatible changes in the code base. If they break some tests, one fixes the tests as well. Once your ContinuousIntegration is green again, you can integrate.

There is however a hidden assumption: one needs an instant access to all the code and perform the whole change at once. This is far from reality when it comes to APIDesign and maintaining frameworks. APIs are like Stars and building them feels like building Universe, not a house. You never know who's watching (e.g. using) your API and you can't fix all the usages at once. It makes no sense to claim: Everything is OK, my ContinuousIntegration server is green! It may be, when you had to fix your tests. If your API is any popular, there are tons of code similar to your tests out there and there is nobody to fix them as part of your refactoring. Better to learn and invest in keeping a bit of BackwardCompatibility.

As a result of that API maintainers need to invest in defining and following API-like lifecycle of their framework:

Both approaches satisfy the major goal of avoiding complete Big Bang rewrites and provide some interval when downstream code works with multiple versions. In any case, if you have to modify your tests, there is something fishy with your change!

Conclusion

Having ContinuousIntegration is a must for any project (including ones that provide an API). However in contrast to building final applications one cannot fit the whole code into the ContinuousIntegration check. As a result of that one needs to apply also other techniques when judging correctness of a change. Keeping BackwardCompatibility (at least temporarily) is a useful initial step that greatly pays off in a long run.

We have just found another paradox of APIDesign: relying only on ContinuousIntegration is bad!

Personal tools
buy