Proximity

From APIDesign

Often there is more than one type of "users" of an API. As 3SidesToEveryAPI explains, there is always at least the API author point of view and API user point of view. But from time to time there may be even more parties involved in design and consumption of some API. Depending on how closely related to the API specification and implementation those are, we can group them into API proximity categories.

Let's ignore the most distant group of developers for now. Those use ClientAPI - e.g. only make calls into existing API objects, but don't implement them (to provide their implementations to others). Let's only remember that there is always much more users of an API and let's call their amount many in the following paragraphs.

None to Many

Some libraries, let's name them simple libraries, have just one implementation. The designer that defines what the API should do also implements it. Obviously, the proximity between the API and the provider of the implementation is almost zero - it is the same person or the same team.

Simple libraries are usually self contained. They combine the API specification together with the implementation. There is no reason, no way to plug into the library and change the way it behaves (for other clients of its API). Examples of such simple library include most of java.util package (with classes like ArrayList or Collections)), or other utility classes like Math class.

Of course, there may be some attempts to re-implement these simple libraries (like GNU Classpath or Harmony show), but those are usually licensing driven efforts, not purely technical ones.

Especially when the simple library is available as open source. It is much easier to contribute back than to fork. If there is some flaw in the library, it seems easier to create an API Patch and donate it back to the simple library owners. In general, there is little to no reason to re-implement simple library APIs.

One to Many

The simplest kind of library with a non-trivial proximity between the author of the API specification and author of the implementation is so-called vendor library. This kind of library is based on standardized specification. Usually multiple vendors with similar functionality sit down and agree on a specification. Each of them then provides their own different implementation. This was the case for EJB, JAX-RS, OSGi and possibly many others. On the other hand, it is almost always the case that there is only a single implementation of such specification in a running system.

The API for XML processing in Java (with entry points in DocumentBuilderFactory and SAXParserFactory) is one of the most well-known examples of a vendor library. Especially in the beginning of 21st century it was a common hobby among many programmers to write own implementation of an XML parser. Often the implementation was hidden behind the facade of SAX or DOM API. Such parsers also package their copy of the API (as providers of vendor library are supposed to do).

The close proximity of the vendor of the implementation and the API (as they are packaged together) leads to so called final interface design. This kind of design caused heavy nightmares as soon as the XML parsing API become part of JDK and as soon DOM2 was evolved into DOM3. These problems lead me to conclusion that final interface is a total anti-pattern (also expressed in chapter 7 of TheAPIBook) and we (me and others in the NetBeans project) invested a lot in inventing a better alternative. Other people however had different opinion and invested a bunch of energy to find out under which conditions final interface is an acceptable pattern.

After visiting OSGiCon in 2012 and having few chats with folks like Peter Kriens I can confirm I see the value of vendor library proximity now. I can understand OSGi better. I could almost say that OSGi has been designed to make final interface a good design pattern. However one has to keep in mind that this API design style works well only with One to Many (and possibly Few to Many) proximity. As soon as Many to Many proximity is needed, even OSGi needs to choose more flexible design.

Few to Many

Semantic versioning is a natural interpretation of dependencies as described in manifesto at semver.org. It is also a base for designing libraries with few to many proximity between their ClientAPI and ProviderAPI sides.

This proximity is best explained by another semantic versioning proposal defined by OSGi guys. It is using range dependencies to distinguish between ClientAPI (always uses major dependencies range like 1.x to 2.0) and ProviderAPI (which is supposed to always depend on 1.x to 1.(x+1) micro range). This style of versioning opens the vendor library proximity category up to a bit more distant providers. If the OSGi semantic versioning is obeyed, one can use Final interface as a contract between clients of your API and providers of your API. It is expected that in case there is an incompatible change in the Final interface, you bump up the micro version.

Althrough common design practise in OSGi world the drawbacks are obvious. Every change into the API renders all existing providers invalid (as the new version of library is out of their micro dependency range). With every such change all providers need to be rewritten. Should there be two providers written against different version of the library they are unlikely to co-exist together (because of versioning, otherwise they very likely could as commonly the case in modular libraries).

Due to these limitations I call this library style a few to many proximity. It is possible to have multiple providers, but as they use micro range, there can never be as much providers as clients. In case you seek real many to many proximity consider using modular library design.

Many to Many

In some cases one needs to have multiple different implementations of some API being used at once. These implementations are being written in distributed development environment, they may not be written against the same version of the library at all. The proximity between the API and the implementations is very weak. As a result trying to mimic the principles used while designing vendor library will not work and even semantic versioning may be too restricting.

In case of modular library the proximity of the API designer and the implementers is almost the same as the proximity of the API designer and the users of the API. This requires the API vendor to prepare evolution plan for both - the ClientAPI as well as ProviderAPI.

As my discussion at OSGiCon revealed, people facing this situation realize the difference between the Many to Many and One to Many and Few to Many cases. However, without having experience with the Many to Many case, they fallback to most obvious solution: Use of abstract classes (or these days interfaces with DefaultMethods). While extending abstract classes with new methods is definitely more compatible than extending interfaces, it can never be 100% safe. The safest solution is to separate API for clients and providers (taken to extreme, while demonstrating all benefits, in the extensible visitor case).

Weak proximity changes everything. An API designer can simplify its life by claiming all providers are close. However sometimes there is just no way around than to admit they are distant. Then the best thing to do is to accept the design style suitable for modular library.

A Final Question

It is clear that the Many to Many case is a superset of One to Many case as well as Few to Many one. It does not impose any significant complexity on the users (both clients as well as providers). Given that I'd like to ask: Why not use the Many to Many style all the time and treat your users and providers fair?