EnforcingProperUsage

From APIDesign

Current revision

API design is about communication between the API designer and its API users. Good API can be recognized by quality of the communication between the designer and the users. Does the vision of proper API usage (in designer's head) recreates properly in heads of the users? If not, your API is probably suffering from a poor clarity. Do you users hack around your API - e.g. use things clearly stated as not being part of the API? Can you prevent them to do so? Those are questions this essay is about to analyze.

Striving for Clarity

If an API reaches clarity - e.g. everything in the API has a single meaning - then there is no reason to enforce anything - users either use something (method, class) or not. If they use something (and as everything - methods, classes - in the API has a single meaning), they have to (or at least chances are very high) be using it correctly. That is the reason why it is important to follow the API design patterns for Clarity and focus on ClarityOfAccessModifiers, ClarityOfTypes and separate API from a service provider interfaces.

Trying to Hack

On the other hand, when striving for clarity, it often happens that the API is more restrictive than its users might want it to be. What the users do then? They hack!

Of course good API users shouldn't hack - they should contribute! As described in Chapter 16 (Teamwork), proper API designing project should have guidelines for accepting patches (like netbeans:APIReviews) and should be ready to accept contributions from its users.

However it is way easier to hack than to contribute an API patch! Some projects (for example OpenJDK comes to my mind) are hostile to random community contributions and getting simple patch in is almost impossible (as my own experience with small enhancements to ReferenceQueue and ServiceLoader shows), however even project open to contribution like NetBeans requires one to justify the change, write a test, document it and make sure it is BackwardCompatible. That is still quite a lot of work.

No wonder users resolve to hacking. How can API designers prevent that?

Engineering solution

API designers are primarily engineers and thus it is no surprise they seek a technical solution. Using language encapsulation constructs like private or package private in Java to hide something from the eyes of API users works as it makes such elements (types or methods or fields) inaccessible (unless one resorts to reflection). Even languages without access modifiers allow ways to declare things privately - for example in JavaScript one can use scope to make objects inaccessible for external world.

However sometimes one structures implementation code to split multiple packages. Then some implementation classes need to be publicly visible. With the help of FriendPackages pattern, one can still forbid users of the API to hook into them, but not everyone knows the FriendPackages pattern and as such it is common to find Javadoc like: Implementation only, don't use! Guess what your hacking ready API users will do?

Enough to mention the famous JDK's sun.misc.Unsafe & co. You can put warnings all over the documention, you can even modify Javac to print warnings that the API is going to disappear. No chance, if the non-public API is useful, hackers will continue to use it.

To prevent that one can play tricks with ClassLoaders. NetBeans Runtime Container, OSGi and Jigsaw allow one to differentiate between public and private packages of your JAR. The runtime containers then enforce the rules by loading user classes with different ClassLoader which does not see the classes in private packages. This prevents direct linkage against such classes.

Still not enough: the shameless hackers are not afraid to use reflection (and taking away the right to use reflection would seriously hurt most of existing libraries)!

Legal solution

So the technical solutions are limited. For many years I tried to prevent hacking of NetBeans internals and I have to admit: unless you take away the reflective permission from the code, there is no engineering way to do it.

However recently I started to work on Truffle API and I noticed interesting thing: the Truffle Hg repository is using two licenses: GPL and GPLwithClassPathException. The packages that are supposed to contain API are licensed under GPLwithClassPathException. The implementation parts of the project (like tests) are kept under GPL.

We all know WhyGPL is useful - it is viral. It is also clear why GPLwithClassPathException is useful - as it is not viral, it increases adoption of your API. Mixing them in a single project like Truffle makes sense. It separates the public parts from the implementation only ones.

However we can go even further: we can use these licenses to prevent hacks! If there is a class in a non-public package (like sun.misc.Unsafe) just license it under GPL. If there are package private classes in otherwise public API package - give them GPL license as well!

No API user should be linking against implementation classes. Releasing them under GPL license will effectively prevent that (unless one accepts the virality of the license and releases own hacks to public under GPL license as well). Not only this prevents normal (e.g. non-GPL users) from a compilation against such non-public classes, but it effectively prevents reflective access as well. Usually the GPL is interpreted as being linkage-viral, but the text talks in general about derived work and it is clear that calling hidden implementation methods of classes is derived from a knowledge of these classes - e.g. GPL remains viral.

Summary

Labelling classes considered public API with GPLwithClassPathException and labelling all others (package private, accidentally visible, etc.) with GPL is a cute way to give hackers a freedom to experiment, but prevent the hacks to get to production (non-GPL) systems.

Re-licensing sun.misc.Unsafe & co. from GPLwithClassPathException to GPL seems to me like the best thing OpenJDK team should do. Unless it fears such change might break the unspoken contract with the Java community at large: Hacking around JDK internals is good and nobody should use lawyers to prevent that!