APIFest08:Report

From APIDesign

Would you win?

The celebration of 10 years of NetBeans releases is in progress and that is why it is also time to celebrate all those who contributed to the NetBeans architecture, and design practices which make NetBeans platform the most stable Java rich client application framework.

The best way to have fun is to play games. The best way to learn is to play as well. The best way to teach is to organize such game. When me and CZJUG lead Jakub Podlešák realized that, we decided to organize similar game as described in Chapter 17 of TheAPIBook. It is a game to teach, learn and have fun while designing architecture and APIs and practising BackwardCompatibility principles in software interfaces.

There were four base rounds (1, 2, 3, 4) in the APIFest08. We started with fourteen participants. However as the competition advanced further and further, only those who were really eager to win managed to finish all four design tasks. Still, six solutions advanced to the final judgment day/week.

The judgment round is in fact the start of the real competition. However it is also much more difficult than the first four rounds. Instead of looking into own solution, one needs to dig into solutions provided by others and seek for left compatibility problems which is different and time consuming. Still I am glad that two participants, Petr Šmíd and Jan Žák did not give up and sent me their hacks showing BackwardCompatibility evolution problems in many solutions.

Problems in Solution 04

Both guys managed to find some problem in solution 04. Actually there were multiple problems as demonstrated by Petr, however the most obvious one was source incompatibility caused by the addition of new method into a subclassable interface as shown by Jan and me (my solutions are just illustrative, I am not one of the competitors). The lesson to take is that if one is seeking for 100% BackwardCompatibility one needs to prevent modifications to classes that can be subclassed.

Problems in Solution 06

The author of solution six did few compatibility mistakes. First of all, as Jan demonstrates, the once public field Convertor.one disappeared in subsequent revision, which is clearly incompatible for clients using it.

I found an interesting source incompatibility caused by adding overloaded version of constructor. If there are two methods with the same name and number of parameters, but different types, one can try to call them with null and the compiler will refuse the compilation:

against-solution06/test/apifest/CurrencyTest.java:18: 
reference to Convertor is ambiguous, both method
  Convertor(BigDecimal,java.util.Currency,java.util.Currency) 
and method
  Convertor(Convertor.RateProvider,java.util.Currency,java.util.Currency)
match
        Convertor c = new Convertor(null, cur1, cur2);

Obviously the code compiled before the second constructor was introduced. It failed with later version. This is not that big incompatibility in practise, but it is enough to loose in APIFest08.

Petr exploited another functional incompatibility. The text of a thrown exception changed between two releases. As such it is enough to generate the exception and compare the text. A thing to remember? There are different kinds of APIs, text messages being one of them. For purpose of APIFest08 all of them are equal.

Problems in Solution 07

When I checked the state of all solutions after task 3, I found a perfect way to win. The solution 07 is an example of absolute API stability. The author of this solution managed to guess my future requirements and he changed nothing in the original API in first three rounds! Indeed, it is impossible to find an inconsistency in two versions of an API if they are identical. That is why my own goal for task 4 was to force the author of solution 07 to make some changes. I succeeded, the API version 4 is different than the previous versions, however as the solution 07 is based on a variation of request/response pattern, it still seems unbreakable.

And actually nobody managed to question BackwardCompatibility of the solution. But Jan Žák noticed an important problem. To quote his email: Sometimes - like in the Task2 - the author even added nothing to the API and implemented the requested feature directly in the test case. I'm not convinced, that this approach is strictly fair.. When I look at the quest of the second round and see the complex subclassing classing code provided by solution 07...

/** Merge all currency rates of convertor 1 with convertor 2.
* Implement this using your API, preferably this method just delegates
* into some API method which does the actual work, without requiring
* API clients to code anything complex.
*/
public static Convertor merge( final Convertor one, final Convertor two ) {
  // quite a complex code with subclassing
}

...I have to agree. All other solutions got this correctly. As such I am giving Jan one point for finding a problem in solution 07 - finding that it does not meet the requirements.

Problems in Solution 11

This is the solution designed by Jan Žák and it seemed completely unbreakable for me. However, Petr had other opinion. He realized that when a code like:

private String k1, k2;
public Constructor(String k1, String k2) {
  this.k1 = k1; this.k2 = k2;
}
public boolean canConvert(String s1, String s2) {
  if (k1.equals(s1) && k2.equals(s2)) {
    return true;
  }
  if (k2.equals(s1) && k1.equals(s2)) {
    return true;
  }
  return false;
}

is modified to be more general:

private List<String> keys = new ArrayList<String>();
public Constructor(String k1, String k2) {
  keys.add(k1); keys.add(k2);
}
public boolean canConvert(String s1, String s2) {
  if (keys.contains(s1) && keys.contains(s2)) {
    return true;
  }
  return false;
}

The result is not the same. Petr managed to exploit this with his test. Congratulation, this is very clever!

This shows how carefully we need to evolve APIs of our libraries. Even slight change in Runtime Aspects of APIs can lead to incompatibilities.

Problems in Solution 13

This solution suffered with source as well functional compatibility problems. Jan managed to exploit both, Petr concentrated on the source incompatibility and I on the functional one.

The thing to remember is that as soon as you implement equals, you need to stick with its implementation. Changing it can be exploited either by careful hackers or by everyone who puts the objects into List or other collection classes.

Problems in Solution 14

This is the solution provided by Petr Šmíd. As Petr is one of the last standing hackers, his solution is not easy to break. I managed to exploit change in implementation classes by using obj.getClass().getInterfaces() check. This does not require any additional permissions, everyone can do this, however it is not really fair. If this was the only way to break Petr's solution, I would still consider his work unbreakable.

However Jan Žák created more intrinsic exploit that generates ArrayStoreException, if executed with newer version of the API. The reason for that is that the task4 version wraps all CurrencyRate with own TimeLimitedCurrencyRate objects. As these objects are not used only internally, but also exposed to clients with a getter getCurrencyRates, it is enough to pass in own objects and check that the returned values are of the same type. Excellent work Jan!

The lesson to remember is to Separate APIs for Clients and Providers as advocated in Chapter 8.

Results

All solutions are flawed in one or other way. As such nobody gets ten points for creating an unbreakable solution. The only points awarded are only for exploits:

Petr Šmíd

Petr is one of the winners of APIFest08.

Petr broke solutions 4, 6, 11, 13.

Jan Žák

Jan broke solutions 4, 6, 13, 14 and challenged correctness of solution 7

Celebration

Both Petr and Jan visited in Sun's Prague office on Thursday 30, 2008 and got their copy of TheAPIBook. Jan is the absolute winner of APIFest08 - he broke four solutions, plus correctly challenged another one. However, I'd like to congratulate you all, dear participants. I enjoyed the game and I hope you are not sorry for participating either.

Thanks, Jaroslav Tulach!

<comments/>