DefaultMethods

From APIDesign

Revision as of 06:03, 28 September 2020

DefaultMethods is a new feature of JDK8. It allows one to specify (default) implementation of Java interface method.

Adding methods with some implementation breaks the clear separation between Java interface (used to specify a contract only) and class (provides some implementation). Many members of the Java community were crying for having a way to add methods into already published interface in a backward compatible way for ages. Of course, as usual in Java, only when the JDK team felt the need itself (because of adding a lot of new Lambda methods into Collection & co. classes), it listened to the general request.

On the other hand, there were people claiming that DefaultMethods are bad - that an interface should be a code-less specification and the change would have consequences. And yes, it does. This page describes some of them.

Useful?

From one point of view, it is a useful concept. Especially if one paints himself into a corner by not following advices of TheAPIBook when designing the first version of the API. If one mixes client and provider API into one type - like List, then one gets into trouble. On one side people call into the type (e.g. use it as ClientAPI). On the other hand, there are people who implement it - e.g. use the List type as a ProviderAPI. They create standard classes like ArrayList, but also numerous custom implementations written by developers around the globe.

Later an evolution of the API may require one to add new methods into the ClientAPI side of the contract. In such situation one is going to welcome default methods as a nice way to get out of the corner. By adding default methods one can enhance the ClientAPI part of the interface, while keeping (most of) the compatibility for those who implement the interface.

However the concept of default methods also comes with drawbacks.

By-Passing Your Interface

When I was implementing netbeans:Html4Java API, I had to create an observable list - so I did it and created JSONList. I've carefully overwritten each public method that modified the list state and called a change notification handler. What could go wrong?

@Override
public boolean add(T e) {
  boolean ret = super.add(e);
  notifyChange();
  return ret;
}

I thought I did everything correctly, as all the methods of the List interface were properly overwritten. But (as you can probably guess) a problem appeared with DefaultMethods.

The Sorting Problem

If you write this code:

People p = new People();
List<String> names = p.getNicknames(); // returns the JSONList implementation
Collections.sort(names);

it works properly on JDK7, but it gets broken on JDK8. The code needs to compile on JDK7, so no DefaultMethods (introduced in JDK8) are called. In spite of that the code behaves differently on JDK8 and the notification change isn't delivered!

The problem is that the static sort method in Collections does the sorting by itself in JDK7, but in JDK8 it delegates to List.sort:

public static <T extends Comparable<? super T>> void sort(List<T> list) {
    list.sort(null);
}

In addition to that there is an optimized implementation of sort in ArrayList that bypasses all existing (at time of JDK7) methods and sorts directly the internal array:

@Override
@SuppressWarnings("unchecked")
public void sort(Comparator<? super E> c) {
    final int expectedModCount = modCount;
    Arrays.sort((E[]) elementData, 0, size, c);
    if (modCount != expectedModCount) {
        throw new ConcurrentModificationException();
    }
    modCount++;
}

as such the JSONList can be sorted without notifying about changes. None of the methods known in JDK7 is called and yet the content of the array is altered. This is a similar problem as the one with delegation and adding new methods into existing types as discussed in Chapter 8 of TheAPIBook.

The solution in this case is to overwrite the sort method, but this is the kind of problems we can expect with DefaultMethods - interface no longer represents a snapshot of a protocol at a time, it evolves (which is what we wanted), but also with all the (sometimes unwanted) consequences.

Increasing Fuzziness

The problem with default methods is that they increase fuzziness. interface is no longer pure it used to be. Java interface with default methods is fuzzier than pure interface (without any implementation). interface with some (default) implementation no longer clearly defines a contract - e.g. its use for ProviderAPI is no longer as sharp as it could be.

When somebody implements such interface, one can choose to implement the default methods or ignore them. That increases fuzziness. Moreover when we look at the concept from the evolution point of view, it may get even fuzzier.

Envision a pure interface defined in an initial version. Imagine the interface being implemented by various 3rd party developers. Later the interface gets enriched with few default methods. When you look at an implementation of the interface which doesn't override the default methods: What can you conclude?

Either they don't implement the default methods as they weren't existing when the implementations were written - e.g. they compiled against original version of the API. Or they don't implement the default methods because they are happy with their default implementation. That may make (and does make as the Collections.sort example shows) a difference! But it is impossible to separate these two cases apart!

As such I am suggesting to not use default methods when designing API. For example when designing a visitor, it is better to get along without them. A good API designed with cluelessness in mind, shall avoid default methods!

Can you disagree?

I also believe default methods can be useful, so I fully understand if you are hesitating to accept the above arguments. However the conclusions are inevitable from the following chain of thoughts:

• Do you agree that mixing client and provider APIs is causing troubles?
• Do you agree that clarity is an important aspect of an API?
• Are you willing to search for ClarityOfTypes in the API that you design?

Then it is clear: Use final classes to represent ClientAPI types and use pure interface to represent ProviderAPI. There is no place for default methods in this ideal setup... but of course, we are humans, so we always find excuses for using/doing what we like, right?

Even Binary Incompatible

DefaultMethods are useful when one desperately needs to add a method into an existing interface. There are reasons to strive to avoid doing that, however sometimes there is just no way around doing so. Then it comes handy that DefaultMethods are available since JDK8. However, use with care. It has recently been demonstrated that adding DefaultMethods can compromise BinaryCompatibility.

Binary Incompatibility in CharSequence

Recently Emilian Bold asked me to participate in a tweeting about binary incompatibility caused by adding new CharSequence.isEmpty default method in JDK15. An interesting case. To summarize: Following code compiles and runs on JDK8 to JDK14:

public interface ArrayLike {
    int length();
 
    default boolean isEmpty() {
        return length() == 0;
    }
}
 
final class CharArrayLike implements CharSequence, ArrayLike {
    private final char[] chars;
 
    CharArrayLike(char... chars) {
        this.chars = chars;
    }
 
    @Override
    public int length() {
        return chars.length;
    }
 
    @Override
    public char charAt(int index) {
        return chars[index];
    }
 
    @Override
    public CharSequence subSequence(int start, int end) {
        return new String(chars, start, end);
    }
 
    public static void main(String... args) {
        boolean empty = new CharArrayLike('E', 'r', 'r', 'o', 'r', '!').isEmpty();
        System.err.println("not empty: " + empty);
    }    
}

However trying to run the code on JDK15, gives a linkage error:

$ /jdk-14/bin/javac ArrayLike.java
$ /jdk-14/bin/java CharArrayLike 
not empty: false
$ /jdk-15/bin/java CharArrayLike 
Exception in thread "main" java.lang.IncompatibleClassChangeError: Conflicting default methods: java/lang/CharSequence.isEmpty ArrayLike.isEmpty
        at CharArrayLike.isEmpty(ArrayLike.java)
        at CharArrayLike.main(ArrayLike.java:32)

Why it fails? Since JDK15 there is the new CharSequence.isEmpty() default method. As such, when the JVM processes the CharArrayLike class it doesn't know whether to dispatch to the ArrayLike.isEmpty() method or to the newly added CharSequence.isEmpty() method.

Dispatch Directly

It is unfortunate that a code that used to run on older JDKs, doesn't run and leads to a linkage error on JDK15. The fix is however simple - just override the method and dispatch manually:

final class CharArrayLike implements CharSequence, ArrayLike {
    @Override
    public boolean isEmpty() {
        return ArrayLike.super.isEmpty();
    }
}

Such fix has to be made across all affected places. Then all applications using affected libraries need to update to their latest versions. Only then such applications can run on JDK15. While the fix is simple, the organizational effort no longer sounds as simple, right? Clearly DefaultMethods aren't a heaven sent solution, they have some cost! If you are starting to write a new API and you want to avoid your customers paying that cost, then separate API from SPI and don't mix the two concepts (especially in types that are frequently subclassed/implemented by users of your API).

Why now? Why not then?

It would be possible to end the story here, but let's go on. The CharSequence.isEmpty() case is not the first time a CharSequence interface was enhanced with additional DefaultMethods. In JDK8 the interface got two new methods:

public default IntStream chars();
public default IntStream codePoints();

No problem happened back then. No clash appeared. How come? Was it just a pure luck?

No! It wasn't a luck. The JDK8 addition of chars() and codePoints() was BinaryCompatible! Because adding new methods into an existing type cannot create a clash, if one of the arguments, or the return type is a newly introduced type!

The IntStream return type of both methods was also added in JDK8 - as such the methods couldn't clash with any code compiled on JDK7 (a method invocation encodes name of methods and types of parameters as well as return type in the bytecode). As such it was impossible on JDK7 to compile method with exactly the same signature as chars() or codePoints().

Moral of the story? When trying to add DefaultMethods into existing interfaces in a 100% BinaryCompatible way, also add a new type! Adding simple signatures like boolean isEmpty() may clash. Adding complex signature like IntStream codePoints() cannot!

Apply the Knowledge

Now, when we have seen a way to enhance interfaces with DefaultMethods compatibly, let's continue the mental experiment and apply the know-how to the CharSequence.isEmpty() case. Let's be super modern and let's use record feature (not really necessary, any final class will do as well):

interface CharSequence {
  /** @since 15 */
  record Status(boolean empty);
 
  /** @since 15 */
  default Status is() {
     return Status(length() == 0);
  }
}

That'd be completely BinaryCompatible evolution. People would have to write seq.is().empty() instead of simpler seq.isEmpty(). But does that matter? Not from a performance perspective - every good JIT compiler eliminates any overhead. Then it is just about the will to be 100% BinaryCompatible or the lack of it.

Future Benefits

Moreover this kind of evolution "mounts" an open space of ClientAPI on the CharSequence interface. E.g. in future versions it will be easy to expand the Status class with new attributes. For example in JDK23 one could add:

interface CharSequence {
  record Status(boolean empty, boolean large, boolean asciiOnly);
 
  default Status is() {
     return Status(length() == 0, length() > 255, codePoints().allMatch((ch) -> ch <= 127);
  }
}

That'd be a BinaryCompatible change (if one follows RecordEvolution rules properly). And so on, so on: New attributes could be added to the Status over time. Adding DefaultMethods to interfaces in a 100% BinaryCompatible way is possible at the end!

Reactions to some input obtained after publishing this essay are available here...

Dušan can: Default Listener Methods

Recently I tried to apply the DefaultMethods API Design warning to work done by my colleagues and they (obviously) objected. That's OK, they object all the time, but the worst thing this time was: they were right!

JavaBean Listener and Adapter

Do you remember the classical JavaBean pattern called listener and its associated adapter? Imagine for example MouseListener and MouseAdapter. The pair of types used to be created to save users from always implementing all the listener methods and offer them to save some typing by rather using the adapter. Turns out DefaultMethods allow us to merge these two types into one:

public interface MouseListener extends EventListener {
    public default void mouseClicked(MouseEvent e) {}
    public default void mousePressed(MouseEvent e) {}
    public default void mouseReleased(MouseEvent e) {}
    public default void mouseEntered(MouseEvent e) {}
    public default void mouseExited(MouseEvent e) {}
}

Such listener type is both a listener as well as an adaptor and there is nothing wrong with that!

Why?

Why does the philippic against using DefaultMethods in an API doesn't apply? The code that runs after one of the listener methods isn't (unless an unexpected exception happens or unless it never returns or runs too long - e.g. all the usual warnings when calling foreign code are applicable) influenced by the code in the method - e.g. it shouldn't matter what the method does and thus adding new method with default empty implementation seems to be OK.

There is a necessary condition for that: such methods have to return void. If the methods returned anything observable, then we'd be back in the DefaultMethods situation. If the default body would look like this:

/** @return true to "consume" the event */
public default boolean mouseExited(MouseEvent e) {
    return false;
}

then all the arguments against using these DefaultMethods would apply again. It would again be impossible to distinguish between methods having default implementation or DefaultMethods overridden with intention to keep the default implementation.

Up Side Down

Thanks Dušane for pointing the listener case out! As mentioned in TheAPIBook, slight change in the initial situation may turn the final advice up side down. Thus I gladly (have to) admit: using DefaultMethods in listener JavaBean pattern is perfectly fine.