InfoQReview

From APIDesign

What was the motivation behind writing the "Practical API Design" book?

The book is based on the notes I've been collecting over the past ten years while designing and maintaining the NetBeans APIs. Also, they're based on the process of transferring the NetBeans API knowledge to the rest of the NetBeans development team. I started to think about converting my notes into a book about five years ago. However, it took a bit of time before I really got started. At various points, other tasks got in the way. Other times I was simply afraid of not being able to finish the book, as well as the fear of being rejected by publishers, etc.

However, in the summer of 2007, I chatted to my wife's cousin at a family party. When I told him of my doubts, he said: "You know what to write about; you know it is an interesting topic; you know you are an expert in it. So... why haven't you started writing yet?".

I was always reminded of this comment whenever my motivation to finish the "Practical API Design" book flagged, which was to be expected in this kind of long running project.

By the way, let me take this opportunity to thank everyone again who helped me finish my book: http://thanks.apidesign.org/

You discussed the Component Injection technique to achieve a

Modular Architecture. What role design concepts like Dependency Injection

(DI), Aspect-Oriented Programming (AOP), and Annotations can play in

software development in general and API design in particular?

I believe that modularizing applications into distinct parts is highly desirable. It's even better when those pieces aren't aware of each other, so that someone can assemble an application from the various independent parts. To allow this kind of assembly from loosely coupled pieces, it is essential to have some form of dependency injection. DI in the Spring style is fine. java.util.ServicesLookup works as well. I dedicated chapter 7 of my book to a comparison of those and other alternatives.

There used to be a time when people went nuts on hearing the phrase "bytecode manipulation". That's no longer the case. People aren't afraid to execute bytecode different to that produced by the Java compiler anymore. However, if you tell them to go directly into the .class file and manipulate the bits, the fear returns. Why is that? I believe this is due to AOP. In fact, AOP can be seen as a high level language for bytecode manipulation. It is not as powerful as making changes to the .class file directly, yet it is approachable to the masses and generally understandable.

This perfectly illustrates the principle that good abstractions make everything more usable. The principle is true in the world of bytecode manipulation as well as in that of API design.

We started making heavy use of annotations during the development of the next version of NetBeans IDE. Basically, we are defining new annotations as a facade over our old XML-based API. During compilation the annotations are processed by our annotation processors. At the end of the processing, the correct (old and complex) XML is generated. I cannot begin to describe how happy we are with this solution! Our APIs are suddenly nicer. Registration information is now part of the Java source code. Code completion automatically works in IDEs. All registrations are verified for correctness during compilation. Based on these experiences, I can only recommend compile-time annotation to all API designers!

You wrote

in the book about how to check the quality of an API library. Can you

elaborate on how the teams can have an on-going assessment and validation

of the quality of software being written?

There is a common wisdom among programmers saying that design cannot be done by committee. However how can we design bigger and bigger systems without designing in teams? Does not that hurt consistency? Yes, partially. Most people seem to be able to keep design consistency when they work alone, however software projects of today and of the future are designed by teams. Keeping consistency in such environment is much harder, but it is possible.

As usually, one has two choices: either detect regressions in quality when they happen, or prevent such problems before they are integrated. I have dedicated full Chapter 16 to this topics. It describes what aspects need to be verified and checked, and how to check them automatically. This can warn us when something goes bad. Also it introduces an "API Review" process which the NetBeans team is using to do review API changes before integration.

How can Agile and lean

software development methodologies like SCRUM, XP and Kanban help a project

team involved in the design and development of API frameworks?

The question is whether agile methodologies help design and development of API frameworks, or whether properly modularizing an application and splitting it into many libraries with APIs makes use of agile methodologies easier? Both is likely true.

There are some good rules in the world of API design. You need to be ready that first version is never perfect. You need to envision your users. Unit test coverage is almost a must. One should be ready for evolution.

Many of the above advices are close to big rules of agile methodologies. As such I think that proper API design and agile methodologies can only strengthen together.

Can you

discuss how your team conducted architecture and design reviews in NetBeans

development project?

We have two modes. Standard review and Fast-Track review. The latter one is used for small, incremental, compatible, non-controversial changes. It is based on "optimistic locking strategy": one prepares the code change diff and attaches it to our issue tracking system. People then have a week to comment or veto the change. If nobody objects, it can be applied then. This works well for single method or class extensions to already existing libraries.

The Standard review is more targeted to reviewing a whole new library or subsystem. It is two round review. First of all we review the concept, and if accepted the result is then reviewed again before integration to main code repository.

All the gory details are available at http://openide.netbeans.org/tutorial/reviews/

What are the best practices and "gotchas" that

software architects should keep in mind when working on creating reusable

component libraries?

It slightly hard to enumerate them all, it took me 400 pages in the Practical API Design book. But let's pick up something inspiring:

What do you think the term API includes? Names of classes? Probably. Names of their fields or methods? Indeed, if they are public or protected. Is that all?

No, it is not. Did you ever think about the files your application reads as an API? About open sockets being an API? Environment variables? Localized messages? Text output that it produces? This all can influence the behaviour of your application/library and also can be observed externally. As such, it is some kind of API. It is important to realize that and keep that in mind.

What is the role of software architects in the

current economic and market conditions?

Tough question. The general answer is to learn to offer what the market needs. But I suspect nobody is really sure what that is. Except...

One of the problems in software development is associated with "big bang" changes, that is, situations where you find that your product has too many bugs and design problems, to the extent that it is unfixable, requiring a complete rewrite. Coming to this point takes a very long time. Usually the evidence needs to grow over several releases. However, there is a point in the lifecycle of most software projects where someone offers to perform a "bigbang" change. Initially, the offer is declined because everyone understands the rewrite will be painful and costly. Yet over the next releases, while the evidence mounts, the team realizes there is no escape. At that point, the whole project is stopped, an incompatible rewrite of a subsystem is forced, and then the other teams adapt to the change. In spite of how realistic the original plans were, this process usually takes much longer than expected. And, at the end we now have a shiny new product which is less buggy than before but handles only about half the functionality of the original. Needless to say, this is not an effective process at all.

The Practical API Design book describes best practices to avoid need for such "big bang" changes. First of all it explains what compatibility is and advocates small, incremental, backward compatible changes, always focusing on future evolution. Such mode does not need distruptive "big bang" changes at all. On the other hand, sometimes they are necessary. That is why the book discusses ways to provide alternative co-existent behaviours and explains how to bridge them together, if necessary.

If we modularize our applications and treat each piece as a library with an API, we can minimize the need for stop-the-world "big bang" rewrites and replace them with continuous distributed improvements. This is may be slightly more expensive at first, but in the long term cost of ownership is definitely lower.

All that said leads to following answer to your original question. Maybe software architects should learn more about proper API design and use it to create more cost effective product teams.

What do you think about the new

features and APIs in the upcoming JDK Version 7 and the dropped features

like Closures?

The #1 thing that Java needs is a standard module system adopted by everyone. Recently I was arguing with a Ruby guy about benefits of API design in Ruby. Originally it looked like "duck typing" is the big advantage, but then we found out that real advantage is that Ruby has gems with all the library dependencies and Java has nothing. This needs to be fixed.

I know there are existing module systems. Btw. one of them runs behind NetBeans, but by no means each library in the world is designed with modularity in mind. This has to be changed, there needs to be support for modularity in the Java language itself. I am even willing to wait next few years for closures, if I can get the modularity in the Java now.

If you have to pick a feature that you like the most in

Java language, what would it be? And the feature that you least like about?

Java is said to have many weaknesses. However I guess one of its weaknesses is also its biggest strength.

Java is said to be verbose. True, you can write shorter programs doing the same in many other languages. So Java is verbose. However the benefit is that people can not only write the program, but also read it. Java programs can be read and understood. Moreover this can be done on a printed page, without "go declaration" and "code completion" in an IDE.

I guess I am not satisfied how verbose Java is. But I am really glad that it is readable.

Lastly, other than your own book, do you have an IT and a non-IT book

recommendation for our readers?