InfoQReview

From APIDesign

What was the motivation behind writing the "Practical API Design" book?

The book is based on the notes I've been collecting over the past ten years while designing and maintaining the NetBeans APIs. Also, they're based on the process of transferring the NetBeans API knowledge to the rest of the NetBeans development team. I started to think about converting my notes into a book about five years ago. However, it took a bit of time before I really got started. At various points, other tasks got in the way. Other times I was simply afraid of not being able to finish the book, as well as the fear of being rejected by publishers, etc.

However, in the summer of 2007, I chatted to my wife's cousin at a family party. When I told him of my doubts, he said: "You know what to write about; you know it is an interesting topic; you know you are an expert in it. So... why haven't you started writing yet?".

I was always reminded of this comment whenever my motivation to finish the "Practical API Design" book flagged, which was to be expected in this kind of long running project.

By the way, let me take this opportunity to thank everyone again who helped me finish my book: http://thanks.apidesign.org/

You discussed the Component Injection technique to achieve a

Modular Architecture. What role design concepts like Dependency Injection

(DI), Aspect-Oriented Programming (AOP), and Annotations can play in

software development in general and API design in particular?

I believe that modularizing applications into distinct parts is highly desirable. It's even better when those pieces aren't aware of each other, so that someone can assemble an application from the various independent parts. To allow this kind of assembly from loosely coupled pieces, it is essential to have some form of dependency injection. DI in the Spring style is fine. java.util.ServicesLookup works as well. I dedicated chapter 7 of my book to a comparison of those and other alternatives.

There used to be a time when people went nuts on hearing the phrase "bytecode manipulation". That's no longer the case. People aren't afraid to execute bytecode different to that produced by the Java compiler anymore. However, if you tell them to go directly into the .class file and manipulate the bits, the fear returns. Why is that? I believe this is due to AOP. In fact, AOP can be seen as a high level language for bytecode manipulation. It is not as powerful as making changes to the .class file directly, yet it is approachable to the masses and generally understandable.

This perfectly illustrates the principle that good abstractions make everything more usable. The principle is true in the world of bytecode manipulation as well as in that of API design.

We started making heavy use of annotations during the development of the next version of NetBeans IDE. Basically, we are defining new annotations as a facade over our old XML-based API. During compilation the annotations are processed by our annotation processors. At the end of the processing, the correct (old and complex) XML is generated. I cannot begin to describe how happy we are with this solution! Our APIs are suddenly nicer. Registration information is now part of the Java source code. Code completion automatically works in IDEs. All registrations are verified for correctness during compilation. Based on these experiences, I can only recommend compile-time annotation to all API designers!

You wrote

in the book about how to check the quality of an API library. Can you

elaborate on how the teams can have an on-going assessment and validation

of the quality of software being written?

Common wisdom among programmers is that design cannot be done by committee. However, how can we design systems of increasing size without designing in teams? Doesn't working in a team hurt consistency? Yes, partly. Most people seem to be able to keep design consistency when they work alone, although software projects of today and of the future are designed in teams. Keeping consistency in such environments is much harder, but is certainly possible.

As usual, one has two choices: either detect regressions in quality when they happen or prevent such problems from occurring before they are integrated. I have dedicated the complete chapter 16 to these topics. There I describe what aspects need to be verified and checked and how to check them automatically. In this way, we can be prepared for when things start going bad. Also, the chapter introduces an "API Review" process, which the NetBeans team is following to review API changes before integration.

How can Agile and lean

software development methodologies like SCRUM, XP and Kanban help a project

team involved in the design and development of API frameworks?

The question is whether agile methodologies help design and development of API frameworks... or whether properly modularizing an application and splitting it into many libraries with APIs simplifies the use of agile methodologies? Both are probably true.

There are several sound rules in the world of API design. For example, you need to be aware at the outset that the first version of your API is not going to be perfect. Also, you need to envision who your users are. Two other basic principles are that unit test coverage is almost a must and that you should always design your API in such a way that it is ready to be evolved further.

Many of these pieces of advice are close to being the governing rules of agile methodologies as well. As such, I think that proper API design and agile methodologies can only strengthen each other.

Can you

discuss how your team conducted architecture and design reviews in NetBeans

development project?

We have two modes: the standard review and the fast-track review. The latter is used for small, incremental, compatible, and non-controversial changes. It is based on the "optimistic locking strategy": you prepare the code change diff and attach it to the issue tracking system. People then have a week to comment or veto the change. If nobody objects in that time frame, the change can immediately be applied. This works well for single method changes or class extensions to already existing libraries.

The standard review is targeted at reviewing a whole new library or subsystem. It is a two-round review. Firstly, we review the concept. Then, if the concept is accepted, the result is reviewed again before integration into the main code repository.

All the gory details are available at http://openide.netbeans.org/tutorial/reviews/

What are the best practices and "gotchas" that

software architects should keep in mind when working on creating reusable

component libraries?

It is difficult to enumerate them all... it took me 400 pages in the Practical API Design book. :-) But let's pick up on something interesting: "What do you think is encompassed by the letters API?" The names of classes? Probably. The names of their fields or methods? Yes, if they are public or protected. But is that all?

No, it isn't. Did you ever think of the files your application reads as being an API? And aren't the open sockets also API? What about the environment variables? Localized messages? How about the text output produced by your code? All these can influence the behavior of your application or library. They can also be observed and used externally. As such, it is therefore some kind of API. It is very important to realize that and to always keep it in mind.

What is the role of software architects in the

current economic and market conditions?

Tough question. The general answer is to learn to offer what the market needs. But I suspect nobody is really sure what that is. Except...

One of the problems in software development is associated with "big bang" changes, that is, situations where you find that your product has too many bugs and design problems, to the extent that it is unfixable, requiring a complete rewrite. Coming to this point takes a very long time. Usually the evidence needs to grow over several releases. However, there is a point in the lifecycle of most software projects where someone offers to perform a "bigbang" change. Initially, the offer is declined because everyone understands the rewrite will be painful and costly. Yet over the next releases, while the evidence mounts, the team realizes there is no escape. At that point, the whole project is stopped, an incompatible rewrite of a subsystem is forced, and then the other teams adapt to the change. In spite of how realistic the original plans were, this process usually takes much longer than expected. And, at the end we now have a shiny new product which is less buggy than before but handles only about half the functionality of the original. Needless to say, this is not an effective process at all.

The Practical API Design book describes best practices to avoid need for such "big bang" changes. First of all it explains what compatibility is and advocates small, incremental, backward compatible changes, always focusing on future evolution. Such mode does not need distruptive "big bang" changes at all. On the other hand, sometimes they are necessary. That is why the book discusses ways to provide alternative co-existent behaviours and explains how to bridge them together, if necessary.

If we modularize our applications and treat each piece as a library with an API, we can minimize the need for stop-the-world "big bang" rewrites and replace them with continuous distributed improvements. This is may be slightly more expensive at first, but in the long term cost of ownership is definitely lower.

All that said leads to following answer to your original question. Maybe software architects should learn more about proper API design and use it to create more cost effective product teams.

What do you think about the new

features and APIs in the upcoming JDK Version 7 and the dropped features

like Closures?

The #1 thing that Java needs is a standard module system adopted by everyone. Recently I was arguing with a Ruby guy about benefits of API design in Ruby. Originally it looked like "duck typing" is the big advantage, but then we found out that real advantage is that Ruby has gems with all the library dependencies and Java has nothing. This needs to be fixed.

I know there are existing module systems. Btw. one of them runs behind NetBeans, but by no means each library in the world is designed with modularity in mind. This has to be changed, there needs to be support for modularity in the Java language itself. I am even willing to wait next few years for closures, if I can get the modularity in the Java now.

If you have to pick a feature that you like the most in

Java language, what would it be? And the feature that you least like about?

Java is said to have many weaknesses. However I guess one of its weaknesses is also its biggest strength.

Java is said to be verbose. True, you can write shorter programs doing the same in many other languages. So Java is verbose. However the benefit is that people can not only write the program, but also read it. Java programs can be read and understood. Moreover this can be done on a printed page, without "go declaration" and "code completion" in an IDE.

I guess I am not satisfied how verbose Java is. But I am really glad that it is readable.

Lastly, other than your own book, do you have an IT and a non-IT book

recommendation for our readers?