Blogs:JaroslavTulach:Theory

From APIDesign

Jaroslav Tulach's Theory Thoughts

More on Ruby, Gems and Modularity

The Ruby anonymous coward which brought duck-typing into attention of readers of this blog, added new comments to our discussion. The note presents Gems and I have to admit that by having Gems as standard packaging and dependency management tool, the Ruby gets far ahead standard offerings of Java. Of course unless you decide to use NetBeans runtime container or similar...

--JaroslavTulach 09:32, 30 September 2008 (UTC)

Are APIs like diamonds or like stars?

Is it uncommon that the same invention is discovered multiple times? Multiple times by different people? At the same time? It is indeed surprising to see something like that, however if you look back at the history of science, it is not that uncommon. I know that lightning rod has been independently invented by at least two people in the middle of 18th century. What was so special then that allowed such independent break-through?

For a centuries great mathematicians were troubled by Euclid's fifth postulate. It felt somewhat unnatural compared to the first four, the general expectation was that it is not necessary and it can be derived from the four others. Many tried, yet nobody succeeded. However, at the begging of 19th century things changed. Independently János Bolyai, Nikolaj Lobačevsky and maybe also Gauss discovered that fifth postulate is independent on the others. As such we can have geometries accepting and denying it and yet they'll make sense. Why at that time? Why three people at once?

There are many more cases that exhibit such coincidence. I do not think anyone has reasonable explanation for that, my personal feeling is that each era has something in the air that turns people's attention towards similar problems and tunes their mind to frequencies helping discover similar solutions.

I've been thinking about the laws of proper API design since 2001 and for a long time I believed that I am the only one who cares about such topic. I was pleasantly surprised during the Java One 2005 fully crowded BOF. However I still believed NetBeans is the only organization that does some research in this area. You can imagine how much I was surprised when I found out, at the end of 2005, that Josh Bloch had spent some time thinking about API design too. And that was not enough, my surprise even grew, when I found out that 80% of his observation in his presentation are similar to mine. There must have been something in the air, mustn't it?

As one comment stated: These things are a lot in Jaroslav Tulach's new book. Only real difference is that 'diamonds' above are 'stars' there. This comment brings us to the title of this post. Things looking similar at first sight may not be same underneath. At the end of 2005, most of the ideas were ready. They just waited for someone to put them into a book. When I finally began to write TheAPIBook, I was searching proper allegory to introduce the reader to the special context of API design. I knew the diamonds methaphor, but I could not use it, as I believe it is missing something important!

There is a significant difference between diamonds and stars. While diamonds are said to be forever, nobody considers stars eternal. As such the allegories are not the same. They are in fact quite different. If you get through the Practical API Design book to chapter 15 and chapter 19, you'll find out that if you have good support from runtime container, properly versioned APIs and you know how to allow co-existence of multiple versions of similar APIs, you can make your old APIs disappear, yet keep BackwardCompatibility. Of course, this is not a common operation, just like stars do not burn out everyday. However, if you really need to, you can send your API (aka your star) towards a black hole and make it disappear there. Moreover, you can do it in a completely user driven way, where the speed of dying is driven by number of remaining users of the old API, e.g. observers of your star. This is all possible and the NetBeans project done that few times.

In short, although APIs look eternal, they are not forever, they are more like stars.

--JaroslavTulach 03:34, 26 September 2008 (UTC)

Can duck-typing help with API evolution?

I've just received a comment mentioning Ruby's duck-typing as a solution to evolution problem in API discussed herein earlier. Interesting food for thought! Thanks! However so far I have to admit that there is no obvious benefit, at least not in the particular case. Read more and feel free to comment...

--JaroslavTulach 15:54, 22 September 2008 (UTC)

getDate() method evolution thoughts by Tim Band

Imagine that you have a class with method getDate() which returns the date of creation of that object. Can you add setDate() in new version? Can that change be compatible? This is topic discussed recently by Tim Band at LtU discussion forums:

If the get_date function had a comment stating "Returns the date at which the object was created" then you have to update that comment -- and that semantic change to get_date is the break.

It is hard to think of a semantic meaning for a get_date function that covers the setting of an arbitrary date, so let's imagine that we want to add "refresh_date" and that the original documentation of get_date said "Returns some date between the creation of the object and the calling of the function", then adding refresh_date is OK.

If get_date is undocumented or ambiguously documented, you need some process to resolve this. At Symbian we have a board of beard-strokers (of whom I am an occaisional member) who deliberate on the difficult cases and inform the customers of our decisions, and get feedback on certain issues. This process is considered integral to our backwards compatibility promise.

It is tempting to dream of a world where a programming language or supporting infrastructure would free us of the compatibility burden, turning it into just another language feature. However, it is in the mucky world of semantics, where interfaces meet the world of human use and abuse where this stuff really matters. You can break compatibility if you know it won't matter (we once changed our formerly weird floating-point implementation to match IEEE with no-one even noticing), but you can't make some compatible changes (changing priorities of processes that have no stated dependency on each other is a classic).

Having said all that, there is much work to be done on merely keeping interfaces stable. You can get quite far just by assuming that the source interface (your class and function signatures in your .h files in C++) matches the semantics, and picking up incompatible changes to that over time (removal of functions for example) and alterations of the binary interface in relation to the source interface (adding a virtual function for example).

Of course, occasionally you must allow customers to override common sense; I'm sure many readers here are familiar with the story of Microsoft Windows' memory manager running in a different mode if it detected Sim City running -- this is always a risk and you have to be aware that you might be forced to retract changes in the face of politics.

I should probably stop now, although really I'm just getting started! Tim Band

--JaroslavTulach 07:32, 17 September 2008 (UTC)

Joel Neely on Enums in APIs

Today I found out that there is an interesting comment to the blog entry published by Andrei a week ago. Andrei shared with us few thoughts on the use of enums in API. Joel Neely noted that it all depends on how the enum is used in the API. I cannot do anything else than agree with his words, yes it depends on whether the enum is used covariantly or contravariantly. If the enum is used covariantly only, then extending it with new constant in new version of the API is OK.

The only problem is that as soon as you publish an enum, one can immediately write a switch statement over it. And the switch statement's behaviour is contravariant. That is no good news for the API writers and compatible extensibility of their enum based APIs...

--JaroslavTulach 21:22, 26 August 2008 (UTC)

Are there any Languages Ready for API Evolution?

When I was describing my API design adventures in TheAPIBook, I could not realize that many of these problems would not even appear if we had better languages, or systems more suitable for the DistributedDevelopment.

I may be wrong, but I do not know about any language that would support modularity. And here I mean not compilation piece by piece, but also modularity in time. Because that is the kind of modularity that is needed quite often during development of APIs for our libraries.

At one point of time we release a version of a library and distribute it to others by publishing it on the website. Now people unknown to us, distributed all around the world download it and use it. As a result, the amount of applications built on top of such library is increasing, which implies also that the amount of bugs and feature requests is growing too. After a while the time to release new version of the library comes. However what if the first version used to have a class:

public abstract class View {
  public abstract String getDisplayName();
}

What if one of the feature requests demands to add an HTML title to each view. Can we change the view class to following form:

public abstract class View {
  public abstract String getDisplayName();
  public abstract String getHTMLTitle();
}

Indeed, this cannot be done. Existing subclasses of View would no longer compile, because the change is not source compatible. Also, even if someone links already compiled subclass with the new version of the library, the Java virtual machine will complain and throw some kind of linkage error, as definition of an abstract super class method is missing.

I would love to see a language or system that would fail the compilation whenever I want to modify my already released classes in a style that could prevent some of their users to continue to use them in previously working way. This would be the real modularity, which is ready for changes in time. So far it seems to me that the current languages do not deal with changes in time really well. Just like Andrei described in his Enums in APIs blog, it seems that our languages do not think about the process of API evolution much and include constructs that one needs to avoid to make DistributedDevelopment possible.

Releasing new version of libraries with modified APIs is really common and our software engineering practices cannot live without it, yet it seems that there is no language in the world that would make this easy and clueless. Or am I wrong?

Also lively discussed at Lambda the Ultimate.

--JaroslavTulach 06:50, 24 August 2008 (UTC)

Visual Aspects of an API

The usual consensus is that visual aspects that are presented just to the end user are not part of API of some application. This is usually well justified and correct, except in some situations...

--JaroslavTulach 11:40, 11 August 2008 (UTC)

Dependencies Are Important Type of API

New type of API has been discovered!

JaroslavTulach 07:05, 15 July 2008 (UTC)