Hotswap

From APIDesign

Hotswap is the most important technology that makes developers productive. Why people think that PHP makes them more productive than Java? Because it is enough to change a single line of code in your .php file, save it and reload the webpage. Doing the same with Java requires compilation, packaging, deployment. All of this takes ages and it has never been flexible enough. At least it has not been until Thomas Würthinger released his patched Hotspot!

Modular systems and Hotswap

In fact, the goal of modular systems is to allow a kind of hotswap. System components are supposed to be installed, enabled, disabled, upgraded or removed from the system on the fly. I believe the modular system succeeded in their mission. NetBeans Runtime Container as well as OSGi containers helped us find the necessary practices to allow this kind of hotswapping to happen and also helped us realize the limits. We know what is beyond the scope.

Modularity works fine for services. For implementation of shared interface that some components/modules in the system implement and others look up and use. Really look up and use everytime, you are not allowed to hold a reference to individual implementations. Such implementations are dynamic, change everytime and keeping reference to them may result in two objects implementing the same interface (old and new) not being assignable to each other.

In spite of that restriction, modularity helps developers a lot. Instead of reloading the whole application after change of one service implementation, one can reload just the implementing component/module. Still, all the objects created by the module need to be released and recreated.

Web vs. Desktop

Some kinds of applications are more suitable for Hotswap than others. Some can release all user objects after swapping their implementation classes easily. This is the case of web applications for example. Most of the objects persist only during handling of an individual request. Changing their implementation, throwing them away and pressing F5 to reload a page in browser to start over is easy.

However hotswapping the desktop application is more complicated. Objects usually live for arbitrary long time. Nobody can be sure that objects allocated by the hotswaped component are not referenced by other parts of the system. True hotswap system for this kind of environment needs to be able to traverse all existing objects of existing component and upgrade them to new ones. Sometimes this is easy (code changes), sometimes it is harder (adding fields, removing fields), sometimes it may be almost impossible (drastic changes to interface hierarchy).

This kind of problems may appear in web applications too (there are objects that span the life cycle of a request). But changes of this kind are not as often and as disastrous as in case of desktop. The session wide or application wide objects are usually just configuration data and they don't change that often anyway.

The JDK Hotswap

When you run the JDK in debug mode, it immediately comes with support for hotswap. You can change your code, recompile your classes and upload modified version to the already running VM. The whole system has one big problem: As of JDK6, you can only change method bodies, not the class definitions. You cannot add new fields, new methods, change method parameters, etc. If you do, then the system shouts at you when applying the changes.

There are two outcomes of this: first of all users don't use JDK's hotswap at all, as they know it is almost always useless. Instead of incorporating instant code changes into common development lifecycle, they rather shut the system down and restart.

The second outcome of this inflexibility of JDK is a proliferation of various projects (open source like Javaleon) as well as commercial ones that play various classloading and bytecode manipulation tricks to make real hotswap work on top of old JDK. The downside is that you need to start the VM with some special parameters (not always supported by all IDEs) and that often the support is not general - for some frameworks it works better than for other Java applications.

There used to be a saying that Microsoft supports innovation by providing poor MS-DOS, so others can implement improvements and earn some money on it. This time Sun is in the same position, instead of improving the JDK to handle real hotswap (something that SmallTalk can do for decades), the company sponsors segmented efforts to workaround its own limitations.

JDK for Developers

However there is no need to despair any longer. Thomas Würthinger fixed HotSpot to handle the real hotswap by itself. No need to download libraries and change your launch scripts. Just download Thomas's work from http://ssw.jku.at/dcevm/, execute the installer and your JDK suddenly becomes true JDK for Developers.

Btw. note that the D in middle of JDK stays for development, but it takes one extra Thomas to make the developer real focus of the JDK!

Thomas's version of HotSpot VM solves the old vs. new objects problem. As soon as you upload new version of a class, it re-casts all existing objects from the old class to new. Your code does not need to be ready for this at all, everything changes automatically. Classes get new methods, implement different interfaces, objects allocate new fields.

One might see this as a hotswaping nirvana, a place where we always wanted to be. Now the hotswap is finally ready for production!

Changing the done state

Wait. It is too early to celebrate. Debugging Hotswap can update the data structures easily, but there are still some remaining issues to make hotswap ready for production. There are things that shall be updated (to really replace the previous version of a component by its new behavior) and some cannot be updated yet.

First problem is a old method on a stack. This is not that big problem for web-like request scope computation, which respond to a request and exit. However for real application (for example the web server infrastructure) this is very desirable. Often there are main loops that endlessly process incoming data over and over.

// old, never-ending loop
for (;;) {
  Runnable getJob = queue.poll();
  getJob.run();
}
// new interruptible loop
while (!Thread.interrupted()) {
  Runnable getJob = queue.poll();
  getJob.run();
}

When the code of such loops is updated, it needs to be replaced while there is a thread having the method on its execution stack (either in poll or run method). Not all hotswap technologies support this, it is complex task, but definitely not impossible. Thomas mentioned his university has plans to work on that.

However problems don't end with method on stack issue, that is just the beginning. More complex problem is the need to re-run already executed algorithms during hotswap. Without this capability old objects can remain around being (from current point of view) improperly initialized. For example the newly added fields can be null, in spite they are always initialized in constructor to non-null values. Or the fields can have different value than expected. Consider following trivial case:

// old version of the class
class Data {
  final int version = 0;
  public void checkVersion() { assert version = 0; }
}
// new version of the class
class Data {
  final int version = 1;
  public void checkVersion() { assert version = 1; }
}

All instances of the Data class existing prior the swapping to new version will throw an assert from the checkVersion() method. To some extend this can be eliminated if the hotswap knows more. For example all fields initialized in constructor could be re-initialized or the hotswap could get some hints from previously made refactorings. But in general, this is unsolvable problem. To demonstrate that consider code creating and shwoing a window with a label:

JFrame f = new JFrame();
f.getContentPane().add(new JLabel());
f.setVisible(true);

Now imagine the application is running, the window remains visible. Now change the code to create JButton instead of JLabel! After swapping the code, the window shall update its state, shall it not? Of course, window will remain unchanged, as the hotswap by itself has no chance of knowing the code change affects already existing objects (of completely unrelated type). Are we stuck? Yes, until we learn how to write hotswap friendly code.

Matisse Example

Here is a screencast video showing how the hotswap friendly code can look like for a desktop application. The whole work (DCEVM and also patch for NetBeans Matisse GUI builder) has been done by Thomas and his Linz university colleagues. I just recorded the screencast.

The basic trick is to generate repeatable initialization code for the form class. Then, on each upload of new version of the class, one just needs to run this re-initialization code again for all instances of the form. The whole thing is relatively simple(!?), yet very powerful as the screencast hopefully shows (regardless of few confused comments I made during the recording).

Ready for Hotswapping?

Hotswap or not? That is the question. The answer depends on what you want to use hotswap for.

True Hotswap is a must for every developer. It improves development habbits rapidly, speeds up edit/try cycle. Now when Thomas improved the JDK, and the hotswap really works in 99% of cases, I can't imagine living without it. Now it is easier to change any class in my program than to modify content of Bundle.properties (surprisingly this observation expands to whole separate topic)! Improve your JDK too: http://ssw.jku.at/dcevm/

Do you want to hotswap components in production systems? Only after a heavy testing. There is so many things that can go wrong! Our current languages are not ready to write hotswap-ready code. One cannot rely on cluelessness, one needs to think a lot (or limit oneself to small changes). Perhaps people will start changing their coding style just to get those live updates into the running system. Javeleon's Allan hopes they will not! However I still believe people have to adjust the coding habits somehow to reach the ultimate hotswap nirvana. Either they need to be able to undo already executed computations (like in Prolog) or all your actions needs to be repeatable (like rebuilding UI of a component or serving a web page). Not all code we write in Java these days falls into these two categories.

The chances this could improve in future are not high, imho. Even functional languages are unlikely to help (as they loose track of what has been computed so far). I guess only Prolog could provide reliable way of coding hotswappable programs (as it usually keeps backtracking history of computation and could recompute all the parts of execution affected by the changed rule). However as not many production systems are written in Prolog the chances are high, massive use of hotswap will always be limited in production and people will rather find other ways (like virtualization) to allow them reach high availablity.

Hotswap for production is dead! Long live Hotswap for development!

<comments/>