OSGiAndNetBeans

From APIDesign

Revision as of 07:50, 24 March 2011

OSGi and NetBeans Runtime Container can now co-exist!

Those who read Chapter 15 of TheAPIBook know that I am not afraid to create bridges. I'd rather spend time to create a bridge between two APIs to allow easily upgradability and co-existence between old and new version, than organize a Big Bang rewrite that stops the world for few weeks/months and introduces new, shiny API. I am always afraid to find out that the new shiny replacement is buggy and incomplete (which is quite common).

I am not afraid to write bridges. Still it took me a while before I created this alien bridge. Not that it would be too much work (at the end it did not took more than few weeks of my time), but everyone (and aspecially OSGi guys) were always claiming that writing bridge between two module systems is close to impossible. Also there was an attempt to provide yet another module system for Java (JSR 277 and friends) which always delayed my attempts to start.

Netigso

Last Chrismas break I said to myself: enough! Let's really try to create a bridge between NetBeans Runtime Container and OSGi.

For a while I considered an option of sneaking OSGi under the NetBeans module system (especially OSGi guys were always suggesting this as the right alternative, not surprisingly). However this would be horribly incompatible solution - an earthquake for everyone who already wrote anything on top of NetBeans Runtime Container. Moreover I found out that there are some aspects in Dependencies between NetBeans modules that cannot be easily expressed in OSGi. Basically 90% of both module systems are quite similar, but the devil lies in the details. Some things just cannot be mapped naturally (see Richard S. Hall's excellent yet complex 1:1 mapping).

Thus I decided to resolve to writing a bridge. The basic idea (as shown in Chapter 15) is to let two independent APIs to live along each other and bridge information between them in a way that allows them to talk to each other.

Thus Netigso runs two containers in the same Java virtual machine, along each other. One is the regular NetBeans Runtime Container and the second is Felix - an OSGi implementation written by Richard Hall. The bridge goes through all registered JAR files and decides whether they represent an OSGi or NetBeans JAR (unsurpisingly both systems use JARs as basic packaging units and also store additional informations about the JARs in their manifest). Each JAR is then passed to the right container to receive appropriate runtime environment.

Moreover each JAR has a shadow - a fake peer registered in the other container - that just sits there and is ready to do bridging. What kind of bridging? Well, mostly class or resource loading. As soon as there is NetBeans module that has module dependency (aka requires bundle) on some OSGi bundle, the shadow associated with that bundle gets activated. The NetBeans module then delegates all class or resource loading to that bundle. Similiarly, if there is a NetBeans module exposing some public packages, any OSGi bundle can import them which activates the shadow bundle associated with that module. The activation is then intercepted and alternative bundle's loading mechanism is injected into it, that delegates all resource loading to the NetBeans module.

Compatibility

The beauty of this solution is that it remains fully compatible for both worlds. OSGi bundles existing so far see exactly the environment they are used to (e.g. Felix). NetBeans modules written so far are also managed the same way they used to be until now - e.g. by NetBeans Runtime Container. This means absolute BackwardCompatibility.

Only when people start to write new NetBeans modules or OSGi bundles that have dependencies on bundles and modules, we are entering the new world, world of cooperation. However as this world never existed before, there are no compatibility requirements. There is no threat to BackwardCompatibility. Things obviously need to work, but whatever works in the first release version, will define the actual amoeba shape of the whole symbiosis. This is much safer than trying to disassemble NetBeans Runtime Container and rebuilt it on top of plain OSGi.

Moreover this opens door to two directions. Over time, as people find out that OSGi is the way to go, they can replace individual NetBeans modules with bundles offering the same API, just packaged as OSGi bundles. Or, if people find out that OSGi has no future, they may rewrite all the bundles to NetBeans modules and run the whole system easily in Netigso ;-)

This exactly shows why APIs in modular systems can be seen as stars. They can be slowly and fully compatibly sent into a blackhole. As soon as people stop to use one of the module systems, the whole parallel execution will no longer be necessary. Only its half will be activated during runtime, the rest may stay disabled (just in case some particle of the vanished module system escapes from its blackhole and wants to get executed in proper runtime environment).

Tricks and Tooling

Writing the bridge between these two aliens was not that complicated. It was easy on the side of NetBeans, as I have full right to do changes into the code base to open backdoors for integration of Netigso. But not many changes were necessary anyway, the NetBeans Runtime Container already had a ModuleFactory concept which allowed other module providers to be plugged in.

The changes on the Felix side were slightly harder to do, as I did not want to modify the code base. But three reflection calls made the trick. I still need to turn these into a patch and convince Richard Hall to accept them into Felix (not OSGi) API for some future version of Felix. If the OSGi aliance accepts similar resource provider injection API in future, the Netigso could then run with any OSGi container.

Surprisingly the biggest amount of work was in the tooling space. Basically I needed to convince the NetBeans IDE apisupport modules to deal with OSGi bundles. Whenever they expected or generated NetBeans module, I needed to provide a branch in code to also understand the Bundle-SymbolicName and similar attributes. This was a lot of work and although the system seems to work acceptably right now, not everything is finished. More work will be needed to polish it into first class citizen.

On the other hand: Do you know how big difference is between NetBeans module and OSGi bundle in source form? Minimal. One line. By default NetBeans module projects have a skeletal manifest.mf file and it contains:

OpenIDE-Module: org.apidesign.your.module
# plus some additional tags

all that one has to do in Netigso to convert such project into OSGi bundle is to change that line to:

Bundle-SymbolicName: org.apidesign.your.module

then the build system (based on Ant) gets it and instead of NetBeans module JAR generates OSGi JAR. No additional change needed, all the other manifest meta-data are inferred from already existing NetBeans module project information. Another indication how similar these two module systems are.

NetBeans and Equinox

Some people asked for support of Equinox instead of Felix in the Netigso project. Good news are that progress has been made and there is now Netbinox subproject available for fans of Equinox OSGi container.

Resources

• List of bugs - report new bug!
• mailing list - subscribe, view archives, etc.
• Dev page...
• Sources are part of standard NetBeans Hg repository since release 6.9.
• Builds made daily

Legal Issues

The following paragraphs contain just my personal speculations. I know that at least one OSGi guy does not find them amusing (which I learned here). Also now I know that these opinions do not match those of my employer (as I have found out via other channels). Take it as oversimplification of complex and not easily understandable actions seen from the other side of Atlantic ocean.

An interesting aspect of the whole Netigso effort is its legal implications. I know US is (much more than Europe) full of IP issues. I am not in the inner circle to know, but the strong Sun's pushback towards accepting OSGi as the default module system for Java (e.g. all the attempts with JSR 277, JSR 294, Jigsaw, etc.) might be motivated by fear that as soon as OSGi is essential part of Java, the IBM knocks on the door and says: "Hey, do you know that by using OSGi you are violating 137 our patents? Shall we bring the course to court or will you give the control over Java to us without such hassle?"

As NetBeans Runtime Container has been working sooner than OSGi finished its (first usable) specification and as Netigso clearly shows that there is 1:1 mapping for most of the concepts in both systems, it is now possible to challenge the patents because there is a prior art. No need to be afraid of bringing module system into Java, everything has already been invented by NetBeans and has been available for free (without any patents) for last decade.

The only concept not invented by NetBeans is package dependency, e.g. ability to import a package, not the whole module/bundle. But it leads to strange coding practices anyway, so there is no harm if it is omitted. Anyway good luck, bringing modularity to Java!

<comments/>