OSGi is a specification for a modular system for Java. It defines a packaging format (JAR file with additional manifest entries and associated troubles), runtime behavior, and a way to register and discover services. In contrast to the NetBeans Runtime Container, it supports package dependencies. For the rest, the systems seem similar, which has been demonstrated by the implementation of the OSGiAndNetBeans bridge.
OSGi and NetBeans Runtime Container are similar, however the devil lies, as always, in the details! From time to time I manage to learn more about OSGi and then I just cannot help but wonder what the motivation was behind designing such behavior! Maybe this is all caused by my preoccupation (as a designer of the NetBeans system), but I just cannot overcome the feeling that certain design decisions in OSGi are just weird.
I'll nitpick on OSGi here, hoping somebody will be able to explain the history and possibly also the rationale behind these (mis)behaviors.
Start/Stop a Bundle
I believe it makes sense to tell a module that it is going to be used or that it is used no more. Of course, I prefer Declarative Programming, but sometimes there is just no other way and some parts of the infrastructure need to be turned on programmatically. Thus, it is makes sense to have a BundleActivator or ModuleInstall (in the NetBeans case). However, then one finds that people can use an OSGi bundle without starting it!
That would be a dangerous situation. It would mean one cannot put any initialization logic into the activator/installer at all. Peter Kriens spent significant amount of his time and patience to explain me the rationale behind this behavior and possible fix:
- Just add Activation-Policy: lazy into your bundle which performs important initialization in BundleActivator and it is guaranteed that as soon as somebody loads a class from your bundle, it is going to be started
- This is not the default to keep things backward compatible - OK, I can understand the value of BackwardCompatibility. Through at this case it might have been better to change the semantics when the bundle manifest version was changed to two (or change it to three). The NetBeans ModuleInstall is guaranteed to be called whenever somebody tries to use a module (by having a dependency on it and trying to get enabled). Although it is recommended to use declarative registrations, when activator/installer is present calling it as soon as the bundle is being used is the natural/expected behavior.
- Peter (as far as I understand) rather advocates splitting the API and the implementation into separate bundles however. The service should rather be registered by independent implementation. I still don't know how my code using the API will force the system to initialize the implementation too, but I expect the answer to be OSGi declarative services (which I am not familiar with).
Possibly Eclipse is using OSGi in wrong way. The singleton initialization in activator complicates the OSGiAndNetBeans bridge. Basically it means we don't have 1:1 mapping between enabling a module in the NetBeans sense and starting a bundle in the OSGi sense. As we discover more and more OSGi based systems (like JDeveloper), we need to add more and more flexibility to cover all the possible ways people deal with the dichotomy between used and started. OSGi is very flexible and people tend to use it in many innovative ways.
At least I know now: the suitable fix is to use Activation-Policy: lazy. The only question remains: How do I convince Eclipse to modify their core bundle manifests to include this tag?
Another thing related to activators is the order of starting and stopping bundles. Again, if we think about the starting to be associated with use, then it makes sense to start the modules that are used by others sooner, then starting the others. Do a topological sort based on interbundle dependencies and then start the bottom bundles first (with as less dependencies as possible). When stopping a set of bundles, do it from the top. Natural? I think so. Do you think OSGi does that? No, not at all.
I've just needed to deal with a NullPointerException caused by the fact that org.eclipse.equinox.registry is stopped sooner than org.eclipse.equinox.security when calling OSGi framework.stop(). What happened? There is a singleton in the registry and BundleActivator.stop() clears it. But still the module remains available for others to call. When the system tries to stop the security bundle, it calls into the registry trying to obtain the singleton and guess, it is not there!
Why could not the great minds behind OSGi create a specification that would sort the modules using a topological sort before shutting them down? Is that unnatural? No. But rather than that OSGi introduced start levels (which we now have to support in NetBeans). The only conclusion one can have: OSGi is not well thought out.
One of the characteristics of OSGi is use of RangeDependencies which seems like a clever good idea supporting cluelessness in its way, but if not done properly it is just a way to mess around with NP-Complete problems. And yes, there are such problems in OSGi as demonstrated by Equinox coming with a 3SAT solver.
On the other hand, this is not complete fault of RangeDependencies. Recent analysis shows that it depends how the RangeDependencies are used. If they are used properly, the NP-Complete problems can be eliminated.
Looking from this point of view: OSGi's high shot (decision to use powerful dependency system) missed the sweet spot (produced underspecified system with corners full of bad consequences).
I value OSGi for its ability to provide a common ground and allow interoperability between various module systems (like Glassfish, Eclipse, NetBeans, JDeveloper, etc.). Only by having OSGi around could we propose to the JDeveloper team to consider the idea of reusing Netbinox and sharing bits of NetBeans. OSGi is good for interoperability.
However, beyond that, I just can't help but think that OSGi is an overdesigned, and sometimes misdesigned, technology. In addition to that, I'd like to point out that OSGi is so 2000-late. The OSGi spec 4.2 still runs on JDK 1.3... and it shows! These days Java is different. With the help of AnnotationProcessors it can do magic (and it does not need poor and misleading tricks like MagicalStrings).
I fully understand the value of pioneers. Somebody just has to be the first (well second), walk the road, and find out that this is not the right path. I believe OSGi succeeded in this mission. Everyone should understand at this point the value of modularity. Everyone should also see, however, that the OSGi path is not the right one to follow. I just hope the next attempt to standardize a module system for Java will learn from the mistakes of its predecessors (OSGi and NetBeans Runtime Container).
Let's use OSGi for interop for now, and then let's forget about it later.
Accusing OSGi deficiencies generated some interesting comments and observations. I am including them here.
Neil Bartlett said ...
Hello Neil, thanks for your comment. Yes, you are right, the page is unlikely good start for a useful discourse. On the other hand, it nicely illustrates my frustration with OSGi. Still, I'd be glad to be enlighten about the design misconceptions.
--JaroslavTulach 10:02, 21 November 2011 (UTC)
Mirko Jahn said ...
A comment by Jesse Glick left at bug 205019
I think this is a bit off the mark. Seems like Eclipse is not really using OSGi as designed. Proper OSGi bundles are (acc. to spec) supposed to work regardless of what other bundles have or have not been started; that is why dynamic services exist, and why bundles are not started in topological order. In other words, we are working around misdesign in Eclipse, not OSGi per se. The problem presumably does not affect Eclipse and its RCP apps because they are not attempting to load code from an arbitrary compliant OSGi container, and so can make assumptions about runtime behavior not guaranteed by the spec.
--Jesse Glick 14:00, 23 November 2011 (CET)
Reply to Mirko's Post
Hello Mirko, first of all - thank you for your time to write down such a long post here. I value your effort.
"I agree that OSGi is not perfect" - sure, I know that NetBeans Runtime Container is not perfect either. Over years in production you just have to come up with a lot of compromises which, when looking backward, may not seem perfect.
"distinguish between a mere library and a state aware module" - yes, I am aware that nobody needs to use Activator - however that is not the problem. I am puzzled by OSGi allowing to load classes from modules with Activator which has not been called! What kind of strange idea is that? In my opinion the fact that I provide an Activator should mean that whenever somebody wants to use me (e.g. activate me), I want that Activator to be called. Why this is not true in OSGi?
"ugly overhead of needing to know what bundles need to be started" - actually this may be the problem. NetBeans is worshiping so called Injectable Singletons. In this use-case, what ever API you link against means that the API is ready to work and is fully configured. The way Eclipse is using Activators violates this (which may be Eclipse fault as Mirko and Jesse indicate). You say "simply do not start the bundle" - sure, this is what works in NetBeans as well if you don't need a module with implementation of an API, just don't enable it. But still, I don't understand why there is the dichotomy between using and starting a bundle. To allow an API to be bundled with its implementation that is registered in Activator (so others can use the API without the implementation)? That sounds weird!
"reverse order they were actually started" - see, I did not know this. But then the problem is the start order. Obviously, those bundles with as less dependencies as possible, should be started first. However I understand the OSGi may have problem - there does not seem to be a bulk start operation. One can start just a single bundle, not a set of bundles. In NetBeans Runtime Container one just asks to enable a set of bundles and the system starts them in the correct order. The system always guarantee Activators are called on modules needed by somebody. Sounds like a deficiency in OSGi where determining the start order is fully left in hands of poor users of the OSGi API.
"Range dependencies are tricky" - RangeDependencies don't really belong into my most recent rant, but there are issues associated with them - althrough they may be mitigated to avoid NP-Complete problems.
"certainly not easy to adhere" - if a concept is not easy to adhere, then the system is not designed for clueless use. And that is a problem as most of us are completely clueless when using others APIs. Should we threat OSGi as an assembly language for modularity hoping somebody will wrap it with higher level concepts? That would support my conclusions - OSGi helped us see the value of modularity, but proper, widely used modularity should rather not expose OSGi at all.
--JaroslavTulach 18:23, 24 November 2011 (UTC)