OSGi
From APIDesign
(→Range Dependencies) |
(→Range Dependencies) |
||
| Line 29: | Line 29: | ||
On the other hand, this is not complete fault of [[RangeDependencies]]. [[RangeDependenciesAnalysed|Recent analysis]] shows that it depends how the [[RangeDependencies]] are used. If they are used properly, the [[NP-Complete]] problems can be eliminated. | On the other hand, this is not complete fault of [[RangeDependencies]]. [[RangeDependenciesAnalysed|Recent analysis]] shows that it depends how the [[RangeDependencies]] are used. If they are used properly, the [[NP-Complete]] problems can be eliminated. | ||
| - | Looking from this point of view: [[OSGi]] high shot (decision to use powerful [[dependency]] system) missed the sweet spot (produced underspecified system with corners full of bad consequences). | + | Looking from this point of view: [[OSGi]]'s high shot (decision to use powerful [[dependency]] system) missed the sweet spot (produced underspecified system with corners full of bad consequences). |
== Executive Summary == | == Executive Summary == | ||
Revision as of 10:10, 9 January 2012
OSGi is a specification for a modular system for Java. It defines a packaging format (JAR file with additional manifest entries and associated troubles), runtime behavior, and a way to register and discover services. In contrast to the NetBeans Runtime Container, it supports package dependencies. For the rest, the systems seem similar, which has been demonstrated by the implementation of the OSGiAndNetBeans bridge.
Contents |
OSGi Weirdness
OSGi and NetBeans Runtime Container are similar, however the devil lies, as always, in the details! From time to time I manage to learn more about OSGi and then I just cannot help but wonder what the motivation was behind designing such behavior! Maybe this is all caused by my preoccupation (as a designer of the NetBeans system), but I just cannot overcome the feeling that certain design decisions in OSGi are just weird.
I'll nitpick on OSGi here, hoping somebody will be able to explain the history and possibly also the rationale behind these (mis)behaviors.
Start/Stop a Bundle
I believe it make sense to tell a module that it is going to be used or that it is no longer being used. Of course, I'd prefer Declarative Programming, but sometimes there is just no other way and some parts of the infrastructure need to be turned on programmatically. Thus, it is makes sense to have a BundleActivator or ModuleInstall (in the NetBeans case). However, then one finds that people can use an OSGi bundle without starting it!
What kind of crazy idea is that? It basically means I cannot put any initialization logic into the activator/installer at all. I need to be ready for somebody using my module without starting me! I really don't get the purpose of this construct at all. The NetBeans ModuleInstall is guaranteed to be called whenever somebody tries to enable a module, or use it (by having a dependency on it and trying to get enabled). This sounds logical to me.
Obviously these misconceptions complicate the OSGiAndNetBeans bridge. Basically it means there is no 1:1 mapping between enabling a module in the NetBeans sense and starting a bundle in the OSGi sense. As we discover more and more OSGi based systems (like JDeveloper), we need to add more and more flexibility to cover all the possible ways people deal with the dichotomy between used and started. OSGi is more flexible, but not logical.
Stopping Order
Another thing related to activators is the order of starting and stopping bundles. Again, if we think about the starting to be associated with use, then it makes sense to start the modules that are used by others sooner, then starting the others. Do a topological sort based on interbundle dependencies and then start the bottom bundles first (with as less dependencies as possible). When stopping a set of bundles, do it from the top. Natural? I think so. Do you think OSGi does that? No, not at all.
I've just needed to deal with a NullPointerException caused by the fact that org.eclipse.equinox.registry is stopped sooner than org.eclipse.equinox.security when calling OSGi framework.stop(). What happened? There is a singleton in the registry and BundleActivator.stop() clears it. But still the module remains available for others to call. When the system tries to stop the security bundle, it calls into the registry trying to obtain the singleton and guess, it is not there!
Why could not the great minds behind OSGi create a specification that would sort the modules using a topological sort before shutting them down? Is that unnatural? No. But rather than that OSGi introduced start levels (which we now have to support in NetBeans). The only conclusion one can have: OSGi is not well thought out.
Range Dependencies
One of the characteristics of OSGi is use of RangeDependencies which seems like a clever good idea supporting cluelessness in its way, but if not done properly it is just a way to mess around with NP-Complete problems. And yes, there are such problems in OSGi as demonstrated by Equinox coming with a 3SAT solver.
On the other hand, this is not complete fault of RangeDependencies. Recent analysis shows that it depends how the RangeDependencies are used. If they are used properly, the NP-Complete problems can be eliminated.
Looking from this point of view: OSGi's high shot (decision to use powerful dependency system) missed the sweet spot (produced underspecified system with corners full of bad consequences).
Executive Summary
I value OSGi for its ability to provide a common ground and allow interoperability between various module systems (like Glassfish, Eclipse, NetBeans, JDeveloper, etc.). Only by having OSGi around could we propose to the JDeveloper team to consider the idea of reusing Netbinox and sharing bits of NetBeans. OSGi is good for interoperability.
However, beyond that, I just can't help but think that OSGi is an overdesigned, and sometimes misdesigned, technology. In addition to that, I'd like to point out that OSGi is so 2000-late. The OSGi spec 4.2 still runs on JDK 1.3... and it shows! These days Java is different. With the help of AnnotationProcessors it can do magic (and it does not need poor and misleading tricks like MagicalStrings).
I fully understand the value of pioneers. Somebody just has to be the first (well second), walk the road, and find out that this is not the right path. I believe OSGi succeeded in this mission. Everyone should understand at this point the value of modularity. Everyone should also see, however, that the OSGi path is not the right one to follow. I just hope the next attempt to standardize a module system for Java will learn from the mistakes of its predecessors (OSGi and NetBeans Runtime Container).
Let's use OSGi for interop for now, and then let's forget about it later.
<comments/>
Defending OSGi
Accusing OSGi deficiencies generated some interesting comments and observations. I am including them here.
Neil Bartlett said ...
Hello Neil, thanks for your comment. Yes, you are right, the page is unlikely good start for a useful discourse. On the other hand, it nicely illustrates my frustration with OSGi. Still, I'd be glad to be enlighten about the design misconceptions.
--JaroslavTulach 10:02, 21 November 2011 (UTC)
Mirko Jahn said ...
First of all, I agree that OSGi is not perfect. In fact I do not know any technology that reached some sort of complexity that doesn't have its flaws. I guess that's nature. However, I think you're missing some background/ experience in the underlying concepts and frameworks you're referring too.
First of all, start/stop of a bundle. Here it is important to distinguish between a mere library and a state aware module. For libraries it is perfectly fine to not have an initialization (static initialization is fine here as well, but no "object" related one). In the OSGi world you then do not need to create an Activator. For all modules that are in fact state aware or need some sort of initialization (like Service dependencies), you could create you Activator (or use a declarative way of expressing that). In case a module does not comply with this contract this is a bug in the bundle code. As a general rule, you only expose API's and provide the instances as services, as long as the bundle is not started, no services will be available and no harm is done. Of course there is the "ugly" overhead of needing to know what bundles need to be started and which don't (you could also start all of them - no overhead in case of library bundles, because they won't execute any code). This seems not entirely clean as an approach, but it is powerful. Just imagine you have one bundle providing the API and a simple default service with the implementation. Once you realized the implementation is not good enough, you could just go ahead, reuse the API and not the exposed service (simply do not start the bundle) and provide your own bundle implementing the service in a way you require it.
Second, stopping order. There is an order in which bundles are stopped and they are stopped in the reverse order they were actually started, so what you saw might be a timing issue or an error in the implementation. Also Eclipse is not a perfect OSGi citizen. They did a lot to improve that but they are still not there. For instance many bundles still do not use the service registry, but a more static eclipse specific construct that has flaws with the dynamism of OSGi. I guess this might explain the problem you were experiencing. Also when done correctly (with services), a stopped bundle would not expose the service any longer and the bundle consuming such service would be able to call such bean (because it wouldn't find it) - no harm could be done here. Unfortunately the developer has to account for that or us a framework that does it for him like BluePrint for instance.
Range dependencies are tricky, I agree. Also the approach on handling the correct version, updating handles in the container, refreshing the wiring between bundles. All that is not trivial and error prone. However, I haven't seen anything even close to what is possible with OSGi. You could by designing the exposed and consumed packages correctly, a powerful and future prove API. Implementers know base on version changes when they have to take a closer look (minor version change) and simple service consumers can rely on the major version as an indicator that something might have changed that they should be aware of. Especially with many modules, keeping track of changes is hell. With such ranges, it at least gives you a way to better track where to look and what to adapt in case something happens. This is still not perfect and I am not sure there is a silver bullet at all, but it's the best thing I've seen so far.
To summarize what I am trying to say. OSGi, like many other specs, is not perfect and there might be a better one in the future, but for the time being, it is pretty good and well thought through. Unfortunately the concepts are not simple and certainly not easy to adhere. Especially with the latest version 4.3, many improvements have been done to make it easier and remove potential for errors, but like every complex system, one first has to know it to fully appreciate it. ;-)
--Mirko Jahn 19:59, 23 November 2011 (CET)
A comment by Jesse Glick left at bug 205019
I think this is a bit off the mark. Seems like Eclipse is not really using OSGi as designed. Proper OSGi bundles are (acc. to spec) supposed to work regardless of what other bundles have or have not been started; that is why dynamic services exist, and why bundles are not started in topological order. In other words, we are working around misdesign in Eclipse, not OSGi per se. The problem presumably does not affect Eclipse and its RCP apps because they are not attempting to load code from an arbitrary compliant OSGi container, and so can make assumptions about runtime behavior not guaranteed by the spec.
--Jesse Glick 14:00, 23 November 2011 (CET)
Reply to Mirko's Post
Hello Mirko, first of all - thank you for your time to write down such a long post here. I value your effort.
"I agree that OSGi is not perfect" - sure, I know that NetBeans Runtime Container is not perfect either. Over years in production you just have to come up with a lot of compromises which, when looking backward, may not seem perfect.
"distinguish between a mere library and a state aware module" - yes, I am aware that nobody needs to use Activator - however that is not the problem. I am puzzled by OSGi allowing to load classes from modules with Activator which has not been called! What kind of strange idea is that? In my opinion the fact that I provide an Activator should mean that whenever somebody wants to use me (e.g. activate me), I want that Activator to be called. Why this is not true in OSGi?
"ugly overhead of needing to know what bundles need to be started" - actually this may be the problem. NetBeans is worshiping so called Injectable Singletons. In this use-case, what ever API you link against means that the API is ready to work and is fully configured. The way Eclipse is using Activators violates this (which may be Eclipse fault as Mirko and Jesse indicate). You say "simply do not start the bundle" - sure, this is what works in NetBeans as well if you don't need a module with implementation of an API, just don't enable it. But still, I don't understand why there is the dichotomy between using and starting a bundle. To allow an API to be bundled with its implementation that is registered in Activator (so others can use the API without the implementation)? That sounds weird!
"reverse order they were actually started" - see, I did not know this. But then the problem is the start order. Obviously, those bundles with as less dependencies as possible, should be started first. However I understand the OSGi may have problem - there does not seem to be a bulk start operation. One can start just a single bundle, not a set of bundles. In NetBeans Runtime Container one just asks to enable a set of bundles and the system starts them in the correct order. The system always guarantee Activators are called on modules needed by somebody. Sounds like a deficiency in OSGi where determining the start order is fully left in hands of poor users of the OSGi API.
"Range dependencies are tricky" - RangeDependencies don't really belong into my most recent rant, but there are issues associated with them - althrough they may be mitigated to avoid NP-Complete problems.
"certainly not easy to adhere" - if a concept is not easy to adhere, then the system is not designed for clueless use. And that is a problem as most of us are completely clueless when using others APIs. Should we threat OSGi as an assembly language for modularity hoping somebody will wrap it with higher level concepts? That would support my conclusions - OSGi helped us see the value of modularity, but proper, widely used modularity should rather not expose OSGi at all.
--JaroslavTulach 18:23, 24 November 2011 (UTC)
Sergey said ...
/ If I do anything in Linux for pmnigomrrag, I use Eclipse if I need an IDE. I also like doing as much as I can in terminal or even a note pad-like program. On Windows (for school), I really like using Notepad++, so the best alternative in Linux for NP++ that I've found is Geany.
--Sergey 14:15, 23 October 2013 (CEST)
Malnormalulo said ...
"In my opinion the fact that I provide an Activator should mean that whenever somebody wants to use me (e.g. activate me), I want that Activator to be called. Why this is not true in OSGi?"
In OSGi, a bundle which has not been started can provide packages (classes), but not services (objects). To me at least, this makes intuitive sense; classes don't need to be initialized*, but objects do.
You may object that a class might, e.g., make use of a singleton which does need initialization. This would be one reason that an OSGi best practice is to only expose an API (i.e., interfaces, exceptions, maybe some simple beans) through your exported packages, and to leave the implementation to service objects defined by private classes. This way, your exported packages remain initialization-free, and references may be made to them without worry regardless of the state of the system.
You may then ask what the point is in allowing an API to be used when the implementation is not available. Even if the API does not, in itself, require initialization, it doesn't necessarily seem useful to have access to it when no implementation is available. In many systems, there's no point at all, and you'd be entirely right to note this as a shortcoming!
But other systems take advantage of OSGi's dynamic nature by doing things like watching for services that implement the API. In the whiteboard pattern (an OSGi-specific variant of the observer pattern) an "observable" object registers no services, instead listening for observers to register services that act as callbacks. When it boots up, the observable code is probably not yet aware of any concrete implementations of the callback interface. Nevertheless, it needs to know about the interface so that it can listen for new implementations that later become available. It is therefore necessary that the API be available even when implementors are not.
* This isn't always true, but if a class does need to be initialized, it can usually just be a singleton object instead, which is what OSGi would recommend.
"However I understand the OSGi may have problem - there does not seem to be a bulk start operation."
This is indeed a problem. Launching OSGi applications is tedious. Various abstractions, like Karaf's "features" and (I think) WebSphere Application Server's "Enterprise Bundle Archives", have been layered on top of OSGi to express structures composed of multiple bundles which may be started and stopped together. There is no standard, though, and I can't vouch for how well any of them actually accomplish the goal of simplifying OSGi deployment.
"Should we threat OSGi as an assembly language for modularity hoping somebody will wrap it with higher level concepts? That would support my conclusions - OSGi helped us see the value of modularity, but proper, widely used modularity should rather not expose OSGi at all."
I, for one, agree. OSGi is too complex to ask every application developer to fully understand it. The assembly language analogy is spot on – OSGi is largely a good thing, but it's in desperate need of higher-level abstractions. Until we have those, we should regard it as a useful way to solve specific problems, but only to be used when actually necessary.
--Malnormalulo 16:39, 15 March 2017 (CET)
Follow
In the introduction you appear to be asking to be enlightened about OSGi, but then you go on to attack it from a position of ignorance and with heaps of bad attitude. Sorry but this is not the way to engage with people when you're asking for help.
If you have some specific questions about how or why OSGi works the way it does, then I and many others will be delighted to help you on the osgi-dev mailing list. Unfortunately this particular page is little more than a rant and can't be the basis for useful discourse.
--Neil Bartlett 00:07, 20 November 2011 (CET)