OSGi
From APIDesign
(→OSGi Weirdness) |
|||
| (14 intermediate revisions not shown.) | |||
| Line 1: | Line 1: | ||
| - | [[wikipedia::OSGi|OSGi]] is a specification | + | [[wikipedia::OSGi|OSGi]] is a specification for a modular system for [[Java]]. It defines a packaging format ([[JAR]] file with additional manifest entries and [[PropertyFiles|associated troubles]]), runtime behavior, and a way to register and discover services. In contrast to the [[NetBeans Runtime Container]], it supports package dependencies. For the rest, the systems seem similar, which has been demonstrated by the implementation of the [[OSGiAndNetBeans]] bridge. |
== [[OSGi]] Weirdness == | == [[OSGi]] Weirdness == | ||
| - | [[OSGi]] and [[NetBeans Runtime Container]] are similar, however the devil lies in details! From time to time I manage to learn more about [[OSGi]] and then I just cannot | + | [[OSGi]] and [[NetBeans Runtime Container]] are similar, however the devil lies, as always, in the details! From time to time I manage to learn more about [[OSGi]] and then I just cannot help but wonder what the motivation was behind designing such behavior! Maybe this is all caused by my preoccupation (as a designer of the [[NetBeans]] system), but I just cannot overcome the feeling that certain design decisions in [[OSGi]] are just weird. |
| - | I'll nitpick on [[OSGi]] here hoping somebody will be able to explain the history and possibly also rationale behind these (mis)behaviors. | + | I'll nitpick on [[OSGi]] here, hoping somebody will be able to explain the history and possibly also the rationale behind these (mis)behaviors. |
=== Start/Stop a Bundle === | === Start/Stop a Bundle === | ||
| - | I believe it | + | I believe it makes sense to tell a module that it is going to be used or that it is used no more. Of course, I prefer [[Declarative Programming]], but sometimes there is just no other way and some parts of the infrastructure need to be turned on programmatically. Thus, it is makes sense to have a '''BundleActivator''' or {{NB|org-openide-modules|org/openide/modules|ModuleInstall}} (in the [[NetBeans]] case). However, then one finds that people can use an [[OSGi]] bundle without starting it! |
| - | + | That would be a dangerous situation. It would mean one cannot put any initialization logic into the ''activator/installer'' at all. [[Peter Kriens]] spent significant amount of his time and patience to explain me the rationale behind this behavior and possible fix: | |
| + | # Just add '''Activation-Policy: lazy''' into your bundle which performs important initialization in ''BundleActivator'' and it is guaranteed that as soon as somebody loads a class from your bundle, it is going to be started | ||
| + | # This is not the default to keep things backward compatible - OK, I can understand the value of [[BackwardCompatibility]]. Through at this case it might have been better to change the semantics when the [[PropertyFiles|bundle manifest version]] was changed to two (or change it to three). The [[NetBeans]] {{NB|org-openide-modules|org/openide/modules|ModuleInstall}} is guaranteed to be called whenever somebody tries to ''use'' a module (by having a [[dependency]] on it and trying to get enabled). Although it is recommended to use [[Declarative Programming|declarative registrations]], when ''activator/installer'' is present calling it as soon as the bundle is being ''used'' is the natural/expected behavior. | ||
| + | # Peter (as far as I understand) rather advocates splitting the [[API]] and the implementation into separate bundles however. The service should rather be registered by independent implementation. I still don't know how my code using the [[API]] will force the system to initialize the implementation too, but I expect the answer to be [[OSGi]] ''declarative services'' (which I am not familiar with). | ||
| - | + | Possibly [[Eclipse]] is using [[OSGi]] in wrong way. The [[singleton]] initialization in activator complicates the [[OSGiAndNetBeans]] bridge. Basically it means we don't have 1:1 mapping between ''enabling'' a module in the [[NetBeans]] sense and ''starting'' a bundle in the [[OSGi]] sense. As we discover more and more [[OSGi]] based systems (like [[JDeveloper]]), we need to add more and more flexibility to cover all the possible ways people deal with the dichotomy between ''used'' and ''started''. [[OSGi]] is very flexible and people tend to use it in many ''innovative'' ways. | |
| + | |||
| + | At least I know now: the suitable fix is to use '''Activation-Policy: lazy'''. The only question remains: How do I convince [[Eclipse]] to modify their core bundle manifests to include this tag? | ||
=== Stopping Order === | === Stopping Order === | ||
| Line 25: | Line 30: | ||
=== Range Dependencies === | === Range Dependencies === | ||
| - | One of characteristics of [[OSGi]] is use of [[RangeDependencies]] which seems like a clever good idea supporting [[cluelessness]] in its way, but | + | One of the characteristics of [[OSGi]] is use of [[RangeDependencies]] which seems like a clever good idea supporting [[cluelessness]] in its way, but if not done properly it is just a way to mess around with [[NP-Complete]] problems. And yes, there are such problems in [[OSGi]] as demonstrated by [[Equinox]] coming with a [[3SAT]] solver. |
| + | |||
| + | On the other hand, this is not complete fault of [[RangeDependencies]]. [[RangeDependenciesAnalysed|Recent analysis]] shows that it depends how the [[RangeDependencies]] are used. If they are used properly, the [[NP-Complete]] problems can be eliminated. | ||
| + | |||
| + | Looking from this point of view: [[OSGi]]'s high shot (decision to use powerful [[dependency]] system) missed the sweet spot (produced underspecified system with corners full of bad consequences). | ||
== Executive Summary == | == Executive Summary == | ||
| - | I value [[OSGi]] for its ability to | + | I value [[OSGi]] for its ability to provide a [[module system|common ground]] and allow interoperability between various [[module system]]s (like [[Glassfish]], [[Eclipse]], [[NetBeans]], [[JDeveloper]], etc.). Only by having [[OSGi]] around could we propose to the [[JDeveloper]] team to consider the idea of reusing [[Netbinox]] and sharing bits of [[NetBeans]]. [[OSGi]] is good for interoperability. |
| - | However beyond that I just can't help | + | However, beyond that, I just can't help but think that [[OSGi]] is an overdesigned, and sometimes misdesigned, technology. In addition to that, I'd like to point out that [[OSGi]] is so ''2000-late''. The [[OSGi]] spec 4.2 still runs on [[JDK]] 1.3... and it shows! These days [[Java]] is different. With the help of [[AnnotationProcessor]]s it can do magic (and it does not need poor and misleading tricks like [[MagicalStrings]]). |
| - | I fully understand the value of pioneers. Somebody just has to be the first (well [[NetBeans Runtime Container|second]]), walk the road and find out that this is not the right path. I believe [[OSGi]] succeeded in this mission. Everyone should understand | + | I fully understand the value of pioneers. Somebody just has to be the first (well [[NetBeans Runtime Container|second]]), walk the road, and find out that this is not the right path. I believe [[OSGi]] succeeded in this mission. Everyone should understand at this point the value of [[modularity]]. Everyone should also see, however, that the [[OSGi]] path is not the right one to follow. I just hope the next attempt to standardize a [[module system]] for [[Java]] will learn from the mistakes of its predecessors ([[OSGi]] and [[NetBeans Runtime Container]]). |
| - | Let's use | + | Let's use [[OSGi]] for interop for now, and then let's forget about it later. |
<comments/> | <comments/> | ||
| + | |||
| + | == Defending [[OSGi]] == | ||
| + | |||
| + | Accusing [[OSGi]] deficiencies generated some interesting [[Talk:OSGi|comments and observations]]. I am including them here. | ||
| + | |||
| + | {{:Talk:OSGi}} | ||
Current revision
OSGi is a specification for a modular system for Java. It defines a packaging format (JAR file with additional manifest entries and associated troubles), runtime behavior, and a way to register and discover services. In contrast to the NetBeans Runtime Container, it supports package dependencies. For the rest, the systems seem similar, which has been demonstrated by the implementation of the OSGiAndNetBeans bridge.
Contents |
OSGi Weirdness
OSGi and NetBeans Runtime Container are similar, however the devil lies, as always, in the details! From time to time I manage to learn more about OSGi and then I just cannot help but wonder what the motivation was behind designing such behavior! Maybe this is all caused by my preoccupation (as a designer of the NetBeans system), but I just cannot overcome the feeling that certain design decisions in OSGi are just weird.
I'll nitpick on OSGi here, hoping somebody will be able to explain the history and possibly also the rationale behind these (mis)behaviors.
Start/Stop a Bundle
I believe it makes sense to tell a module that it is going to be used or that it is used no more. Of course, I prefer Declarative Programming, but sometimes there is just no other way and some parts of the infrastructure need to be turned on programmatically. Thus, it is makes sense to have a BundleActivator or ModuleInstall (in the NetBeans case). However, then one finds that people can use an OSGi bundle without starting it!
That would be a dangerous situation. It would mean one cannot put any initialization logic into the activator/installer at all. Peter Kriens spent significant amount of his time and patience to explain me the rationale behind this behavior and possible fix:
- Just add Activation-Policy: lazy into your bundle which performs important initialization in BundleActivator and it is guaranteed that as soon as somebody loads a class from your bundle, it is going to be started
- This is not the default to keep things backward compatible - OK, I can understand the value of BackwardCompatibility. Through at this case it might have been better to change the semantics when the bundle manifest version was changed to two (or change it to three). The NetBeans ModuleInstall is guaranteed to be called whenever somebody tries to use a module (by having a dependency on it and trying to get enabled). Although it is recommended to use declarative registrations, when activator/installer is present calling it as soon as the bundle is being used is the natural/expected behavior.
- Peter (as far as I understand) rather advocates splitting the API and the implementation into separate bundles however. The service should rather be registered by independent implementation. I still don't know how my code using the API will force the system to initialize the implementation too, but I expect the answer to be OSGi declarative services (which I am not familiar with).
Possibly Eclipse is using OSGi in wrong way. The singleton initialization in activator complicates the OSGiAndNetBeans bridge. Basically it means we don't have 1:1 mapping between enabling a module in the NetBeans sense and starting a bundle in the OSGi sense. As we discover more and more OSGi based systems (like JDeveloper), we need to add more and more flexibility to cover all the possible ways people deal with the dichotomy between used and started. OSGi is very flexible and people tend to use it in many innovative ways.
At least I know now: the suitable fix is to use Activation-Policy: lazy. The only question remains: How do I convince Eclipse to modify their core bundle manifests to include this tag?
Stopping Order
Another thing related to activators is the order of starting and stopping bundles. Again, if we think about the starting to be associated with use, then it makes sense to start the modules that are used by others sooner, then starting the others. Do a topological sort based on interbundle dependencies and then start the bottom bundles first (with as less dependencies as possible). When stopping a set of bundles, do it from the top. Natural? I think so. Do you think OSGi does that? No, not at all.
I've just needed to deal with a NullPointerException caused by the fact that org.eclipse.equinox.registry is stopped sooner than org.eclipse.equinox.security when calling OSGi framework.stop(). What happened? There is a singleton in the registry and BundleActivator.stop() clears it. But still the module remains available for others to call. When the system tries to stop the security bundle, it calls into the registry trying to obtain the singleton and guess, it is not there!
Why could not the great minds behind OSGi create a specification that would sort the modules using a topological sort before shutting them down? Is that unnatural? No. But rather than that OSGi introduced start levels (which we now have to support in NetBeans). The only conclusion one can have: OSGi is not well thought out.
Range Dependencies
One of the characteristics of OSGi is use of RangeDependencies which seems like a clever good idea supporting cluelessness in its way, but if not done properly it is just a way to mess around with NP-Complete problems. And yes, there are such problems in OSGi as demonstrated by Equinox coming with a 3SAT solver.
On the other hand, this is not complete fault of RangeDependencies. Recent analysis shows that it depends how the RangeDependencies are used. If they are used properly, the NP-Complete problems can be eliminated.
Looking from this point of view: OSGi's high shot (decision to use powerful dependency system) missed the sweet spot (produced underspecified system with corners full of bad consequences).
Executive Summary
I value OSGi for its ability to provide a common ground and allow interoperability between various module systems (like Glassfish, Eclipse, NetBeans, JDeveloper, etc.). Only by having OSGi around could we propose to the JDeveloper team to consider the idea of reusing Netbinox and sharing bits of NetBeans. OSGi is good for interoperability.
However, beyond that, I just can't help but think that OSGi is an overdesigned, and sometimes misdesigned, technology. In addition to that, I'd like to point out that OSGi is so 2000-late. The OSGi spec 4.2 still runs on JDK 1.3... and it shows! These days Java is different. With the help of AnnotationProcessors it can do magic (and it does not need poor and misleading tricks like MagicalStrings).
I fully understand the value of pioneers. Somebody just has to be the first (well second), walk the road, and find out that this is not the right path. I believe OSGi succeeded in this mission. Everyone should understand at this point the value of modularity. Everyone should also see, however, that the OSGi path is not the right one to follow. I just hope the next attempt to standardize a module system for Java will learn from the mistakes of its predecessors (OSGi and NetBeans Runtime Container).
Let's use OSGi for interop for now, and then let's forget about it later.
<comments/>
Defending OSGi
Accusing OSGi deficiencies generated some interesting comments and observations. I am including them here.
Neil Bartlett said ...
Hello Neil, thanks for your comment. Yes, you are right, the page is unlikely good start for a useful discourse. On the other hand, it nicely illustrates my frustration with OSGi. Still, I'd be glad to be enlighten about the design misconceptions.
--JaroslavTulach 10:02, 21 November 2011 (UTC)
Mirko Jahn said ...
First of all, I agree that OSGi is not perfect. In fact I do not know any technology that reached some sort of complexity that doesn't have its flaws. I guess that's nature. However, I think you're missing some background/ experience in the underlying concepts and frameworks you're referring too.
First of all, start/stop of a bundle. Here it is important to distinguish between a mere library and a state aware module. For libraries it is perfectly fine to not have an initialization (static initialization is fine here as well, but no "object" related one). In the OSGi world you then do not need to create an Activator. For all modules that are in fact state aware or need some sort of initialization (like Service dependencies), you could create you Activator (or use a declarative way of expressing that). In case a module does not comply with this contract this is a bug in the bundle code. As a general rule, you only expose API's and provide the instances as services, as long as the bundle is not started, no services will be available and no harm is done. Of course there is the "ugly" overhead of needing to know what bundles need to be started and which don't (you could also start all of them - no overhead in case of library bundles, because they won't execute any code). This seems not entirely clean as an approach, but it is powerful. Just imagine you have one bundle providing the API and a simple default service with the implementation. Once you realized the implementation is not good enough, you could just go ahead, reuse the API and not the exposed service (simply do not start the bundle) and provide your own bundle implementing the service in a way you require it.
Second, stopping order. There is an order in which bundles are stopped and they are stopped in the reverse order they were actually started, so what you saw might be a timing issue or an error in the implementation. Also Eclipse is not a perfect OSGi citizen. They did a lot to improve that but they are still not there. For instance many bundles still do not use the service registry, but a more static eclipse specific construct that has flaws with the dynamism of OSGi. I guess this might explain the problem you were experiencing. Also when done correctly (with services), a stopped bundle would not expose the service any longer and the bundle consuming such service would be able to call such bean (because it wouldn't find it) - no harm could be done here. Unfortunately the developer has to account for that or us a framework that does it for him like BluePrint for instance.
Range dependencies are tricky, I agree. Also the approach on handling the correct version, updating handles in the container, refreshing the wiring between bundles. All that is not trivial and error prone. However, I haven't seen anything even close to what is possible with OSGi. You could by designing the exposed and consumed packages correctly, a powerful and future prove API. Implementers know base on version changes when they have to take a closer look (minor version change) and simple service consumers can rely on the major version as an indicator that something might have changed that they should be aware of. Especially with many modules, keeping track of changes is hell. With such ranges, it at least gives you a way to better track where to look and what to adapt in case something happens. This is still not perfect and I am not sure there is a silver bullet at all, but it's the best thing I've seen so far.
To summarize what I am trying to say. OSGi, like many other specs, is not perfect and there might be a better one in the future, but for the time being, it is pretty good and well thought through. Unfortunately the concepts are not simple and certainly not easy to adhere. Especially with the latest version 4.3, many improvements have been done to make it easier and remove potential for errors, but like every complex system, one first has to know it to fully appreciate it. ;-)
--Mirko Jahn 19:59, 23 November 2011 (CET)
A comment by Jesse Glick left at bug 205019
I think this is a bit off the mark. Seems like Eclipse is not really using OSGi as designed. Proper OSGi bundles are (acc. to spec) supposed to work regardless of what other bundles have or have not been started; that is why dynamic services exist, and why bundles are not started in topological order. In other words, we are working around misdesign in Eclipse, not OSGi per se. The problem presumably does not affect Eclipse and its RCP apps because they are not attempting to load code from an arbitrary compliant OSGi container, and so can make assumptions about runtime behavior not guaranteed by the spec.
--Jesse Glick 14:00, 23 November 2011 (CET)
Reply to Mirko's Post
Hello Mirko, first of all - thank you for your time to write down such a long post here. I value your effort.
"I agree that OSGi is not perfect" - sure, I know that NetBeans Runtime Container is not perfect either. Over years in production you just have to come up with a lot of compromises which, when looking backward, may not seem perfect.
"distinguish between a mere library and a state aware module" - yes, I am aware that nobody needs to use Activator - however that is not the problem. I am puzzled by OSGi allowing to load classes from modules with Activator which has not been called! What kind of strange idea is that? In my opinion the fact that I provide an Activator should mean that whenever somebody wants to use me (e.g. activate me), I want that Activator to be called. Why this is not true in OSGi?
"ugly overhead of needing to know what bundles need to be started" - actually this may be the problem. NetBeans is worshiping so called Injectable Singletons. In this use-case, what ever API you link against means that the API is ready to work and is fully configured. The way Eclipse is using Activators violates this (which may be Eclipse fault as Mirko and Jesse indicate). You say "simply do not start the bundle" - sure, this is what works in NetBeans as well if you don't need a module with implementation of an API, just don't enable it. But still, I don't understand why there is the dichotomy between using and starting a bundle. To allow an API to be bundled with its implementation that is registered in Activator (so others can use the API without the implementation)? That sounds weird!
"reverse order they were actually started" - see, I did not know this. But then the problem is the start order. Obviously, those bundles with as less dependencies as possible, should be started first. However I understand the OSGi may have problem - there does not seem to be a bulk start operation. One can start just a single bundle, not a set of bundles. In NetBeans Runtime Container one just asks to enable a set of bundles and the system starts them in the correct order. The system always guarantee Activators are called on modules needed by somebody. Sounds like a deficiency in OSGi where determining the start order is fully left in hands of poor users of the OSGi API.
"Range dependencies are tricky" - RangeDependencies don't really belong into my most recent rant, but there are issues associated with them - althrough they may be mitigated to avoid NP-Complete problems.
"certainly not easy to adhere" - if a concept is not easy to adhere, then the system is not designed for clueless use. And that is a problem as most of us are completely clueless when using others APIs. Should we threat OSGi as an assembly language for modularity hoping somebody will wrap it with higher level concepts? That would support my conclusions - OSGi helped us see the value of modularity, but proper, widely used modularity should rather not expose OSGi at all.
--JaroslavTulach 18:23, 24 November 2011 (UTC)
Sergey said ...
/ If I do anything in Linux for pmnigomrrag, I use Eclipse if I need an IDE. I also like doing as much as I can in terminal or even a note pad-like program. On Windows (for school), I really like using Notepad++, so the best alternative in Linux for NP++ that I've found is Geany.
--Sergey 14:15, 23 October 2013 (CEST)
Malnormalulo said ...
"In my opinion the fact that I provide an Activator should mean that whenever somebody wants to use me (e.g. activate me), I want that Activator to be called. Why this is not true in OSGi?"
In OSGi, a bundle which has not been started can provide packages (classes), but not services (objects). To me at least, this makes intuitive sense; classes don't need to be initialized*, but objects do.
You may object that a class might, e.g., make use of a singleton which does need initialization. This would be one reason that an OSGi best practice is to only expose an API (i.e., interfaces, exceptions, maybe some simple beans) through your exported packages, and to leave the implementation to service objects defined by private classes. This way, your exported packages remain initialization-free, and references may be made to them without worry regardless of the state of the system.
You may then ask what the point is in allowing an API to be used when the implementation is not available. Even if the API does not, in itself, require initialization, it doesn't necessarily seem useful to have access to it when no implementation is available. In many systems, there's no point at all, and you'd be entirely right to note this as a shortcoming!
But other systems take advantage of OSGi's dynamic nature by doing things like watching for services that implement the API. In the whiteboard pattern (an OSGi-specific variant of the observer pattern) an "observable" object registers no services, instead listening for observers to register services that act as callbacks. When it boots up, the observable code is probably not yet aware of any concrete implementations of the callback interface. Nevertheless, it needs to know about the interface so that it can listen for new implementations that later become available. It is therefore necessary that the API be available even when implementors are not.
* This isn't always true, but if a class does need to be initialized, it can usually just be a singleton object instead, which is what OSGi would recommend.
"However I understand the OSGi may have problem - there does not seem to be a bulk start operation."
This is indeed a problem. Launching OSGi applications is tedious. Various abstractions, like Karaf's "features" and (I think) WebSphere Application Server's "Enterprise Bundle Archives", have been layered on top of OSGi to express structures composed of multiple bundles which may be started and stopped together. There is no standard, though, and I can't vouch for how well any of them actually accomplish the goal of simplifying OSGi deployment.
"Should we threat OSGi as an assembly language for modularity hoping somebody will wrap it with higher level concepts? That would support my conclusions - OSGi helped us see the value of modularity, but proper, widely used modularity should rather not expose OSGi at all."
I, for one, agree. OSGi is too complex to ask every application developer to fully understand it. The assembly language analogy is spot on – OSGi is largely a good thing, but it's in desperate need of higher-level abstractions. Until we have those, we should regard it as a useful way to solve specific problems, but only to be used when actually necessary.
--Malnormalulo 16:39, 15 March 2017 (CET)
Follow
In the introduction you appear to be asking to be enlightened about OSGi, but then you go on to attack it from a position of ignorance and with heaps of bad attitude. Sorry but this is not the way to engage with people when you're asking for help.
If you have some specific questions about how or why OSGi works the way it does, then I and many others will be delighted to help you on the osgi-dev mailing list. Unfortunately this particular page is little more than a rant and can't be the basis for useful discourse.
--Neil Bartlett 00:07, 20 November 2011 (CET)