'. '

ThreadContextClassLoader

From APIDesign

(Difference between revisions)
Jump to: navigation, search
(Happy End)
Current revision (19:52, 8 September 2014) (edit) (undo)
(Isolating ClassLoader)
 
(4 intermediate revisions not shown.)
Line 5: Line 5:
The need for enhanced [[ThreadContextClassLoader]] in [[modular system]]s is clear as documented by [[eclipse:Context_Class_Loader_Enhancements|Equinox class loader enhancements]] document. It summarizes the need as well as the solution taken by [[Equinox]] guys. Everything is built around '''buddies''' - e.g. each module can define its own '''buddy''' loading policy and let others to register is '''buddies'''.
The need for enhanced [[ThreadContextClassLoader]] in [[modular system]]s is clear as documented by [[eclipse:Context_Class_Loader_Enhancements|Equinox class loader enhancements]] document. It summarizes the need as well as the solution taken by [[Equinox]] guys. Everything is built around '''buddies''' - e.g. each module can define its own '''buddy''' loading policy and let others to register is '''buddies'''.
-
The [[NetBeans]] way is slightly different. It knows packages of all ''enabled'' modules and the [[ThreadContextClassLoader]] can load any such classes (even non-public, e.g. not exported). It is build around observation that it does not make much sense in a running application to have the same implementation module loaded twice (if such duplication happens, the [[ThreadContextClassLoader]] yields an error).
+
The [[NetBeans]] way is slightly different. It knows packages of all ''enabled'' modules and the [[ThreadContextClassLoader]] can load any such classes (even non-public, e.g. not exported). It is build around observation that it does not make much sense in a running application to have the same implementation module loaded twice (if such duplication happens, the [[ThreadContextClassLoader]] yields an error) and that dynamically loading classes shall not be automatic, but it shall be relatively easy (no definition of any policies needed).
-
 
+
== Best of Both Worlds? ==
== Best of Both Worlds? ==
Line 13: Line 12:
Everything looked fine, until I started to execute existing tests in this new system. The testing framework (built around [[JUnit]] just like [[NetBeans]] [[NbJUnit]]), has the classes visible twice. Once on application classpath and once in the available bundles. This is a problem. The here-in described [[Netbinox]] [[ThreadContextClassLoader]] can can successfully find a class, if it can be found by at least one of the systems. However it does not guarantee it will be the right one.
Everything looked fine, until I started to execute existing tests in this new system. The testing framework (built around [[JUnit]] just like [[NetBeans]] [[NbJUnit]]), has the classes visible twice. Once on application classpath and once in the available bundles. This is a problem. The here-in described [[Netbinox]] [[ThreadContextClassLoader]] can can successfully find a class, if it can be found by at least one of the systems. However it does not guarantee it will be the right one.
-
The problem is that [[NetBeans]] [[ThreadContextClassLoader]] by default delegates to parent [[ClassLoader]] first (e.g. the class is usually loaded from the application classpath), while the [[Equinox]] [[ThreadContextClassLoader]] prefers the bundle version. When there are two version of a class visible (like in case of unit testing), having union of both worlds is a way to hell. It leads to randomness and bunch of ''ClassCastException''s (as classes of the same name loaded by different [[ClassLoader]]s cannot be cast to each other).
+
The problem is that [[NetBeans]] [[ThreadContextClassLoader]] by default delegates to parent [[ClassLoader]] first (e.g. the class is usually loaded from the application classpath), while the [[Equinox]] [[ThreadContextClassLoader]] prefers the bundle version. When there are two versions of a class visible (like in case of unit testing), having union of both worlds is a way to hell. It leads to randomness and bunch of ''ClassCastException''s (as classes of the same name loaded by different [[ClassLoader]]s cannot be cast to each other).
== Isolating [[ClassLoader]] ==
== Isolating [[ClassLoader]] ==
-
I don't see much reasons for real system to duplicated classes in a module/bundle and on the classpath. It is useful in rare situations (like providing newer ''javac'' or ''JAX-WS'' than available in [[JDK]]), but having all classes twice is an unusual situation. But hard to avoid in the testing environment, if one relies on standard [[Ant]] [[JUnit]] task.
+
I don't see much reasons for real system to duplicate it's classes in a module/bundle and on the classpath. It is useful in rare situations (like providing newer ''javac'' or ''JAX-WS'' than available in [[JDK]]), but having all classes twice is an unusual situation. But hard to avoid in the testing environment, if one relies on standard [[Ant]] [[JUnit]] task.
Thus I decided to write new ''isolating'' [[ClassLoader]]. One that does not load anything from classpath (except own [[JUnit]] classes). By using this [[ClassLoader]] as a parent for [[NetBeans]] or for [[Equinox]], one can make sure the system basically sees only one class and thus the union again becomes the best of ''both worlds''.
Thus I decided to write new ''isolating'' [[ClassLoader]]. One that does not load anything from classpath (except own [[JUnit]] classes). By using this [[ClassLoader]] as a parent for [[NetBeans]] or for [[Equinox]], one can make sure the system basically sees only one class and thus the union again becomes the best of ''both worlds''.
Line 36: Line 35:
</source>
</source>
-
Looks like there is a bug in [[Equinox]] that sometimes (surprise!) prefers class on classpath rather than one from its '''buddy'' bundle. The fix was then a simple one liner change:
+
Looks like there is a bug in [[Equinox]] that sometimes (surprisingly the behavior is really random) prefers class on classpath rather than one from its '''buddy'' bundle. The fix was then a simple one liner change:
<source lang="java">
<source lang="java">
Line 42: Line 41:
</source>
</source>
-
Then the [[Equinox]] finally completely ignored the classpath classes and really uses the ''isolating'' [ClassLoader]]. Majority of the tests started to work.
+
Then the [[Equinox]] finally completely ignored the classpath classes and really used the ''isolating'' [[ClassLoader]]. Majority of the tests started to work.
== Happy End ==
== Happy End ==

Current revision

Java provides way to associate a special ClassLoader with each executing thread. This is handy, as often the executing class needs more classes than those visible by its own ClassLoader. However it is also accompanied with problems. Especially in general purpose modular systems, when it is not easy to control thread management. If any part of application can create and destroy threads on the fly, it is hard to make sure the ThreadContextClassLoader behavior is specified properly.

Things get even more complicated as each modular runtime container may have different idea what ThreadContextClassLoader shall do. I know what I am talking about, I have spend last month trying to reimplement Netbinox OSGi container to provide ThreadContextClassLoader which is friendly to NetBeans modules as well as bundles comming from Equinox world.

The need for enhanced ThreadContextClassLoader in modular systems is clear as documented by Equinox class loader enhancements document. It summarizes the need as well as the solution taken by Equinox guys. Everything is built around buddies - e.g. each module can define its own buddy loading policy and let others to register is buddies.

The NetBeans way is slightly different. It knows packages of all enabled modules and the ThreadContextClassLoader can load any such classes (even non-public, e.g. not exported). It is build around observation that it does not make much sense in a running application to have the same implementation module loaded twice (if such duplication happens, the ThreadContextClassLoader yields an error) and that dynamically loading classes shall not be automatic, but it shall be relatively easy (no definition of any policies needed).

Best of Both Worlds?

It is not hard to union the behavior of both systems. I did that quite easily: If the requested class is not found the NetBeans way, then Netbinox tries the classical Equinox buddy style. This guarantees that if the class is present and one of the systems can load it, it will be loaded. So classes that belong to enabled modules or are accessible via the buddy policy are available.

Everything looked fine, until I started to execute existing tests in this new system. The testing framework (built around JUnit just like NetBeans NbJUnit), has the classes visible twice. Once on application classpath and once in the available bundles. This is a problem. The here-in described Netbinox ThreadContextClassLoader can can successfully find a class, if it can be found by at least one of the systems. However it does not guarantee it will be the right one.

The problem is that NetBeans ThreadContextClassLoader by default delegates to parent ClassLoader first (e.g. the class is usually loaded from the application classpath), while the Equinox ThreadContextClassLoader prefers the bundle version. When there are two versions of a class visible (like in case of unit testing), having union of both worlds is a way to hell. It leads to randomness and bunch of ClassCastExceptions (as classes of the same name loaded by different ClassLoaders cannot be cast to each other).

Isolating ClassLoader

I don't see much reasons for real system to duplicate it's classes in a module/bundle and on the classpath. It is useful in rare situations (like providing newer javac or JAX-WS than available in JDK), but having all classes twice is an unusual situation. But hard to avoid in the testing environment, if one relies on standard Ant JUnit task.

Thus I decided to write new isolating ClassLoader. One that does not load anything from classpath (except own JUnit classes). By using this ClassLoader as a parent for NetBeans or for Equinox, one can make sure the system basically sees only one class and thus the union again becomes the best of both worlds.

Still my problems were not over. From time to time the system managed to load a class from classpath, in spite it was available in a bundle (and the classpath shall not be visible). It took me next few days to track the random problem and attribute that to following code in Equinox's BundleLoader.java:

// hack to support backwards compatibiility for bootdelegation
// or last resort; do class context trick to work around VM bugs
if (parentCL != null && !bootDelegation && ((checkParent && 
bundle.getFramework().compatibiltyBootDelegation) || isRequestFromVM()))
  // we don't need to continue if a CNFE is thrown here.
  try {
    return parentCL.loadClass(name);
  } catch (ClassNotFoundException e) {
    // we want to generate our own exception below
}

Looks like there is a bug in Equinox that sometimes (surprisingly the behavior is really random) prefers class on classpath rather than one from its 'buddy bundle. The fix was then a simple one liner change:

System.setProperty("osgi.parentClassloader", "fwk");

Then the Equinox finally completely ignored the classpath classes and really used the isolating ClassLoader. Majority of the tests started to work.

Happy End

Later I found out that small amount of tests was still failing when using the "fwk" value. I switched it back to "app" (to see classes from classpath) and often that helped. Morale of this story? ThreadContextClassLoader is an important type of API. If you are writing a framework, make sure you define its behavior.

Personal tools
buy