ThreadContextClassLoader - Revision history

78.45.39.147: /* Isolating ClassLoader */

78.45.39.147 — Mon, 08 Sep 2014 19:52:30 GMT

Isolating ClassLoader

←Older revision		Revision as of 19:52, 8 September 2014
Line 16:		Line 16:
	== Isolating [[ClassLoader]] ==		== Isolating [[ClassLoader]] ==

-	I don't see much reasons for real system to ~~duplicated~~ classes in a module/bundle and on the classpath. It is useful in rare situations (like providing newer ''javac'' or ''JAX-WS'' than available in [[JDK]]), but having all classes twice is an unusual situation. But hard to avoid in the testing environment, if one relies on standard [[Ant]] [[JUnit]] task.	+	I don't see much reasons for real system to duplicate it's classes in a module/bundle and on the classpath. It is useful in rare situations (like providing newer ''javac'' or ''JAX-WS'' than available in [[JDK]]), but having all classes twice is an unusual situation. But hard to avoid in the testing environment, if one relies on standard [[Ant]] [[JUnit]] task.

	Thus I decided to write new ''isolating'' [[ClassLoader]]. One that does not load anything from classpath (except own [[JUnit]] classes). By using this [[ClassLoader]] as a parent for [[NetBeans]] or for [[Equinox]], one can make sure the system basically sees only one class and thus the union again becomes the best of ''both worlds''.		Thus I decided to write new ''isolating'' [[ClassLoader]]. One that does not load anything from classpath (except own [[JUnit]] classes). By using this [[ClassLoader]] as a parent for [[NetBeans]] or for [[Equinox]], one can make sure the system basically sees only one class and thus the union again becomes the best of ''both worlds''.

JaroslavTulach at 10:40, 9 November 2010

JaroslavTulach — Tue, 09 Nov 2010 10:40:16 GMT

←Older revision		Revision as of 10:40, 9 November 2010
Line 5:		Line 5:
	The need for enhanced [[ThreadContextClassLoader]] in [[modular system]]s is clear as documented by [[eclipse:Context_Class_Loader_Enhancements\|Equinox class loader enhancements]] document. It summarizes the need as well as the solution taken by [[Equinox]] guys. Everything is built around '''buddies''' - e.g. each module can define its own '''buddy''' loading policy and let others to register is '''buddies'''.		The need for enhanced [[ThreadContextClassLoader]] in [[modular system]]s is clear as documented by [[eclipse:Context_Class_Loader_Enhancements\|Equinox class loader enhancements]] document. It summarizes the need as well as the solution taken by [[Equinox]] guys. Everything is built around '''buddies''' - e.g. each module can define its own '''buddy''' loading policy and let others to register is '''buddies'''.

-	The [[NetBeans]] way is slightly different. It knows packages of all ''enabled'' modules and the [[ThreadContextClassLoader]] can load any such classes (even non-public, e.g. not exported). It is build around observation that it does not make much sense in a running application to have the same implementation module loaded twice (if such duplication happens, the [[ThreadContextClassLoader]] yields an error).	+	The [[NetBeans]] way is slightly different. It knows packages of all ''enabled'' modules and the [[ThreadContextClassLoader]] can load any such classes (even non-public, e.g. not exported). It is build around observation that it does not make much sense in a running application to have the same implementation module loaded twice (if such duplication happens, the [[ThreadContextClassLoader]] yields an error) and that dynamically loading classes shall not be automatic, but it shall be relatively easy (no definition of any policies needed).
-		+
	== Best of Both Worlds? ==		== Best of Both Worlds? ==

JaroslavTulach: /* Isolating ClassLoader */

JaroslavTulach — Tue, 09 Nov 2010 10:30:07 GMT

Isolating ClassLoader

←Older revision		Revision as of 10:30, 9 November 2010
Line 36:		Line 36:
	</source>		</source>

-	Looks like there is a bug in [[Equinox]] that sometimes (~~surprise!~~) prefers class on classpath rather than one from its '''buddy'' bundle. The fix was then a simple one liner change:	+	Looks like there is a bug in [[Equinox]] that sometimes (surprisingly the behavior is really random) prefers class on classpath rather than one from its '''buddy'' bundle. The fix was then a simple one liner change:

	<source lang="java">		<source lang="java">

JaroslavTulach: /* Best of Both Worlds? */

JaroslavTulach — Tue, 09 Nov 2010 10:27:54 GMT

Best of Both Worlds?

←Older revision		Revision as of 10:27, 9 November 2010
Line 13:		Line 13:
	Everything looked fine, until I started to execute existing tests in this new system. The testing framework (built around [[JUnit]] just like [[NetBeans]] [[NbJUnit]]), has the classes visible twice. Once on application classpath and once in the available bundles. This is a problem. The here-in described [[Netbinox]] [[ThreadContextClassLoader]] can can successfully find a class, if it can be found by at least one of the systems. However it does not guarantee it will be the right one.		Everything looked fine, until I started to execute existing tests in this new system. The testing framework (built around [[JUnit]] just like [[NetBeans]] [[NbJUnit]]), has the classes visible twice. Once on application classpath and once in the available bundles. This is a problem. The here-in described [[Netbinox]] [[ThreadContextClassLoader]] can can successfully find a class, if it can be found by at least one of the systems. However it does not guarantee it will be the right one.

-	The problem is that [[NetBeans]] [[ThreadContextClassLoader]] by default delegates to parent [[ClassLoader]] first (e.g. the class is usually loaded from the application classpath), while the [[Equinox]] [[ThreadContextClassLoader]] prefers the bundle version. When there are two ~~version~~ of a class visible (like in case of unit testing), having union of both worlds is a way to hell. It leads to randomness and bunch of ''ClassCastException''s (as classes of the same name loaded by different [[ClassLoader]]s cannot be cast to each other).	+	The problem is that [[NetBeans]] [[ThreadContextClassLoader]] by default delegates to parent [[ClassLoader]] first (e.g. the class is usually loaded from the application classpath), while the [[Equinox]] [[ThreadContextClassLoader]] prefers the bundle version. When there are two versions of a class visible (like in case of unit testing), having union of both worlds is a way to hell. It leads to randomness and bunch of ''ClassCastException''s (as classes of the same name loaded by different [[ClassLoader]]s cannot be cast to each other).

	== Isolating [[ClassLoader]] ==		== Isolating [[ClassLoader]] ==

JaroslavTulach: /* Isolating ClassLoader */

JaroslavTulach — Mon, 08 Nov 2010 09:09:14 GMT

Isolating ClassLoader

←Older revision		Revision as of 09:09, 8 November 2010
Line 42:		Line 42:
	</source>		</source>

-	Then the [[Equinox]] finally completely ignored the classpath classes and really ~~uses~~ the ''isolating'' [ClassLoader]]. Majority of the tests started to work.	+	Then the [[Equinox]] finally completely ignored the classpath classes and really used the ''isolating'' [[ClassLoader]]. Majority of the tests started to work.

	== Happy End ==		== Happy End ==

JaroslavTulach: /* Happy End */

JaroslavTulach — Mon, 08 Nov 2010 09:08:39 GMT

Happy End

←Older revision		Revision as of 09:08, 8 November 2010
Line 46:		Line 46:
	== Happy End ==		== Happy End ==

-	Later I found out that small amount of tests was still failing when using the "fwk" value. I ~~tried to switch~~ it back to "app" and often that helped. Morale of this story? [[ThreadContextClassLoader]] is an important [[APITypes\|type of API]]. If you are writing a framework make sure you define [[ThreadContextClassLoader\|its]] behavior.	+	Later I found out that small amount of tests was still failing when using the "fwk" value. I switched it back to "app" (to see classes from classpath) and often that helped. Morale of this story? [[ThreadContextClassLoader]] is an important [[APITypes\|type of API]]. If you are writing a framework, make sure you define [[ThreadContextClassLoader\|its]] behavior.

	[[Category:APITypes]]		[[Category:APITypes]]

JaroslavTulach: /* The NbJUnit ClassLoader */

JaroslavTulach — Mon, 08 Nov 2010 09:07:37 GMT

The NbJUnit ClassLoader

←Older revision		Revision as of 09:07, 8 November 2010
Line 15:		Line 15:
	The problem is that [[NetBeans]] [[ThreadContextClassLoader]] by default delegates to parent [[ClassLoader]] first (e.g. the class is usually loaded from the application classpath), while the [[Equinox]] [[ThreadContextClassLoader]] prefers the bundle version. When there are two version of a class visible (like in case of unit testing), having union of both worlds is a way to hell. It leads to randomness and bunch of ''ClassCastException''s (as classes of the same name loaded by different [[ClassLoader]]s cannot be cast to each other).		The problem is that [[NetBeans]] [[ThreadContextClassLoader]] by default delegates to parent [[ClassLoader]] first (e.g. the class is usually loaded from the application classpath), while the [[Equinox]] [[ThreadContextClassLoader]] prefers the bundle version. When there are two version of a class visible (like in case of unit testing), having union of both worlds is a way to hell. It leads to randomness and bunch of ''ClassCastException''s (as classes of the same name loaded by different [[ClassLoader]]s cannot be cast to each other).

-	== ~~The [[NbJUnit]]~~ [[ClassLoader]] ==	+	== Isolating [[ClassLoader]] ==

-	I don't see much reasons ~~why classes in~~ real system ~~shall be~~ duplicated in a bundle and on classpath ~~except~~ in rare situations (like providing newer ''javac'' or ''JAX-WS'' than available in [[JDK]])~~. Thus the [[JUnit]] class duplication~~ is an unusual situation. But hard to ~~overcome~~ if one relies on standard [[Ant]] [[JUnit]] task.	+	I don't see much reasons for real system to duplicated classes in a module/bundle and on the classpath. It is useful in rare situations (like providing newer ''javac'' or ''JAX-WS'' than available in [[JDK]]), but having all classes twice is an unusual situation. But hard to avoid in the testing environment, if one relies on standard [[Ant]] [[JUnit]] task.

-	Thus I decided to write new ''isolating'' [[ClassLoader]]. One that does not load anything from classpath (except own [[JUnit]] classes). By using this [[ClassLoader]] as a parent for [[NetBeans]] or for [[Equinox]], one can make sure the system basically sees only one class and thus the best of ''both worlds'' ~~again becomes beneficial~~.	+	Thus I decided to write new ''isolating'' [[ClassLoader]]. One that does not load anything from classpath (except own [[JUnit]] classes). By using this [[ClassLoader]] as a parent for [[NetBeans]] or for [[Equinox]], one can make sure the system basically sees only one class and thus the union again becomes the best of ''both worlds''.

-	Still my problems were not over. From time to time the system managed to load a class from classpath, in spite it was available in a bundle (and the classpath shall not be visible). It took me next few days to ~~find out I have~~ to ~~use~~:	+	Still my problems were not over. From time to time the system managed to load a class from classpath, in spite it was available in a bundle (and the classpath shall not be visible). It took me next few days to track the random problem and attribute that to following code in [[Equinox]]'s BundleLoader.java:
		+
		+	<source lang="java">
		+	// hack to support backwards compatibiility for bootdelegation
		+	// or last resort; do class context trick to work around VM bugs
		+	if (parentCL != null && !bootDelegation && ((checkParent &&
		+	bundle.getFramework().compatibiltyBootDelegation) \|\| isRequestFromVM()))
		+	// we don't need to continue if a CNFE is thrown here.
		+	try {
		+	return parentCL.loadClass(name);
		+	} catch (ClassNotFoundException e) {
		+	// we want to generate our own exception below
		+	}
		+	</source>
		+
		+	Looks like there is a bug in [[Equinox]] that sometimes (surprise!) prefers class on classpath rather than one from its '''buddy'' bundle. The fix was then a simple one liner change:

	<source lang="java">		<source lang="java">
Line 27:		Line 42:
	</source>		</source>

-	~~Only then~~ the [[Equinox]] ~~stops loading classes from~~ the classpath and really uses the ''isolating'' [ClassLoader]]. ~~Tests~~ started to ~~almost~~ work.	+	Then the [[Equinox]] finally completely ignored the classpath classes and really uses the ''isolating'' [ClassLoader]]. Majority of the tests started to work.

	== Happy End ==		== Happy End ==

JaroslavTulach at 08:59, 8 November 2010

JaroslavTulach — Mon, 08 Nov 2010 08:59:05 GMT

←Older revision		Revision as of 08:59, 8 November 2010
Line 9:		Line 9:
	== Best of Both Worlds? ==		== Best of Both Worlds? ==

-	It is ~~easy~~ to ~~create a~~ union behavior ~~that will support~~ both ~~ways~~. ~~And~~ I did it quite easily. If the class is not found the [[NetBeans]] way, then [[Netbinox]] tries the classical [[Equinox]] ~~way~~. This guarantees that if the class is present ~~in the system~~ and ~~either belongs~~ to enabled ~~module~~ or ~~it is~~ accessible via '''buddy''' policy~~, it will be loaded~~.	+	It is not hard to union the behavior of both systems. I did that quite easily: If the requested class is not found the [[NetBeans]] way, then [[Netbinox]] tries the classical [[Equinox]] '''buddy''' style. This guarantees that if the class is present and one of the systems can load it, it will be loaded. So classes that belong to enabled modules or are accessible via the '''buddy''' policy are available.

-	Everything looked fine, until I started to execute existing tests in this new system. The testing framework (built around [[JUnit]] just like [[NetBeans]] [[NbJUnit]]), has the classes visible twice. Once on application classpath and once in the available bundles. ~~I claimed that I~~ can successfully find a class, if it can be found ~~just a paragraph ago~~. ~~Yes, I can, but I cannot~~ guarantee it will be the right one. The problem is [[NetBeans]] [[ThreadContextClassLoader]] by default delegate to parent [[ClassLoader]] first (e.g. the class is usually loaded from the application classpath), while the [[Equinox]] [[ThreadContextClassLoader]] prefers the bundle version.	+	Everything looked fine, until I started to execute existing tests in this new system. The testing framework (built around [[JUnit]] just like [[NetBeans]] [[NbJUnit]]), has the classes visible twice. Once on application classpath and once in the available bundles. This is a problem. The here-in described [[Netbinox]] [[ThreadContextClassLoader]] can can successfully find a class, if it can be found by at least one of the systems. However it does not guarantee it will be the right one.

-	When there are two version of a class visible (like in case of unit testing), having ~~best~~ of both worlds is a way to hell. It leads to randomness and bunch of ''ClassCastException''s.	+	The problem is that [[NetBeans]] [[ThreadContextClassLoader]] by default delegates to parent [[ClassLoader]] first (e.g. the class is usually loaded from the application classpath), while the [[Equinox]] [[ThreadContextClassLoader]] prefers the bundle version. When there are two version of a class visible (like in case of unit testing), having union of both worlds is a way to hell. It leads to randomness and bunch of ''ClassCastException''s (as classes of the same name loaded by different [[ClassLoader]]s cannot be cast to each other).

	== The [[NbJUnit]] [[ClassLoader]] ==		== The [[NbJUnit]] [[ClassLoader]] ==

JaroslavTulach at 08:51, 8 November 2010

JaroslavTulach — Mon, 08 Nov 2010 08:51:04 GMT

←Older revision		Revision as of 08:51, 8 November 2010
Line 1:		Line 1:
-	[[Java]] provides way to associate a special ~~class loader~~ with each of executing ~~threads~~. This is handy, as often the executing class needs ~~access to~~ more ~~than~~ classes visible by its own [[ClassLoader]]. However it is also accompanied with problems. Especially in general purpose [[modular system]]s, when it is not easy to control thread management. If any part of application can create and destroy threads on the fly, it is hard to make sure the [[ThreadContextClassLoader]] behavior is specified properly.	+	[[Java]] provides way to associate a special [[ClassLoader]] with each executing thread. This is handy, as often the executing class needs more classes than those visible by its own [[ClassLoader]]. However it is also accompanied with problems. Especially in general purpose [[modular system]]s, when it is not easy to control thread management. If any part of application can create and destroy threads on the fly, it is hard to make sure the [[ThreadContextClassLoader]] behavior is specified properly.

	Things get even more complicated as each [[NetBeans Runtime Container\|modular runtime container]] may have different idea what [[ThreadContextClassLoader]] shall do. I know what I am talking about, I have spend last month trying to reimplement [[Netbinox]] [[OSGi]] container to provide [[ThreadContextClassLoader]] which is friendly to [[NetBeans]] modules as well as bundles comming from [[Equinox]] world.		Things get even more complicated as each [[NetBeans Runtime Container\|modular runtime container]] may have different idea what [[ThreadContextClassLoader]] shall do. I know what I am talking about, I have spend last month trying to reimplement [[Netbinox]] [[OSGi]] container to provide [[ThreadContextClassLoader]] which is friendly to [[NetBeans]] modules as well as bundles comming from [[Equinox]] world.

-	The need for enhanced [[ThreadContextClassLoader]] is clear as ~~can be seen summary about~~ [[eclipse:Context_Class_Loader_Enhancements\|Equinox class loader enhancements]]. It summarizes the need as well as the solution taken by [[Equinox]] guys. Everything is built around '''buddies''' - e.g. each module can define its own '''buddy''' loading policy and let others to register is '''buddies'''.	+	The need for enhanced [[ThreadContextClassLoader]] in [[modular system]]s is clear as documented by [[eclipse:Context_Class_Loader_Enhancements\|Equinox class loader enhancements]] document. It summarizes the need as well as the solution taken by [[Equinox]] guys. Everything is built around '''buddies''' - e.g. each module can define its own '''buddy''' loading policy and let others to register is '''buddies'''.

-	The [[NetBeans]] ~~ways~~ is slightly different. It knows packages of all ''enabled'' modules and the [[ThreadContextClassLoader]] can load any classes ~~from any of their packages~~ (even non-public, e.g. not exported). It is build around observation that it does not make much sense in a running application to have the same implementation module loaded twice.	+	The [[NetBeans]] way is slightly different. It knows packages of all ''enabled'' modules and the [[ThreadContextClassLoader]] can load any such classes (even non-public, e.g. not exported). It is build around observation that it does not make much sense in a running application to have the same implementation module loaded twice (if such duplication happens, the [[ThreadContextClassLoader]] yields an error).

	== Best of Both Worlds? ==		== Best of Both Worlds? ==

JaroslavTulach at 08:23, 8 November 2010

JaroslavTulach — Mon, 08 Nov 2010 08:23:46 GMT

←Older revision		Revision as of 08:23, 8 November 2010
Line 1:		Line 1:
-	[[Java]] provides way to associate a special class loader with each of executing threads. This is handy, as often the executing class needs access to more than classes visible by its own ~~classloader~~. However it is also accompanied with problems. Especially in general purpose [[modular system]]s, when it is not easy to control thread management. If any part of application can create and destroy threads on the fly, it is hard to make sure the [[ThreadContextClassLoader]] behavior is specified properly.	+	[[Java]] provides way to associate a special class loader with each of executing threads. This is handy, as often the executing class needs access to more than classes visible by its own [[ClassLoader]]. However it is also accompanied with problems. Especially in general purpose [[modular system]]s, when it is not easy to control thread management. If any part of application can create and destroy threads on the fly, it is hard to make sure the [[ThreadContextClassLoader]] behavior is specified properly.

	Things get even more complicated as each [[NetBeans Runtime Container\|modular runtime container]] may have different idea what [[ThreadContextClassLoader]] shall do. I know what I am talking about, I have spend last month trying to reimplement [[Netbinox]] [[OSGi]] container to provide [[ThreadContextClassLoader]] which is friendly to [[NetBeans]] modules as well as bundles comming from [[Equinox]] world.		Things get even more complicated as each [[NetBeans Runtime Container\|modular runtime container]] may have different idea what [[ThreadContextClassLoader]] shall do. I know what I am talking about, I have spend last month trying to reimplement [[Netbinox]] [[OSGi]] container to provide [[ThreadContextClassLoader]] which is friendly to [[NetBeans]] modules as well as bundles comming from [[Equinox]] world.
Line 11:		Line 11:
	It is easy to create a union behavior that will support both ways. And I did it quite easily. If the class is not found the [[NetBeans]] way, then [[Netbinox]] tries the classical [[Equinox]] way. This guarantees that if the class is present in the system and either belongs to enabled module or it is accessible via '''buddy''' policy, it will be loaded.		It is easy to create a union behavior that will support both ways. And I did it quite easily. If the class is not found the [[NetBeans]] way, then [[Netbinox]] tries the classical [[Equinox]] way. This guarantees that if the class is present in the system and either belongs to enabled module or it is accessible via '''buddy''' policy, it will be loaded.

-	Everything looked fine, until I started to execute existing tests in this new system. The testing framework (built around [[JUnit]] just like [[NetBeans]] [[NbJUnit]]), has the classes visible twice. Once on application classpath and once in the available bundles. I claimed that I can successfully find a class, if it can be found just a paragraph ago. Yes, I can, but I cannot guarantee it will be the right one. The problem is [[NetBeans]] [[ThreadContextClassLoader]] by default delegate to parent ~~classloader~~ first (e.g. the class is usually loaded from the application classpath), while the [[Equinox]] [[ThreadContextClassLoader]] prefers the bundle version.	+	Everything looked fine, until I started to execute existing tests in this new system. The testing framework (built around [[JUnit]] just like [[NetBeans]] [[NbJUnit]]), has the classes visible twice. Once on application classpath and once in the available bundles. I claimed that I can successfully find a class, if it can be found just a paragraph ago. Yes, I can, but I cannot guarantee it will be the right one. The problem is [[NetBeans]] [[ThreadContextClassLoader]] by default delegate to parent [[ClassLoader]] first (e.g. the class is usually loaded from the application classpath), while the [[Equinox]] [[ThreadContextClassLoader]] prefers the bundle version.

	When there are two version of a class visible (like in case of unit testing), having best of both worlds is a way to hell. It leads to randomness and bunch of ''ClassCastException''s.		When there are two version of a class visible (like in case of unit testing), having best of both worlds is a way to hell. It leads to randomness and bunch of ''ClassCastException''s.

-	== The [[NbJUnit]] ClassLoader ==	+	== The [[NbJUnit]] [[ClassLoader]] ==

	I don't see much reasons why classes in real system shall be duplicated in a bundle and on classpath except in rare situations (like providing newer ''javac'' or ''JAX-WS'' than available in [[JDK]]). Thus the [[JUnit]] class duplication is an unusual situation. But hard to overcome if one relies on standard [[Ant]] [[JUnit]] task.		I don't see much reasons why classes in real system shall be duplicated in a bundle and on classpath except in rare situations (like providing newer ''javac'' or ''JAX-WS'' than available in [[JDK]]). Thus the [[JUnit]] class duplication is an unusual situation. But hard to overcome if one relies on standard [[Ant]] [[JUnit]] task.

-	Thus I decided to write new ''isolating'' ~~classloader~~. One that does not load anything from classpath (except own [[JUnit]] classes). By using this ~~classloader~~ as a parent for [[NetBeans]] or for [[Equinox]], one can make sure the system basically sees only one class and thus the best of ''both worlds'' again becomes beneficial.	+	Thus I decided to write new ''isolating'' [[ClassLoader]]. One that does not load anything from classpath (except own [[JUnit]] classes). By using this [[ClassLoader]] as a parent for [[NetBeans]] or for [[Equinox]], one can make sure the system basically sees only one class and thus the best of ''both worlds'' again becomes beneficial.

	Still my problems were not over. From time to time the system managed to load a class from classpath, in spite it was available in a bundle (and the classpath shall not be visible). It took me next few days to find out I have to use:		Still my problems were not over. From time to time the system managed to load a class from classpath, in spite it was available in a bundle (and the classpath shall not be visible). It took me next few days to find out I have to use:
Line 27:		Line 27:
	</source>		</source>

-	Only then the [[Equinox]] stops loading classes from the classpath and really uses the ''isolating'' ~~classloader~~. Tests started to almost work.	+	Only then the [[Equinox]] stops loading classes from the classpath and really uses the ''isolating'' [ClassLoader]]. Tests started to almost work.

	== Happy End ==		== Happy End ==