I have just find out a trick to prevent the ''victims'' mentioned in the previous paragraph. The [[OpenJDK]] is considering to change the specification of the [[Java]] [[HotSpot|virtual machine]] to allow a class with not fully resolved methods to be loaded into the runtime and work fine. Everything is supposed to work until one calls the method which is not resolvable (for example '''AppletInitializer''' class would be missing for one of '''Beans''' methods), then an exception shall be thrown.

I have just find out a trick to prevent the ''victims'' mentioned in the previous paragraph. The [[OpenJDK]] is considering to change the specification of the [[Java]] [[HotSpot|virtual machine]] to allow a class with not fully resolved methods to be loaded into the runtime and work fine. Everything is supposed to work until one calls the method which is not resolvable (for example '''AppletInitializer''' class would be missing for one of '''Beans''' methods), then an exception shall be thrown.

Very useful trick. This shows what one can achieve when thinking without boundaries and also, when one is allowed to ''change the rules of the game''. When I did the modularization of [[NetBeans]] [[API]]s, I obviously could not change the [[HotSpot|virtual machine]] spec. I was left with only one option: I had to ''sneak simplicity'' in. I had to deprecate classes with too large ''conceptual surface'' and replace them with similar classes without unneeded [[dependencies]]. This was fine in case of '''Beans''' like classes that no other [[API]] referred to (in signatures). However as this deprecation is transitive, it becomes quite hard to replace '''java.awt.Image''' - one would need to deprecate also '''java.awt.Toolkit''' and '''java.awt.Graphics''', and so on, so on transitively. It is impossible to get rid of a single method in '''java.lang.String''' using this style, obviously.

Very useful trick. This shows what one can achieve when thinking without boundaries and also, when one is allowed to ''change the rules of the game''. When I did the modularization of [[NetBeans]] [[API]]s, I obviously could not change the [[HotSpot|virtual machine]] spec. I was left with only one option: I had to ''sneak simplicity'' in. I had to deprecate classes with too large [[conceptual surface]] and replace them with similar classes without unneeded [[dependencies]]. This was fine in case of '''Beans''' like classes that no other [[API]] referred to (in signatures). However as this deprecation is transitive, it becomes quite hard to replace '''java.awt.Image''' - one would need to deprecate also '''java.awt.Toolkit''' and '''java.awt.Graphics''', and so on, so on transitively. It is impossible to get rid of a single method in '''java.lang.String''' using this style, obviously.

Thus it is good there will be chance to do ''partial resolution'' of classes by the [[HotSpot|virtual machine]]. Especially when one faces a ''sneak in simplicity'' problem with too big transitive closure, this can be very helpful. On the other hand I can imagine problems with complicated (e.g. not enough [[Cluelessness|clueless]]) [[Javadoc]] that will mention ''conditional'' method and conditions when they are allow to work, etc. That is why in the '''Beans''' case, it still may be better to ''sneak the simplicity in'' and classically deprecate the old class and provide a modern replacement.

Thus it is good there will be chance to do ''partial resolution'' of classes by the [[HotSpot|virtual machine]]. Especially when one faces a ''sneak in simplicity'' problem with too big transitive closure, this can be very helpful. On the other hand I can imagine problems with complicated (e.g. not enough [[Cluelessness|clueless]]) [[Javadoc]] that will mention ''conditional'' method and conditions when they are allow to work, etc. That is why in the '''Beans''' case, it still may be better to ''sneak the simplicity in'' and classically deprecate the old class and provide a modern replacement.

The purpose of such module is to keep [[BackwardCompatibility]] only. It will have dependencies on all other modules in your system, and as such it can contain classes that do not fit anywhere else. Users of previous version of your [[API]] should see this module by default, so their previous dependencies are satisfied.

The purpose of such module is to keep [[BackwardCompatibility]] only. It will have dependencies on all other modules in your system, and as such it can contain classes that do not fit anywhere else. Users of previous version of your [[API]] should see this module by default, so their previous dependencies are satisfied.

On the other hand users of your new version, shall not care and shall use other classes in properly modularized [[API]]s that have smaller ''conceptual surface'' and smaller compile type and runtime [[dependencies]].

On the other hand users of your new version, shall not care and shall use other classes in properly modularized [[API]]s that have smaller [[conceptual surface]] and smaller compile type and runtime [[dependencies]].

For those interested, here is the final diff of the java.beans and java.applet separation:

For those interested, here is the final diff of the java.beans and java.applet separation:

So things are looking good. With just one problem: There is a static method in '''Beans''' class that takes '''AppletInitializer''' parameter. Right now it is commented out, but for the sake of [[BackwardCompatibility]] I need to return it back? Another puzzle! What shall I do now?

So things are looking good. With just one problem: There is a static method in '''Beans''' class that takes '''AppletInitializer''' parameter. Right now it is commented out, but for the sake of [[BackwardCompatibility]] I need to return it back? Another puzzle! What shall I do now?

Well, the basic trick is to ''sneak in simplicity''. Of course ''simplicity'' can have various meanings, but in this context it means number of outgoing [[dependencies]]. The '''Beans''' class is not simple, because it has dependency on beans, as well as applet classes. If we can replace it with some other class, that does not depend on applet, then we will simplify the [[API]]. Sometimes this is called [conceptual surface] - the amount of concepts one needs to understand when dealing with an [[API]]. By removing the need for users of a class to know anything about applet, we ''simplify'' its surface. Not only that, we also allow it to be compilable without applet being around (which is actually the most important goal when modularizing an [[API]]).

Well, the basic trick is to ''sneak in simplicity''. Of course ''simplicity'' can have various meanings, but in this context it means number of outgoing [[dependencies]]. The '''Beans''' class is not simple, because it has dependency on beans, as well as applet classes. If we can replace it with some other class, that does not depend on applet, then we will simplify the [[API]]. Sometimes this is called [[conceptual surface]] - the amount of concepts one needs to understand when dealing with an [[API]]. By removing the need for users of a class to know anything about applet, we ''simplify'' its surface. Not only that, we also allow it to be compilable without applet being around (which is actually the most important goal when modularizing an [[API]]).

The only question is how to ''simplify'' the ''Beans'' class? Of course, the simplest way is to remove the one static method that references '''Applet''' - however this is horribly [[BackwardCompatibility|backward incompatible]] and compatibility for existing clients in our highest priority. Thus the only compile time option is to deprecate the whole Beans class and replace it with some other, ''simplified'' one.

The only question is how to ''simplify'' the ''Beans'' class? Of course, the simplest way is to remove the one static method that references '''Applet''' - however this is horribly [[BackwardCompatibility|backward incompatible]] and compatibility for existing clients in our highest priority. Thus the only compile time option is to deprecate the whole Beans class and replace it with some other, ''simplified'' one.

So things are looking good. With just one problem: There is a static method in '''Beans''' class that takes '''AppletInitializer''' parameter. Right now it is commented out, but for the sake of [[BackwardCompatibility]] I need to return it back? Another puzzle! What shall I do now?

So things are looking good. With just one problem: There is a static method in '''Beans''' class that takes '''AppletInitializer''' parameter. Right now it is commented out, but for the sake of [[BackwardCompatibility]] I need to return it back? Another puzzle! What shall I do now?

Well, the basic trick is to ''sneak in simplicity''. Of course ''simplicity'' can have various meanings, but in this context it means number of outgoing [[dependencies]]. The '''Beans''' class is not simple, because it has dependency on beans, as well as applet classes. If we can replace it with some other class, that does not depend on applet, then we will simplify the [[API]]. Sometimes this is called ''conceptual surface'' - the amount of concepts one needs to understand when dealing with an [[API]]. By removing the need for users of a class to know anything about applet, we ''simplify'' its surface. Not only that, we also allow it to be compilable without applet being around (which is actually the most important goal when modularizing an [[API]]).

Well, the basic trick is to ''sneak in simplicity''. Of course ''simplicity'' can have various meanings, but in this context it means number of outgoing [[dependencies]]. The '''Beans''' class is not simple, because it has dependency on beans, as well as applet classes. If we can replace it with some other class, that does not depend on applet, then we will simplify the [[API]]. Sometimes this is called [conceptual surface] - the amount of concepts one needs to understand when dealing with an [[API]]. By removing the need for users of a class to know anything about applet, we ''simplify'' its surface. Not only that, we also allow it to be compilable without applet being around (which is actually the most important goal when modularizing an [[API]]).

The only question is how to ''simplify'' the ''Beans'' class? Of course, the simplest way is to remove the one static method that references '''Applet''' - however this is horribly [[BackwardCompatibility|backward incompatible]] and compatibility for existing clients in our highest priority. Thus the only compile time option is to deprecate the whole Beans class and replace it with some other, ''simplified'' one.

The only question is how to ''simplify'' the ''Beans'' class? Of course, the simplest way is to remove the one static method that references '''Applet''' - however this is horribly [[BackwardCompatibility|backward incompatible]] and compatibility for existing clients in our highest priority. Thus the only compile time option is to deprecate the whole Beans class and replace it with some other, ''simplified'' one.

By trivially splitting a [[JAR]] into ten smaller ones, one can only increase the amount of work done by the system. Instead of opening just one [[JAR]] and reading list of its entries, one needs to do this operation ten times, and this is obviously slower, especially when operating system caches are empty (e.g. after boot). Also the mutual communication between the newly separated pieces of the application will require some overhead. Not much, but certainly something.

By trivially splitting a [[JAR]] into ten smaller ones, one can only increase the amount of work done by the system. Instead of opening just one [[JAR]] and reading list of its entries, one needs to do this operation ten times, and this is obviously slower, especially when operating system caches are empty (e.g. after boot). Also the mutual communication between the newly separated pieces of the application will require some overhead. Not much, but certainly something.

I have faced this when I split the monolithic '''openide.jar''' - a [[JAR]] with majority of [[NetBeans]] [[API]]s back in 2005 (see the [http://openide.netbeans.org/proposals/arch/modularize.html project page] for more details). When I divided the big [[JAR]] into fifteen smaller, the start time of [[NetBeans]] IDE increased by 5%. I was seeking for reasons of such slowdown for the whole release and managed to eliminate it somehow, but the proper fix was still waiting to be discovered.

I have faced this when I split the monolithic '''openide.jar''' - a [[JAR]] with majority of [[NetBeans]] [[API]]s back in 2005 (see the [[Modularization of NetBeans Platform]] page for more details). When I divided the big [[JAR]] into fifteen smaller, the start time of [[NetBeans]] IDE increased by 5%. I was seeking for reasons of such slowdown for the whole release and managed to eliminate it somehow, but the proper fix was still waiting to be discovered.

These days the [[NetBeans]] IDE starts faster then it used to before its ''modularization'' - we improved the infrastructure and made it more ''module friendly''. We created various caches (for content of META-INF/MANIFEST.MF files of all modules, for META-INF/services, for classes loaded during start, for layout of files on disk, [[NetBeansLayers]], etc.) and these days ([[NetBeans]] IDE 6.5, or 6.7) we don't open the modularized [[JAR]]s at all. Thus we have the deployment benefits as claimed in [[Modularity|manifesto of modular programming]], while during runtime the system behaves like a monolithic application.

These days the [[NetBeans]] IDE starts faster then it used to before its ''modularization'' - we improved the infrastructure and made it more ''module friendly''. We created various caches (for content of META-INF/MANIFEST.MF files of all modules, for META-INF/services, for classes loaded during start, for layout of files on disk, [[NetBeansLayers]], etc.) and these days ([[NetBeans]] IDE 6.5, or 6.7) we don't open the modularized [[JAR]]s at all. Thus we have the deployment benefits as claimed in [[Modularity|manifesto of modular programming]], while during runtime the system behaves like a monolithic application.

I like puzzles that tease my mind (a bit). Last week I've been introduced to one. [[Modularize]] JDK (as described at [http://blogs.sun.com/mr/entry/modular_java_platform Mark Reinhold's blog]). This page will capture my thoughts on this topic.

I like puzzles that tease my mind (a bit). Last week I've been introduced to one. [[Modularize]] JDK (as described at [http://blogs.sun.com/mr/entry/modular_java_platform Mark Reinhold's blog]). This page will capture my thoughts and experiments on this topic. I don't know how much my work influenced actual [[Jigsaw]] work, but I know that at least Mandy has seen my results.

There can be many reasons for having [[module system|modular]] [[JDK]], but to simplify things, let's stick with one: we want to reach a point in future, when it will be enough to download just a limited set of [[JDK]] to execute an applet/application/server, etc.

There can be many reasons for having [[module system|modular]] [[JDK]], but to simplify things, let's stick with one: we want to reach a point in future, when it will be enough to download just a limited set of [[JDK]] to execute an applet/application/server, etc.

I like puzzles that tease my mind (a bit). Last week I've been introduced to one. Modularize JDK (as described at [http://blogs.sun.com/mr/entry/modular_java_platform Mark Reinhold's blog]). This page will capture my thoughts on this topic.

I like puzzles that tease my mind (a bit). Last week I've been introduced to one. [[Modularize]] JDK (as described at [http://blogs.sun.com/mr/entry/modular_java_platform Mark Reinhold's blog]). This page will capture my thoughts on this topic.

There can be many reasons for having [[module system|modular]] [[JDK]], but to simplify things, let's stick with one: we want to reach a point in future, when it will be enough to download just a limited set of [[JDK]] to execute an applet/application/server, etc.

There can be many reasons for having [[module system|modular]] [[JDK]], but to simplify things, let's stick with one: we want to reach a point in future, when it will be enough to download just a limited set of [[JDK]] to execute an applet/application/server, etc.

The first achievement of this effort is narrowing the scope and expectations. The modularization is beneficial, but itself alone cannot speed up loading time of a Hello World! application.

The first achievement of this effort is narrowing the scope and expectations. The modularization is beneficial, but itself alone cannot speed up loading time of a Hello World! application.

To get the best from the modularization, it is not enough to split the code to modules. One also needs to improve the infrastructure behind (at least in long term). Proper runtime container needs various caches to optimize load time, runtime behaviour, etc.

To get the best from the modularization, it is not enough to split the code to modules. One also needs to improve the infrastructure behind (at least in long term). Proper runtime container needs [[CacheForModularity|various caches]] to optimize load time, runtime behaviour, etc.

This project demonstrated how to reuse [[Ant]] and create a reusable build infrastructure. With such infrastructure it is then easy to experiment with various groupings of classes into modules and immediately verify that these groupings are sane, remain compilable and usable.

This project demonstrated how to reuse [[Ant]] and create a reusable build infrastructure. With such infrastructure it is then easy to experiment with various groupings of classes into modules and immediately verify that these groupings are sane, remain compilable and usable.

There were two problems I had to face. '''java.util.prefs''' and '''java.util.Properties''' can be read and written into [[XML]]. Again I used the [[CodeInjection]], but in this case it may not be necessary. Writing the [[XML]] output is easy while using simple '''java.io.PrintWriter''' (see ''export'' method in following [http://source.apidesign.org/hg/jdk/rev/24b6c30fbf71 patch]). The only question is how to stick to the old behaviour as close a possible. Here I used [[ImplementationCompatibilityTest]] which writes the values of a '''Preferences''' into two strings (using the new and old implementation) and then compares the results are same (also available in the [http://source.apidesign.org/hg/jdk/rev/24b6c30fbf71 patch]). The test is extensible: Every time one notices a divergence in behaviour, one can easily demonstrate it in the test. Then it can be fixed for once and ever (stiffing the shape of its implementation [[Amoeba Model|amoeba]]).

There were two problems I had to face. '''java.util.prefs''' and '''java.util.Properties''' can be read and written into [[XML]]. Again I used the [[CodeInjection]], but in this case it may not be necessary. Writing the [[XML]] output is easy while using simple '''java.io.PrintWriter''' (see ''export'' method in following [http://source.apidesign.org/hg/jdk/rev/24b6c30fbf71 patch]). The only question is how to stick to the old behaviour as close a possible. Here I used [[ImplementationCompatibilityTest]] which writes the values of a '''Preferences''' into two strings (using the new and old implementation) and then compares the results are same (also available in the [http://source.apidesign.org/hg/jdk/rev/24b6c30fbf71 patch]). The test is extensible: Every time one notices a divergence in behaviour, one can easily demonstrate it in the test. Then it can be fixed for once and ever (stiffing the shape of its implementation [[Amoeba Model|amoeba]]).

Reading [[XML]] is slightly harder, and I definitely did not want to do it manually. In the name of [[cluelessness]] I wanted to assemble the solution, not write it in scratch. Thus I searched around and found a [http://devkix.com/nanoxml.php nanoxml]] parser. Small, few kilobytes of code, that I used to parse the [[XML]] stream. The code remains the same and keeps the same functionality (I again used the [[ImplementationCompatibilityTest]]) and instead of 2MB of [[Xerces]] I need just about 10KB (e.g. [[cluelessness]] is good, but better to be [[cluelessness|clueless]] and small).

Reading [[XML]] is slightly harder, and I definitely did not want to do it manually. In the name of [[cluelessness]] I wanted to assemble the solution, not write it in scratch. Thus I searched around and found a [http://devkix.com/nanoxml.php nanoxml] parser. Small, few kilobytes of code, that I used to parse the [[XML]] stream. The code remains the same and keeps the same functionality (I again used the [[ImplementationCompatibilityTest]]) and instead of 2MB of [[Xerces]] I need just about 10KB (e.g. [[cluelessness]] is good, but better to be [[cluelessness|clueless]] and small).

== Executive Summary ==

== Executive Summary ==

Sometimes people expect to get better performance just by modularizing their application. This is probably a false expectation, at least in the initial step.

Sometimes people expect to get better performance just by modularizing their application. This is probably a false expectation, at least in the initial step.

By trivially splitting a [[JAR]] into ten smaller ones, one can only increase the amount of work done by the system. Instead of opening just one [[JAR]] and reading list of its entries, one needs to do this operation ten times, and this is obviously slower, especially when operating system caches are empty (e.g. after boot). Also the mutual communication between the newly separated pieces of the application will require some some overhead. Not much, but certainly something.

By trivially splitting a [[JAR]] into ten smaller ones, one can only increase the amount of work done by the system. Instead of opening just one [[JAR]] and reading list of its entries, one needs to do this operation ten times, and this is obviously slower, especially when operating system caches are empty (e.g. after boot). Also the mutual communication between the newly separated pieces of the application will require some overhead. Not much, but certainly something.

I have faced this when I split the monolithic '''openide.jar''' - a [[JAR]] with majority of [[NetBeans]] [[API]]s back in 2005 (see the [http://openide.netbeans.org/proposals/arch/modularize.html project page] for more details). When I divided the big [[JAR]] into fifteen smaller, the start time of [[NetBeans]] IDE increased by 5%. I was seeking for reasons of such slowdown for the whole release and managed to eliminate it somehow, but the proper fix was still waiting to be discovered.

I have faced this when I split the monolithic '''openide.jar''' - a [[JAR]] with majority of [[NetBeans]] [[API]]s back in 2005 (see the [http://openide.netbeans.org/proposals/arch/modularize.html project page] for more details). When I divided the big [[JAR]] into fifteen smaller, the start time of [[NetBeans]] IDE increased by 5%. I was seeking for reasons of such slowdown for the whole release and managed to eliminate it somehow, but the proper fix was still waiting to be discovered.