Incremental deployment - Revision history

JaroslavTulach at 18:31, 12 August 2011

2011-08-12T18:31:11Z

←Older revision		Revision as of 18:31, 12 August 2011
Line 1:		Line 1:
-	When talking about the need for [[BackwardCompatibility]] I often talk in context of [[distributed development]]. Without some sort of [[BackwardCompatibility]] you can't expect independent groups to develop on top of each other's libraries. This might imply, that [[BackwardCompatibility]] is only important when multiple ([[distributed development\|distributed]]) parties are involved. [[Paradox]]ically ~~That~~ would be a false conclusion. Hear the following story!	+	When talking about the need for [[BackwardCompatibility]] I often talk in context of [[distributed development]]. Without some sort of [[BackwardCompatibility]] you can't expect independent groups to develop on top of each other's libraries. This might imply, that [[BackwardCompatibility]] is only important when multiple ([[distributed development\|distributed]]) parties are involved. [[Paradox]]ically that would be false conclusion. Hear the following story!

	== [[NetBeans]] 7.0.1 Bugfix Release ==		== [[NetBeans]] 7.0.1 Bugfix Release ==

JaroslavTulach: /* Being compatible vs. Update it all! */

2011-08-12T18:26:23Z

Being compatible vs. Update it all!

←Older revision		Revision as of 18:26, 12 August 2011
Line 39:		Line 39:
	We used to believe we control [[API]] that we don't expose to public, but the latest adventures with [[incremental deployment]] show that even such assumption is over estimated. In truly distributed system you have little control and the only thing to rely on is (self) [[BackwardCompatibility\|compatibility]].		We used to believe we control [[API]] that we don't expose to public, but the latest adventures with [[incremental deployment]] show that even such assumption is over estimated. In truly distributed system you have little control and the only thing to rely on is (self) [[BackwardCompatibility\|compatibility]].

-	[[~~TBD~~]].	+	In such wild setup there is just one advice: be [[BackwardCompatibility\|backward compatible]]. If you are things will work, once you relax and give up on the [[BackwardCompatibility]] you'll be punished.
		+

	[[Category:APIDesignPatterns:Evolution]]		[[Category:APIDesignPatterns:Evolution]]

JaroslavTulach: /* Easy fix */

2011-08-12T18:24:29Z

Easy fix

←Older revision		Revision as of 18:24, 12 August 2011
Line 31:		Line 31:
	Obviously (or surprisingly from certain point of view), it turned out that the simplest fix is to eliminate incompatible changes and solve the problem in [[BackwardCompatible]] way. At the end it was not that hard. Rather than adding new methods into interface, or changing signatures of existing methods, it was enough to create a new sub-interface (as described in [[ExtendingInterfaces]]).		Obviously (or surprisingly from certain point of view), it turned out that the simplest fix is to eliminate incompatible changes and solve the problem in [[BackwardCompatible]] way. At the end it was not that hard. Rather than adding new methods into interface, or changing signatures of existing methods, it was enough to create a new sub-interface (as described in [[ExtendingInterfaces]]).

-	This is not terribly nice solution, but who cares!? ~~Millions~~ of users ~~are~~ waiting for new version of [[NetBeans]] and are we arguing whether one guy can change three modules/source files in compatible way? C'mon, look at the problems from global perspective - just sacrifice yourself (as a writer of an [[API]]), the rest (e.g. million minus yourself) will benefit.	+	This is not terribly nice solution, but who cares!? Million of users is waiting for new version of [[NetBeans]] and are we arguing whether one guy can change three modules/source files in compatible way? C'mon, look at the problems from global perspective - just sacrifice yourself (as a writer of an [[API]]), the rest (e.g. million minus yourself) will benefit.

	== Being compatible vs. Update it all! ==		== Being compatible vs. Update it all! ==

JaroslavTulach: /* Easy fix */

2011-08-12T18:23:39Z

Easy fix

←Older revision		Revision as of 18:23, 12 August 2011
Line 27:		Line 27:
	=== Easy fix ===		=== Easy fix ===

-	For a moment this looked like unsolvable situation. We could fix the deployment to be less incremental. But that would require changes to the overall infrastructure, with the risk of being more disruptive than helpful. We could return back to 6.9 state, but we knew it had it own problems. We could try to load two incompatible version of the same library into the same [[JVM]] and hope for the best. But we had just one day to fix the problem!	+	For a moment this looked like unsolvable situation. We could fix the deployment to be less incremental. But that would require changes to the overall infrastructure, with the risk of being more disruptive than helpful. We could return back to 6.9 state (e.g. update even disabled modules), but we knew it had it own problems. We could try to load two incompatible version of the same library into the same [[JVM]] and hope for the best. But we had just one day to fix the problem!

	Obviously (or surprisingly from certain point of view), it turned out that the simplest fix is to eliminate incompatible changes and solve the problem in [[BackwardCompatible]] way. At the end it was not that hard. Rather than adding new methods into interface, or changing signatures of existing methods, it was enough to create a new sub-interface (as described in [[ExtendingInterfaces]]).		Obviously (or surprisingly from certain point of view), it turned out that the simplest fix is to eliminate incompatible changes and solve the problem in [[BackwardCompatible]] way. At the end it was not that hard. Rather than adding new methods into interface, or changing signatures of existing methods, it was enough to create a new sub-interface (as described in [[ExtendingInterfaces]]).

JaroslavTulach: /* An incompatible change */

2011-08-12T18:22:46Z

An incompatible change

←Older revision		Revision as of 18:22, 12 August 2011
Line 23:		Line 23:
	Now combine this incompatible change with the [[incremental deployment]] system we had in place in release 7.0! Under some circumstances users of [[NetBeans]] 7.0 could be using the library and just one module depending on it. That is why, when we offered update to release 7.0.1, only those two would be upgraded. However, then the user might have a need to enable other installed functionality (comming from [[NetBeans]] 7.0) including the other module depending on the incompatibly changed library.		Now combine this incompatible change with the [[incremental deployment]] system we had in place in release 7.0! Under some circumstances users of [[NetBeans]] 7.0 could be using the library and just one module depending on it. That is why, when we offered update to release 7.0.1, only those two would be upgraded. However, then the user might have a need to enable other installed functionality (comming from [[NetBeans]] 7.0) including the other module depending on the incompatibly changed library.

-	That did not work, as there ~~would be~~ one module requesting library in old version, another module requesting library in new version and only one of these library versions could be active. The system collapsed and users were forced to delete it and reinstall from scratch!	+	That did not work, as there was one module requesting library in old version, another module requesting library in new version and only one of these library versions could be active. The system collapsed and users were forced to delete it and reinstall from scratch!

	=== Easy fix ===		=== Easy fix ===

JaroslavTulach: /* An incompatible change */

2011-08-12T18:22:23Z

An incompatible change

←Older revision		Revision as of 18:22, 12 August 2011
Line 21:		Line 21:
	During bugfixing we felt one of the bugs is most easily fixable by doing an incompatible change to one of our [[API]] libraries. The library had a restricted list of users, so we knew that just two modules depended on it. Morever all three modules were maintained by the same guy, thus we faced no problem during compilation. It was easy to make all the necessary changes at once, during one [[Mercurial]] changeset. The change was properly marked as being incompatible, so the [[NetBeans Runtime Container]] knew whether the classes will or will not link. So far so good.		During bugfixing we felt one of the bugs is most easily fixable by doing an incompatible change to one of our [[API]] libraries. The library had a restricted list of users, so we knew that just two modules depended on it. Morever all three modules were maintained by the same guy, thus we faced no problem during compilation. It was easy to make all the necessary changes at once, during one [[Mercurial]] changeset. The change was properly marked as being incompatible, so the [[NetBeans Runtime Container]] knew whether the classes will or will not link. So far so good.

-	Now combine this incompatible change with the [[incremental deployment]] system we had in place in release 7.0! Under some circumstances users of [[NetBeans]] 7.0 could be using the library and just one module depending on it. That is why, when we offered update to release 7.0.1, only those two would be upgraded. However, then the user might have a need to enable other installed functionality (comming from [[NetBeans]] 7.0) including the other module depending on the incompatibly ~~change~~ library.	+	Now combine this incompatible change with the [[incremental deployment]] system we had in place in release 7.0! Under some circumstances users of [[NetBeans]] 7.0 could be using the library and just one module depending on it. That is why, when we offered update to release 7.0.1, only those two would be upgraded. However, then the user might have a need to enable other installed functionality (comming from [[NetBeans]] 7.0) including the other module depending on the incompatibly changed library.

	That did not work, as there would be one module requesting library in old version, another module requesting library in new version and only one of these library versions could be active. The system collapsed and users were forced to delete it and reinstall from scratch!		That did not work, as there would be one module requesting library in old version, another module requesting library in new version and only one of these library versions could be active. The system collapsed and users were forced to delete it and reinstall from scratch!

JaroslavTulach: /* An incompatible change */

2011-08-12T18:21:22Z

An incompatible change

←Older revision		Revision as of 18:21, 12 August 2011
Line 19:		Line 19:
	=== An incompatible change ===		=== An incompatible change ===

-	During bugfixing we felt one of the bugs is most easily fixable by doing an incompatible change to one of our [[API]] libraries. The library had a ~~restrict~~ list of users, so we knew that just two modules depended on it. Morever all three modules were maintained by the same guy, thus we faced no problem during compilation. It was easy to make all the necessary changes at once, during one [[Mercurial]] changeset. The change was properly marked as being incompatible, so the [[NetBeans Runtime Container]] knew whether the classes will or will not link. So far so good.	+	During bugfixing we felt one of the bugs is most easily fixable by doing an incompatible change to one of our [[API]] libraries. The library had a restricted list of users, so we knew that just two modules depended on it. Morever all three modules were maintained by the same guy, thus we faced no problem during compilation. It was easy to make all the necessary changes at once, during one [[Mercurial]] changeset. The change was properly marked as being incompatible, so the [[NetBeans Runtime Container]] knew whether the classes will or will not link. So far so good.

	Now combine this incompatible change with the [[incremental deployment]] system we had in place in release 7.0! Under some circumstances users of [[NetBeans]] 7.0 could be using the library and just one module depending on it. That is why, when we offered update to release 7.0.1, only those two would be upgraded. However, then the user might have a need to enable other installed functionality (comming from [[NetBeans]] 7.0) including the other module depending on the incompatibly change library.		Now combine this incompatible change with the [[incremental deployment]] system we had in place in release 7.0! Under some circumstances users of [[NetBeans]] 7.0 could be using the library and just one module depending on it. That is why, when we offered update to release 7.0.1, only those two would be upgraded. However, then the user might have a need to enable other installed functionality (comming from [[NetBeans]] 7.0) including the other module depending on the incompatibly change library.

JaroslavTulach: /* An incompatible change */

2011-08-12T18:20:59Z

An incompatible change

←Older revision		Revision as of 18:20, 12 August 2011
Line 19:		Line 19:
	=== An incompatible change ===		=== An incompatible change ===

-	During bugfixing we felt one of the bugs is most easily fixable by doing an incompatible change one of our [[API]] libraries. The library had a restrict list of users, so we knew that just two modules depended on it. Morever all three modules were maintained by the same guy, thus we faced no problem during compilation. It was easy to make all the necessary changes at once, during one [[Mercurial]] changeset. The change was properly marked as being incompatible, so the [[NetBeans Runtime Container]] knew whether the classes will or will not link. So far so good.	+	During bugfixing we felt one of the bugs is most easily fixable by doing an incompatible change to one of our [[API]] libraries. The library had a restrict list of users, so we knew that just two modules depended on it. Morever all three modules were maintained by the same guy, thus we faced no problem during compilation. It was easy to make all the necessary changes at once, during one [[Mercurial]] changeset. The change was properly marked as being incompatible, so the [[NetBeans Runtime Container]] knew whether the classes will or will not link. So far so good.

	Now combine this incompatible change with the [[incremental deployment]] system we had in place in release 7.0! Under some circumstances users of [[NetBeans]] 7.0 could be using the library and just one module depending on it. That is why, when we offered update to release 7.0.1, only those two would be upgraded. However, then the user might have a need to enable other installed functionality (comming from [[NetBeans]] 7.0) including the other module depending on the incompatibly change library.		Now combine this incompatible change with the [[incremental deployment]] system we had in place in release 7.0! Under some circumstances users of [[NetBeans]] 7.0 could be using the library and just one module depending on it. That is why, when we offered update to release 7.0.1, only those two would be upgraded. However, then the user might have a need to enable other installed functionality (comming from [[NetBeans]] 7.0) including the other module depending on the incompatibly change library.

JaroslavTulach: /* Update only needed pieces */

2011-08-12T18:20:23Z

Update only needed pieces

←Older revision		Revision as of 18:20, 12 August 2011
Line 13:		Line 13:
	At the time of release 6.1 we realized that although we are offering various downloads (like [[Java]] edition, [[C]]/C++ edition, [[PHP]] edition, etc.), about 80% of people are downloading the full edition. Most of them never used all the technologies, but when the price is good (I mean [[NetBeans]] is for free) why not take it all? I knew why, because it would be bigger, slower and contain more bugs. However this was not what our customers realized, thus we had to do something with it. We invented [[netbeans:FitnessForever\|Features on Demand]]! As a result people pay only for what they use even if they download everything.		At the time of release 6.1 we realized that although we are offering various downloads (like [[Java]] edition, [[C]]/C++ edition, [[PHP]] edition, etc.), about 80% of people are downloading the full edition. Most of them never used all the technologies, but when the price is good (I mean [[NetBeans]] is for free) why not take it all? I knew why, because it would be bigger, slower and contain more bugs. However this was not what our customers realized, thus we had to do something with it. We invented [[netbeans:FitnessForever\|Features on Demand]]! As a result people pay only for what they use even if they download everything.

-	Two releases later we realized that [[netbeans:FitnessForever\|Features on Demand!]] can be broken by updates of new modules. Such modules, often by default turned on, turn on also pieces of the [[netbeans:FitnessForever\|Features on Demand!]] functionality effectively rendering it useless and in some cases even broken. To mitigate this problem we decided to updates only that part of functionality that is enabled. That is [[good]] solution, as ~~the~~ lowers ~~the~~ requirements on network bandwidth (only pieces that you really use are downloaded) and it also solves all previously known problems.	+	Two releases later we realized that [[netbeans:FitnessForever\|Features on Demand!]] can be broken by updates of new modules. Such modules, often by default turned on, turn on also pieces of the [[netbeans:FitnessForever\|Features on Demand!]] functionality effectively rendering it useless and in some cases even broken. To mitigate this problem we decided to updates only that part of functionality that is enabled. That is [[good]] solution, as it lowers requirements on network bandwidth (only pieces that you really use are downloaded) and it also solves all previously known problems.

	This was the state of the system at the time of [[NetBeans]] 7.0 release - [[modular]] and fully [[incremental deployment]] and enablement of all [[module]]s.		This was the state of the system at the time of [[NetBeans]] 7.0 release - [[modular]] and fully [[incremental deployment]] and enablement of all [[module]]s.

JaroslavTulach: /* Update only needed pieces */

2011-08-12T18:19:46Z

Update only needed pieces

←Older revision		Revision as of 18:19, 12 August 2011
Line 13:		Line 13:
	At the time of release 6.1 we realized that although we are offering various downloads (like [[Java]] edition, [[C]]/C++ edition, [[PHP]] edition, etc.), about 80% of people are downloading the full edition. Most of them never used all the technologies, but when the price is good (I mean [[NetBeans]] is for free) why not take it all? I knew why, because it would be bigger, slower and contain more bugs. However this was not what our customers realized, thus we had to do something with it. We invented [[netbeans:FitnessForever\|Features on Demand]]! As a result people pay only for what they use even if they download everything.		At the time of release 6.1 we realized that although we are offering various downloads (like [[Java]] edition, [[C]]/C++ edition, [[PHP]] edition, etc.), about 80% of people are downloading the full edition. Most of them never used all the technologies, but when the price is good (I mean [[NetBeans]] is for free) why not take it all? I knew why, because it would be bigger, slower and contain more bugs. However this was not what our customers realized, thus we had to do something with it. We invented [[netbeans:FitnessForever\|Features on Demand]]! As a result people pay only for what they use even if they download everything.

-	Two releases later we realized that [[netbeans:FitnessForever\|Features on Demand!]] can be broken by updates of new modules. Such modules, often by default turned on, turn on also pieces of the [[netbeans:FitnessForever\|Features on Demand!]] functionality effectively rendering it useless and some cases broken. To mitigate this problem we decided to updates only that part of functionality that is enabled. That is [[good]] solution, as the lowers the requirements on network bandwidth (only pieces that you really use are downloaded) and it also solves all previously known problems.	+	Two releases later we realized that [[netbeans:FitnessForever\|Features on Demand!]] can be broken by updates of new modules. Such modules, often by default turned on, turn on also pieces of the [[netbeans:FitnessForever\|Features on Demand!]] functionality effectively rendering it useless and in some cases even broken. To mitigate this problem we decided to updates only that part of functionality that is enabled. That is [[good]] solution, as the lowers the requirements on network bandwidth (only pieces that you really use are downloaded) and it also solves all previously known problems.

	This was the state of the system at the time of [[NetBeans]] 7.0 release - [[modular]] and fully [[incremental deployment]] and enablement of all [[module]]s.		This was the state of the system at the time of [[NetBeans]] 7.0 release - [[modular]] and fully [[incremental deployment]] and enablement of all [[module]]s.