TruffleSigtest - Revision history

JaroslavTulach: /* Notification of Daily Changes */

2015-11-23T10:27:39Z

Notification of Daily Changes

←Older revision		Revision as of 10:27, 23 November 2015
Line 84:		Line 84:

	then it is just a matter of sending the output of the last command to an observer mailing list, so humans can decide whether the change is intended or not.		then it is just a matter of sending the output of the last command to an observer mailing list, so humans can decide whether the change is intended or not.
		+
		+	[[I]] believe that with infrastructure like this the [[Truffle]] project will be able to stick to its [http://lafo.ssw.uni-linz.ac.at/javadoc/truffle/latest/overview-summary.html promise of compatibility] and better preserve investments done by [[language]] writers porting their languages to the [[RubySpeed\|fastest (J)VM on the Planet]]!

JaroslavTulach: /* Notification of Daily Changes */

2015-11-23T10:24:05Z

Notification of Daily Changes

←Older revision		Revision as of 10:24, 23 November 2015
Line 72:		Line 72:
	<source lang="bash">		<source lang="bash">
	$ mx sigtest --generate		$ mx sigtest --generate
		+	$ zip snapshots.zip truffle//sigtest
	</source>		</source>

JaroslavTulach: /* Notification of Daily Changes */

2015-11-23T10:22:50Z

Notification of Daily Changes

←Older revision		Revision as of 10:22, 23 November 2015
Line 64:		Line 64:
	==== Notification of Daily Changes ====		==== Notification of Daily Changes ====

-	In addition to catching [[BackwardCompatible\|incompatible]] changes, it is useful to track compatible ones as well Some of the [[API]] changes happen by an accident and when they aren't noticed in time, they may become part of a release. But then they have to stay (as they are like [[star]]s) regardless how bad they are. Nobody wants to maintain erroneous code or [[API]], so it is better to catch such changes as soon as possible and revert them before they propagate to a release.	+	In addition to catching [[BackwardCompatible\|incompatible]] changes, it is useful to track [[BackwardCompatible\|compatible ones]] as well.
		+
		+	Some of the [[API]] changes happen by an accident and when they aren't noticed in time, they may become part of a release. But then they have to stay (as they are like [[star]]s) regardless how bad they are. Nobody wants to maintain erroneous code or [[API]], so it is better to catch such changes as soon as possible and revert them before they propagate to a release.

	Having a [[Hudson]] job to watch daily compatible changes and notify us about them is the best way to prevent accidental [[API]] changes. The job shall have two parts: generate new snapshot of today's [[API]]s:		Having a [[Hudson]] job to watch daily compatible changes and notify us about them is the best way to prevent accidental [[API]] changes. The job shall have two parts: generate new snapshot of today's [[API]]s:

144.24.20.230: /* Notification of Daily Changes */

2015-11-23T10:21:55Z

Notification of Daily Changes

←Older revision		Revision as of 10:21, 23 November 2015
Line 64:		Line 64:
	==== Notification of Daily Changes ====		==== Notification of Daily Changes ====

-	[[~~TBD~~]]: [[I]] ~~will set~~ a [[Hudson]] job to watch daily compatible changes and notify us about them ~~- so we can~~ prevent accidental [[API]] changes.	+	In addition to catching [[BackwardCompatible\|incompatible]] changes, it is useful to track compatible ones as well Some of the [[API]] changes happen by an accident and when they aren't noticed in time, they may become part of a release. But then they have to stay (as they are like [[star]]s) regardless how bad they are. Nobody wants to maintain erroneous code or [[API]], so it is better to catch such changes as soon as possible and revert them before they propagate to a release.
		+
		+	Having a [[Hudson]] job to watch daily compatible changes and notify us about them is the best way to prevent accidental [[API]] changes. The job shall have two parts: generate new snapshot of today's [[API]]s:
		+
		+	<source lang="bash">
		+	$ mx sigtest --generate
		+	</source>
		+
		+	and make them persisted artifacts of the job. The other part is to take these artifacts from the last successful build and compare them against current state:
		+
		+	<source lang="bash">
		+	$ wget $HUDSON_URL/lastSuccessfulBuild/snaphots.zip
		+	$ unzip snapshots.zip
		+	$ mx sigtest --check all
		+	</source>
		+
		+	then it is just a matter of sending the output of the last command to an observer mailing list, so humans can decide whether the change is intended or not.

JaroslavTulach: /* Setting the Snapshots Up */

2015-11-20T19:06:04Z

Setting the Snapshots Up

←Older revision		Revision as of 19:06, 20 November 2015
Line 60:		Line 60:
	* Adjusting to binary incompatible change of (on)returnExceptional parameters [http://lafo.ssw.uni-linz.ac.at/hg/truffle/rev/7273b139fff2 7273b139fff2] - this was an intentional and agreed on change (in spite of being against the [http://lafo.ssw.uni-linz.ac.at/javadoc/truffle/latest/overview-summary.html compatibility policy])		* Adjusting to binary incompatible change of (on)returnExceptional parameters [http://lafo.ssw.uni-linz.ac.at/hg/truffle/rev/7273b139fff2 7273b139fff2] - this was an intentional and agreed on change (in spite of being against the [http://lafo.ssw.uni-linz.ac.at/javadoc/truffle/latest/overview-summary.html compatibility policy])
	* And at last, there was an [[API]] [http://lafo.ssw.uni-linz.ac.at/hg/truffle/rev/23d2b5513c83 design problem] - a return type of an [[API]] visible method was not public - e.g. visible		* And at last, there was an [[API]] [http://lafo.ssw.uni-linz.ac.at/hg/truffle/rev/23d2b5513c83 design problem] - a return type of an [[API]] visible method was not public - e.g. visible
-	In all of these cases [[I]] manually updated the signature files to match the expected state. Then the compatibility check finally succeeded.	+	In all of these cases [[I]] manually updated the signature files to match the expected state. Then the compatibility check finally succeeded. Since today it will be way easier to keep [[BackwardCompatibility]] in our [[Truffle]] project!

	==== Notification of Daily Changes ====		==== Notification of Daily Changes ====

	[[TBD]]: [[I]] will set a [[Hudson]] job to watch daily compatible changes and notify us about them - so we can prevent accidental [[API]] changes.		[[TBD]]: [[I]] will set a [[Hudson]] job to watch daily compatible changes and notify us about them - so we can prevent accidental [[API]] changes.

JaroslavTulach: /* Setting the Snapshots Up */

2015-11-20T19:04:54Z

Setting the Snapshots Up

←Older revision		Revision as of 19:04, 20 November 2015
Line 56:		Line 56:

	Here is how such snapshot files look like: [http://lafo.ssw.uni-linz.ac.at/hg/truffle/rev/b07854a19ab4 changeset]. Of course, it turned out that not everything is perfect (how it could be! we weren't running the tests and it is so hard to reason about [[BackwardCompatibility]] without testing):		Here is how such snapshot files look like: [http://lafo.ssw.uni-linz.ac.at/hg/truffle/rev/b07854a19ab4 changeset]. Of course, it turned out that not everything is perfect (how it could be! we weren't running the tests and it is so hard to reason about [[BackwardCompatibility]] without testing):
-	* Removing API elements that were deprecated at the time of truffle-0.9 release from the list of required API elements: [http://lafo.ssw.uni-linz.ac.at/hg/truffle/rev/2ce4c49bc131 2ce4c49bc131] - [[Truffle]] project is still in its early phases, so according to our [http://lafo.ssw.uni-linz.ac.at/javadoc/truffle/latest/overview-summary.html compatibility policy] ~~this~~ is OK.	+	* Removing API elements that were deprecated at the time of truffle-0.9 release from the list of required API elements: [http://lafo.ssw.uni-linz.ac.at/hg/truffle/rev/2ce4c49bc131 2ce4c49bc131] - [[Truffle]] project is still in its early phases, so according to our [http://lafo.ssw.uni-linz.ac.at/javadoc/truffle/latest/overview-summary.html compatibility policy] removing deprecated elements is OK. Once we reach release 1.0, we won't do things like this.
	* The check discovered a removed constructor [http://lafo.ssw.uni-linz.ac.at/hg/truffle/rev/5033b980cc68 5033b980cc68] which was clearly a bug		* The check discovered a removed constructor [http://lafo.ssw.uni-linz.ac.at/hg/truffle/rev/5033b980cc68 5033b980cc68] which was clearly a bug
	* Adjusting to binary incompatible change of (on)returnExceptional parameters [http://lafo.ssw.uni-linz.ac.at/hg/truffle/rev/7273b139fff2 7273b139fff2] - this was an intentional and agreed on change (in spite of being against the [http://lafo.ssw.uni-linz.ac.at/javadoc/truffle/latest/overview-summary.html compatibility policy])		* Adjusting to binary incompatible change of (on)returnExceptional parameters [http://lafo.ssw.uni-linz.ac.at/hg/truffle/rev/7273b139fff2 7273b139fff2] - this was an intentional and agreed on change (in spite of being against the [http://lafo.ssw.uni-linz.ac.at/javadoc/truffle/latest/overview-summary.html compatibility policy])

JaroslavTulach: /* Setting the Snapshots Up */

2015-11-20T19:03:31Z

Setting the Snapshots Up

←Older revision		Revision as of 19:03, 20 November 2015
Line 48:		Line 48:
	<source lang="bash">		<source lang="bash">
	$ hg up -C truffle-0.9		$ hg up -C truffle-0.9
		+	$ mx build
	$ mx sigtest --generate		$ mx sigtest --generate
	$ hg up default		$ hg up default

JaroslavTulach: /* Setting the Snapshots Up */

2015-11-20T19:03:00Z

Setting the Snapshots Up

←Older revision		Revision as of 19:03, 20 November 2015
Line 49:		Line 49:
	$ hg up -C truffle-0.9		$ hg up -C truffle-0.9
	$ mx sigtest --generate		$ mx sigtest --generate
		+	$ hg up default
	$ hg add .		$ hg add .
	$ hg ci -m "Adding API snapshots as for version truffle-0.9 and enabling their check in the gate"		$ hg ci -m "Adding API snapshots as for version truffle-0.9 and enabling their check in the gate"

JaroslavTulach: /* Integrate into Build Process */

2015-11-20T19:01:50Z

Integrate into Build Process

←Older revision		Revision as of 19:01, 20 November 2015
Line 22:		Line 22:
	# generate - which will take a snapshot of your [[API]] at a certain point of time (usually at a release time)		# generate - which will take a snapshot of your [[API]] at a certain point of time (usually at a release time)
	# check - which will compare the current state of your [[API]] with the existing snapshot		# check - which will compare the current state of your [[API]] with the existing snapshot
-	~~The way to~~ invoke these ~~operation~~ in [[Truffle]] repository ~~are like~~:	+	To invoke these operations in [[Truffle]] repository use:
	<source lang="bash">		<source lang="bash">
	$ mx sigtest --generate		$ mx sigtest --generate

JaroslavTulach: /* Integrate into Build Process */

2015-11-20T19:00:35Z

Integrate into Build Process

←Older revision		Revision as of 19:00, 20 November 2015
Line 17:		Line 17:
	==== Integrate into Build Process ====		==== Integrate into Build Process ====

-	[[SignatureTests\|Signature testing]] needs to be integrated into build process of your project - in case there is a violation with respect to [[BackwardCompatibility]], the build should fail. As (almost) every big project invents its own build harness, the ways to integrate ~~differs~~. For example, [[NetBeans]] is using [[Ant]] based harness and thus one deals with the signature testing tool via [[Ant]] as described at [[netbeans:SignatureTest]] page.	+	[[SignatureTests\|Signature testing]] needs to be integrated into build process of your project - in case there is a violation with respect to [[BackwardCompatibility]], the build should fail. As (almost) every big project invents its own build harness, the ways to integrate differ. For example, [[NetBeans]] is using [[Ant]] based harness and thus one deals with the signature testing tool via [[Ant]] as described at [[netbeans:SignatureTest]] page.

	The [[Truffle]] project (as well as [[Graal]]) is using [[Python]] based harness. As such [[I]] had to write a bit of [[Python]] code to integrate the [[netbeans:APITest\|APITest]] into it (see [https://bitbucket.org/allr/mx/src/e7fa425baca686c418ffa3cb521fa3f853a994a6/mx_sigtest.py here]). You basically need two operations:		The [[Truffle]] project (as well as [[Graal]]) is using [[Python]] based harness. As such [[I]] had to write a bit of [[Python]] code to integrate the [[netbeans:APITest\|APITest]] into it (see [https://bitbucket.org/allr/mx/src/e7fa425baca686c418ffa3cb521fa3f853a994a6/mx_sigtest.py here]). You basically need two operations: