HgParallelProjectIntegration

From APIDesign

Recovered from web 2010 archive

This is an implementation of distributed development and integration for NetBeans. This page reflects the current state and describes why it is useful to guarantee success of daily production builds, while allowing projects to continuously validate their work and integrate only when it is known to be stable.

Motto

Are you tired of waiting for ages before every integration for the following set of targets to finish?

ant clean
ant build
ant commit-validation

Switch to a project repository, integrate immediately, work on other things and let the project builder verify that your integration breaks nothing essential!

How it works?

The solution is based on creation pre-tested integration repository populated automatically and in parallel from each registered project's repository (like main, core-main, web-main, ergonomics and many more). But I guess a picture is better than thousand words. This is view of the system from a single developer point of view:

Work on your local repository, do changes, verify they are sort of OK (no need to build everything or test everything). When you feel your work deserves integration: push. When you feel you'd like to see work done by other teams (tested and verified), or by your team members (untested, but you trust them, don't you?) do: pull.

As soon as your changes are in your project repository (like main, core-main, web-main, ergonomics and many more), your dedicated builder picks them up and verifies them and validates them (this is customizable, it is your builder, tell it what test it shall run). Failures on this level are OK and acceptable. It is your team's responsibility to deal with them (or at least evaluate them and report a bug in case the changes infiltrated from somewhere else).

If everything is OK, then once a while the integration server decides to pick your project's last known stable changes and merge them with already approved changes done by others. After verifying that the merge results in sane bits that compile and run it pushes the change to main-silver repository. Failures on this level shall be rare. If they happen it is sign of instability of the system or the fact that the previous builder did bad job. Your team is supposed to evaluate the failure and (very likely) report a bug against the infrastructure.

After an integration server's successful build the verified changes are distributed back to each project's repository (the green arrow).

Twice a day the bits in main-silver repository are taken by the production builder. It builds various targets not verified sooner (jnlp, javadoc, nbms, etc.) and if things are OK, it publishes the bits and integrates the changes to the main-golden for use by those who want absolutely stable revisions only. It also marks the latest stable revision with "golden" tag (use hg log -r golden to see which revision is stable in your repository).

The automated push made on various levels guarantees that basic quality criteria are fulfilled while individual developers do not need to waste time with individual validation themselves. Btw. the we have many tens of developers and about ten teams, so the above picture is simplified slice, to get the whole image just multiply the project builders by ten and individual developers by hundred.