TestNG

From APIDesign

Current revision

TestNG is a testing harness created to improve the JUnit3 experience. Tests can be annotated with annotations and as such the design looked more modern when Java5 (and support for annotations) was new. For a while it seemed TestNG would take over the Java testing market. It was new, slick, and it offered a way to execute JUnit3 tests in a compatible way.

However, the JUnit guys struck back by creating JUnit4, which basically copies the way TestNG uses annotations. As such, those who had previous investments in JUnit3 could just upgrade to JUnit4, keep compatibility and offering annotations to those who wished to be free-cool-in. Since then, the adoption of TestNG slowed down, at least from my point of view, as an external observer.

Still there are reasons to use TestNG. The JUnit project decided to stick with archaic license, long time superseded by EPL, and at least Oracle is not comfortable with redistributing software under such license. TestNG comes with the widely accepted Apache license.

Also, JUnit claims to support primarily unit testing. Sometimes it leads to ridiculous stories when the JUnit project decided to prefer enforcing test isolation even when it leads to irreproducible random test failures (probably to be resolved in next version). When one needs more than just unit testing, TestNG can still be a viable alternative.

Migrating from JUnit4 to TestNG

It has already been mentioned that TestNG always supported execution of JUnit3 tests. That was fine in early years of the century, when there were no new investments in JUnit4, but these days this is horribly insufficient! Almost everyone I know switched from JUnit3 to JUnit4 (possibly still writing mostly JUnit3 style tests, like NetBeans does) and the JUnit4 features are slowly infiltrating the test code base.

Switching to TestNG in such situation seemed almost impossible. JUnit4 and TestNG projects have been diverging for years and in spite there are source conversion tools to migrate the JUnit tests to TestNG, they handle only trivial cases. However poor users of JUnit4, don't despair! There is a smooth migration path since February 2012!

We are proud to announce an improved version of TestNG that handles an automatic mixed mode execution of TestNG, JUnit3 and JUnit4 tests. No reason to wait with the switch. Just place 6.4-NEXT version of TestNG on classpath and modify your build script to invoke TestNG. All your existing tests will run in their natural environment and you can start to create new generation test using TestNG.

Moreover you can start to incrementally rewrite existing tests to TestNG. No need to hurry, but once you rewrite them all, you can remove JUnit from test classpath and lower proliferation of Open source licenses by getting rid of yet another obsolete license in your code base.

Drop-in Bridge

The whole system works like a drop-in bridge. The TestNG is compiled against JUnit4 sources, but itself does not contain the JUnit classes (otherwise we could not get rid of the obsolete license).

If TestNG JAR is used alone, it will not care about JUnit at all. If a JUnit JAR is on the classpath as well, one of the two JUnit-related subsystems is initialized.

One subsystem handles execution against JUnit3 binary and is just a slightly modified version of the original code written by Cédric. This one will be instantiated if old JUnit classes are found. If you are running TestNG with JUnit3 JAR, this subsystem guarantees complete compatibility to what you are used to.

The new subsystem requires JUnit4 binaries to link and be instantiated. It uses the JUnitCore API of JUnit4 to guarantee standard JUnit4 support for JUnit3 and JUnit4 tests. As soon as you drop-in JUnit4 JAR on your classpath, this is the bridge that will get instantiated.

More details available here.

Credits

The mixed mode as well as various other improvements has been written by Lukáš Jungmann as part of his bachelor thesis. This is amazing, in the epilogue of TheAPIBook I called for bigger participation of universities in real software projects. Lukáš's work is an example how (master) thesis should in my opinion look like.

Cédric Beust, the creator of TestNG, deserves huge credit for accepting Lukáš's changes in spite of being busy with other tasks. The patch is not simple and the time is ticking - Lukáš needs to finish his thesis and the NetBeans release 7.2 is close as well. Also thanks for the nice words, Cédric:

At any rate, I certainly agree that Lukáš deserves much praise for being
able to dive into the (complex) code base of TestNG and being able to add
such a core integration. In my opinion, this kind of feature is much, much
harder than writing your own code from scratch, so congratulations Lukáš.

I fully agree and I believe these words guarantee A for Lukáš's thesis! Which is what happened on Jun 20, 2012 Lukáš's thesis was accepted and as it was the best provided on that date, it has been recommended for a special prize (which I don't know what it is yet). Congratulation Lukáši!

Time to Migrate

And what about you? Now, when it is so easy to migrate and when TestNG has such promising contributor like Lukáš, is not it a time to start incrementallly writing your new generation tests too? NetBeans plans to do so by end of 2012.