TestNG

From APIDesign

Revision as of 15:23, 31 March 2012

TestNG is a testing harness created to improve JUnit3 experience. Tests can be annotated with annotations and as such the design looked more modern when Java5 (and support for annotations was new). For a while it seemed TestNG would take over the Java testing market. It was new, slick and it offered a way to execute JUnit3 tests in a compatible way.

However the JUnit guys struck back by creating JUnit4, which basically copies the way TestNG uses annotations. As such those who had previous investments in JUnit3 could just upgrade to JUnit4, keep compatibility and offer annotations to those who wished to be free-cool-in. Since then the adoption of TestNG slowed down, at least from point of me, an external observer.

Still there are reason to use TestNG. JUnit project decided to stick with archaic license, long time superseded by EPL, and at least Oracle is not comfortable with redistributing software under such license. TestNG comes with widely accepted Apache license.

Also JUnit claims to support primarily unit testing. Sometimes it leads to ridiculous stories when JUnit project decides to prefer enforcing test isolation even it leads to irreproducible random test failures (probably to be resolved in next version). When one needs more than just unit testing, TestNG can still be viable alternative.

Migrating from JUnit4 to TestNG

It has already been mentioned that TestNG always supported execution of JUnit3 tests. That was fine in early years of the century, when there were no new investments in JUnit4, but these days this is horribly insufficient! Almost everyone I know switched from JUnit3 to JUnit4 (possibly still writing mostly JUnit3 style tests, like NetBeans does) and the JUnit4 features are slowly infiltrating the test code base.

Switching to TestNG in such situation seemed almost impossible. JUnit4 and TestNG projects have been diverging for years and in spite there are source conversion tools to migrate the JUnit tests to TestNG, they handle only trivial cases. However poor users of JUnit4, don't despair! There is a smooth migration path since February 2012!

We are proud to announce an improved version of TestNG that handles an automatic mixed mode execution of TestNG, JUnit3 and JUnit4 tests. No reason to wait with the switch. Just place 6.4-NEXT version of TestNG on classpath and modify your build script to invoke TestNG. All your existing tests will run in their natural environment and you can start to create new generation test using TestNG.

Moreover you can start to incrementally rewrite existing tests to TestNG. No need to hurry, but once you rewrite them all, you can remove JUnit from test classpath and lower proliferation of Open source licenses by getting rid of yet another obsolete license in your code base.

Drop-in Bridge

The whole system works like a drop-in bridge. The TestNG is compiled against JUnit4 sources, but itself does not contain the JUnit classes (otherwise we could not get rid of the obsolete license).

If TestNG JAR is used alone, it will not care about JUnit at all. If a JUnit JAR is on the classpath as well, one of the two JUnit-related subsystems is initialized.

One subsystem handles execution against JUnit3 binary and is just a slightly modified version of the original code written by Cédric. This one will be instantiated if old JUnit classes are found. If you are running TestNG with JUnit3 JAR, this subsystem guarantees complete compatibility to what you are used to.

The new subsystem requires JUnit4 binaries to link and be instantiated. It uses the JUnitCore API of JUnit4 to guarantee standard JUnit4 support for JUnit3 and JUnit4 tests. As soon as you drop-in JUnit4 JAR on your classpath, this is the bridge that will get instantiated.

More details available here.

Credits

The mixed mode as well as various other improvements has been written by Lukáš Jungmann as part of his bachelor thesis. This is amazing, in the epilogue of TheAPIBook I called for bigger participation of universities in real software projects. Lukáš's work is an example how (master) thesis should in my opinion look like.

Cédric Beust, the creator of TestNG, deserves huge credit for accepting Lukáš's changes in spite of being busy with other tasks. The patch is not simple and the time is ticking - Lukáš needs to finish his thesis and the NetBeans release 7.2 is close as well. Also thanks for the nice words, Cédric:

At any rate, I certainly agree that Lukáš deserves much praise for being
able to dive into the (complex) code base of TestNG and being able to add
such a core integration. In my opinion, this kind of feature is much, much
harder than writing your own code from scratch, so congratulations Lukáš.

I fully agree and I believe these words will guarantee A for Lukáš's thesis!

Time to Migrate

And what about you? Now, when it is so easy to migrate and when TestNG has such promising contributor like Lukáš, is not it a time to start incrementallly writing your new generation tests too? NetBeans plans to do so by end of 2012.