Parasoft's Jtest 6.0

What does that mean? Beck explains that test-first code is an analysis technique, a design technique, a way to reduce code defects, a way to communicate with your pair-programming partner, and a way to document your thinking. At the same time, Beck admits that writing user interfaces test-first is too hard to be effective.

When you write unit tests, you have to imagine all the cases your code will have to handle, think up good tests that will cover all cases, and know the right answers. What if you're not that smart (like Beck faced with UI code), or don't have the time to generate full coverage (like most of us, most of the time)? Jtest can help by automatically generating Java test cases for JUnit, by automatically checking Java code for conformance to standards, and by automatically checking your code coverage.Enter Jtest

Jtest 6.0, from Parasoft, is available for Windows 2000, Windows XP, Linux, and Solaris. Pricing starts at $3,495 for a single user, machine-locked license. I reviewed Jtest 3.0 in 1999, for PC Week, just as XP was beginning to take off. At the time, I moaned about the price (then almost $5,000), but said "Organizations that are currently performing rigorous unit testing manually are chewing up valuable human resources for an easily mechanized task. Unless your software development and QA staff are paid third-world wages, their improved productivity should easily justify equipping them with automated test tools."

Jtest 6.0 looks a lot different than that version, which I rated "C" for usability and "B" in all other categories. Parasoft abandoned its old stand-alone user interface sometime in the last 6 years and made Jtest an integrated add-on to Eclipse, giving it much improved usability. If you happen to use Eclipse as your primary Java development environment, then having Jtest right there as an Eclipse perspective with its own menu is great; it's not quite as wonderful if you spend most of your time running NetBeans or any of the other Java IDEs, and have to switch to Eclipse to run Jtest.

Jtest can import existing projects from specific folders or detect their layouts starting from a base folder. I did not do extensive testing of the automatic layout detection feature -- my projects tend to be on the small side, and my layouts are fairly standard. Jtest can also import projects from JBuilder, Together, and Ant.

Coding StandardsA Jtest run typically starts by checking your Java code against a set of rules. Jtest 6 has about 500 built-in rules for coding standards, organized into about 30 categories, ranging from Beans, Coding Conventions, and Design by Contract, to Servlets, Threads and Synchronization, and Unused Code. It does have rules for EJBs, J2ME, and Java Server Pages; it doesn't have specific rules for Struts, Spring, or any of the other recent Java Frameworks. You can still use Jtest with those frameworks, however, since they use standard technologies like Java Servlets, JavaBeans, and XML.

You can test individual modules, subtrees, or your entire project tree. It might make sense to test individual modules as you work on them, test your own subtrees nightly, and regression test the entire project prior to a build.

You don't have to apply all the rules to every module, nor would it make sense to do so. Jtest 6 includes about 20 built-in configurations; you can parameterize rules and configure multiple rule sets to meet your team's development objectives. Running too many rules often results in so many useless messages that the coders develop a "shoot-the-messenger" syndrome; running too few rules lets bad code slip through the first net.

Test Case Generation

The second step in a Jtest run is to automatically generate test cases. Jtest is set up to support white-box (reliability) tests, black-box (functional) tests, and regression tests. It does a pretty good job of generating white-box tests automatically, although you might have to supplement its tests with your own or tweak the test generation if the coverage reported is not 100%.Jtest can generate functional tests automatically if you include Design by Contract comments in your code. Otherwise, you might need to write your own functional tests based on the code specification. Sometimes you can start with automatically-generated tests and make them more realistic. Other times, you can use Jtest's test-case sniffer to generate realistic tests by monitoring an actual run of the program.

Running Test Cases

Jtest runs JUnit test cases, both cases it has generated and cases you have written, and monitors the execution for coverage, memory leaks, and uncaught runtime exceptions. Since Jtest remembers every test it has ever run, it automatically becomes an effective regression tester as a project matures. You can automatically rerun specific tests later, or schedule a set of tests for execution at a specific time.

Correcting your Code

When you run Jtest, it generates a task list of problems found. You can then fix the problems at your own pace. In simple cases, like obvious typos in your code, Jtest offers you a "quick fix" item in the context menu for the task item. Jtest has quick fixes for about 200 of its rules.A class of tasks comes from the requirement to verify the outcome of automatically generated tests, which Jtest marks with a "// jtest_unverified" comment. Once you have examined the test and are sure that the specified outcome is correct, you can use the quick fix to change the comment to show that you verified the outcome (see screen shot). Alternatively, you can change the expected outcome, ignore the test outcome, or suppress the test.

Conclusions

JTest 6.0, unlike test-first coding, is a testing tool. It can be used effectively in a test-first coding scenario, especially if you write stubs with Design by Contract comments, but it really shines in scenarios where tests need to be generated after large amounts of code have been written.

Overall, my conclusions from 6 years ago still hold true. JTest 6.0 is a relatively expensive tool, but it can pay for itself fairly quickly in a medium-to-large Java development shop that needs to do significant amounts of testing, especially when there are contractual or regulatory requirements for 100% testing coverage.

Martin Heller is a Web and Windows programming consultant, an author, and a Senior Contributing Editor at BYTE.com. His most recent publicly viewable project is PC Pitstop. You can reach Martin at [email protected].0