Binding Free-form Projects With the JUnit Output Window

By Marián Petráš (mpetras@netbeans.org)

To bind your free-form project with the JUnit output window, the tests must be executed using the Ant's <junit> task and the task parameters must follow several requirements. I will explain the requirements on the following example.

Example

<target name="test" depends="compile-tests">
    <mkdir dir="${test.results.dir}"/>
    <junit showoutput="true"
           fork="true"
           failureproperty="tests.failed"
           errorproperty="tests.failed">
        <batchtest todir="${test.results.dir}">
            <fileset dir="test">
                <include name="**/*Test.java"/>
            </fileset>
        </batchtest>
        <classpath path="${classes.dir}:${junit.jar}:${test.classes.dir}"/>
        <formatter usefile="false" type="brief"/>
        <formatter type="xml"/>
    </junit>
</target>

Summary of the Requirements

1. The target's name must be "test", "run-tests" or it must begin with "test", followed by a non-letter character (e.g. "test-all" or "test-project" but not "testall").
   This is only necessary in NetBeans 5.x; since NetBeans 6.0, this is not necessary.
2. The tests must be executed using the Ant's <junit> task.
3. The <junit> element must have an attribute showoutput="true". Values "yes" and "on" are equivalent and are accepted as well.
4. The <junit> element must have a nested element <formatter usefile="false" type="brief">. Formatter types "plain" and "xml" are acceptable, too.
5. Unless the <formatter> element described above uses the "xml" formatter, an additional nested element <formatter type="xml"> must be present.
   If this requirement is not met, the output window will only display information about failed tests.
6. If the <formatter type="xml"> element is present, test report files will be generated during runtime of the tests. It is recommended that these files are generated to a separate directory. This directory is specified by the todir=... attribute of the <batchtest> (or <test>) element. If this attribute is missing, the report files will be generated to the current directory (at runtime).
7. The directory for storage of test report files must exist before the tests start. Make sure the directory exists by placing an <mkdir> element before the appropriate <junit> element.

Technical Background

1. Name of the Ant target is used for recognition of targets running tests in the current implementation.
   This is true for NetBeans 5.x; since NetBeans 6.0, name of the Ant target is not taken into account.
2. The tests must be run using the Ant's <junit> task because the JUnit module relies on the <junit> task's output and ignores output from other tasks.
3. Attribute showoutput="true" ensures that output generated by the tests (System.out.println(...), System.err.println(...), etc.) is passed to the output during runtime of the tests. The JUnit module reads the output and immediately passes it to the Output tab of the JUnit output window. This also allows for mixing of error and non-error lines in the output window. If the attribute is not present or if it is set to false (the default value), no output will be displayed in the Output tab.
4. The reason is similar to the previous one, but this time, it is necessary for providing the JUnit module with the data collected by the JUnit framework during tests runtime. Attribute usefile="false" determines that the test results are not saved to a file, but that they are sent to the standard output instead. The JUnit module reads the output, parses it and displays the data in the Statistics panel of the results window. The output is also visible in the IDE's default Output window.
5. Report formatted by the "brief" and "plain" formatters is not complete. For example, the brief formatter does not show information about passed tests. To get complete information, report generated by the "xml" formatter is created (and saved to a report file) and used by the JUnit module for displaying complete data. More precisely, data stored in these XML reports override data obtained from the default output. If these reports files are not available, only data obtained from the default output is used.
6. This is only necessary to keep the test report files in a dedicated directory, separated from all other files.
7. If the directory for storage of test report files does not exist, the formatter which should write to it might fail.