Outputting XML Test Reports

Note

New in version 0.2

Output test reports in junit-xml format.

This plugin implements startTest(), testOutcome() and stopTestRun() to compile and then output a test report in junit-xml format. By default, the report is written to a file called nose2-junit.xml in the current working directory.

You can configure the output filename by setting path in a [junit-xml] section in a config file. Unicode characters which are invalid in XML 1.0 are replaced with the U+FFFD replacement character. In the case that your software throws an error with an invalid byte string.

By default, the ranges of discouraged characters are replaced as well. This can be changed by setting the keep_restricted configuration variable to True.

By default, the arguments of parametrized and generated tests are not printed. For instance, the following code:

# a.py

from nose2 import tools

def test_gen():
    def check(a, b):
        assert a == b, '{}!={}'.format(a,b)

    yield check, 99, 99
    yield check, -1, -1

@tools.params('foo', 'bar')
def test_params(arg):
    assert arg in ['foo', 'bar', 'baz']

Produces this XML by default:

<testcase classname="a" name="test_gen:1" time="0.000171"
 timestamp="2021-12-09T21:28:09.686611">
    <system-out />
</testcase>
<testcase classname="a" name="test_gen:2" time="0.000202"
 timestamp="2021-12-09T21:28:09.686813">
    <system-out />
</testcase>
<testcase classname="a" name="test_params:1" time="0.000159"
 timestamp="2021-12-09T21:28:09.686972">
    <system-out />
</testcase>
<testcase classname="a" name="test_params:2" time="0.000163"
 timestamp="2021-12-09T21:28:09.687135">
    <system-out />
</testcase>

But if test_fullname is True, then the following XML is produced:

<testcase classname="a" name="test_gen:1 (99, 99)" time="0.000213"
 timestamp="2021-12-09T21:28:09.686611">
    <system-out />
</testcase>
<testcase classname="a" name="test_gen:2 (-1, -1)" time="0.000194"
 timestamp="2021-12-09T21:28:09.687105">
    <system-out />
</testcase>
<testcase classname="a" name="test_params:1 ('foo')" time="0.000178"
 timestamp="2021-12-09T21:28:09.687283">
    <system-out />
</testcase>
<testcase classname="a" name="test_params:2 ('bar')" time="0.000187"
 timestamp="2021-12-09T21:28:09.687470">
    <system-out />
</testcase>

Enable this Plugin

This plugin is built-in, but not loaded by default.

Even if you specify always-on = True in the configuration, it will not run unless you also enable it. You can do so by putting the following in a unittest.cfg or nose2.cfg file

[unittest]
plugins = nose2.plugins.junitxml

The plugins parameter may contain a list of plugin names, including nose2.plugins.junitxml

Configuration [junit-xml]

always-on

Default:: False
Type:: boolean

keep_restricted

Default:: False
Type:: boolean

path

Default:: nose2-junit.xml
Type:: str

test_fullname

Default:: False
Type:: boolean

test_properties

Default:: None
Type:: str

Sample configuration

The default configuration is equivalent to including the following in a unittest.cfg file.

[junit-xml]
always-on = False
keep_restricted = False
path = nose2-junit.xml
test_fullname = False

Command-line options

--junit-xml-path FILE: Output XML filename

-X DEFAULT, --junit-xml DEFAULT: Generate junit-xml output report

Plugin class reference: JUnitXmlReporter

class nose2.plugins.junitxml.JUnitXmlReporter(*args, **kwargs)[source]

Output junit-xml test report to file

handleArgs(event)[source]: Read option from command line and override the value in config file when necessary

startTest(event)[source]: Count test, record start time

stopTestRun(event)[source]: Output xml tree to file

testOutcome(event)[source]: Add test outcome to xml tree

Sample output

The XML test report for nose2’s sample scenario with tests in a package looks like this:

<testsuite errors="1" failures="5" name="nose2-junit" skips="1" tests="25" time="0.004">
  <testcase classname="pkg1.test.test_things" name="test_gen:1" time="0.000141" />
  <testcase classname="pkg1.test.test_things" name="test_gen:2" time="0.000093" />
  <testcase classname="pkg1.test.test_things" name="test_gen:3" time="0.000086" />
  <testcase classname="pkg1.test.test_things" name="test_gen:4" time="0.000086" />
  <testcase classname="pkg1.test.test_things" name="test_gen:5" time="0.000087" />
  <testcase classname="pkg1.test.test_things" name="test_gen_nose_style:1" time="0.000085" />
  <testcase classname="pkg1.test.test_things" name="test_gen_nose_style:2" time="0.000090" />
  <testcase classname="pkg1.test.test_things" name="test_gen_nose_style:3" time="0.000085" />
  <testcase classname="pkg1.test.test_things" name="test_gen_nose_style:4" time="0.000087" />
  <testcase classname="pkg1.test.test_things" name="test_gen_nose_style:5" time="0.000086" />
  <testcase classname="pkg1.test.test_things" name="test_params_func:1" time="0.000093" />
  <testcase classname="pkg1.test.test_things" name="test_params_func:2" time="0.000098">
    <failure message="test failure">Traceback (most recent call last):
  File "nose2/plugins/loader/parameters.py", line 162, in func
    return obj(*argSet)
  File "nose2/tests/functional/support/scenario/tests_in_package/pkg1/test/test_things.py", line 64, in test_params_func
    assert a == 1
AssertionError
</failure>
  </testcase>
  <testcase classname="pkg1.test.test_things" name="test_params_func_multi_arg:1" time="0.000094" />
  <testcase classname="pkg1.test.test_things" name="test_params_func_multi_arg:2" time="0.000089">
    <failure message="test failure">Traceback (most recent call last):
  File "nose2/plugins/loader/parameters.py", line 162, in func
    return obj(*argSet)
  File "nose2/tests/functional/support/scenario/tests_in_package/pkg1/test/test_things.py", line 69, in test_params_func_multi_arg
    assert a == b
AssertionError
</failure>
  </testcase>
  <testcase classname="pkg1.test.test_things" name="test_params_func_multi_arg:3" time="0.000096" />
  <testcase classname="" name="test_fixt" time="0.000091" />
  <testcase classname="" name="test_func" time="0.000084" />
  <testcase classname="pkg1.test.test_things.SomeTests" name="test_failed" time="0.000113">
    <failure message="test failure">Traceback (most recent call last):
  File "nose2/tests/functional/support/scenario/tests_in_package/pkg1/test/test_things.py", line 17, in test_failed
    assert False, "I failed"
AssertionError: I failed
</failure>
  </testcase>
  <testcase classname="pkg1.test.test_things.SomeTests" name="test_ok" time="0.000093" />
  <testcase classname="pkg1.test.test_things.SomeTests" name="test_params_method:1" time="0.000099" />
  <testcase classname="pkg1.test.test_things.SomeTests" name="test_params_method:2" time="0.000101">
    <failure message="test failure">Traceback (most recent call last):
  File "nose2/plugins/loader/parameters.py", line 144, in _method
    return method(self, *argSet)
  File "nose2/tests/functional/support/scenario/tests_in_package/pkg1/test/test_things.py", line 29, in test_params_method
    self.assertEqual(a, 1)
AssertionError: 2 != 1
</failure>
  </testcase>
  <testcase classname="pkg1.test.test_things.SomeTests" name="test_skippy" time="0.000104">
    <skipped />
  </testcase>
  <testcase classname="pkg1.test.test_things.SomeTests" name="test_typeerr" time="0.000096">
    <error message="test failure">Traceback (most recent call last):
  File "nose2/tests/functional/support/scenario/tests_in_package/pkg1/test/test_things.py", line 13, in test_typeerr
    raise TypeError("oops")
TypeError: oops
</error>
  </testcase>
  <testcase classname="pkg1.test.test_things.SomeTests" name="test_gen_method:1" time="0.000094" />
  <testcase classname="pkg1.test.test_things.SomeTests" name="test_gen_method:2" time="0.000090">
    <failure message="test failure">Traceback (most recent call last):
  File "nose2/plugins/loader/generators.py", line 145, in method
    return func(*args)
  File "nose2/tests/functional/support/scenario/tests_in_package/pkg1/test/test_things.py", line 24, in check
    assert x == 1
AssertionError
</failure>
  </testcase>
</testsuite>