Change test runner to display full dotted name of test

[Chris Jerdonek]

At least twice in the past (#26659 and #23332), it has been suggested to change Django's test runner to display the full dotted name of tests in test output, for example:

FAIL: test_runner.test_discover_runner.DiscoverRunnerTest.test_output

instead of:

FAIL: test_output (test_runner.test_discover_runner.DiscoverRunnerTest)

The reasoning is that this is more convenient for developers: a failing test could be rerun simply by copying and pasting the test name directly from the test output, as is.

Previously, in #23332, this suggestion was closed with the following reasoning:

Django uses the built-in unittest library of Python, and the test output is entirely handled by unittest. So there is nothing in Django which could be changed to modify this output.

However, I don't think this summary is accurate because unittest makes it very easy to change the test output (I would even say it facilitates this), for example by exposing various subclass hooks on its test runner and friends. Django indeed is ​already doing this and modifying the test output in certain cases using DebugSQLTextTestResult. Django also customizes unittest's test running in other ways.

The customizability of test output is also supported by the ​following comment by the author of unittest, Michael Foord, in response to a request that this change be made in unittest itself:

I agree with Robert that the text output of the default runner should not be considered a part of the "api" that we make backwards compatible guarantees about. People who want to customise that should be customising the text runner/result.

I hope you can reconsider. I put together a patch with tests and will submit a PR shortly. You will see that the change is non-intrusive and uses the customizability exposed by unittest.

[Tim Graham]

We reconsider wontfix tickets by reopening the original ticket after a discussion on the DevelopersMailingList yields a consensus to do so. In this case, I'd suggest you spend time contributing a fix to Python so that more people can ultimately benefit from your work.

[Chris Jerdonek]

The Python core developers don't consider this as a bug (and the default output needs to remain the same anyways to preserve backwards compatibility), and unittest already permits this to be easily customized, so it's not clear to me what there is to "fix."

The final comment says this:

Yes, making customising the output easier is a good thing. One way is to use e.g. subunit.run (which can work with all unittest versions since 2.6) and write a custom filter. Or a custom TestResult and TextTestRunner can work too :)

The pull request I was planning to submit was to use a custom TestResult as this commenter (as well as Michael Foord's comment) suggests.

[Tim Graham]

That wasn't my understanding of the state of the Python issue. The title is still "Change format of test runner output" and statements like "I agree with Robert that the text output of the default runner should not be considered a part of the "api" that we make backwards compatible guarantees about." suggest to me that backwards compatibility isn't a concern.

[Chris Jerdonek]

I posted a message to the django-developers list through the web interface after joining, but the message didn't show up in the web interface. Should the message have appeared already, or does the message need to be approved first?

[Tim Graham]

The mailing list is moderated for first-time posters.