Opened 7 years ago

Closed 7 years ago

#29105 closed Cleanup/optimization (wontfix)

Make Default Test Output More Verbose (And Formatted Prettier)

Reported by: Rich Jones Owned by: nobody
Component: Testing framework Version: 2.0
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: yes

Description

This ticket is part of the Django Test Output Beautification initiative: https://code.djangoproject.com/ticket/29101

The current default test output provides no information about what test is currently running. This makes the test output pretty useless if you're actually trying to determine which test is taking so long, or failing, or misbehaving, etc.

The current test output is a wall of unformatted text - bleh! With --verbosity=3, it includes information about which test is running, and the description of the test (yay), but the formatting is still horrible.

test_bad_file_grouping (data_refinery_workers.downloaders.test_transcriptome_index.DownloadTranscriptomeIndexTestCase)
Raises exception if both files don't have the same download_url. ... 2018-02-02 15:31:50,587 local/MainProcess data_refinery_workers.downloaders.transcriptome_index ERROR [downloader_job: 12]: A Batch's file doesn't have the same download URL as the other batch's file.
ok
test_good_file_grouping (data_refinery_workers.downloaders.test_transcriptome_index.DownloadTranscriptomeIndexTestCase)
Raises exception if both files don't have the same download_url. ...
ok

We should make the _default_ test output to be something like:

========================
Testing 'test_bad_file_grouping'
(data_refinery_workers.downloaders.test_transcriptome_index.DownloadTranscriptomeIndexTestCase)
_Raises exception if both files don't have the same download_url._
-----------------------------------------------------
...
ERROR [downloader_job: 12]: A Batch's file doesn't have the same download URL as the other batch's file.
Ok!

========================
Testing 'test_good_file_grouping'
(data_refinery_workers.downloaders.test_transcriptome_index.DownloadTranscriptomeIndexTestCase)
_Makes sure that proper file group runs the pipeline._
-----------------------------------------------------
...
Ok!

Where test title, test description and Ok/Failure are also colorized appropriately.

I'm not totally certain what else the 'verbosity' argument is controlling, (perhaps log level?), but either way I think the way to improve this experience is clear: Move to a more structured, human-friendly text formatting and away from barfing out a wall of unformatted text.

Related tickets:
https://code.djangoproject.com/ticket/29101
https://code.djangoproject.com/ticket/22449

Change History (2)

comment:1 by Chris Jerdonek, 7 years ago

I like that the default output prints just a single . for each test. For example, if you have 1000 tests, you wouldn't want 5 lines per test, or even one line.

However, I agree the format can be improved when the verbosity is increased. I do think one of the available verbosity levels should show the "minimal" information needed to know what test is being run -- so without any divider lines, test description, etc. It can be as simple as a single line that looks something like:

Testing: <test name>

or perhaps prefixed with the number of the test:

1. Testing: <test name>

Higher verbosity levels can show more like the description, divider lines, etc.

comment:2 by Tim Graham, 7 years ago

Resolution: wontfix
Status: newclosed

Django is delegating to Python's testing output here. I'm not immediately convinced that we should implement our own test output. As I mentioned in the other tickets, there are a number of third-party testing tools that offer other outputs. Do we need to spend our time doing something similar? Feel free to start a thread on the DevelopersMailingList to determine if there's a consensus.

Note: See TracTickets for help on using tickets.
Back to Top