Opened 18 months ago

Last modified 15 months ago

#34657 closed New feature

Testing assertions `assertContains` and `assertInHTML` should output the haystack on failure — at Version 1

Reported by: Thibaud Colas Owned by: nobody
Component: Testing framework Version: dev
Severity: Normal Keywords: HTML, assertions, testing
Cc: Chinmoy Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Thibaud Colas)

If I use the vanilal Python assertIn or assertRegex, when the assertion fails, it’s very simple to assess what went wrong from the test error only:

    self.assertIn("<b>hey</b>", "<p>Howdy!</p>")
AssertionError: '<b>hey</b>' not found in '<p>Howdy!</p>'

With Django’s assertContains and assertInHTML, this gets much harder:

    self.assertInHTML("<b>hey</b>", "<p>Howdy!</p>")
  File "/Users/thibaudcolas/Dev/django/html-testing-with-django/.venv/lib/python3.11/site-packages/django/test/testcases.py", line 1076, in assertInHTML
    self.assertTrue(
AssertionError: False is not true : Couldn't find '<b>
hey
</b>' in response

and:

    self.assertContains(res, "<b>hey</b>")
  File "/Users/thibaudcolas/Dev/django/html-testing-with-django/.venv/lib/python3.11/site-packages/django/test/testcases.py", line 660, in assertContains
    self.assertTrue(
AssertionError: False is not true : Couldn't find '<b>hey</b>' in response

---

In both cases, Django doesn’t display the haystack – so I have to waste a lot of time going back to my HTML templates to check what they might be outputting / or potentially load the same scenario in a browser / or manually add print statements to my test cases. This is all very time-consuming. Instead, it’d be much better if the haystack was just present.

Additionally for assertInHTML – it’s annoying that needle as displayed in the failure message is the parser’s output, which is therefore broken up over multiple lines. It’d be much nicer if the example above resulted in AssertionError: False is not true : Couldn't find '<b>hey</b>' in response.

---

Test suite I used to compare output for reference: ​https://github.com/thibaudcolas/html-testing-with-django/blob/39c82c410cce4bea71ac54be27be67057f4d8dd8/testing_tests/tests.py#L5

And link to Python’s assertIn / assertNotIn implementations: ​https://github.com/python/cpython/blob/101d5ec7d7fe122fa81a377c8ab8b562d1add9ee/Lib/unittest/case.py#L1147-L1159

Change History (1)

comment:1 by Thibaud Colas, 18 months ago

Description: modified (diff)
Note: See TracTickets for help on using tickets.
Back to Top