Add an assertFormSetError function to django.test.TestCase

[martin_speleo]

As well having the function assertFormError in TransactionTestCase see (source:django/trunk/django/test/testcases.py), a similar function for checking for errors in formsets would be usefull. It would look for

• field errors
• non-field errors
• non-form errors

This is a similar request to ticket:11241 but may be neater and have a larger scope. A possible solution is this modification of assertFormError:

    def assertFormSetError(self, response, formset, index, field, errors):
        """
        Asserts that a formset used to render the response has a specific error.
        """
        # Put context(s) into a list to simplify processing.
        contexts = to_list(response.context)
        if not contexts:
            self.fail('Response did not use any contexts to render the response')

        # Put error(s) into a list to simplify processing.
        errors = to_list(errors)

        # Search all contexts for the error.
        found_formset = False
        for i,context in enumerate(contexts):
            if formset not in context:
                continue
            found_formset = True
            for err in errors:
                if field:
                    if field in context[formset].forms[index].errors:
                        field_errors = context[formset].forms[index].errors[field]
                        self.failUnless(err in field_errors,
                                "The field '%s' on formset '%s' [%s] in"
                                " context %d does not contain the"
                                " error '%s' (actual errors: %s)" %
                                        (field, formset, index, i, err,
                                        repr(field_errors)))
                    elif field in context[formset].forms[index].fields:
                        self.fail("The field '%s' on formset '%s' [%s] in context %d"
                                  " contains no errors" % (field, form, index, i))
                    else:
                        self.fail("The formset '%s' [%s] in context %d does not"
                                  " contain the field '%s'" %
                                        (formset, index, i, field))
                elif index is not None:
                    non_field_errors = context[formset].forms[index].non_field_errors()
                    self.failUnless(err in non_field_errors,
                        "The formset '%s' [%s] in context %d does not contain the"
                        " non-field error '%s' (actual errors: %s)" %
                        (formset, index, i, err, non_field_errors))
                else:
                    print dir(context[formset])
                    non_form_errors = context[formset].non_form_errors()
                    self.failUnless(err in non_form_errors,
                        "The formset '%s' in context %d does not contain the"
                        " non-form error '%s' (actual errors: %s)" %
                        (formset, i, err, non_form_errors))
        if not found_formset:
            self.fail("The formset '%s' was not used to render the response" % formset)

Let me know if this is a sensible approach, and if so I could work on diffs and unittests for it.

[martin_speleo]

My first attempt at contibuting to Django, I think the patch should be ready to be checked in.

It could possibly do with some unittesting on checking for more than one error at once. assertFormError also does not have unittests for dealing with multiple errors, along with a few other little bits and pieces. But if I understand correctly, exceptions are normally used to raise errors in forms/formsets so mulitple errors will be uncommon.

I have not tested that my changes to the documentation do not break everything, as I am running Vista and the make file for the documentation used a -p command on mkdir which did not appear to be understood. If this is a problem let me know and I will try to make the documenetation again.

[Julien Phalip]

Thanks for your patch, it looks pretty good. It would need to be updated to apply to trunk though. Also, the tests could be made more elegant by using assertRaises and the with statement (​http://docs.python.org/library/unittest.html#unittest.TestCase.assertRaises) instead of all those try/except/else's. By the way, you can import the with statement in Python 2.5 by doing: from __future__ import with_statement

[martin_speleo]

I have updated the patch to include support for the optional msq_prefix keyword argument in trunk. The tests now use assertRaises and the with statement. The documentation states that it is a new feature in SVN, let me know if I need to change this to 1.4, and add to the 1.4 release notes.

[martin_speleo]

Documentation changed assuming that this will be added to version 1.4, along with a minor feature entry in the v1.4 release notes.

[Russell Keith-Magee]

I just tried to apply this patch, and it failed 4 of its own tests.

On a closer examination, I'm also inclined to suggest that the form argument should be named form_index; there are other assertions that have a form argument that take an actual form instance, so form_index makes it clear that you're asking for an integer, not a form.

[anonymous]

Replying to russellm:

I just tried to apply this patch, and it failed 4 of its own tests.

I am unable to reproduce these test failures, could you send me the errors that you get, and any relevant version information.

Thanks,
Martin

[Russell Keith-Magee]

As requested in a private chat, here are the errors I see running against r16707, Python 2.6.1, under OSX Snow Leopard.

django)kronkite:tests rkm$ ./runtests.py --settings=sqlite test_client test_client_regress
Creating test database for alias 'default'...
Creating test database for alias 'other'...
.................................................FF...F..FF..................................................................
======================================================================
FAIL: test_no_nonfield_error (regressiontests.test_client_regress.models.AssertFormsetErrorTests)
An assertion is raised if the formsets non-field errors doesn't contain any errors.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rkm/projects/django/hg/tests/regressiontests/test_client_regress/models.py", line 606, in test_no_nonfield_error
    prefix + "The formset 'my_formset', form 1 in "
AssertionError: "True is not False : The formset 'my_formset', form 1 in context 0 does not contain any non-field errors." != "The formset 'my_formset', form 1 in context 0 does not contain any non-field errors."

======================================================================
FAIL: test_no_nonform_error (regressiontests.test_client_regress.models.AssertFormsetErrorTests)
An assertion is raised if the formsets non-form errors doesn't contain any errors.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rkm/projects/django/hg/tests/regressiontests/test_client_regress/models.py", line 647, in test_no_nonform_error
    "The formset 'my_formset' in context 0 "
AssertionError: "True is not False : The formset 'my_formset' in context 0 does not contain any non-form errors." != "The formset 'my_formset' in context 0 does not contain any non-form errors."

======================================================================
FAIL: test_unknown_error (regressiontests.test_client_regress.models.AssertFormsetErrorTests)
An assertion is raised if the field doesn't contain the specified error
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rkm/projects/django/hg/tests/regressiontests/test_client_regress/models.py", line 579, in test_unknown_error
    prefix + "The field 'email' "
AssertionError: "False is not True : The field 'email' on formset 'my_formset', form 0 in context 0 does not contain the error 'Some error.' (actual errors: [u'Enter a valid e-mail address.'])" != "The field 'email' on formset 'my_formset', form 0 in context 0 does not contain the error 'Some error.' (actual errors: [u'Enter a valid e-mail address.'])"

======================================================================
FAIL: test_unknown_nonfield_error (regressiontests.test_client_regress.models.AssertFormsetErrorTests)
An assertion is raised if the formsets non-field errors doesn't contain the provided error.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rkm/projects/django/hg/tests/regressiontests/test_client_regress/models.py", line 621, in test_unknown_nonfield_error
    prefix + "The formset 'my_formset', form 0 in "
AssertionError: "False is not True : The formset 'my_formset', form 0 in context 0 does not contain the non-field error 'Some error.' (actual errors: [u'Non-field error.'])" != "The formset 'my_formset', form 0 in context 0 does not contain the non-field error 'Some error.' (actual errors: [u'Non-field error.'])"

======================================================================
FAIL: test_unknown_nonform_error (regressiontests.test_client_regress.models.AssertFormsetErrorTests)
An assertion is raised if the formsets non-form errors doesn't contain the provided error.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rkm/projects/django/hg/tests/regressiontests/test_client_regress/models.py", line 661, in test_unknown_nonform_error
    "The formset 'my_formset' in context 0 does not "
AssertionError: "False is not True : The formset 'my_formset' in context 0 does not contain the non-form error 'Some error.' (actual errors: [u'Forms in a set must have distinct E-mail addresses.'])" != "The formset 'my_formset' in context 0 does not contain the non-form error 'Some error.' (actual errors: [u'Forms in a set must have distinct E-mail addresses.'])"

----------------------------------------------------------------------
Ran 125 tests in 3.638s

FAILED (failures=5)

[martin_speleo]

Fixed failing unittests, changed form parameter to form_index

[martin_speleo]

I have modified the code and documentation, as suggested, to take a form_index parameter rather than a form parameter.

I have resolved the failures of the test. The issue was the default value of the longMessage variable (Boolean) in the unittest module. I have now checked that the tests work using python 2.6 and python 2.7, with and without unittest2 available. The documentation building has also been checked.

For AssertFormsetError the error messages are slightly clearer with longMessage set to False, however I have not set this parameter in order to stay consistent with the other test functions eg. AssertFormError.

[Jacob]

Milestone 1.4 deleted

[martin_speleo]

Maintance update of patch, and removal of entry in release notes for 1.4

[Tim Graham]

I've used this in a project with good success. I've updated the patch to apply cleanly to trunk and submitted a pull request.

​https://github.com/django/django/pull/708

[Tim Graham]

I've updated the pull request above to ensure the tests pass on Python 3 and think this is RFC, but would appreciate if another core dev could take a quick look and +1 since my contributions have been limited to documentation.

[Tim Graham <timograham@…>]

In d194714c0a707773bd470bffb3d67a60e40bb787:

Fixed #11603 - Added django.test.SimpleTestCase.assertFormsetError

Thank-you Martin Green for the patch.