Unittest of a form with Multiselect not filling correctly

[Trevor Woolley]

When I run a unit test on a form containing a MultipleChoiceField, the contents of the MultipleChoiceField are being filled from the production database not the test database. When the form is displayed(printed) it does not appear to be refreshed when called, and only appears to be filled with data, taken before the test database was actually created.

The production database in this case is empty, with appropriate data in the production database this test passes, but the values are the production based ones.

Test Output: Print statements as per code elements lower down.

$ ./manage.py test issue.tests.test_forms.DispLayIssueFormTests.test_valid_save2
REAL --> []
In Form --> []
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
START
MOCKED --> [{'id': '1', 'displayname': 'foo1 (bar1)'}, {'id': '2', 'displayname': 'foo2 (bar2)'}]
ERRORS --> <ul class="errorlist"><li>stuff<ul class="errorlist"><li>Select a valid choice. 1 is not one of the available choices.</li></ul></li></ul>
FORMS --> <tr><th><label for="id_description">Description:</label></th><td><input type="text" name="description" value="test3" id="id_description" maxlength="199" required /></td></tr>
<tr><th><label for="id_stuff">Stuff:</label></th><td><ul class="errorlist"><li>Select a valid choice. 1 is not one of the available choices.</li></ul><select name="stuff" id="id_stuff" size="10" multiple="multiple"></select><input type="hidden" name="db_id" value="0" id="id_db_id" /></td></tr>
F
======================================================================
FAIL: test_valid_save2 (issue.tests.test_forms.DispLayIssueFormTests)
Test to ensure a valid save succeeds - with stuff
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\projects\issue\lib\site-packages\mock\mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "c:\projects\issue\issue\tests\test_forms.py", line 288, in test_valid_save2
    self.assertTrue(testform.is_valid())
AssertionError: False is not true

----------------------------------------------------------------------
Ran 1 test in 0.538s

FAILED (failures=1)
Destroying test database for alias 'default'...
$

python and django version

$ ./manage.py shell
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:43:06) [MSC v.1600 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import django
>>> django.get_version()
'1.11'
>>>

forms.py snippet

# appropriate imports

def get_my_list():
    """ Function to get the complete stuff list for multiselect """
    result = []
    my_stuff = Stuff.objects.all()
    for stuff in my_stuff:
        displayname = "%s (%s)" % (stuff.name1, stuff.name2)
        result.append({"id": str(stuff.id), "displayname": displayname})
    print("REAL --> %s" % result)
    return sorted(result, key=itemgetter("displayname"))


class DispLayIssueForm(forms.Form):
    """ Class to manage the changes / adds for the extra Stuff model """
    description = forms.CharField(
        label='Short Description',
        max_length=199,
        strip=True
    )
    my_choices = [(m['id'], m['displayname']) for m in get_my_list()]
    print("In Form --> %s" % my_choices)

    stuff = forms.MultipleChoiceField(
        widget=forms.SelectMultiple(attrs={'size': '10'}),
        choices=my_choices,
        label="My Stuff",
        required=False,
    )
    db_id = forms.CharField(widget=forms.HiddenInput)

    def clean(self):
        """ Form Clean function """
        # <Snip>

    def save(self):
        """ Form Save function """
        # <Snip>

    def update(self):
        """ Form Update function """
        # <Snip>

test_forms.py

# appropriate imports
# django.test import TestCase


def mocked_get_my_list():
    """ Mocking the function so that it grabs the test data """
    result = []
    my_stuff = Stuff.objects.all()
    for stuff in my_stuff:
        displayname = "%s (%s)" % (stuff.name1, stuff.name2)
        result.append({"id": str(stuff.id), "displayname": displayname})
    print("MOCKED --> %s" % result)
    return result

class DispLayIssueFormTests(TestCase):
    """ Class to test the display issue form """

    def setUp(self):
        """ Initial setup just creates two stuff entries """
        self.name1 = create_stuff("foo1", "bar1")
        self.name2 = create_stuff("foo2", "bar2")

    def tearDown(self):
        """ Final tear downs """
        self.name1.delete()
        self.name2.delete()

    # Other tests that leave the stuff fields empty pass

    @mock.patch('form.get_my_list')
    def test_valid_save2(self, mock_list):
        """ Test to ensure a valid save succeeds - with stuff """
        print("START")
        mock_list.side_effect = mocked_get_my_list()
        testform = DispLayIssueForm({
            'description': 'test3',
            'stuff': [str(self.name1.id), ],
            'db_id': 0,
        })
        print("ERRORS --> %s" % testform.errors)
        print("FORMS --> %s" % testform)
        self.assertTrue(testform.is_valid())
        testform.save()
        dummy = ExtraStuff.objects.get(description='test3')

[Tim Graham]

This is because you have a module level query at my_choices = [(m['id'], m['displayname']) for m in get_my_list()].

There's a documentation warning about this; see the "Finding data from your production database when running tests?" note in ​this section.

[Trevor Woolley]

I was just about to update this ticket, with additional comments, these being.

1) This also occurs outside of testing when running the test server.
and
2) I have noted that a similar issue occurs on apache production based servers.

It appears the choices are only loaded on restart, and not when called.

So the Subject should change to --> Dynamic MulitpleChoiceFields not updating with new data until restarted.

[Tim Graham]

Those symptoms are also because you're running module level queries which are cached until the server is restarted.

p.s. Next time, you might want to try our support channels and open a ticket after you've confirmed a bug in Django.

[Trevor Woolley]

Thanks Tim

FYI - I did just get the testform to load the updated data by using the following. But I do understand your response, and the below may be invalid in the long run, but I'll add it here for reference.

    @mock.patch('form.get_my_list')
    def test_valid_save2(self, mock_list):
        """ Test to ensure a valid save succeeds - with stuff """
        print("START")
        mock_list.side_effect = mocked_get_my_list()
        testform = DispLayIssueForm({
            'description': 'test3',
            'stuff': [str(self.name1.id), ],
            'db_id': 0,
        })
        testform.fields['stuff'].choices = [
            (m['id'], m['displayname']) for m in mocked_get_my_list()
        ]
        print("ERRORS --> %s" % testform.errors)
        print("FORMS --> %s" % testform)
        self.assertTrue(testform.is_valid())
        testform.save()
        dummy = ExtraStuff.objects.get(description='test3')