random TransactionManagementError during tests in _fixture_setup

[joao@…]

Automated Django 1.5.1 tests randomly fail with TransactionManagementError. It is hard to track down because a failed test can run again, without any changes, and pass.

I am NOT using the @transaction.commit_manually decorator and am aware of the need for manual commit/rollback when it is used.

One view does have the @transaction.commit_on_success decorator. It does NOT have any commands for manual commit/rollback and from the docs I believe this is correct. But this might be irrelevant:

1. the view is not called in every fail case
2. error raised during fixture setup

Test database is set to django.db.backends.sqlite3

Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/django/test/testcases.py", line 259, in __call__
    self._pre_setup()
  File "/usr/lib64/python2.7/site-packages/django/test/testcases.py", line 479, in _pre_setup
    self._fixture_setup()
  File "/usr/lib64/python2.7/site-packages/django/test/testcases.py", line 518, in _fixture_setup
    **{'verbosity': 0, 'database': db_name, 'skip_validation': True})
  File "/usr/lib64/python2.7/site-packages/django/core/management/__init__.py", line 161, in call_command
    return klass.execute(*args, **defaults)
  File "/usr/lib64/python2.7/site-packages/django/core/management/base.py", line 255, in execute
    output = self.handle(*args, **options)
  File "/usr/lib64/python2.7/site-packages/django/core/management/commands/loaddata.py", line 234, in handle
    transaction.leave_transaction_management(using=using)
  File "/usr/lib64/python2.7/site-packages/django/db/transaction.py", line 66, in leave_transaction_management
    connection.leave_transaction_management()
  File "/usr/lib64/python2.7/site-packages/django/db/backends/__init__.py", line 131, in leave_transaction_management
    "This code isn't under transaction management")
TransactionManagementError: This code isn't under transaction management

Selenium Tests

• Using Selenium webdriver 2.32.0 (latest)
• BrowserTestCase extends django.test.LiveServerTestCase
• Test classes extend BrowserTestCase and inherit the same fixtures

Unit Tests

• Error does NOT occur for tests extending django.test.TestCase or django.utils.unittest
• Use the same fixtures as above

[Anssi Kääriäinen]

It is very likely that you have a test running two code paths concurrently:

1. The test server is generating a response for a request generated by selenium
2. The tests do not wait for that response to be generated, instead the tests continue parallel to the request

When using in-memory SQLite3 the connection is shared between the test server and the test itself. As a result weird concurrency errors are possible. Most likely a test just before the failing one has an ongoing request, so that the next test's fixture loading happens concurrently to the request processing.

A cause for this could be pretty much anything done via selenium scripting (for example click) but not waiting for the results of that action to be generated. See 50677b29af39ca670274fb45087415c883c78b04 and #19856 for a similar issue in Django's test suite.

I will mark this as invalid as this is most likely not an issue in Django itself.

[xme]

I can confirm problem with tests (the same error message) and running in background server. Such a situation happens even if there is another port for dev and tests. Stopping dev server makes that tests pass.

[anonymous]

Same here, I've seen people asking about it on SO as well.