Documentation of "test execution order" in 1.6

[Florian Apolloner]

Current docs (​https://docs.djangoproject.com/en/dev/topics/testing/overview/#order-in-which-tests-are-executed) suggest that there is no particular order defined in which tests are executed; this is not true according to ​https://github.com/django/django/blob/master/django/test/runner.py#L18 which reorders the tests to execute TestCase before TransactionTestCase

[Aymeric Augustin]

Note that the deprecated DjangoTestSuiteRunner behaves according to the documentation: ​https://github.com/django/django/blob/master/django/test/simple.py#L253. This could be an unintended side effect of the new test discovery.

However, the behavior of the current code is useful for fixing #20579, and I'd prefer to keep it if possible at all.

[Carl Meyer]

It was not intended for discovery to change test case reordering, and based on my reading of the code there is no change. In both the old and new runners, the call that is made is reorder_suite(suite, (TestCase,)). The only difference is that in the new runner the value (TestCase,) is pulled out into a runner class attribute for easier modification by subclasses, but the default reordering behavior is the same in both runners.

[Aymeric Augustin]

There's a small perversity here, it isn't the same TestCase in the two classes:

• in the old (deprecated) runner, TestCase is unittest.TestCase (it's explicitly spelled like that).
• in the new runner, TestCase is from django.test import TestCase.

[Andrew Godwin <andrew@…>]

In e26b589b8cd2c46c0e6af360abaacdb2fb0af27d:

Fixed #20590: Documented new test case ordering