Confusing message id=1_6.W001

[Paolo Dente]

When migrating an old project, django.core.checks.compatibility.django_1_6_0._check_test_runner() will show a message regarding the tests, while what's actually being checked is settings.py.
I have spent quite some time trying to ensure all my tests were properly located and executed correctly, trying to get rid of the warning - to no avail, of course, since the check never actually looks at them. I suggest that either the message is phrased so it's more clear what's going on, or the page that it links to explains that, once you are sure about your tests, you can get rid of the message by explicitly setting a test runner.

[Russell Keith-Magee]

Fair point - The message doesn't mention the TEST_RUNNER setting, and the link provided in the message points at the release note describing the new feature, rather than the backwards compatibility guide saying how to compensate for the problem. There's a link from the release note _to_ the backwards compatibility guide, but that's a step that a user shouldn't have to make.

[mardini]

PR: ​https://github.com/django/django/pull/2580

[Tim Graham <timograham@…>]

In 214d1e1b0f1574ecab856f0096e409109efa9523:

Fixed #22454 - Changed compatibility warning hint

The warning hint of _check_test_runner of 1.6 compatibility had a link
to a general release note. The link should be edited to refer the
relevant "Backwards incompatible changes in 1.6" section that documents
the cause and the possible solutions and workarounds of the warning.

[Tim Graham <timograham@…>]

In fa60ac76c032aae884c3c277e89cb323a463994c:

[1.6.x] Fixed #22454 - Changed compatibility warning hint for test runner.

Backport of 214d1e1b0f from master

[Tim Graham <timograham@…>]

In 4df5dc42dd29ce2bba1ac3b7c6b45aac5b14fad1:

[1.7.x] Fixed #22454 - Changed compatibility warning hint

The warning hint of _check_test_runner of 1.6 compatibility had a link
to a general release note. The link should be edited to refer the
relevant "Backwards incompatible changes in 1.6" section that documents
the cause and the possible solutions and workarounds of the warning.

Backport of 214d1e1b0f from master

[Tai Lee]

I get this warning in a new 1.7 project, started with startproject and with the settings module customised (added SITE_ID, because I use django.contrib.sites; defined TEMPLATE_LOADERS, because I use a custom template loader; and defined MANAGERS, because I use mail_managers). Django thinks this project has been migrated, but it hasn't. I had to check the source to find out why, and even the docstring says it checks if we have not overridden TEST_RUNNER. The link from the warning talks about either updating my test suite OR using a deprecated test runner to maintain the old behaviour. I don't need to do either, but I can force Django to skip this check by explicitly defining TEST_RUNNER to be the same value that it is already set to by default. Seems a bit of a hack.

The linked docs should mention that Django checks a few key markers in the settings module to guess what version of Django you might have run in the past (not a very good check to begin with), and provide a new short-term setting specifically to disable the check if you know it doesn't apply to you, instead of forcing you to explicitly set the test runner to the default value.

Or even better, check the markers and if they trigger then use the old test runner to actually detect tests that aren't found with the new runner and report that there "are" (instead of "may be") tests that won't run.

[Russell Keith-Magee]

@mrmachine - The problem we have here is that the best we can do is guess. There's no canonical way to tell the difference between a project that was started under 1.6, and a project that was started under 1.5. The absence of TEST_RUNNER means either the project was started in 1.6, or it was started in 1.5, and hasn't been upgraded.

We check for SITE_ID, TEMPLATE_LOADERS etc because they're settings that were taken out` of the default settings.py for 1.6; but obviously, this technique is prone to false positives.

Using the old test runner and comparing to the new won't work - if you've got a new project and a moderately complex test suite, you won't have all the __init__.py tricks in place to cause the tests to be discovered, so you'll get false positives again.

If you're happy you don't have a problem, you can silence any system check using ​SILENCED_SYSTEM_CHECKS.