Django's TestCase should reset the cache

[andrewfong]

In between test cases, the cache should be flushed in order to prevent one test case from polluting another, similar to how the database is currently flushed (or transactions rolled back) in Django's test case.

This will be easier to implement if #11503 is complete

[andrewfong]

Adds cache flushing after every testcase + a simple regression test

[andrewfong]

Documentation of cache flushing

[Mikhail Korobov]

There is a pattern to run tests on production machine. With this patch e.g. production memcached instance can be flushed after every commit. Databases are different because one DB server can handle several databases and test database can be created during test setup. I think it can't be done for 'test memcached instance'.

Memcached key space can be easily shared between production and test environment using custom cache backend that prepends all key with fixed string (giving us an equivalent of 'test database' inside single memcached instance). But invalidation is nearly impossible.

[Julien Phalip]

#16401 is a duplicate and contains a much more thorough patch.

[Jim Dalton]

Replying to julien:

#16401 is a duplicate and contains a much more thorough patch.

Thanks julien for consolidating the new ticket I opened back to this one.

I'm uploading the latest patch from that ticket here. In a nutshell, this approach monkey patches cache and get_cache during test setup and keeps track of any keys used during the test. After each test, those keys are flushed. We also prepend '_test' to the key prefix to ensure existing values are not overwritten. The patch is tested and working. I'm leaving it as "needs improvement" for now mainly because there is one test in the suite that is failing and I haven't determined why yet. It's minor whatever is is though.

In sum, the patch works great and solves the problem. There was a bit of discussion here about the approach, however: ​https://groups.google.com/forum/#!topic/django-developers/zlaPsP13dUY . Jacob stated he wasn't necessarily a fan of monkey patching and suggested an alternative. I had some thoughts which I posted in response as well.

If anyone reading this comment is interested in help push this ticket toward resolution, a great way would be to review this patch and then weigh in on that django dev discussion. If we can reach a consensus on django-dev I'd be happy to put in some time to implement whatever approach is agreed upon.

[Jim Dalton]

Adding a new patch for this, which extends the modified cache approach to cache middleware. I had noticed on some of my projects that certain tests were failing when using cache middleware, which turned out to be because the cache was returning the response and thus the context was not included in the response to the test client. Tests reproducing this error state as well as a fix are included in the latest patch. The full test suite passes as well (minus 5 unrelated failures that are on trunk right now).

The failures to me are a good illustration of why this patch is important. The system should ideally be as clean as possible in between each test; cache values carrying over from test to test create problematic behavior when a full test suite is run.

I might post again to the dev list if there's no movement on this. If anyone on the CC list here would like to review this patch and give any feedback please do so. At this point, all that's really required from what I can see is a few notes in the documentation.

[Aymeric Augustin]

In [17042]:

Fixed several problems that hid one another in the cache tests and code.

1 - Used django.test.TestCase instead of unittest.TestCase so that the override_settings decorator works.
2 - Reverted parts of r17039 that caused failures (masked until 1).
3 - Isolated tests by clearing the cache in tear down (masked until 1). Refs #11505.
4 - Fixed a bug in cache key generation (revealed by 3).
5 - Fixed a test that relied on this bug -- hardcoding the generated cache keys in tests isn't a very good idea anyway.
6 - Uniformized some parts of the cache tests.

[Thomas C]

I ran into another example of tests depending on each other execution, because of the way django.contrib.sites.models.Site instances are cached in a global variable SITE_CACHE

Reproductible with the command python tests/runtests.py contenttypes_tests.tests.ContentTypesViewsTests.test_shortcut_with_absolute_url syndication_tests

[Tim Graham <timograham@…>]

In 3bb78c5e7a330ed70c518e77899bf8a5ac6bf766:

Cleanup cache in contrib.sites to prevent test interference -- refs #11505

[Tim Graham <timograham@…>]

In 4135d837027eac43ec416856d9476c478167d8a6:

Isolated a flatpages test; refs #11505.

[Tim Graham <timograham@…>]

In 1806e059f6127e3e4572db09f297984c96ea9d02:

[1.8.x] Isolated a flatpages test; refs #11505.

Backport of 4135d837027eac43ec416856d9476c478167d8a6 from master

[Tim Graham <timograham@…>]

In e0b3926026984dccc86a09c0c4f32e8bec6f3fe1:

Isolated auth_tests from contenttypes_tests; refs #11505.

[Tim Graham <timograham@…>]

In 259259a819a12a9ecfd0448c646899fad78e23e0:

[1.8.x] Isolated auth_tests from contenttypes_tests; refs #11505.

Backport of e0b3926026984dccc86a09c0c4f32e8bec6f3fe1 from master

[Tim Graham <timograham@…>]

In f668bac9d223408627ca92b2281cf7110039510b:

Fixed #24345 -- Isolated sitemaps_tests from contenttypes_tests; refs #11505

[Tim Graham <timograham@…>]

In bc2eb6bfef00fd8ea4395e3462366b6571f88601:

[1.8.x] Fixed #24345 -- Isolated sitemaps_tests from contenttypes_tests; refs #11505

Backport of f668bac9d223408627ca92b2281cf7110039510b from master

[Tim Graham <timograham@…>]

In 2a9bcb5:

Refs #26085, #11505 -- Cleared Site cache in contenttypes_tests.

[Kayvan Sylvan]

You can do this automatically for all tests by adding a pytest fixture like this. The operative part is autouse=True

from django.conf import settings
from django.core.cache import cache

@pytest.fixture(autouse=True)
def reset_cache():
    yield
    assert (
        settings.CACHES['default']['BACKEND']
        == 'django.core.cache.backends.locmem.LocMemCache'
    )
    cache.clear()