Cache should be reset after each test without flushing

[Jim Dalton]

At present, when cache values are set during a test, those values carry over from test to test. This breaks proper test isolation and can making testing code that uses the cache cumbersome, since cache values have to be manually flushed.

Using cache.clear() would be on option to handle this, however this clears *all* the data in a cache, not just for your application. This could be considered impolite at best and dangerous at worst, so a proper solution would not resort to this measure.

Here are some guidelines/requirements for a solution:

• Any cache(s) used during a test run should ideally be restored to their original state when the test run is finished.
• Any cache values set during an individual test should be cleared when that test is finished.

Some discussion on the developers list here: ​https://groups.google.com/forum/#!topic/django-developers/zlaPsP13dUY

[Jim Dalton]

Okay, I sat down to work on a patch. I think it worked out fine. The tests I added are pretty thorough and most everything seems to be working. A few notes:

• During global setup, it seems that the cache is hit a few times before I do the cache monkey patching. The result is the cache is not absolutely 100% pristine after a run. I think that's just fine though -- no sense overcomplicating things in a senseless pursuit of perfection.
• The full test suite passes all tests, save for 3. There was one error which was due to the dummy cache's set_many() not implementing the proper method signature from base cache. i just fixed this in my patch. The other are filebased cache related. For some reason the cull count is coming out differently and as a result two cull tests are failing for the file-based cache. These numbers seen somewhat arbitrary to me -- I'm not sure why we expect 29 objects to be saved when the max of 30 is hit. Anyhow, with my patch the number comes out to 28 for the file patch. It's probably something mundane that's causing the bug, like an extra value getting added somehow. If anyone has any ideas let me know. Most likely 28 is the "correct" number, I just want to verify this before I change it. I'll do some checking later as well.
• Note that because we are testing that values don't pass between tests, there is some order dependency to the tests. Unittest runs test in alphanumeric order so I've named the tests appropriately, and added a few assertions to affirm that the order is correct. I'm also using setUpClass() to handle some of the cache preconfiguration we need to do to have meaningful tests.
• We probably need a note in the testing documentation about this stuff assuming it gets accepted.

That's all. It's actually pretty straightforward despite being a bit fiddly with the monkey patching, as that stuff usually is.

Any questions/comments/concerns let me know. Marking as patch needs improvement pending the fix for those two tests and needs documentation, assuming we might want to mention this in test docs.

[Julien Phalip]

I believe this is a duplicate of #11505.