LocMem cache may cause deadlocks

[Rafal Stozek]

LocMem cache uses locks to synchronize access to the cache dict.

The problem is that pickle dumps/loads is executed inside code blocks which acquired a reader/writer lock. Pickle dumps/loads can execute parts of user's code to accomplish it's task. The problem starts when user's code tries to retrieve something from the cache.

In that situation for example set() acquires a lock, calls pickle.dumps() which calls user's code, which calls get(), which tries to acquire a lock - causing a deadlock. Very rare situation but it does happen.

[Rafal Stozek]

The solution would be to move pickle.dumps/loads calls outside of the with self._lock.reader()/writer() blocks.

[ersran9]

I added a pull request for this. Could someone review it? ​https://github.com/django/django/pull/1296

[Jaap Roes]

Looked at your pull request and found one issue, I've added a line comment on GitHub. Did you run the (cache) tests?

[ersran9]

I have made the changes that you had suggested. The commit was squashed into the previous pull, not sure if that come out okay. The cache tests does pass.

[anonymous]

Fixed in 126250a4c422f66469eb70d792dcfd65b1ac5178

[Tim Graham <timograham@…>]

In 6c5a30b4e7f51e8c255dc104714d5748e5b5870c:

Added tests for LocalMemCache deadlocks. refs #20613 and refs #18541.

Thanks Zach Smith for the patch.