Opened 2 years ago

Closed 2 years ago

#34276 closed Bug (invalid)

LocMemCache not working for multiple threads

Reported by: D Bersan Owned by: nobody
Component: Core (Cache system) Version: 4.1
Severity: Normal Keywords: LocMemCache
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by D Bersan)

I have a thread that is initialized on apps.py in a given app in django.

This thread updates the value of a counter that is saved in cache, using django.core.cache.

Also I have a view that returns the value of the same cache, in an endpoint:

# jobs.py
import threading
import time
from django.core.cache import cache
 
class MyThread:
  def __init__(self) -> None:
    self.t = threading.Thread(target=MyThread.myLoop, args=(self,))
    self.t.daemon = True
    self.count = 1
 
  def myLoop(self):
    while self.running:
      time.sleep(3)
      self.count += 1
      cache.set('count', self.count)
 
  def start(self):  
    self.running = True 
    self.t.start()
 
  def stop(self):
    self.running = False
 
# --------------
# apps.py
from django.apps import AppConfig
import os
 
class MyAppConfig(AppConfig):
    default_auto_field = "django.db.models.BigAutoField"
    name = "MyApp"
 
    def ready(self) -> None:
        from .jobs import MyThread
        t= MyThread()
 
        if os.environ.get('RUN_MAIN', None) != 'true':
            t.start()
        # return super().ready()
 
# --------------
# views.py
from django.http import JsonResponse
from django.core.cache import cache
# ... other imports
 
def getCurrentCount(request):
  global cache
  if request.method == 'GET':
    return JsonResponse({
      'count': cache.get('count')
    })

I am using (LocMemCache)https://docs.djangoproject.com/en/4.1/topics/cache/#local-memory-caching in settings.py:

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
        'LOCATION': 'unique-snowflake',
    }
}

But when I try to log the result of cache.get('count'), it returns None.

The documentation clearly states:

"This cache is per-process (see below) and thread-safe."

"Note that each process will have its own private cache instance, which means no cross-process caching is possible. "

I have the impression that they meant the cache is per-thread , not per-process . Both my threads are on the same process, so they should share the same caching, according to the documentation. It also emphasizes, no cross-process caching is possible. It doesn't say no cross-threading caching is possible. But reading other resources I found online, it seems this is not shared even among threads, so the documentation is wrong, or there is a bug in this version of Django.

Change History (3)

comment:1 by D Bersan, 2 years ago

Description: modified (diff)

comment:2 by D Bersan, 2 years ago

Description: modified (diff)

in reply to:  description comment:3 by Mariusz Felisiak, 2 years ago

Component: UncategorizedCore (Cache system)
Resolution: invalid
Status: newclosed

The documentation clearly states:

"This cache is per-process (see below) and thread-safe."
"Note that each process will have its own private cache instance, which means no cross-process caching is possible. "

I have the impression that they meant the cache is per-thread , not per-process .

The LocMemCache cache is safe -- each process and thread gets its own cache, so there's no chances of conflicts (see comment).

... Both my threads are on the same process, so they should share the same caching, according to the documentation ...

There is nothing in the docs about sharing the same cache instance between processes or threads.

It also emphasizes, no cross-process caching is possible. It doesn't say no cross-threading caching is possible. But reading other resources I found online, it seems this is not shared even among threads, so the documentation is wrong, or there is a bug in this version of Django.

I would argue that documentation doesn't need to mention everything that it's not possible. It's also documented:

"To provide thread-safety, a different instance of the cache backend will be returned for each thread."

Note: See TracTickets for help on using tickets.
Back to Top