Changes between Initial Version and Version 3 of Ticket #33533


Ignore:
Timestamp:
Mar 15, 2022, 3:35:59 PM (3 years ago)
Author:
Michael
Comment:

Replying to Tim Graham:

See TicketClosingReasons/UseSupportChannels for ways to get help if you cannot debug the issue yourself. Please only use this ticket tracker for reporting confirmed bugs. Thanks!

Hi, I have done some more digging with some help: ​https://forum.djangoproject.com/t/should-it-be-impossible-for-a-session-dict-to-return-more-than-one-value-for-a-key/12298/13

I think the issue is this: if it receives multiple requests at the same time (I have a service worker that requests many pages at installation so it can cache them), and if SESSION_SAVE_EVERY_REQUEST = True, then the way it handles session updates it just falls over in many spectacular ways. My for now my fix is to set SESSION_COOKIE_AGE to over a year, and have SESSION_SAVE_EVERY_REQUEST = False

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #33533

    • Property Resolution → invalid
    • Property Status new → closed

  • Ticket #33533 – Description

    initial v3  
    1 In my project, I store the serial number of the device that logs in it's session. However in the code that gets the value from the session:
     1In my project, there are some methods on a custom user model that require the `request` to calculate certain values. This simple middleware does the trick:
    22
    33{{{
    4 SESSION_SERIAL_NUMBER = 'terminal_serial_number'
     4class AttachRequestToUserMiddleware:
     5    """Adds the request to the user object, so session information can be looked
     6    up by the custom user model.
    57
    6 user._serial_number = request.session.get(SESSION_SERIAL_NUMBER, '')
     8    Must come after django.contrib.auth.middleware.AuthenticationMiddleware which adds the user"""
     9
     10    def __init__(self, get_response):
     11        self.get_response = get_response
     12
     13    def __call__(self, request):
     14        request.user.request = request
     15        return self.get_response(request)
    716}}}
    817
    9 Very occasionally throws an error. The part that baffles me is:
     18In production, when there are multiple workers running parrallel by uWSGI, if one has `SESSION_SAVE_EVERY_REQUEST = True`, then if one makes async requests from JavaScript (e.g. say a Service Worker caching pages on install), then the way it saves/retrieves sessions on every request fails in many spectacular ways.
    1019
    11 > get() returned more than one Session -- it returned more than 20!
     20Here are some example trace backs:
     21{{{
     22Django Version: 4.0.1
     23Python Version: 3.8.5
    1224
    13 How can a dict return more than one result, surely that is a bug for a dict to return more than one value?
     25Exception Type: ProgrammingError
     26Exception Value:       
     27no results to fetch
     28Exception Location:     /home/michael/.venv/project/lib/python3.8/site-packages/django/db/utils.py, line 97, in inner
     29/home/michael/.venv/project/lib/python3.8/site-packages/django/db/utils.py, line 97, in inner
     30                return func(*args, **kwargs)
    1431
    15 Note: When I query the database `django_sessions` table with the given `sessionid`, it only returns one row.
    1632
    17 Heres the error (from the email):
     33Exception Type: RuntimeError
     34Exception Value:       
     35generator raised StopIteration
     36/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/auth/__init__.py, line 60, in _get_user_session_key
     37return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])
    1838
    19 {{{
    20 Internal Server Error: /accounts/login/
    21 
    22 ProgrammingError at /accounts/login/
    23 no results to fetch
     39Exception Type: MultipleObjectsReturned
     40Exception Value:       
     41get() returned more than one Session -- it returned 2!
     42/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py, line 180, in _get_session
     43            return self._session_cache
     44/home/michael/.venv/project/lib/python3.8/site-packages/django/core/handlers/exception.py, line 47, in inner
     45                response = get_response(request)
     46./core/accounts/middleware.py, line 33, in __call__
     47        request.user.request = request
     48/home/michael/.venv/project/lib/python3.8/site-packages/django/utils/functional.py, line 278, in __setattr__
     49                self._setup()
     50/home/michael/.venv/project/lib/python3.8/site-packages/django/utils/functional.py, line 384, in _setup
     51        self._wrapped = self._setupfunc()
     52/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/auth/middleware.py, line 25, in <lambda>
     53        request.user = SimpleLazyObject(lambda: get_user(request))
     54/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/auth/middleware.py, line 11, in get_user
     55        request._cached_user = auth.get_user(request)
     56/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/auth/__init__.py, line 177, in get_user
     57        user_id = _get_user_session_key(request)
     58/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/auth/__init__.py, line 60, in _get_user_session_key
     59    return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])
     60/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py, line 50, in __getitem__
     61        return self._session[key]
     62/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py, line 185, in _get_session
     63                self._session_cache = self.load()
     64/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/db.py, line 43, in load
     65        s = self._get_session_from_db()
     66/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/db.py, line 32, in _get_session_from_db
     67            return self.model.objects.get(
     68/home/michael/.venv/project/lib/python3.8/site-packages/django/db/models/manager.py, line 85, in manager_method
     69                return getattr(self.get_queryset(), name)(*args, **kwargs)
     70/home/michael/.venv/project/lib/python3.8/site-packages/django/db/models/query.py, line 443, in get
     71        raise self.model.MultipleObjectsReturned(
    2472}}}
    25 
    26 And the traceback:
    27 
    28 {{{
    29 Traceback (most recent call last):
    30   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py", line 180, in _get_session
    31     return self._session_cache
    32 
    33 During handling of the above exception ('SessionStore' object has no attribute '_session_cache'), another exception occurred:
    34   File "/home/company/.venv/project/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    35     response = get_response(request)
    36   File "./dist/terminals/middleware.py", line 11, in __call__
    37     user._serial_number = request.session.get(SESSION_SERIAL_NUMBER, '')
    38   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py", line 65, in get
    39     return self._session.get(key, default)
    40   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py", line 185, in _get_session
    41     self._session_cache = self.load()
    42   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/db.py", line 43, in load
    43     s = self._get_session_from_db()
    44   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/db.py", line 32, in _get_session_from_db
    45     return self.model.objects.get(
    46   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
    47     return getattr(self.get_queryset(), name)(*args, **kwargs)
    48   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/models/query.py", line 443, in get
    49     raise self.model.MultipleObjectsReturned(
    50 
    51 During handling of the above exception (get() returned more than one Session -- it returned more than 20!), another exception occurred:
    52   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py", line 180, in _get_session
    53     return self._session_cache
    54 
    55 During handling of the above exception ('SessionStore' object has no attribute '_session_cache'), another exception occurred:
    56   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/utils.py", line 97, in inner
    57     return func(*args, **kwargs)
    58 
    59 The above exception (no results to fetch) was the direct cause of the following exception:
    60   File "/home/company/.venv/project/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    61     response = get_response(request)
    62   File "/home/company/.venv/project/lib/python3.8/site-packages/django/utils/deprecation.py", line 126, in __call__
    63     response = response or self.get_response(request)
    64   File "/home/company/.venv/project/lib/python3.8/site-packages/django/core/handlers/exception.py", line 49, in inner
    65     response = response_for_exception(request, exc)
    66   File "/home/company/.venv/project/lib/python3.8/site-packages/django/core/handlers/exception.py", line 110, in response_for_exception
    67     response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
    68   File "/home/company/.venv/project/lib/python3.8/site-packages/django/core/handlers/exception.py", line 149, in handle_uncaught_exception
    69     return callback(request)
    70   File "./core/base/views.py", line 152, in error_500_view
    71     response = render(request, "core.base/500.html", context=context)
    72   File "/home/company/.venv/project/lib/python3.8/site-packages/django/shortcuts.py", line 19, in render
    73     content = loader.render_to_string(template_name, context, request, using=using)
    74   File "/home/company/.venv/project/lib/python3.8/site-packages/django/template/loader.py", line 62, in render_to_string
    75     return template.render(context, request)
    76   File "/home/company/.venv/project/lib/python3.8/site-packages/django/template/backends/django.py", line 61, in render
    77     return self.template.render(context)
    78   File "/home/company/.venv/project/lib/python3.8/site-packages/django/template/base.py", line 174, in render
    79     with context.bind_template(self):
    80   File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
    81     return next(self.gen)
    82   File "/home/company/.venv/project/lib/python3.8/site-packages/debug_toolbar/panels/templates/panel.py", line 46, in _request_context_bind_template
    83     context = processor(self.request)
    84   File "./core/base/context_processors.py", line 13, in project
    85     menu = get_plug_menu(request.user)
    86   File "./dist/plug/menu.py", line 58, in get_plug_menu
    87     if user.is_anonymous or user.is_terminal:
    88   File "/home/company/.venv/project/lib/python3.8/site-packages/django/utils/functional.py", line 248, in inner
    89     self._setup()
    90   File "/home/company/.venv/project/lib/python3.8/site-packages/django/utils/functional.py", line 384, in _setup
    91     self._wrapped = self._setupfunc()
    92   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/auth/middleware.py", line 25, in <lambda>
    93     request.user = SimpleLazyObject(lambda: get_user(request))
    94   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/auth/middleware.py", line 11, in get_user
    95     request._cached_user = auth.get_user(request)
    96   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/auth/__init__.py", line 177, in get_user
    97     user_id = _get_user_session_key(request)
    98   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/auth/__init__.py", line 60, in _get_user_session_key
    99     return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])
    100   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py", line 50, in __getitem__
    101     return self._session[key]
    102   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py", line 185, in _get_session
    103     self._session_cache = self.load()
    104   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/db.py", line 43, in load
    105     s = self._get_session_from_db()
    106   File "/home/company/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/db.py", line 32, in _get_session_from_db
    107     return self.model.objects.get(
    108   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
    109     return getattr(self.get_queryset(), name)(*args, **kwargs)
    110   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get
    111     num = len(clone)
    112   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/models/query.py", line 262, in __len__
    113     self._fetch_all()
    114   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/models/query.py", line 1354, in _fetch_all
    115     self._result_cache = list(self._iterable_class(self))
    116   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/models/query.py", line 51, in __iter__
    117     results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
    118   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1234, in execute_sql
    119     return list(result)
    120   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1678, in cursor_iter
    121     for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
    122   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1678, in <lambda>
    123     for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
    124   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/utils.py", line 97, in inner
    125     return func(*args, **kwargs)
    126   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
    127     raise dj_exc_value.with_traceback(traceback) from exc_value
    128   File "/home/company/.venv/project/lib/python3.8/site-packages/django/db/utils.py", line 97, in inner
    129     return func(*args, **kwargs)
    130 
    131 Exception Type: ProgrammingError at /accounts/login/
    132 Exception Value: no results to fetch
    133 Request information:
    134 USER: AnonymousUser
    135 }}}
    136 
    137 This is the middleware it sits in:
    138 {{{
    139 from dist.terminals.apps import SESSION_SERIAL_NUMBER
    140 
    141 class UserSerialNumberMiddleware:
    142     def __init__(self, get_response):
    143         self.get_response = get_response
    144         # One-time configuration and initialization.
    145 
    146     def __call__(self, request):
    147         user = request.user
    148         user._serial_number = request.session.get(SESSION_SERIAL_NUMBER, '')
    149         response = self.get_response(request)
    150         return response
    151 }}}
Back to Top