Opened 3 years ago

Last modified 3 years ago

#32815 closed Uncategorized

Failed to reset ContextVars in sync/async middlewares — at Initial Version

Reported by: Michael Manganiello Owned by: nobody
Component: Uncategorized Version: 3.2
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When using a middleware that can process both sync and async requests, and trying to set and reset a ContextVar (in different methods of its request lifecycle), Python fails with error:
ValueError: <Token var=<ContextVar name='current_context' at 0x7f9a8b9ad900> at 0x7f9a68575180> was created in a different Context

This is a simple middleware example to reproduce the mentioned issue:

@sync_and_async_middleware
class TemplateResponseMiddleware(BaseMiddleware):
    def process_view(self, request, view_func, view_args, view_kwargs):
        request.META['_CONTEXT_RESET_TOKEN'] = current_context.set(id(request))

    def process_template_response(self, request, response):
        current_context.reset(request.META['_CONTEXT_RESET_TOKEN'])
        response.context_data['mw'].append(self.__class__.__name__)
        return response

This use case is what the OpenTelemetry integration uses for spans to be traced in Django: https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/middleware.py

  • In process_request, a ContextVar is set, and the generated token is persisted in the request.META object.
  • In process_response, the ContextVar is reset, by using the persisted token.

This approach works correctly for synchronous requests. However, as part of adding ASGI support to the Django integration for OpenTelemetry (in https://github.com/open-telemetry/opentelemetry-python-contrib/pull/391), we found that the ContextVar triggers the mentioned error when we want to reset it to its previous value. OpenTelemetry inherits from MiddlewareMixin, but I'm attaching a diff for a simple test scenario that reproduces the issue, using the new Middleware format.

The main suspects here are the calls to sync_to_async, which adapt the middleware methods to the async flow. However, both those calls explicitly set thread_sensitive=True.

Traceback for the attached test scenario:

$ ./runtests.py -k MiddlewareSyncAsyncTests.test_async_process_template_response
# ...
ERROR: test_async_process_template_response (middleware_exceptions.tests.MiddlewareSyncAsyncTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mike/.virtualenvs/django/lib/python3.9/site-packages/asgiref/sync.py", line 222, in __call__
    return call_result.result()
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 438, in result
    return self.__get_result()
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 390, in __get_result
    raise self._exception
  File "/home/mike/.virtualenvs/django/lib/python3.9/site-packages/asgiref/sync.py", line 287, in main_wrap
    result = await self.awaitable(*args, **kwargs)
  File "/mnt/data/Proyectos/third_party/django/django/test/utils.py", line 423, in inner
    return await func(*args, **kwargs)
  File "/mnt/data/Proyectos/third_party/django/tests/middleware_exceptions/tests.py", line 319, in test_async_process_template_response
    response = await self.async_client.get(
  File "/mnt/data/Proyectos/third_party/django/django/test/client.py", line 911, in request
    self.check_exception(response)
  File "/mnt/data/Proyectos/third_party/django/django/test/client.py", line 580, in check_exception
    raise exc_value
  File "/home/mike/.virtualenvs/django/lib/python3.9/site-packages/asgiref/sync.py", line 458, in thread_handler
    raise exc_info[1]
  File "/mnt/data/Proyectos/third_party/django/django/core/handlers/exception.py", line 38, in inner
    response = await get_response(request)
  File "/mnt/data/Proyectos/third_party/django/django/core/handlers/base.py", line 249, in _get_response_async
    response = await middleware_method(request, response)
  File "/home/mike/.virtualenvs/django/lib/python3.9/site-packages/asgiref/sync.py", line 423, in __call__
    ret = await asyncio.wait_for(future, timeout=None)
  File "/usr/lib/python3.9/asyncio/tasks.py", line 442, in wait_for
    return await fut
  File "/home/mike/.virtualenvs/django/lib/python3.9/site-packages/asgiref/current_thread_executor.py", line 22, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/mike/.virtualenvs/django/lib/python3.9/site-packages/asgiref/sync.py", line 462, in thread_handler
    return func(*args, **kwargs)
  File "/mnt/data/Proyectos/third_party/django/tests/middleware_exceptions/middleware.py", line 135, in process_template_response
    current_context.reset(request.META['_CONTEXT_RESET_TOKEN'])
ValueError: <Token var=<ContextVar name='current_context' at 0x7f1dd4ec2720> at 0x7f1db129a880> was created in a different Context

Change History (1)

by Michael Manganiello, 3 years ago

Attachment: django-issue-32815.diff added

Test scenario (diff over main commit f10c52afab)

Note: See TracTickets for help on using tickets.
Back to Top