Opened 3 hours ago

#35945 new Uncategorized

Paginator doesn't work in async mode

Reported by: smiling-watermelon Owned by:
Component: Core (Other) Version: 5.1
Severity: Normal Keywords: Paginator, async, SynchronousOnlyOperation
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

If you provide a QuerySet into a paginator from an async context, the Paginator cannot retrieve count property of the QuerySet resulting in an error.
This can be avoided by making an async version of page method that would use an async version of count property that would refer to acount method of an object, if it exists.

As of now we have to use sync_to_async to run the pagination code in production.

Error traceback below:

(Partially omitted)
Traceback (most recent call last):
  File "/app/MyProjectName/operations/XYZ/read.py", line 23, in get_XYZ_model
    XYZ_list, total = await paginate_queryset(
                           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/MyProjectName/operations/utils/pagination.py", line 12, in paginate_queryset
    page = paginator.page(qs)
               ^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/django/core/paginator.py", line 89, in page
    number = self.validate_number(number)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/django/core/paginator.py", line 70, in validate_number
    if number > self.num_pages:
                ^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/django/utils/functional.py", line 47, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
                                         ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/django/core/paginator.py", line 116, in num_pages
    if self.count == 0 and not self.allow_empty_first_page:
       ^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/django/utils/functional.py", line 47, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
                                         ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/django/core/paginator.py", line 110, in count
    return c()
           ^^^
  File "/app/.venv/lib/python3.12/site-packages/django/db/models/query.py", line 620, in count
    return self.query.get_count(using=self.db)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/django/db/models/sql/query.py", line 630, in get_count
    return obj.get_aggregation(using, {"__count": Count("*")})["__count"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/django/db/models/sql/query.py", line 616, in get_aggregation
    result = compiler.execute_sql(SINGLE)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/cachalot/monkey_patch.py", line 37, in inner
    return original(compiler, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/cachalot/monkey_patch.py", line 96, in inner
    return _get_result_or_execute_query(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/cachalot/monkey_patch.py", line 64, in _get_result_or_execute_query
    result = execute_query_func()
             ^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/cachalot/monkey_patch.py", line 80, in <lambda>
    execute_query_func = lambda: original(compiler, *args, **kwargs)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1572, in execute_sql
    cursor = self.connection.cursor()
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/django/utils/asyncio.py", line 24, in inner
    raise SynchronousOnlyOperation(message)
django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.

P.S. I'm 99.99% certain that cachalot is not at fault here at all, I can create a small repro-code example, if necessary.
P.P.S. I'm uncertain what type of issue this is. It's partially a feature request, partially a bug. I leave the decision on this matter to maintainers.

Change History (0)

Note: See TracTickets for help on using tickets.
Back to Top