QuerySet.update() raises FieldError on queryset ordered by an annotated field.

[Colton Hicks]

A FieldError results if I try to perform a custom a custom action on data in the admin interface IF that data is sorted by an annotated field. The action is obviously not acting on an annotated field as those fields do not exist in the database, but raises a FieldError saying it 'Cannot resolve keyword 'field_that_data_is_currently_sorted_by' into field.' and then lists the choices for fields (all of which are the raw model fields, not the annotated fields in the custom queryset).

My admin model:

@admin.register(Position)
class PositionAdmin(admin.ModelAdmin):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

    def get_queryset(self, request):

        qs = super(PositionAdmin, self).get_queryset(request).prefetch_related(
            'orders').select_related(
                'portfolio', 'current_position').select_related(
                    'current_position').annotate(
                Count('orders', distinct=True),
                Count('short_requests'),
                total_position_value=ExpressionWrapper(
                    F('price') * F('quantity'), output_field=IntegerField()),
                diff_shares=Coalesce(
                    F('quantity')
                    - F('current_position__quantity'),
                    F('quantity')),
                diff_value=ExpressionWrapper(
                    F('diff_shares') * F('price'),
                    output_field=IntegerField()),
                orders_with_errors=Count(
                    Case(When(~Q(orders__error=''), then=1))),
                non_accepted_orders=Count(
                    Case(When(Q(orders__status=''), then=1))),
                order_today=Count(
                    Case(When(
                        orders__created_at__gte=_brokerage_today_start(),
                        then=1))))
        return qs

    # Custom Action
    def approve_position_for_trading(self, request, queryset):
        try:
            num_pos_approved = queryset.update(approved=True)
        except FieldError:
            self.message_user(
                request, "You cannot perform actions when you have sorted by "
                "this column. Please remove your column sortings and then try "
                "your action again.", level=messages.ERROR)
        else:
            if num_pos_approved == 1:
                message_bit = "1 position was was"
            else:
                message_bit = "%s positions were" % num_pos_approved
            self.message_user(
                request, "%s successfully approved for trading." % message_bit)

I had to write code to handle the error so that the user will know how to proceed. However, this seems like bad behavior. Django can successfully perform that action (changing the 'Approved' field to 'True') on the data if it is sorted by any core model field, or if sorted by no field at all. However, if the data is sorted by a field resulting from the annotation of my queryset, then I get the FieldError when I try to perform the action. This seems like a bug.

[Sergey Fedoseev]

Could you provide traceback and try minimize your example?

[Tim Graham]

Yes, a more minimal (but complete, with models) example seems needed.

[Márton Salomváry]

Hey folks, I've run into this bug both on django 3.x and 4.x. Here is a small showcase project: ​https://github.com/salomvary/django_admin_action_bug

Steps to reproduce:

• Run the project
• Sign in to admin
• Create a few "Things"
• Go to Home › My_App › Things and select a Thing using the checkboxes
• Go to Actions > "Update Something"
• Click "Go"

You will see the following error page:

FieldError at /admin/my_app/thing/

Cannot resolve keyword 'other_things_count' into field. Choices are: id, is_something, other_things

Few things to note:

• Removing queryset.order_by eliminates the problem
• Removing annotations that involve joins also eliminates the problem

Stack trace:

Internal Server Error: /admin/my_app/thing/
Traceback (most recent call last):
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/contrib/admin/options.py", line 622, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/views/decorators/cache.py", line 56, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/contrib/admin/sites.py", line 236, in inner
    return view(request, *args, **kwargs)
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
    return bound_method(*args, **kwargs)
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1736, in changelist_view
    response = self.response_action(request, queryset=cl.get_queryset(request))
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1417, in response_action
    response = func(self, request, queryset)
  File "/Users/mrc/Projects/django_admin_action_bug/my_app/admin.py", line 9, in update_something
    queryset.update(is_something=True)
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/db/models/query.py", line 790, in update
    rows = query.get_compiler(self.db).execute_sql(CURSOR)
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1591, in execute_sql
    cursor = super().execute_sql(result_type)
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1189, in execute_sql
    sql, params = self.as_sql()
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1526, in as_sql
    self.pre_sql_setup()
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1626, in pre_sql_setup
    super().pre_sql_setup()
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 60, in pre_sql_setup
    order_by = self.get_order_by()
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 389, in get_order_by
    for expr, is_ref in self._order_by_pairs():
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 373, in _order_by_pairs
    yield from self.find_ordering_name(
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 767, in find_ordering_name
    field, targets, alias, joins, path, opts, transform_function = self._setup_joins(pieces, opts, alias)
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 807, in _setup_joins
    field, targets, opts, joins, path, transform_function = self.query.setup_joins(pieces, opts, alias)
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/db/models/sql/query.py", line 1604, in setup_joins
    path, final_field, targets, rest = self.names_to_path(
  File "/Users/mrc/Projects/django_admin_action_bug/.venv/lib/python3.9/site-packages/django/db/models/sql/query.py", line 1522, in names_to_path
    raise FieldError("Cannot resolve keyword '%s' into field. "
django.core.exceptions.FieldError: Cannot resolve keyword 'other_things_count' into field. Choices are: id, is_something, other_things
[09/Dec/2021 17:00:45] "POST /admin/my_app/thing/ HTTP/1.1" 500 154267

[Mariusz Felisiak]

Thanks for details! QuerySet.update() removes all annotations (see a84344bc539c66589c8d4fe30c6ceaecf8ba1af3) that's why order_by() raises a FieldError. We should probably clear ordering in QuerySet.update() or at least remove all annotated fields from it, e.g.

django/db/models/query.py

@@ -790 +790 @@
         query = self.query.chain(sql.UpdateQuery)
         query.add_update_values(kwargs)
         # Clear any annotations so that they won't be present in subqueries.
+        query.clear_ordering()
         query.annotations = {}
         with transaction.mark_for_rollback_on_error(using=self.db):
             rows = query.get_compiler(self.db).execute_sql(CURSOR)

or

django/db/models/query.py

@@ -790 +790 @@
         query = self.query.chain(sql.UpdateQuery)
         query.add_update_values(kwargs)
         # Clear any annotations so that they won't be present in subqueries.
+        query.order_by = tuple(
+            col for col in query.order_by if col not in query.annotations
+        )
         query.annotations = {}
         with transaction.mark_for_rollback_on_error(using=self.db):
             rows = query.get_compiler(self.db).execute_sql(CURSOR)

As a workaround you can clear ordering in your actions, e.g.

def update_something(admin, request, queryset):
    queryset.order_by().update(is_something=True)

[Márton Salomváry]

Thanks for the workaround, I can confirm it makes the exception go away.

[Saeed Blanchette]

Hi, guys I'd love to work in this, any suggestions ! I need help to understand this more and what can you do next

[Mariusz Felisiak]

Saeed, Have you seen my comment?

[David Wobrock]

The suggested code seems to fix it indeed :)

I could only reproduce when annotating/ordering on a ManyToManyField.
I used the snippet that keeps some order_by and doesn't clear it entirely ¯\_(ツ)_/¯
​PR

[Mariusz Felisiak <felisiak.mariusz@…>]

In f4680a11:

Refs #28897 -- Added test for QuerySet.update() on querysets ordered by inline m2m annotation.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 3ef37a5:

Fixed #28897 -- Fixed QuerySet.update() on querysets ordered by annotations.

[Mariusz Felisiak <felisiak.mariusz@…>]

In ccb24384:

[4.1.x] Refs #28897 -- Added test for QuerySet.update() on querysets ordered by inline m2m annotation.

Backport of f4680a112d01d85540411673eade31f37712d0a6 from main

[Mariusz Felisiak <felisiak.mariusz@…>]

In d44dc31f:

[4.1.x] Fixed #28897 -- Fixed QuerySet.update() on querysets ordered by annotations.

Backport of 3ef37a5245015f69a9b9f884ebc289a35d02c5f6 from main