Opened 2 years ago

Closed 2 years ago

#34012 closed Cleanup/optimization (fixed)

QuerySet.order_by() silently skips non-existing fields on related fields with Meta.ordering.

Reported by: Klaas van Schelven Owned by: David Sanders
Component: Database layer (models, ORM) Version: 4.1
Severity: Normal Keywords:
Cc: David Sanders Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Compare the following desirable behavior:

>>> SomeModel.objects.all().order_by("non_existing_field")

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "......lib/python3.10/site-packages/django/db/models/query.py", line 1149, in order_by
    obj.query.add_ordering(*field_names)
  File "......lib/python3.10/site-packages/django/db/models/sql/query.py", line 2016, in add_ordering
    self.names_to_path(item.split(LOOKUP_SEP), self.model._meta)
  File "......lib/python3.10/site-packages/django/db/models/sql/query.py", line 1562, in names_to_path
    raise FieldError("Cannot resolve keyword '%s' into field. "

django.core.exceptions.FieldError: Cannot resolve keyword 'non_existing_field' into field. Choices are: [redacted]

with the following undesirable behavior:

>>> SomeModel.objects.all().order_by("some_foreign_key__non_existing_field")
<QuerySet .... [ i.e. shows some results ]

Change History (11)

comment:1 by Mariusz Felisiak, 2 years ago

Component: UncategorizedDatabase layer (models, ORM)
Summary: order_by silently skips non-existing fields of existing related fieldsQuerySet.order_by() silently skips non-existing fields on related fields.
Triage Stage: UnreviewedAccepted
Type: UncategorizedCleanup/optimization

Tenatively accepted, I'm not sure it's worth the additional complexity (similar to the #29408, see 440505cb2cadbe1a5b9fba246bcde6c04f51d07e).

comment:2 by David Sanders, 2 years ago

Cc: David Sanders added

fwiw I can't reproduce the issue on 4.1.1 🤔 … 

Here's what I tried (full tracebacks omitted for brevity):

class Parent(Model):
    ...

class Child(Model):
    parent = ForeignKey(Parent, on_delete=CASCADE)


Child.objects.all().order_by('asdf')
FieldError: Cannot resolve keyword 'asdf' into field. Choices are: id, parent, parent_id

Child.objects.all().order_by('parent__id')
<QuerySet []>

Child.objects.all().order_by('parent__asdf')
FieldError: Unsupported lookup 'asdf' for BigAutoField or join on the field not permitted, perhaps you meant df?

During handling of the above exception, another exception occurred:
FieldError: Cannot resolve keyword 'asdf' into field. Choices are: child, id

comment:3 by Klaas van Schelven, 2 years ago

Good catch, this only happens when an explicit ordering is defined on the Parent's Meta:

from django.db import models


class Parent(models.Model):
    class Meta:
        ordering = ["id"]


class Child(models.Model):
    parent = models.ForeignKey(Parent, on_delete=models.CASCADE)

Reproducable up to Django 4.1.1

comment:4 by David Sanders, 2 years ago

felixx this looks like an issue with overriding the foreign model's ordering.

If I repeat the test with Klaas' models and print the query it reveals it still attempts to use parent.id:

class Parent(Model):
    class Meta:
        ordering = ["id"]

class Child(Model):
    parent = ForeignKey(Parent, on_delete=CASCADE)

print(Child.objects.all().order_by('parent__asdf').query)
SELECT "sample_child"."id", "sample_child"."parent_id" FROM "sample_child" INNER JOIN "sample_parent" ON ("sample_child"."parent_id" = "sample_parent"."id") ORDER BY "sample_parent"."id" ASC

Edit: (I guess the question is whether Django should raise an error or continue onto ordering)

Last edited 2 years ago by David Sanders (previous) (diff)

comment:5 by David Sanders, 2 years ago

Has patch: set
Owner: changed from nobody to David Sanders
Status: newassigned

comment:6 by David Sanders, 2 years ago

Version: 4.04.1

comment:7 by Mariusz Felisiak, 2 years ago

Summary: QuerySet.order_by() silently skips non-existing fields on related fields.QuerySet.order_by() silently skips non-existing fields on related fields with Meta.ordering.

comment:8 by Mariusz Felisiak, 2 years ago

Patch needs improvement: set

comment:9 by GitHub <noreply@…>, 2 years ago

In 37a13cc9:

Refs #34012 -- Added test for ordering by transform of related fields.

comment:10 by Mariusz Felisiak, 2 years ago

Patch needs improvement: unset
Triage Stage: AcceptedReady for checkin

comment:11 by Mariusz Felisiak <felisiak.mariusz@…>, 2 years ago

Resolution: fixed
Status: assignedclosed

In 4771a169:

Fixed #34012 -- Made QuerySet.order_by() apply transforms on related fields for models with Meta.ordering.

This makes QuerySet.order_by() no longer ignore trailing transforms for
models with Meta.ordering. As a consequence, FieldError is raised in
such cases for non-existent fields.

Thanks to Klaas van Schelven for the report and Mariusz Felisiak for the
review and advice.

Note: See TracTickets for help on using tickets.
Back to Top