Opened 11 days ago

Closed 10 days ago

Last modified 10 days ago

#36299 closed Bug (fixed)

Calling alias after values_list adds the aliased value to the result set

Reported by: OutOfFocus4 Owned by: Simon Charette
Component: Database layer (models, ORM) Version: 5.2
Severity: Release blocker Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by OutOfFocus4)

ORM calls that worked prior to Django 5.2 now return different results or outright fail in Django 5.2. I have attached a file with proof-of-concept tests.

The first issue I noticed is that calling alias on after values_list adds the aliased value to the result set. I believe the root of this error is in django/db/models/sql/query.py lines 1224 and 1225:

        if self.selected:
            self.selected[alias] = alias

This code adds the alias to selected regardless of the value of the select parameter.

Another issue I found causes queryset evaluation to raise an AttributeError if the database backend supports SELECT ... FOR UPDATE. The following code:

with atomic():
    values = (
        User.objects.select_for_update(of=("self",))
        .values_list(
            Concat(F("first_name"), Value(" "), F("last_name")), "email"
        )
        .get(pk=12)
    )

will fail with a stacktrace ending in AttributeError: 'Concat' object has no attribute 'target' using the postgresql DB backend.

Change History (11)

by OutOfFocus4, 11 days ago

Attachment: tests.py added

comment:1 by OutOfFocus4, 11 days ago

Description: modified (diff)

comment:2 by Simon Charette, 11 days ago

Owner: set to Simon Charette
Severity: NormalRelease blocker
Status: newassigned
Triage Stage: UnreviewedAccepted

Regression in #28900 (65ad4ade74dc9208b9d686a451cd6045df0c9c3a)

Thanks for the detailed report.

The first issue I noticed is that calling alias on after values_list adds the aliased value to the result set. I believe the root of this error is in django/db/models/sql/query.py lines 1224 and 1225:
...
This code adds the alias to selected regardless of the value of the select parameter.

That's effectively the origin of the problem, thanks for investigating.

Currently looking into the second one with select_for_update.

comment:3 by Sarah Boyce, 11 days ago

Summary: ORM Regressions in Django 5.2Calling alias after values_list adds the aliased value to the result set

Thank you for the report

Simon you're too quick!

  • TabularUnified tests/queries/tests.py

    diff --git a/tests/queries/tests.py b/tests/queries/tests.py
    index c429a93af3..f9e831da15 100644
    a b from operator import attrgetter  
    77from django.core.exceptions import EmptyResultSet, FieldError, FullResultSet
    88from django.db import DEFAULT_DB_ALIAS, connection
    99from django.db.models import CharField, Count, Exists, F, Max, OuterRef, Q
    10 from django.db.models.expressions import RawSQL
     10from django.db.models.expressions import RawSQL, Value
    1111from django.db.models.functions import ExtractYear, Length, LTrim
    1212from django.db.models.sql.constants import LOUTER
    1313from django.db.models.sql.where import AND, OR, NothingNode, WhereNode
    class ValuesQuerysetTests(TestCase):  
    27372737        qs = qs.values_list("num")
    27382738        self.assertSequenceEqual(qs, [(72,)])
    27392739
     2740    def test_values_list_with_alias(self):
     2741        qs = Number.objects.values_list("num").alias(extra=Value(1))
     2742        self.assertSequenceEqual(qs, [(72,)])
     2743
    27402744    def test_flat_extra_values_list(self):
    27412745        # testing for ticket 14930 issues

Replicated the alias issue and confirmed it's a regression in 65ad4ade74dc9208b9d686a451cd6045df0c9c3a
If select_for_update is a separate issue, we should create a separate ticket

comment:4 by Sarah Boyce, 11 days ago

Replicated the second issue, also a regression in 65ad4ade74dc9208b9d686a451cd6045df0c9c3a

  • TabularUnified tests/select_for_update/tests.py

    diff --git a/tests/select_for_update/tests.py b/tests/select_for_update/tests.py
    index e8ba8f8b6e..18fca277cb 100644
    a b from django.db import (  
    1313    router,
    1414    transaction,
    1515)
     16from django.db.models import F, Value
     17from django.db.models.functions import Concat
    1618from django.test import (
    1719    TransactionTestCase,
    1820    override_settings,
    class SelectForUpdateTests(TransactionTestCase):  
    122124            list(Person.objects.select_for_update(no_key=True))
    123125        self.assertIs(self.has_for_update_sql(ctx.captured_queries, no_key=True), True)
    124126
     127    @skipUnlessDBFeature("has_select_for_update_of")
     128    def test_for_update_of_values_list(self):
     129        with transaction.atomic():
     130            values = (
     131                Person.objects.select_for_update(
     132                    of=("self",),
     133                )
     134                .values_list(Concat(Value("Dr. "), F("name")), "born")
     135            ).get(pk=self.person.pk)
     136        self.assertTupleEqual(values, ("Dr. Reinhardt", self.city1.pk))
     137
    125138    @skipUnlessDBFeature("has_select_for_update_of")

comment:5 by Simon Charette, 11 days ago

Thanks for the triage Sarah, I'll create a ticket for the second one.

Here's a PR for the alias issue Jeff. I gave you co-author attribute since you pointed at the exact solution and provided tests but I can remove it if you'd prefer just LMK on Github.

comment:6 by Simon Charette, 11 days ago

Has patch: set

comment:7 by Simon Charette, 11 days ago

Moved the select_for_update issue to #36301.

comment:8 by Mariusz Felisiak, 10 days ago

Triage Stage: AcceptedReady for checkin

comment:9 by Mariusz Felisiak <felisiak.mariusz@…>, 10 days ago

Resolution: fixed
Status: assignedclosed

In 12b771a1:

Fixed #36299 -- Prevented field selection on QuerySet.alias() after values().

Regression in 65ad4ade74dc9208b9d686a451cd6045df0c9c3a.

Refs #28900.

Thanks Jeff Iadarola for the report and tests.

Co-Authored-By: OutOfFocus4 <jeff.iadarola@…>

comment:10 by Mariusz Felisiak <felisiak.mariusz@…>, 10 days ago

In cd1aa54f:

[5.2.x] Fixed #36299 -- Prevented field selection on QuerySet.alias() after values().

Regression in 65ad4ade74dc9208b9d686a451cd6045df0c9c3a.

Refs #28900.

Thanks Jeff Iadarola for the report and tests.

Co-Authored-By: OutOfFocus4 <jeff.iadarola@…>

Backport of 12b771a1ec4bbfe82405176f5601e6441855a303 from main

Note: See TracTickets for help on using tickets.
Back to Top