Opened 10 years ago

Closed 10 years ago

#23805 closed Uncategorized (invalid)

query `first` method clears cached queryset for prefetch_related

Reported by: Aleck Landgraf Owned by: nobody
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords: first query
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When a cached queryset is iterated through with prefetch_related, if .first() is called during the iteration, a new DB query is fired off. This is potentially due to the LIMIT 1 [:1] or the order_by added to the queryset with the first method.

https://github.com/django/django/blob/master/django/db/models/query.py#L518

ex.

from django.db import connection
connection.queries = []

pizzas = Pizza.objects.all().prefetch_related('toppings')
print len(connection.queries)  # 2

for pizza in pizzas:
    first_topping = pizza.toppings.first()

print len(connection.queries)  # 2 + number of pizzas

This fixes in my code at least:

from django.db import connection
connection.queries = []

pizzas = Pizza.objects.all().prefetch_related('toppings')
print len(connection.queries)  # 2

for pizza in pizzas:
    try:
        first_topping = pizza.toppings.all()[0]
    except IndexError:
        first_topping = None

print len(connection.queries)  # 2

Change History (2)

comment:1 by Thomas C, 10 years ago

In the first code example, first() is used on an unordered queryset. As first() is intended to be deterministic, it isn't surprising that it tries to add an order condition (which default to order_by('pk').

Your second code example might work, but it isn't deterministic either as there is no guarantee on the ordering of pizza.toppings.all().

Do you need an arbitrary element or the first one ? If you need the first one, there should be some notion of ordering involved, otherwise your second solution is fine.

Also, you might wanna have a look at https://docs.djangoproject.com/en/1.7/ref/models/queries/#django.db.models.Prefetch (I've never used it myself but it should allow more complex prefetch_related queryset, including ORDER clause as described here https://docs.djangoproject.com/en/1.7/ref/models/querysets/#prefetch-related).

comment:2 by Tim Graham, 10 years ago

Resolution: invalid
Status: newclosed

tchaumeny has explained the issue and I don't see any bug here.

Note: See TracTickets for help on using tickets.
Back to Top