lazy prefetch_related

[g00fy]

I suggest, that prefetch_related should be lazy. What I mean is, it would only execute aditional query, when related model needs to be accessed.
Example:

# this would do just one query - prefetch_related is lazy
pizzas = Pizza.objects.all().prefetch_related('toppigns')
for pizza in pizzas:
    print pizza.name

# Proposal A

# now we want to access the related objects, 
# 1 aditional query (intermediate m2m table) is executed
# not 2 since, we only need the id of the related object (which is already in the m2m table)
# the 2nd query would result in a set of 'lazy' objects
for pizza in pizzas:
    for topping in pizza.toppings.all():
        print topping.id

# now 3rd query is made
for pizza in pizzas:
    for topping in pizza.toppings.all():
        print topping.name

# Proposal B


# 2nd query is made (with the current implementation - m2m table join)
for pizza in pizzas:
    for topping in pizza.toppings.all():
        print topping.id

Usecase:

let's consider logic like this:

pizzas = Pizza.objects.all()

print len(pizzas)
if pizzas[0].name == 'pepperoni':
    for pizza in pizzas:
        print [topping.name for topping in pizza.toppings.all()]
else:
    for pizza in pizzas:
        print pizza.name

in otherwords, this would be useful, if there is some complex logic that sometimes may require prefetch_related (and this requirement can be only checked after the query was made - so there is no way to do the checking & then doing prefetch_related).

[Anssi Kääriäinen]

I am going to close this.

There are two reasons for this:

1. I don't see any easy way to achieve this. Implementing this is likely possible, but it is going to cost us in code complexity.
2. Lazy prefetch_related might lead to situations where prefetch_related is littered just in case in get_query_set() of base manager. This has the side effect that if you access toppings.all() for one object, then you are going to do a prefetch for the whole queryset. In short, it will be harder to understand what queries are made and when which isn't a nice property to have in the ORM.

If there is an easy way to make this achievable in 3rd party app, then lets add that into core. But that is separate ticket's issue, and that ticket should be opened only if it is accompanied with an idea how to achieve the wanted result.