Changes between Initial Version and Version 5 of Ticket #34791


Ignore:
Timestamp:
Aug 23, 2023, 2:54:04 PM (13 months ago)
Author:
Maxime Toussaint
Comment:

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #34791 – Description

    initial v5  
    1 There seems to be an issue when using a Prefetch object to fetch something that has already been fetched. Here is an example: so let's say I have a Pizza and some Toppings. Now I want to get all the toppings, but for some reason I also want to separately fetch only the toppings that are out of stock. I could do something like:
     1Note: Edited the description following the discussion
     2
     3There seems to be an issue when using a Prefetch object to fetch something that has already been fetched. The issue only seems to happen when there is depth in the prefetch. Here is an example I made this morning that fails:
     4{{{
     5        pizzas = Pizza.objects.all().prefetch_related(
     6            "toppings__origin",
     7            Prefetch(
     8                "toppings__origin",
     9                queryset=Country.objects.filter(label="China"),
     10                to_attr="china",
     11            ),
     12        )
     13
     14        china = pizzas[0].toppings.all()[0].china
     15}}}
     16Here, when trying to get china, I would assume it to either be a Country object or None. However, I get the message: AttributeError: 'Topping' object has no attribute 'china'
     17
     18Here are the models I set up for my test:
     19{{{
     20class Country(models.Model):
     21    label = models.CharField(max_length=50)
    222
    323
     24class Pizza(models.Model):
     25    label = models.CharField(max_length=50)
     26
     27
     28class Topping(models.Model):
     29    pizza = models.ForeignKey(Pizza, on_delete=models.CASCADE, related_name="toppings")
     30    label = models.CharField(max_length=50)
     31    origin = models.ForeignKey(
     32        Country, on_delete=models.CASCADE, related_name="toppings"
     33    )
     34}}}
     35And here are the queries made when calling the queryset:
    436{{{
    5 queryset = Pizza.objects.all().prefetch_related(
    6     'toppings',
    7     Prefetch('toppings', queryset=Topping.objects.filter(is_in_stock=False), to_attr='out_of_stock_toppings'),
    8 )
     371. SELECT "tests_pizza"."id", "tests_pizza"."label", "tests_pizza"."provenance_id" FROM "tests_pizza"
     382. SELECT "tests_topping"."id", "tests_topping"."pizza_id", "tests_topping"."label", "tests_topping"."origin_id" FROM "tests_topping" WHERE "tests_topping"."pizza_id" IN (1, 2)
     393. SELECT "tests_country"."id", "tests_country"."label", "tests_country"."continent_id" FROM "tests_country" WHERE "tests_country"."id" IN (2, 3, 4)
    940}}}
     41Note that the filter by label='china' has completely disappeared.
    1042
    11 This looks good, but if we run it, it will fail, saying out_of_stock_toppings is not an attribute of Pizza. However, if I were to do it like this instead:
     43Now, if I switch the prefetches around like so:
    1244{{{
    13 queryset = Pizza.objects.all().prefetch_related(
    14     Prefetch('toppings', queryset=Topping.objects.filter(is_in_stock=False), to_attr='out_of_stock_toppings'),
    15     'toppings',
    16 )
     45        pizzas = Pizza.objects.all().prefetch_related(
     46            Prefetch(
     47                "toppings__origin",
     48                queryset=Country.objects.filter(label="China"),
     49                to_attr="china",
     50            ),
     51            "toppings__origin",
     52        )
    1753}}}
     54Fetching china now works, and here are the queries being made:
     55{{{
     561. SELECT "tests_pizza"."id", "tests_pizza"."label", "tests_pizza"."provenance_id" FROM "tests_pizza"
     572. SELECT "tests_topping"."id", "tests_topping"."pizza_id", "tests_topping"."label", "tests_topping"."origin_id" FROM "tests_topping" WHERE "tests_topping"."pizza_id" IN (1, 2)
     583. SELECT "tests_country"."id", "tests_country"."label", "tests_country"."continent_id" FROM "tests_country" WHERE ("tests_country"."label" = 'China' AND "tests_country"."id" IN (2, 3, 4))
     594. SELECT "tests_country"."id", "tests_country"."label", "tests_country"."continent_id" FROM "tests_country" WHERE "tests_country"."id" IN (2, 3, 4)
     60}}}
     61This time, both calls to Country were made.
    1862
    19 then all works fine. Looking at the code, this seems to be because the name used by a field to validate the cache is not the name used to store the data on the model, but rather simply the name of the field, so it collides. I have not tested it, but I think in the second example, the data returned will actually be the filtered data, not the full expected queryset. Note that this is my first time reading that part of the code, so there could be things I missed.
    20 
    21 Now this is a bit of a nonsensical example, but when using rest_framework with serializers, this type of situation could come up, where one serializer needs it formatted a certain way and this issue could arise (it has for me).
    22 
    23 I am not sure what the best way to fix this would be, but I feel like setting a to_attr should make the cache take that new field name into account instead of the field name.
     63I did follow the code a bit yesterday, and I believe it comes from the fact that the ForeignKey field defines the cache name as being simply the name of the field. I am not certain though, and it would likely require someone with more knowledge than me to look into it.
Back to Top