Subsequent calls to prefetch_related_objects() don't work for model instances that have changed.

[Dennis Kliban]

Our project processes instances in a stream. In some cases the instances are repeated. In these cases, we discovered that prefetch_related_objects() does not work correctly if the list of related objects has changed since the first time prefetch_related_objects() on the instance. I've written a simple reproducer here[0].

In the reproducer, if I call delattr(book, "the_authors") after adding the author, everything works.

[0] ​https://github.com/django/django/commit/73a138665ba20915f51680816120727a5e2ba771

[Mariusz Felisiak]

Thanks for the report and a regression test. It's similar to the #32089, but it's not the same issue.

[Simon Charette]

I wonder if this is actually a bug or we should wontfix this one.

The definition of is_fetched for a prefetched relationship via to_attr is the presence of the attribute itself. The ​many-to-many manager already takes care of clearing the prefetched objects that Django knows about when add is called (_prefetched_objects_cache is under Django's control) but unless we introduce a way to do some bookeeping of the form _prefetched_to_attrs: dict[prefetch_cache_name: str, to_attrs: list[str]] when to_attr is used I don't see how we can fix this issue.

Then comes the question of what to do if the property doesn't support del and documenting that this change now breaks patterns of the form

bookings = Booking.objects.prefetch_related(
    'seats',
    Prefetch('seats', to_attr='existing_seats'),
)
sync_bookings(bookings)  # function that alters .seats one way or the other
added_seats = []
removed_seats = []
for booking in bookings:
    added_seats.extend(set(booking.seats.all()) - set(booking.existing_seats))
    removed_seats.extend(set(booking.existing_seats) - set(booking.seats.all())

Since it's not obvious if to_attr should be cleared or not on a prefetched relationship change and we've defaulted to not doing so since the introduction of the feature I'd be inclined to wontfix this one and declare that it's the user responsibility to do the book keeping and clear to_attr fetched relationships prior to attempting a second round of prefetch_related_objects:

tests/prefetch_related/test_prefetch_related_objects.py

@@ -151 +151 @@
             )
             self.assertCountEqual(book.the_authors, [])
         book.authors.add(author)
+        del book.the_authors
         with self.assertNumQueries(1):
             prefetch_related_objects(
                 [book],

[Mariusz Felisiak]

Agreed with Simon, I was torn too.