Add a way to control related fields reverse cache

[dirleyrls]

I have a very specific use-case where we use a database view to represent some data. I've mapeed it using a unmanaged model like this:

# concrete table
class Something(models.Model):
  # ...


class SomethingHelperView(models.Model):
  something = models.OneToOneField(Something, related_name='helper')
  counter = models.PositiveIntegerField()
  
  class Meta:
    managed = False
    db_table = 'something_helper_view'

When I do:

>>> something = Something.objects.get(pk=1)
>>> something.helper.counter
10
>>> # some operation that will update the counter
>>> # ...
>>> something.helper.counter
10

This was not the behavior I wanted to have. Since I'm using a database view, I expect it to always be "fresh". I want that, every time I access my helper, it hits the database, bringing me a fresh value. That's the point of this view.

Poking around through Django's code, I realized that related fields have their back references cached. It's ok, I understand the purpose of that cache. But I got frustrated when I discovered I can't just turn it off. It would be good to be able to turn it off.

And for the record, currently I'm working around this issue doing like this:

Something.fresh_helper = property(lambda self: SomethingHelperView.get(something=self))

[Aymeric Augustin]

Caching is the expected behavior in the general case. See previous tickets on this topic.

What do you mean by "be able to turn it off"? What if a request temporarily "turns in off", crashes, and it's never turned back on?

[dirleyrls]

Replying to aaugustin:

Caching is the expected behavior in the general case. See previous tickets on this topic.

What do you mean by "be able to turn it off"? What if a request temporarily "turns in off", crashes, and it's never turned back on?

Sorry, I didnt make myself clear about the "turn it off". I expect to be able to do something like this:

class SomethingHelperView(models.Model)
  something = models.OneToOneField(Something, related_name='helper', backref_cache=False)
  # ...

Now, whenever I access the Somethig.helper attribute, I'll get a fresh Helper instance.

[Luke Plant]

In Python in general, properties that might be expensive to calculate are often cached, so I don't think this is surprising behaviour.

Personally, I think this is a corner case that can be easily addressed in the manner you specified, or simply by doing SomethingHelperView.get(something=the_thing). I suspect the backref_cache idea would be hard enough to implement correctly that it is not worth it.

I also don't think it is a nice API to define it on the model. It's entirely possible that some types of access of a certain model would benefit from the caching behaviour, and for other types of access on the same model you wouldn't want it. An API like select_related() allows you to tune a similar performance related behaviour and turn it on and off at will, no matter what the defaults on the model are, but this API would not allow it, since we are talking about properties on instances and not methods on a queryset.

[Aymeric Augustin]

In addition to the points made by Luke, I would add that this API creates "action at a distance". A developer would have to look at the model definition to understand the performance characteristics of the code he's working on (SQL queries are a huge factor in the performance of a Django application).

I find it safer to always cache (I fixed several bugs in this area of Django), thereby minimizing SQL queries, and reload objects explicitly (or write your own APIs).

For these reasons, I'm going to reject the API proposed in comment 2.

PS: this isn't limited to backwards relations, the exact same arguments also apply to forward relations.