Feature: Clear cached_property on related DB operations.

[Özer Sahin]

Hey,

I am wondering, why it wouldn't be smarter to let the decorator (somehow) clear related caches whenever a related model changes.

Use Case:

I have a big model with many related models and I need to filter them for a ListView Page. To display those values I would use cached_property for data from other models like images. Since the related instances have not changed, we could use the cached_property. Now if we upload a new image, a signal could clear the cached_property used in the other model. The same goes for update, delete or other relevant operations. That way the system would always be up to date, but max out the potential of caching where ever it makes sense.

Is that clear? I am not an expert like you, but since I am currently learning about all this, I was curious why this hasn't been implemented (yet).

[Tim Graham]

Which cached properties are you referring to? Can you give an example shell session or something so the idea is a bit more concrete?

[Özer Sahin]

Here is a quick example to try to illustrate:

class ModelA(models.Model):

    @cached_property
    def related_count(self):
        return self.related.count()

    @cached_property
    def related_content(self):
        return ', '.join([rel.title for rel in self.related.all()])


class Related(models.Model):

    model = models.ForeignKey(ModelA, related_name='related')
    title = models.CharField()

    def has_changed(self):
        # some method to determine if object changed
        return True

    def save(self):
        if not self.id or self.has_chaged():  # newly created or changed
            del self.model.related_count

    def delete(self):
        del self.model.related_count
   


>>> model = ModelA()
>>> model.related_count
0  # calls first time
>>> model.related_count
0  # does not call

>>> rel = Related()
>>> rel.model = model
>>> rel.title = "Text"
>>> rel.save()

>>> model.related_count
1  # calls first time, because rel added
>>> model.related_count
1  # does not call

>>> model.related_content
Text  # calls first time
>>> model.related_content
Text  # does not call

>>> rel.title = "New Text"
>>> rel.save()
>>> model.related_content
New Text  # calls first time, because rel changed
>>> model.related_content
New Text  # does not call
>>> model.related_count
1  # calls again, because rel changed

>>> rel.delete()
>>> model.related_count
0  # calls, because rel removed
>>> model.related_content
  # calls, return nothing

So what I am actually looking for, are some kind of signals to attach to the decorator so we can input logic, when the cached_property should be removed so it calls it again for an updated value. We could define that it should delete the cached value once related objects were updated, deleted or created. That way we can max out the cache, but keep it always up to date. I think it could be a huge performance booster without the issue of manually deleting cached_properties so they update.

I hope I am not too far off, awaiting your feedback!

[Tim Graham]

If I understand correctly, you want rel.save() to modify (clear the cached properties on) another, unrelated variable model. I'm not aware of any facility in Python to do that.

[Özer Sahin]

I updated the example with save and delete methods and what should happen. Maybe that's more clear. The model is related (ForeignKey in class Related).

[Tim Graham]

Is your idea that Django should automate the behavior in save() and delete() so you don't have to write that logic yourself? I still don't understand what change you want to make in Django. Is it impossible to implement your proposal as a third-party app?

[Özer Sahin]

Yes, so Django should automatically take care of those operations in the background. Of course I could write a mixin or something similar, but that would bring overhead to the project.

Pydanny has a module, that is a configurable property: ​https://github.com/pydanny/cached-property

But that does not cover that case, I already asked and he referred me to the Django project.

I am not sure, if I could write an app that does all that. I thought it would be convenient if the cached_property could be called with kwargs like @cached_property(delete_on_save=True, delete_when_models_change=[ModelA, ModelB]).

Thank you for your time btw, really appreciate it!

[Tim Graham]

Adding model related parameters to cached_property would require adding some sort of specialized cached_property since currently it's a general utility function usable on any Python class. All in all, it seems the ideas in this ticket aren't thought through thoroughly enough to say whether or not adding something to Django is required so I'm going to close the ticket for now. A ​third-party app for this use case might already exist. If you write some code to solve your problem and think it should be added to Django, or if you find that some change in Django is required to accomplish your use case, feel free to write to the DevelopersMailingList to get feedback about where to go from here.