Add after_db_init() hook method to model

[Robert Singer]

I've encountered a need in my personal projects, and when writing ​django-lifecycle, to hook into the moment when a model has been fully initialized from the database.

Overriding the model's init method does NOT work here because the select_related relationships have not yet been added in/cached in the model's FK fields. It would be useful to do things right after the model is fully loaded and initialized from the database. For example, if you have a "type" foreign key field, you may want to apply some polymorphic behavior based on the value of that field. Doing this in __init__ will cause a n+1 query explosion if you load multiple models and iterate over the QuerySet.

Current Problem

This will cause an n+1 issue when iterating a QuerySet:

class CatMixin(object):
    def greet(self):
         return "Meow"

class DogMixin(object):
    def greet(self):
         return "Woof"

class PolymorphicModel(models.Model):
   type = models.ForeignKey(PetType)

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
       # another db query is triggered b/c select_related has populated the instance yet
       if self.type.name == 'cat':
           self.__class__ = CatMixin
       else:
           self.__class__ = DogMixin

The Fix

Add a call to obj.post_db_init() to this line:
​https://github.com/django/django/blob/master/django/db/models/query.py#L91

Then this code will eliminate the n+1 problem (assuming select_related('type') is used):

class PolymorphicModel(models.Model):
   type = models.ForeignKey(PetType)

    def post_db_init(self,):
       if type.name == 'cat':
           self.__class__ = CatMixin
       else:
           self.__class__ = DogMixin

I realize there are other ways to achieve the behavior in this example -- I'm bringing up polymorphism as a general use case. In django-lifecycle, this hook would allow me to track a model instance's initial state across a foreign key relationship without causing users to experience the n+1 problem.

[Mariusz Felisiak]

select_related() is not related with Model's initialization it's a QuerySet method, so I cannot imagine how you would like to connect them. As you pointed out there are many ways to solve your issues in the current Django (e.g. you can ​add select_related() to the initial QuerySet and use @cached_property, there is also ​post_init signal). You can use one of support channels to describe your real use case and get help.

[Robert Singer]

Replying to felixxm:

select_related() is not related with Model's initialization it's a QuerySet method, so I cannot imagine how you would like to connect them. As you pointed out there are many ways to solve your issues in the current Django (e.g. you can ​add select_related() to the initial QuerySet and use @cached_property, there is also ​post_init signal). You can use one of support channels to describe your real use case and get help.

I appreciate the quick response to the ticket, but I feel I may have failed to convey what the issue is. The problem is that a model's initialization lifecycle currently lacks a way to hook into the moment right after the ORM populates FK relations that are specified in select_related. The point where that happens is right here: ​https://github.com/django/django/blob/master/django/db/models/query.py#L91

The nearest point to this moment is __init__, but it occurs before FK relations are populated, so examining FK related field values at that point will cause an n+1 query problem. There is no workaround/support for hooking into this moment currently: modifying an initial queryset or using the @cached_property won't work. I am glad to submit a pull request to add this feature. But it's probably something only advanced users/library authors need, so it may be out of scope. My use case is enhancing my library, ​django-lifecycle, to more efficiently support state transitions based on FK values.

[Mariusz Felisiak]

Adding a hook after each iteration doesn't sound like a good idea to me. You need to take into account that we have different iterable classes e.g. ValuesIterable, ModelIterable, NamedValuesListIterable, etc. and that Queryset is lazy, so you will not be able to add any useful hook without forcing users to iterate through all rows. This looks quite niche and I still cannot imagine any real use case based on a post iteration hook. You can start a discussion on DevelopersMailingList if you don't agree.