Model middleware

Warning: Requires magic-removal!

Warning: Experimental!

This came up on IRC: someone wanted a way to supply versioning of the db contents from a single point in the system (such as an app). The app would then somehow recognize those fields that require version information to be saved and record their changes in the db.

This then requires several problems to be resolved:

1. Gaining access to save() and delete() methods of any model, installed anywhere in a project.
2. Leaving save() and delete() methods of models intact, as overriding them would mean breaking the "pluggable" idea behind this.
3. Allowing for many pre/post callbacks to be inserted into the same model.
4. Keeping the usage simple.

The proposed solution solves the problem in a hackish, but convenient way. To dispel some worries: it doesn't involve any changes to Django's source. After playing with the ideas of method decorators and model inheritance, I've rejected both: decorators are too rigid and spammy, since they require you to explicitely override save() and delete() even when it's not needed; model inheritance would interfere with manually overriding the methods. So the final choice was made in favour of using a metaclass to insert the callbacks.

All Django models already have a metaclass - django.db.models.ModelBase. A new metaclass was defined, inheritting from ModelBase - MetaModelMiddleware. This new metaclass lets your model inherit from custom classes, that can define any or all of pre_save(), post_save(), pre_delete(), post_delete() methods. At the construction of your model class these methods are automatically added to relevant points, so that the actual save() and delete() methods remain untouched. Through multiple inheritance, your model can acquire as many of these callbacks as needed - they all will be called in order. The callback methods receive a single argument - the instance of your model for which save() or delete() is being executed.

In practice it works like this:

# import the model middleware module from wherever it is on your path
from myproject.utils import model_utils

# define a descendant of model_utils.ModelMiddleware with pre/post_* methods

class TestMiddleware(model_utils.ModelMiddleware):
    def pre_save(self):
        print self, "is about to be saved."

    def post_save(self):
        print self, " has been saved."

    def pre_delete(self):
        print self, "is about to be deleted."

    def post_delete(self):
        print self, "has been deleted."

# your model then inherits from the above class
# note that inheritting from models.Model is unneeded

class MyModel(TestMiddleware):
  ...

Here's a more practical example, that I used for ReST parsing

class ReSTMiddleware(ModelMiddleware):
    def pre_save(self):
        try:
            cont = self.content.decode('utf_8')
            parts = build_document(cont, initial_header_level=2)
            self.html_body = parts['html_body'].encode('utf_8')
            self.html_toc = parts['toc'].encode('utf_8')
        except:
            pass
        d = datetime.now()
        pdate = datetime(d.year, d.month, d.day, d.hour, d.minute)
        if self.pub_date is None:
            self.pub_date = pdate
        self.last_modified = pdate

This is still fairly contrived, since you can see it somehow knowing which field must be ReST'ified, as well as setting publication and modification times (and how does it know how to do that?). To solve that problem the metaclass can also save your model's "middleware settings", much like those for Admin are saved. To apply the settings to your model you'll then do the same thing as for Admin, but with a different name:

class MyModel(ModelMiddleware):
    ...
    # fields, custom methods, Admin, etc.

    class Middle:
        ReST = ({"field" : "content", "save_body" : "html_body", "save_toc" : "html_toc", "init_header" : 2},)

My ReST parser takes a string and returns two parts: the ReST version of the same string and the table of contents, generated from that string. Therefore, it needs to know which field to get the raw string from, and where to save the body and toc parts.

The middleware options are accessed through the class object with MyModelKlass._middle. Since the actual middleware class only has access to the instance object, it would need to do self.__class__._middle to get the options. Here's how it looks with that change:

class ReSTMiddleware(ModelMiddleware):
    def pre_save(self):
        try:
            opts = self.__class__._middle["ReST"] # individual options are saved in a dict
        except AttributeError:
            return  # just fail silently, though it might not be a very good idea in practice

        # lets be nice to ourselves and provide a default value for the initial header level
        for opt in opts:
            opt.setdefault("init_header", 1)
        
        # parse for as many fields as we have options for
        for opt in opts:  
            try:
                cont = getattr(self, opt["field"]).decode("utf_8")
                parts = build_document(cont, initial_header_level=opt["init_header"])
                setattr(self, opt["save_body"], parts["html_body"].encode('utf_8'))
                setattr(self, opt["save_toc"], parts["toc"].encode('utf_8'))
            except:
                pass # another silent fail, needs fixing

        d = datetime.now()
        pdate = datetime(d.year, d.month, d.day, d.hour, d.minute)
        if self.pub_date is None:
            self.pub_date = pdate
        self.last_modified = pdate

Now ReST parsing can operate on any model, given that this model is correctly set up for such an operation, e.g. has the needed fields for html parts of every ReST-enabled field and the relevant Middle options.

There's still that "date problem" left though - this mucking has no business being in a ReSTMiddleware class. But there's a positive side to every case of coding negligence. In this particular case, I have an opportunity to show off an important feature of this approach: separation of generic and custom aspects of record changes.

We don't really want to define a model middleware class to handle this date juggling, since this is just a result of lazyness and there's probably a built in way to do the same thing in Django, which I just didn't bother to look up. But nor do I want to go looking for it right now, as I have a highly impatient dog to walk (or a rug to scrub, depending on how soon I finish writting this), so we'll move it inside the model's save() method:

class MyModel(ReSTMiddleware):
    ...

    def save(self):
        d = datetime.now()
        pdate = datetime(d.year, d.month, d.day, d.hour, d.minute)
        if self.pub_date is None:
            self.pub_date = pdate
        self.last_modified = pdate
        super(Model, self).save()

And the final version of ReSTMiddleware then becomes:

class ReSTMiddleware(ModelMiddleware):
    def pre_save(self):
        try:
            opts = self.__class__._middle["ReST"] # individual options are saved in a dict
        except AttributeError:
            return  # just fail silently, though it might not be a very good idea in practice

        # lets be nice to ourselves and provide a default value for the initial header level
        for opt in opts:
            opt.setdefault("init_header", 1)
        
        # parse for as many fields as we have options for
        for opt in opts:  
            try:
                cont = getattr(self, opt["field"]).decode("utf_8")
                parts = build_document(cont, initial_header_level=opt["init_header"])
                setattr(self, opt["save_body"], parts["html_body"].encode('utf_8'))
                setattr(self, opt["save_toc"], parts["toc"].encode('utf_8'))
            except:
                pass # another silent fail, needs fixing

Now when an instance of MyModel class is saved, the ReSTMiddleware.pre_save() method will be called before the actual MyModel.save(), preparing all the ReST'ified fields. And this ReST parser can be applied to as many models as needed, automatically inserting the same pre_save() method into all of them. If you wanted another ModelMiddleware class to be included in your model, you'd just add it to the list of parents from which the model inherits.

Your model can mix ModelMiddleware classes and "straight" Django models in the inheritance list - since the MetaModelMiddleware inherits from the ModelBase metaclass, no inheritance clashes will occur, and since middleware classes are actually removed from the parents list of the inheritting model class (leaving everything else in (hopefully)), you won't get any errors about missing tables in your db.

The provided module includes two model middleware classes: the already described ReSTMiddleware and TimestampMiddleware, which puts that "date mucking" from the first version of code above into a model middleware class. The ReSTMiddleware class won't work without an extra module that includes the function used for parsing ReST strings.

Problems:

• Not really tested yet, in a formal sense of the word.
• No guarantee is given that "mixed" inheritance (ModelMiddleware + models.Model descendants) really works.