This is an attempt to bring together the current state and long term plans of how application loading is done in Django, what it could do, and why.

Application loading refactor

At present application loading is handled in different ways through Django. Because of some of the limitations of Python, and some peculiarities of how models work, modules should not be reimported. Consequently Django maintains a global static, called the application cache. Models are imported from the application cache transparently when interacting with the database. But applications listed in INSTALLED_APPS contain more than just models. Some contain template tags, admin.py files, search_indexes.py files from haystack, template folders and many other features which applications use to inspect each other, or to contribute objects to a site-wide collection.

For the majority of applications, even reusable applications, anything more than the current system is unnecessary. Models are automatically registered, template tags can be loaded, and templates are available. (Tests are also available but this is a separate issue.)

However, not every application has all of these components - there are many useful pluggable applications which do not contain models, but must still be registered in INSTALLED_APPS. A simple example is django-pagination - which consists of a single template tag. This module must ship with an empty models.py module to allow it to be registered as an installed app, for that template tag to be available.

The problem is even more difficult for "meta-applications" - that is to say applications which exist solely to interact with other applications. The canonical example of this is the django admin, but there are many third party applications which do a similar thing. Haystack (the search engine framework) is a good example. Also there are applications which have a plugin archicture, whereby adding a second application to installed apps changes the behaviour of the first application. All of these third party systems have implemented their own ways to do this, with varying degrees of success.

This is an opportunity for Django to do something great for the third party application community. It will be a very powerful tool, and opens up a range of magical things which could be done. App loading is magical, but Magic is not always a bad thing - most Django developers are very pleased they don't have to individually register every model with Django as they do to register a model with the admin. You just put it in the models.py file and it works! This will allow the magic to be more explicit, and ultimately more consistent between third party applications.

This is a big change, and almost redefines what an "App" is within Django. To my mind, an app was originally intended to be a fairly self contained part of the system, normally in fact a self contained part of the Web site.

Specification

Here is a specification of what an application is, and what it can do. Perhaps not all of these points need to be implemented to be able to integrate this refactor into core, but they should be considered to avoid future backwards incompatibilities.

• An application is a self contained piece of code which contributes something to the site. It may contibute any number of the following:
  • models
  • templates
  • template tags
  • static files
  • admin configuration
  and these are just (some of?) the things which are picked up on by other parts of the Django core. Other things (seach indexes, migrations etc) may also be contributed by the application, and processed by other applications.

Note Some things are explicitly registered in settings, such as middleware, database routers and cache backends. These things may be contained within an application's code base, but they will not be registered automatically by the presence of that application in the INSTALLED_APPS.

• An application may contain urls and views. These will generally be explicitly registered, but some applications may choose to attach dynamic urls offer a way of getting the urls from the application rather than directly (similar to how the admin does to it's site class).

• An application may modify another application. That is to say, an application may provide an interface on it's application class for other applications to modify it. This will facilitate a plugin architecture. A classic example of where this will be useful is a Content Management System.

• An application may depend on the presence of another application. It is preferable for the application to depend on a defined contract with another application, rather than explicity depend on that application. This should allow applications to be more loosely coupled, leading potentially to pluggable auth/messages applications. It also allows such applications to be tested perhaps without the presence of the dependant app - a mocked object may suffice.

• An application may inspect other applications to gather information about them. The system should provide tools to easily and safely access this information, whether it is accessing a folder or a python module. Where possible, we should modify parts of Django core & contrib to use this system. Examples would include the app directories template loader, admin site and template tag libraries.

• An exact copy of an application can be deployed within the same project by changing its label. This will have some subtle issues regarding imports.

Implementation plan

Application class. Talk about what APIs would bee needed on that class to achieve the above. Separate by the above points. Ideally, does not need to be contained within the python module it relates to (thus allowing me to subclass an app's Application Class in my own project to tweak how it works, this may or may not be better than passing configuration options)

Application cache. Discuss what hooks this would need to provide, including signals, for the above to work.

Examine any backwards incompatibilities which may result. The app refactor itself shouldn't introduce any, but the new functionality which it offers and the resulting changes in other applications such as contrib.auth may need deprecation paths.

Current status

Preston has merged master into the existing branch with work done by Jannis and Arthur. (​https://github.com/ptone/django/commit/8722bdcc17293baa8b527ae7169b9a465e728d41)

We need to evaluate which of the current features are implemented, and which are not.

Update tests so that everything passes again, including outside of the appcache tests.

Try to work out which parts of the implementation plan are tangential to each other, and therefore could easily be worked on by other people.