Problem running syncdb with a multi-db router that restricts auth to a single database

[Russell Keith-Magee]

Via Klemens Mantzos on django-users:

If i do

python manage.py syncdb --database=default

before i sync the users database this error comes up (which seems ok to me):

django.db.utils.DatabaseError: (1146, "Table 'multidb_user.auth_permission' doesn't exist")

but if i do this:

python manage.py syncdb --database=users

i get that every time (no matter if django_content_type already exists in the default db or not):

django.db.utils.DatabaseError: (1146, "Table 'multidb_user.django_content_type' doesn't exist")

Router config

# don't know but OtherRouter is maybe obsolete.
DATABASE_ROUTERS = ['dbrouter.AuthRouter', 'dbrouter.OtherRouter']

project/dbrouter.py

class AuthRouter(object):
  """A router to control all database operations on models in
  the contrib.auth application"""

  ....

  def allow_syncdb(self, db, model):
      "Make sure the auth app only appears on the 'credentials' db"
      if db == 'users':
          return model._meta.app_label == 'auth'
      elif model._meta.app_label == 'auth':
          return False
      return None


class OtherRouter(object):
  """A router that sets up a simple master/slave configuration"""

  ...

  def allow_syncdb(self, db, model):
      "Explicitly put all models on all databases."
      return True

[Russell Keith-Magee]

The issue appears to be the way that the contenttype post-syncdb handler deals with a multi-db situation.

[Adam Vandenberg]

Hack: blocks command if ContentType isn't allowed to sync into the target db.

[Russell Keith-Magee]

After closer inspection, I'm going to mark this someday/maybe.

The underlying problem here is that auth has foreign keys to contenttypes. This means that you need to deal with referential integrity across databases, which is a non-trivial problem to solve (other than the obvious workaround of putting contenttypes on the same database as auth)

An alternative solution would be to modify the contenttypes syncdb handler so that contenttypes can be synchronized onto multiple databases. However, this would require lots of extra logic to ensure that databases remained in sync, including ensuring that primary key allocation for content types is consistent between databases.

However, I will make a change to the multi-db docs -- they currently suggest that routers can be used to put auth onto a separate database, and while this is technically true, it won't be true for many uses in practice. Better to avoid the implication altogether.

[Russell Keith-Magee]

(In [12751]) Refs #12767 -- Modified the multi-db docs to remove the implication that contrib.auth can be easily put on a separate database.

[fetzig]

maybe its a stupid question but I just don't get it, and need to ask:

@russellm does that mean that its required to keep models with foreign keys to each other together in one db (even with configured router)?

or is it just a "special" problem with the syncdb of contenttypes?

thx for your time russel.

greets,
klemens

[Russell Keith-Magee]

@fetzig -- This is not specific to ContentTypes. Foreign Keys can't cross databases. This is all a consequence of referential integrity -- a database can't guarantee referential integrity if it doesn't have access to the ground truth. There might be ways to relax this restriction, but it isn't trivial.

[Digitalxero]

This bug affects any multi-db config that has django.contrib.contenttpes installed even if the models being synced do not have a ForeignKey to ContentType

My examples, I have a DB of gis data for zip codes, counties / states etc. This DB is used by a number of sites so I want it in it's own DB, and nothing else gets synced there but when I do

manage.py syncdb --database=gis

I get

django.db.utils.DatabaseError: (1146, "Table 'gis.django_content_type' doesn't exist")

None of the models in my gis app have relations to ContentTypes and no app in the rest of the site has relations to the gis app, this is something I ensure with a DB Router. So this error is coming because the ContentTypes app wants to track all installed applications, but it should not be tracking items it can have no relation with.

This patch does fix the error

[Russell Keith-Magee]

The patch doesn't fix the error at all; it hides it. It means that you won't have a content type for the objects in the separate database, which has the potential to cause all sorts of other problems.

I also have a suspicion there might be something wrong with your router; If you're getting errors about "gis.django_content_type", that suggests that something in your routing configuration is pushing contenttype to the gis database, which shouldn't be happening.

[Digitalxero]

I am fairly sure it is because the update_contenttypes method sets using(db) for all db calls (eg ct.save(using=db))

[Russell Keith-Magee]

You may want to update to trunk. The [source:/django/trunk/django/contrib/contenttypes/management.py current source code] doesn't force the database at all. There aren't any calls to using() or save(using=)

[Digitalxero]

Thats what I get for testing with beta1 and not upgrading to trunk when I find a bug. Just upgraded to trunk and verified I no longer see my issue even without this patch.

[Marco Paolini]

maybe related to #14662

[Aymeric Augustin]

This is a duplicate of #16039 that's fixed in 1.5.