Add support for cross-database foreign keys and m2m

[Russell Keith-Magee]

Django's multidb support currently prevents foreign keys from referencing foreign databases. This means the obvious use case of "put User objects in the auth database" et al are not possible.

The full solution requires at least the following:

• Modifying ForeignKey to relax foreign key constraints if it is known that the foreign key will be on another database
• Providing hooks to allow FK and M2M descriptors to look up the right database to use

There has been ​some initial discussion on django users about this subject.

[Zbigniew Braniecki]

I suggest also adding config parameters that would bind apps in a project to a given database. Something like:

settings.py
DATABASES = {

'default': {

'NAME': 'db1',
'ENGINE': 'django.db.backends.mysql',
'USER': 'root',
'PASSWORD': ,

},
'db2': {

'NAME': 'db2',
'ENGINE': 'django.db.backends.mysql',
'USER': 'root',
'PASSWORD': ,

}

}

INSTALLED_APPS = (

{'NAME':'app1', 'DEFAULT_DB': 'db1'},
{'NAME':'app2', 'DEFAULT_DB': 'db2'},

)

this would allow models to be able to discover that the ForeignKey referenced in app1.models is using db2. It would also allow for models in app1 to on save() add required records to models from app2 without having to explicitly use using() on them (which would constrain the app relations)

Does it sounds reasonable? Is it a material for a separate ticket?

[Russell Keith-Magee]

@zbraniecki: This is not a new idea - it has been made by several people over the history of multi-db development. Read up on the archives to see why it hasn't been added.

[Russell Keith-Magee]

First pass at improved cross-database support.

[Russell Keith-Magee]

The patch I've just uploaded is a first pass at implementing a fix for cross-database support. Feedback is welcome.

[Russell Keith-Magee]

Version 2 of a fix for database allocation problems. This time with a public DB Router API.

[Russell Keith-Magee]

(In [12272]) Fixed #12540, #12541 -- Added database routers, allowing for configurable database use behavior in a multi-db setup, and improved error checking for cross-database joins.

[Simon Meers]

The title/description of this ticket haven't actually been "fixed", have they? -- "Django doesn’t currently provide any support for foreign key or many-to-many relationships spanning multiple databases" -- ​https://docs.djangoproject.com/en/dev/topics/db/multi-db/#limitations-of-multiple-databases

Is the addition of such support still being considered, and tracked elsewhere?

[Russell Keith-Magee]

I'm not aware of any plans to change the status quo here. Cross database referential integrity is a Hard Problem™. "Fixed" is perhaps misleading in this case, but it's as fixed as it's likely to get. The "fix" in this case was adding configurable routers, which lets you have a bit more control over where models are synchronised. That doesn't make cross database joins possible, but it does let you actively prevent them.

[Alexander Schepanovski]

Any plans to extend this to make sharding possible? Or should I open a new ticket?