Possible bad interaction between migration dependencies and relabeling apps

[Aymeric Augustin]

As far as I can tell, in migrations, dependencies is a list of (app_label, migration_name).

I'm wondering what happens if one relabels an application with AppConfig.label. How can migrations handle this case?

(I haven't tried to create such a problem. I'm just making a note before I forget.)

[Markus Holtermann]

It partially works. If there are no migrations for an app yet, all migrations will be created with the changed label in the dependencies and all places it's being referenced. But as soon as you already have existing migrations and then change the label of an app, the makemigrations command blows up with:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/markus/Coding/django/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/home/markus/Coding/django/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/markus/Coding/django/django/core/management/base.py", line 390, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/markus/Coding/django/django/core/management/base.py", line 442, in execute
    output = self.handle(*args, **options)
  File "/home/markus/Coding/django/django/core/management/commands/makemigrations.py", line 58, in handle
    loader = MigrationLoader(None, ignore_no_migrations=True)
  File "/home/markus/Coding/django/django/db/migrations/loader.py", line 48, in __init__
    self.build_graph()
  File "/home/markus/Coding/django/django/db/migrations/loader.py", line 295, in build_graph
    _reraise_missing_dependency(migration, parent, e)
  File "/home/markus/Coding/django/django/db/migrations/loader.py", line 265, in _reraise_missing_dependency
    raise exc
  File "/home/markus/Coding/django/django/db/migrations/loader.py", line 291, in build_graph
    self.graph.add_dependency(migration, key, parent)
  File "/home/markus/Coding/django/django/db/migrations/graph.py", line 47, in add_dependency
    parent
django.db.migrations.graph.NodeNotFoundError: Migration testapp.0002_proxy dependencies reference nonexistent parent node ('otherapp', '0001_initial')

[Aymeric Augustin]

I'm afraid we messed up that part of the API and it's too late to fix :-|

Andrew, do you have any thoughts on this?

[Carl Meyer]

To me this seems like a straightforward case of "not every edge case can be automatic." The fix is manual, but simple, right? You just have to replace that app label in all existing migrations that reference it.

[Aymeric Augustin]

That's inconvenient when the affected app is a third-party app.

[Carl Meyer]

A third-party app with models changing its label has always meant changing the name of all its database tables, too (unless it uses Meta.db_table to preserve them). So it's always been the case that changing the label of an app with models is a Fairly Backwards Incompatible Thing To Do. I guess what's new in 1.7 is that a) changing app-label is easier than it used to be (no longer implies changing import path), and b) migrations introduce a new layer of label-dependence that isn't fixed by using a backwards-compatible Meta.db_table.

I'm having trouble seeing a way to make this smoother, other than possibly improving the error message (though the current one isn't terrible) and probably adding a note to the docs in the AppConfig section clarifying that changing app_label midstream is a breaking change for any existing installs of that app. Fundamentally, migrations need some way to reliably and persistently reference an app, and that's always been the purpose of app-label.

[Aymeric Augustin]

A warning in the documentation of AppConfig.label is the way to go then.

[Carl Meyer]

This does mean that, in the new world of required migrations, using AppConfig to relabel third-party apps is pretty much impossible (due to app-label being hardcoded in migration dependencies). So the idea that AppConfig.label can allow using two third-party apps with conflicting labels is dead (if those apps both have models/migrations).

[Carl Meyer]

In theory, it would be possible for migrations to stop using app-label for dependencies and start using full dotted paths instead? Maybe?

And then since db table names aren't hardcoded in migrations, maybe migrations could adapt to the app being re-labeled? (If you did the re-labeling before running any of its migrations; re-labeling midstream would still require manual table renames.)

This would be a big change, but if we wanted to revive the idea of using AppConfig.label for third-party apps, I think that would be the necessary path.

Using full paths of course means that if the import path for an app changes, you have to edit all its migrations. But maybe that's better, since import path is definitely under the control of the app author, always, unlike app label (since the advent of AppConfig.label).

If we don't make this change to migrations, it seems possible that there are no really compelling use-cases remaining for AppConfig.label, just bugs-in-waiting, and we should consider deprecating and removing it.

[Carl Meyer]

Last thought: using AppConfig.label for third-party apps with migrations is still possible, I guess - you just have to use MIGRATION_MODULES also, and regenerate all its migrations. Which gets unfortunate if the app is upgraded and ships a new migration; you'd have to always re-create those yourself.

[Andy Miller]

PR: ​https://github.com/django/django/pull/18324

[Natalia Bidart]

PR looks good, some tests are failing due to Jenkins not being able to access GH:

> git fetch --no-tags --force --progress --depth=1 -- https://github.com/django/django.git +refs/pull/18324/merge:refs/remotes/origin/pull/18324/merge # timeout=10
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from https://github.com/django/django.git

Will merge when all runs are completed.

[Natalia <124304+nessita@…>]

In fb140420:

[5.1.x] Fixed #23790 -- Warned about renaming AppConfig.label in docs/ref/applications.txt.

Backport of aa74c4083e047473ac385753e047e075e8f04890 from main.

[nessita <124304+nessita@…>]

In aa74c40:

Fixed #23790 -- Warned about renaming AppConfig.label in docs/ref/applications.txt.

[Natalia <124304+nessita@…>]

In 4cf71990:

[5.0.x] Fixed #23790 -- Warned about renaming AppConfig.label in docs/ref/applications.txt.

Backport of aa74c4083e047473ac385753e047e075e8f04890 from main.