Opened 10 years ago

Closed 8 years ago

#22931 closed Bug (duplicate)

Migrations cannot rename intermediate models

Reported by: JockeTF Owned by: nobody
Component: Migrations Version: 1.7
Severity: Normal Keywords: migrations migrate makemigrations rename intermediate model
Cc: eschler@…, info+coding@…, vytis.banaitis@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Due to a change in naming conventions I needed to rename an intermediate model. I renamed a model from Subscriber to Subscribe.

from django.db import models
from django.conf import settings


class Channel(models.Model):
        name = models.CharField(max_length=128)
        subscribers = models.ManyToManyField(settings.AUTH_USER_MODEL, through='Subscribe', related_name='subscriptions')
        date_created = models.DateTimeField(auto_now_add=True)


class Subscribe(models.Model):
        user = models.ForeignKey(settings.AUTH_USER_MODEL)
        channel = models.ForeignKey(Channel)
        date_subscribed = models.DateTimeField(auto_now_add=True)

        class Meta():
                unique_together = ('user', 'channel')

Running makemigrations in Django 1.7c1 results in an exception.

  0002_auto_20140701_0817.py:
    - Create model Subscribe
    - Alter unique_together for subscribe (1 constraints)
Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/base.py", line 337, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/commands/makemigrations.py", line 115, in handle
    self.write_migration_files(changes)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/commands/makemigrations.py", line 131, in write_migration_files
    self.stdout.write("    - %s\n" % operation.describe())
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/operations/models.py", line 251, in describe
    return "Alter %s for %s (%s constraints)" % (self.option_name, self.name, len(self.unique_together))
TypeError: object of type 'NoneType' has no len()

In Django 1.7b3 makemigrations actually worked.

from django.db import models, migrations
from django.conf import settings


class Migration(migrations.Migration):
        dependencies = [
                ('api', '0002_auto_20140630_1443'),
        ]

        operations = [
                migrations.RenameModel(
                        old_name='Subscriber',
                        new_name='Subscribe',
                ),
                migrations.AlterField(
                        model_name='channel',
                        name='subscribers',
                        field=models.ManyToManyField(through='api.Subscribe', to=settings.AUTH_USER_MODEL),
                ),
        ]

However, running migrate in Django 1.7b3 results in another exception.

Running migrations:
  Applying api.0013_subscriber_renamed...Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/django/apps/config.py", line 152, in get_model
    return self.models[model_name.lower()]
KeyError: 'subscriber'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/state.py", line 78, in render
    model = self.apps.get_model(lookup_model[0], lookup_model[1])
  File "/usr/local/lib/python3.4/dist-packages/django/apps/registry.py", line 190, in get_model
    return self.get_app_config(app_label).get_model(model_name.lower())
  File "/usr/local/lib/python3.4/dist-packages/django/apps/config.py", line 155, in get_model
    "App '%s' doesn't have a '%s' model." % (self.label, model_name))
LookupError: App 'api' doesn't have a 'subscriber' model.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/base.py", line 337, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/commands/migrate.py", line 146, in handle
    executor.migrate(targets, plan, fake=options.get("fake", False))
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/executor.py", line 62, in migrate
    self.apply_migration(migration, fake=fake)
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/executor.py", line 96, in apply_migration
    migration.apply(project_state, schema_editor)
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/migration.py", line 107, in apply
    operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/operations/models.py", line 141, in database_forwards
    new_apps = to_state.render()
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/state.py", line 88, in render
    model=lookup_model,
ValueError: Lookup failed for model referenced by field api.Channel.subscribers: api.Subscriber

The same exception is thrown if I move the migration generated in Django 1.7b3 to Django 1.7c1.

Change History (13)

comment:1 by bernie@…, 10 years ago

Hi. I'm getting the same bug. I've done some digging and have some more details on the exact conditions that cause it.

First of all, a minimal reproduction. Create a project with an app and add this to the models.py. Run makemigrations, then rename the class ModelToBeRenamed, then run it again.

class FKTarget(models.Model):
    prop = models.TextField()

class ModelToBeRenamed(models.Model):
    fk = models.ForeignKey(FKTarget)
    val = models.TextField()
    val2 = models.TextField()

    class Meta:
        unique_together = ('val', 'val2')

Interesting notes:

  1. The class being renamed must have both a foreign key field and a unique_together constraint, but the unique_together doesn't have to contain the foreign key
  2. If there is no unique_together constraint, the makemigrations behaviour is to handle te renaming by deleting the old model and creating the new one
  3. If there is no foreign key field, the makemigrations detects the rename and prompts the user to confirm that they have renamed the model
  4. You can trigger the same bug by removing the unique_together constraint instead of renaming the model.

comment:2 by bernie@…, 10 years ago

Not sure why it's saying that I've unset various flags (Needs documentation etc). It wasn't deliberate!

comment:3 by Tim Graham, 10 years ago

Triage Stage: Unreviewed → Accepted

comment:4 by Dirk Eschler, 10 years ago

Cc: eschler@… added
Component: Database layer (models, ORM) → Migrations

comment:5 by abraham.martin@…, 10 years ago

Resolution: → worksforme
Status: new → closed

I haven't been able to reproduce the error neither renaming te model or removing the unique_together constraint. Tested against ​https://github.com/django/django/tree/d1c08d4758998318eb1a881a3963b63bc89435b8 with the tests submitted by the creator of the ticket and those in the first comment.

I guess it has been fixed...

comment:7 by JockeTF, 10 years ago

Resolution: fixed
Status: closed → new
Version: 1.7-rc-1 → 1.7-rc-2

I am still affected by this issue. It appears to affect 1.7c2, 1.7.x, master, commit 015496539247b24c73b163f279ae8c8d3ccefc4c, and commit d1c08d4758998318eb1a881a3963b63bc89435b8. I have included the steps I took to reproduce the issue.

Start a new application called 'rename' in Django 1.7c2 with two models.

from django.db import models
from django.conf import settings


class Channel(models.Model):
    name = models.CharField(max_length=128)
    subscribers = models.ManyToManyField(settings.AUTH_USER_MODEL, through='Subscriber', related_name='subscriptions')
    date_created = models.DateTimeField(auto_now_add=True)


class Subscriber(models.Model):
    user = models.ForeignKey(settings.AUTH_USER_MODEL)
    channel = models.ForeignKey(Channel)
    date_subscribed = models.DateTimeField(auto_now_add=True)

    class Meta():
        unique_together = ('user', 'channel')

Make migrations.

$ python3 manage.py makemigrations
Migrations for 'rename':
  0001_initial.py:
    - Create model Channel
    - Create model Subscriber
    - Add field subscribers to channel
    - Add field channel to subscriber
    - Add field user to subscriber
    - Alter unique_together for subscriber (1 constraint(s))

Change the name of the subscriber model. Make sure to also update the subscribers field to reflect the change.

from django.db import models
from django.conf import settings


class Channel(models.Model):
    name = models.CharField(max_length=128)
    subscribers = models.ManyToManyField(settings.AUTH_USER_MODEL, through='Subscribe', related_name='subscriptions')
    date_created = models.DateTimeField(auto_now_add=True)


class Subscribe(models.Model):
    user = models.ForeignKey(settings.AUTH_USER_MODEL)
    channel = models.ForeignKey(Channel)
    date_subscribed = models.DateTimeField(auto_now_add=True)

    class Meta():
        unique_together = ('user', 'channel')

Make migrations.

$ python3 manage.py makemigrations
Migrations for 'rename':
  0002_auto_20140729_0822.py:
    - Create model Subscribe
    - Alter unique_together for subscribe (1 constraint(s))
    - Alter unique_together for subscriber (0 constraint(s))
    - Remove field channel from subscriber
    - Remove field user from subscriber
    - Delete model Subscriber
    - Alter field subscribers on channel

Django failed to detect the rename (perhaps due to some other bug). Replace the automatically generated migration with one generated by Django 1.7b4. I cannot say whether or not this migration would still be valid, but I have no other way of generating a proper migration at the moment.

from django.db import models, migrations
from django.conf import settings


class Migration(migrations.Migration):
    dependencies = [
        ('rename', '0001_initial'),
    ]

    operations = [
        migrations.RenameModel(
            old_name='Subscriber',
            new_name='Subscribe',
        ),
        migrations.AlterField(
            model_name='channel',
            name='subscribers',
            field=models.ManyToManyField(through='rename.Subscribe', to=settings.AUTH_USER_MODEL),
        ),
    ]

Make migrations detects no further changes.

$ python3 manage.py makemigrations
No changes detected

Migrate.

$ python3 manage.py migrate
Operations to perform:
  Apply all migrations: sessions, contenttypes, admin, auth, rename
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying rename.0001_initial... OK
  Applying rename.0002_auto_20140729_0822...Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/django/apps/config.py", line 158, in get_model
    return self.models[model_name.lower()]
KeyError: 'subscriber'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/state.py", line 79, in render
    model = self.apps.get_model(lookup_model[0], lookup_model[1])
  File "/usr/local/lib/python3.4/dist-packages/django/apps/registry.py", line 202, in get_model
    return self.get_app_config(app_label).get_model(model_name.lower())
  File "/usr/local/lib/python3.4/dist-packages/django/apps/config.py", line 161, in get_model
    "App '%s' doesn't have a '%s' model." % (self.label, model_name))
LookupError: App 'rename' doesn't have a 'subscriber' model.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/base.py", line 337, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/commands/migrate.py", line 160, in handle
    executor.migrate(targets, plan, fake=options.get("fake", False))
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/executor.py", line 63, in migrate
    self.apply_migration(migration, fake=fake)
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/executor.py", line 97, in apply_migration
    migration.apply(project_state, schema_editor)
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/migration.py", line 107, in apply
    operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/operations/models.py", line 141, in database_forwards
    new_apps = to_state.render()
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/state.py", line 89, in render
    model=lookup_model,
ValueError: Lookup failed for model referenced by field rename.Channel.subscribers: rename.Subscriber

I also tried to change the order of the operations.

from django.db import models, migrations
from django.conf import settings


class Migration(migrations.Migration):
    dependencies = [
        ('rename', '0001_initial'),
    ]

    operations = [
        migrations.AlterField(
            model_name='channel',
            name='subscribers',
            field=models.ManyToManyField(through='rename.Subscribe', to=settings.AUTH_USER_MODEL),
        ),
        migrations.RenameModel(
            old_name='Subscriber',
            new_name='Subscribe',
        ),
    ]

This results in a slightly different exception.

$ rm db.sqlite3 
$ python3 manage.py migrate
Operations to perform:
  Apply all migrations: rename, admin, auth, contenttypes, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying rename.0001_initial... OK
  Applying rename.0002_auto_20140729_0822...Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/django/apps/config.py", line 158, in get_model
    return self.models[model_name.lower()]
KeyError: 'subscribe'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/state.py", line 79, in render
    model = self.apps.get_model(lookup_model[0], lookup_model[1])
  File "/usr/local/lib/python3.4/dist-packages/django/apps/registry.py", line 202, in get_model
    return self.get_app_config(app_label).get_model(model_name.lower())
  File "/usr/local/lib/python3.4/dist-packages/django/apps/config.py", line 161, in get_model
    "App '%s' doesn't have a '%s' model." % (self.label, model_name))
LookupError: App 'rename' doesn't have a 'subscribe' model.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/base.py", line 337, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.4/dist-packages/django/core/management/commands/migrate.py", line 160, in handle
    executor.migrate(targets, plan, fake=options.get("fake", False))
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/executor.py", line 63, in migrate
    self.apply_migration(migration, fake=fake)
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/executor.py", line 91, in apply_migration
    if self.detect_soft_applied(migration):
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/executor.py", line 134, in detect_soft_applied
    project_state = self.loader.project_state((migration.app_label, migration.name), at_end=True)
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/loader.py", line 268, in project_state
    return self.graph.make_state(nodes=nodes, at_end=at_end, real_apps=list(self.unmigrated_apps))
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/graph.py", line 147, in make_state
    project_state = self.nodes[node].mutate_state(project_state)
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/migration.py", line 76, in mutate_state
    operation.state_forwards(self.app_label, new_state)
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/operations/models.py", line 117, in state_forwards
    apps = state.render(skip_cache=True)
  File "/usr/local/lib/python3.4/dist-packages/django/db/migrations/state.py", line 89, in render
    model=lookup_model,
ValueError: Lookup failed for model referenced by field rename.Channel.subscribers: rename.Subscribe

comment:8 by Andrew Godwin, 10 years ago

On the detecting of the rename: Django does not guarantee to detect renames, it's just a courtesy we offer if we can, so that in itself is not a bug. It can probably be improved, sure, but it's unrelated to this issue.

It looks to me that the likely cause is RenameModel not taking care of all the state changes on related fields - notably, it fixes all "to" references, but no "through" references, so making it do that will likely fix this.

comment:9 by Markus Holtermann, 10 years ago

Cc: info+coding@… added

I'm not sure why there is no stable sorting on the fields in model states, but this fix makes rename detection more reliable:

  • django/db/migrations/autodetector.py 

    diff --git a/django/db/migrations/autodetector.py b/django/db/migrations/autodetector.py
index c8b7db7..441a453 100644
    a b class MigrationAutodetector(object):  
    7373        change during renames)
    7474        """
    7575        fields_def = []
    76         for name, field in fields:
     76        for name, field in sorted(fields):
    7777            deconstruction = self.deep_deconstruct(field)
    7878            if field.rel and field.rel.to:
    7979                del deconstruction[2]['to']

I cannot reproduce the renaming problem on stable/1.7.x or master.

comment:10 by Tim Graham, 10 years ago

Version: 1.7-rc-2 → 1.7

comment:11 by Simon Charette, 8 years ago

#25044 looks very similar and has a patch implementing the fix proposed by Andrew in comment:8.

comment:12 by Vytis Banaitis, 8 years ago

Cc: vytis.banaitis@… added

I have followed the steps in comment:7 on current master and can confirm that the patch from #25044 fixes this issue.

comment:13 by Simon Charette, 8 years ago

Resolution: → duplicate
Status: new → closed

Let's close this ticket as a duplicate in this case.

Note: See TracTickets for help on using tickets.
Back to Top