Altering unique_together sometimes missing deleted fields

[no]

Steps to reproduce:

1. Create an App spam
2. Create two models spam.Eggs and spam.Spam

class Eggs(models.Model):
    pass


class Spam(models.Model):
    class Meta:
        unique_together = (
           ('a', 'b'),
        )
        
    a = models.ForeignKey(Eggs)
    b = models.CharField(max_length=10)

3. Make migrations
4. Create an App ham
5. Create model ham.Ham

class Ham(models.Model): pass

6. Add foreignkey to Spam.Spam: c = models.ForeignKey('ham.Ham')
7. Make Migrations
8. Delete field Spam.a
9. Change Spam.unique_together to ('c', 'b')
10. Make Migrations
11. Run migrations.

Output:

django.db.models.fields.FieldDoesNotExist: Spam has no field named u'a'

I think this might be related to issue #23505 since that is what I was actually trying to reproduce.

[no]

Test project

[Baptiste Mispelon]

Hi,

I can reproduce the issue (I had to use c = models.ForeignKey('ham.Ham', null=True) though otherwise it asked me for a default when running makemigrations).

The traceback under Python 3 provides a little bit more information:

Traceback (most recent call last):
  File "./django/db/models/options.py", line 404, in get_field_by_name
    return self._name_map[name]
AttributeError: 'Options' object has no attribute '_name_map'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./django/db/models/options.py", line 407, in get_field_by_name
    return cache[name]
KeyError: 'a'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "./django/core/management/__init__.py", line 336, in execute_from_command_line
    utility.execute()
  File "./django/core/management/__init__.py", line 328, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "./django/core/management/base.py", line 369, in run_from_argv
    self.execute(*args, **cmd_options)
  File "./django/core/management/base.py", line 419, in execute
    output = self.handle(*args, **options)
  File "./django/core/management/commands/migrate.py", line 193, in handle
    executor.migrate(targets, plan, fake=options.get("fake", False))
  File "./django/db/migrations/executor.py", line 63, in migrate
    self.apply_migration(migration, fake=fake)
  File "./django/db/migrations/executor.py", line 97, in apply_migration
    migration.apply(project_state, schema_editor)
  File "./django/db/migrations/migration.py", line 107, in apply
    operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
  File "./django/db/migrations/operations/fields.py", line 84, in database_forwards
    schema_editor.remove_field(from_model, from_model._meta.get_field_by_name(self.name)[0])
  File "./django/db/backends/sqlite3/schema.py", line 185, in remove_field
    self._remake_table(model, delete_fields=[field])
  File "./django/db/backends/sqlite3/schema.py", line 128, in _remake_table
    self.create_model(temp_model)
  File "./django/db/backends/schema.py", line 253, in create_model
    columns = [model._meta.get_field_by_name(field)[0].column for field in fields]
  File "./django/db/backends/schema.py", line 253, in <listcomp>
    columns = [model._meta.get_field_by_name(field)[0].column for field in fields]
  File "./django/db/models/options.py", line 410, in get_field_by_name
    % (self.object_name, name))
django.db.models.fields.FieldDoesNotExist: Spam has no field named 'a'

Thanks!

[Markus Holtermann]

I opened an initial pull-request: ​https://github.com/django/django/pull/3430

This patch only solves the problem for one situation: removing a field. Renaming a field and keeping it in the index/unique_together option shouldn't be a problem, since _generate_altered_foo_together works on the renamed fields.

[Tim Graham <timograham@…>]

In 5c9c1e029d139bd3d5213804af2ed9f317cd0b86:

Fixed #23614 -- Changed the way the migration autodetector orders unique/index_together

Thanks to Naddiseo for the report and Tim Graham for the review

[Tim Graham <timograham@…>]

In 21358e72253aac0d26af0cff2f597c1ab7bf151c:

[1.7.x] Fixed #23614 -- Changed the way the migration autodetector orders unique/index_together

Thanks to Naddiseo for the report and Tim Graham for the review

Backport of 5c9c1e029d from master