RunSQL migration is run twice (and once for real with "--fake")

[Tino de Bruijn]

I added a cleanup migration to remove the auth_* tables after swapping the user model as follows:

class Migration(migrations.Migration):

    dependencies = [
        ('users', '0001_initial'),
    ]

    operations = [
        migrations.RunSQL(
            "DROP TABLE auth_user_user_permissions;"
            "DROP TABLE auth_group_permissions;"
            "DROP TABLE auth_user_groups;"
            "DROP TABLE auth_group;"
            "DROP TABLE auth_permission;"
            "DROP TABLE auth_user;"
        )
    ]

With python manage.py migrate, the tables are deleted, but this migration is run twice, resulting in an error:

Applying users.0002_delete_auth_tables... OK
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/core/management/base.py", line 393, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/core/management/base.py", line 444, in execute
    output = self.handle(*args, **options)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 226, in handle
    emit_post_migrate_signal(created_models, self.verbosity, self.interactive, connection.alias)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/core/management/sql.py", line 280, in emit_post_migrate_signal
    using=db)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/dispatch/dispatcher.py", line 201, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/contrib/auth/management/__init__.py", line 93, in create_permissions
    "content_type", "codename"
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/models/query.py", line 162, in __iter__
    self._fetch_all()
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/models/query.py", line 965, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/models/query.py", line 1220, in iterator
    for row in compiler.results_iter():
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 794, in results_iter
    results = self.execute_sql(MULTI)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 840, in execute_sql
    cursor.execute(sql, params)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 318, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: auth_permission

Appearently, this also happens when the --fake flag is passed (I removed the django_migrations table row that marked this migration as applied), which is concerning because I would expect the migration not to run at all:

$ python manage.py migrate users 0002 --fake
Operations to perform:
  Target specific migration: 0002_delete_auth_tables, from users
Running migrations:
  Rendering model states... DONE
  Applying users.0002_delete_auth_tables... FAKED
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/core/management/base.py", line 393, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/core/management/base.py", line 444, in execute
    output = self.handle(*args, **options)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 226, in handle
    emit_post_migrate_signal(created_models, self.verbosity, self.interactive, connection.alias)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/core/management/sql.py", line 280, in emit_post_migrate_signal
    using=db)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/dispatch/dispatcher.py", line 201, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/contrib/auth/management/__init__.py", line 93, in create_permissions
    "content_type", "codename"
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/models/query.py", line 162, in __iter__
    self._fetch_all()
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/models/query.py", line 965, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/models/query.py", line 1220, in iterator
    for row in compiler.results_iter():
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 794, in results_iter
    results = self.execute_sql(MULTI)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 840, in execute_sql
    cursor.execute(sql, params)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/Users/tino/Dev/.virtualenvs/test/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 318, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: auth_permission

[Tino de Bruijn]

Hmm, sometimes it does work. But to reproduce, create a project with a 'users' app, with a simple MyUser model, create the initial migration, and the second one as described above, and run:

rm db.sqlite3
python manage.py migrate auth
python manage.py migrate sessions
python manage.py migrate admin
python manage.py migrate users

[Tim Graham]

Does your "users" migration have a dependency on "auth" to ensure that its migrations run after auth's? If you could provide a sample project that will save the ticket triagers some time. Thanks.

[Tino de Bruijn]

Hi Tim, it didn't have that dependency, but I ran the migration in the order as described above. Adding the dependency does not change the behaviour, but does make that you only have to run python manage.py migrate once to have it display.

Repository: ​https://github.com/tino/django-ticket-25442

[Tim Graham]

I think this is a symptom of trying to migrate from the built-in auth user to a custom user model. This currently doesn't work very well as described in #25313.

The tracebacks are you seeing aren't a migration running, but rather the post_migrate signal handler that creates user permissions.