Reverse migrations model state rendering slow with moderate to large migrations

[David Sanders]

Relevant discussion: ​https://forum.djangoproject.com/t/migrating-backwards-takes-orders-of-magnitude-longer-than-migrating-forwards/20873/2

Investigations revealed 2 things.

1. this block of code in db/migrations/excecutor.py was responsible for a large portion of processing time: ​https://github.com/django/django/blob/b8b2f7451201f3ff60891b6ce55f177400700d7a/django/db/migrations/executor.py#L222-L232

In my small project with only a moderate number of migrations, a simple reverse migrate of the last migration on the main app took ~20s. The block below was responsible for ~16s.

        # Generate the post migration state by starting from the state before
        # the last migration is unapplied and mutating it to include all the
        # remaining applied migrations.
        last_unapplied_migration = plan[-1][0]
        state = states[last_unapplied_migration]
        for index, (migration, _) in enumerate(full_plan):
            if migration == last_unapplied_migration:
                for migration, _ in full_plan[index:]:
                    if migration in applied_migrations:
                        migration.mutate_state(state, preserve=False)
                break

2. these 2 lines appeared to be responsible for the slowdown, commenting them out made the block above run almost instantly: ​https://github.com/django/django/blob/b8b2f7451201f3ff60891b6ce55f177400700d7a/django/db/migrations/executor.py#L203-L204

                if "apps" not in state.__dict__:
                    state.apps  # Render all -- performance critical

I have a small patch to move record the unapplied state before these 2 lines and clone it, removing the state.apps attribute.

[Mariusz Felisiak]

Tentatively accepted for the investigation.

[Mariusz Felisiak <felisiak.mariusz@…>]

In b6e2b83:

Fixed #34841 -- Avoided rendering apps on state still requiring mutation.