Unable to unapply a branch of migrations

[Roman Odaisky]

1. Check out the master branch of a project into a new branch
2. Implement a feature
3. Make migrations
4. Apply the migrations
5. Merge updates from master into current branch
6. makemigrations --merge
7. Possibly repeat items 2–6 several times
8. Decide to abandon the feature, look for a way to restore the DB to what the code in master expects

However, there’s no invocation of manage.py migrate that will unapply all the migrations introduced by one branch.

For example:

master ...---0050---0051a---0052a
                 \               \
feature           0051b---0052b---0053m

We’re currently at the merge migration 0053m. We’d like to mark 0053m as unapplied, perform the rollback operations of 0052b and 0051b and end up at 0052a (at which point we can check out master and forget about the existence of the abandoned branch and its migrations). Currently this is only possible (other than rolling back and re-applying the master migrations, which may not even be an option) by doing a number of --fake migrations in a risk-prone manner.

There needs to be an option to run manage.py migrate --before myapp 0051b or something like that, a command that would rollback the specified migration and all others that depend on it but leaving ones from parallel branches unaffected. In other words, while manage.py migrate myapp 1234 ensures that 1234 is applied, the suggested command would ensure that the specified migration is not applied (unapplying other migrations where necessary, but as few as possible).

[Mariusz Felisiak]

We’re currently at the merge migration 0053m. We’d like to mark 0053m as unapplied, perform the rollback operations of 0052b and 0051b and end up at 0052a (at which point we can check out master and forget about the existence of the abandoned branch and its migrations). Currently this is only possible (other than rolling back and re-applying the master migrations, which may not even be an option) by doing a number of --fake migrations in a risk-prone manner.

There needs to be an option to run manage.py migrate --before myapp 0051b or something like that, a command that would rollback the specified migration and all others that depend on it but leaving ones from parallel branches unaffected. In other words, while manage.py migrate myapp 1234 ensures that 1234 is applied, the suggested command would ensure that the specified migration is not applied (unapplying other migrations where necessary, but as few as possible).

If you've already performed migrations from the feature branch you can restore the previous state by ​reversing migrations to the 0050, removing migrations from the feature branch and applying again 0051a and 0052a. You can also add new migrations that will reverse changes from 0051b, 0052b.

Working with migrations on feature branches is difficult and each situation is different, I don't see a way to add an option to the migrate command that can decide for you. Personally I would recommend using a separate db for each feature that can be abandoned if not needed.

[Roman Odaisky]

Replying to Mariusz Felisiak:

Working with migrations on feature branches is difficult and each situation is different, I don't see a way to add an option to the migrate command that can decide for you. Personally I would recommend using a separate db for each feature that can be abandoned if not needed.

Can we at the very least add something like manage.py migrate --unapply-one <app> <migration> that would unapply one migration, having ensured that no currently applied migrations depend on it? This will solve the use case I outlined and maybe some others while not being able to corrupt the DB state.

[Roman Odaisky]

By the way, do I understand correctly that if this code: ​https://github.com/django/django/blob/stable/3.1.x/django/db/migrations/executor.py#L43 is modified so instead of the loop

for node in next_in_app:
    for migration in self.loader.graph.backwards_plan(node):

it simply performs self.loader.graph.backwards_plan(target) it will do exactly what I want, that is, migrate back to the specified migration and then unapply that migration as well?

[Mariusz Felisiak]

Can we at the very least add something like manage.py migrate --unapply-one <app> <migration> that would unapply one migration, having ensured that no currently applied migrations depend on it? This will solve the use case I outlined and maybe some others while not being able to corrupt the DB state.

Migrations can be reversed, see ​docs. However adding an option to reverse a specific migration doesn't sound like a good idea, we will not be able to ensure that a database state is not corrupted. Migrations history must be continuous. You're talking about reversing migrations in a database but without continuous changes reflected in migrations. So you would like to drop a feature branch, reverse its migrations and have a clear path without migrations from the feature branch, and yes you can do this but outside of Django migrations. The proper way do this in Django is to reverse code changes from the feature branch and run makemigrations that will create a new migration reversing db changes.

[Roman Odaisky]

Replying to Mariusz Felisiak:

Can we at the very least add something like manage.py migrate --unapply-one <app> <migration> that would unapply one migration, having ensured that no currently applied migrations depend on it? This will solve the use case I outlined and maybe some others while not being able to corrupt the DB state.

Migrations can be reversed, see ​docs. However adding an option to reverse a specific migration doesn't sound like a good idea, we will not be able to ensure that a database state is not corrupted. Migrations history must be continuous.

If we just do one check: that no children of this particular migration are applied. Does that not allow us to reverse the migration while guaranteeing consistency?

[Roman Odaisky]

It turned out to be rather simple to implement: ​https://github.com/django/django/pull/13781

[Mariusz Felisiak]

Again, you're trying to fix a really specific situation that we don't want to support, IMO. You want to unapply migrations and then (probably) remove reverted migrations, this is not a flow that is supported. Also, this can corrupt a database if you will have any migrations created after a "merge" migration (0053m in your example).

[Mariusz Felisiak]

You can start a discussion on DevelopersMailingList if you don't agree.

[Guðlaugur Stefán Egilsson]

I believe that what is missing here is the ability to migrate back to a point in time based on the migrations table of that particular deployment, instead of relying exclusively on the migrations graph. Migrations are always applied in a linear fashion and should be "unappliable" in the exact reverse order recorded in the migrations table, regardless of merge migrations.

So something like

python manage.py 2022-01-17T09:46:51.444

Should unapply all migrations applied after this timestamp. This would solve this issue here.

An alternative would be to use the ID of the migration.

python manage.py 42

Would unapply migrations with ID greater than 42 in the migrations table for all apps.

Or am I missing something?

[Roman Odaisky]

python manage.py migrate appname 42
Would unapply migrations with ID greater than 42

That’s exactly what would happen, and in the case shown in the ticket, manage.py migrate appname 0050 will unapply both 0051a and 0051b, while what’s desired is to unapply 0051b only.