Gracefull Migration | relation already exists

[Graeme Ford]

I have noticed that Django's migration system cannot gracefully handle a semi-populated DB.

Problem

I am stuck in a situation where we maintain development and deployment environments, each with slightly different migrations due to error and bug fixing and the fact that the DB is not backed up to the dev environment. As a result, we do not track/sync migration files between the two as they would not be compatible with the tracked files in each DB. Recently we have just had to wipe and re-start our migrations due to some hardware complications and it has come up that we cannot simply re-generate migration files as Django will attempt to re-create all migration files and all models from scratch. Obviously this is kicking up a django.db.utils.ProgrammingError: relation "<Table_Name_Here>" already exists which is not very easily fixable.

Solution/My Request:

I could always play around with the migration files or some such and tweak them until the migrations work but that is not ideal, especially in a production environment. It would be more intuitive to me for Django to check against the database whether or not any of the database tables already exist and optionally (Maybe a command flag?), only migrate new model tables/fields?

[Simon Charette]

Recently we have just had to wipe and re-start our migrations due to some hardware complications and it has come up that we cannot simply re-generate migration files as Django will attempt to re-create all migration files and all models from scratch. Obviously this is kicking up a django.db.utils.ProgrammingError: relation "<Table_Name_Here>" already exists which is not very easily fixable.

Is there a reason why you can't regenerate your migrations from scractch and simply run migrate --fake?

I could always play around with the migration files or some such and tweak them until the migrations work but that is not ideal, especially in a production environment. It would be more intuitive to me for Django to check against the database whether or not any of the database tables already exist and optionally (Maybe a command flag?), only migrate new model tables/fields?

That's precisely what initial migration and --fake-initial do ​https://docs.djangoproject.com/en/3.0/topics/migrations/#initial-migrations. If you got to a point where your Django migrations and your production database state diverged I suggest you start from scratch and --fake-initial

I am stuck in a situation where we maintain development and deployment environments, each with slightly different migrations due to error and bug fixing and the fact that the DB is not backed up to the dev environment. As a result, we do not track/sync migration files between the two as they would not be compatible with the tracked files in each DB.

I would advise against doing that as it's an uncommon pattern that is likely to keep breaking in the future. If you want to keep two sets of migrations that are different based on the environment that you are deploying to I'd suggest you use the ​MIGRATION_MODULES setting to point to per-app overrides in production.

I have a feeling this is more of an incomprehension of how the migration system works so I suggest you give another read at ​the migration documentation and start by using support channels before submitting a feature request in the future TicketClosingReasons/UseSupportChannels.

[Graeme Ford]

Many thanks for the feedback. Upon reading your response, I did some reading and did try out --fake-initial, however, it does not seem to have the desired effect. It appears to have only checked some of the models and not created the new models while marking the migration as --fake in the command-line output. I shall raise my questions on the correct support channels as you so tactlessly pointed out.

Regarding your parting words:

I have a feeling this is more of an incomprehension of how the migration system works so I suggest you give another read at ​the migration documentation and start by using support channels before submitting a feature request in the future TicketClosingReasons/UseSupportChannels.

I suggest you take a customer service course in the near future. Your response was construed as incredibly condescending and hostile. Kindness costs you nothing.

[Simon Charette]

Hello Graeme,

Sorry if my answer came out as condescending. We do get a lot of ticket reports from users who use this issue tracker as a support channel even if it's made clear that it shouldn't in the Please read this first section when filling a ticket. It can be hard to provide a firm answer to discourage such behaviour without sounding hostile.