Opened 8 years ago

Closed 8 years ago

#26693 closed Cleanup/optimization (duplicate)

RenameModel causes state.apps rendering leading to massive time increase in makemigrations

Reported by: Simon Percivall Owned by: nobody
Component: Migrations Version: 1.10
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description (last modified by Simon Percivall)

Not 100% sure I have complete understanding of what happens, or why, but it seems a RenameModel migration leads to an evaluation of state.apps, which affects the time it takes for it and all following migration states to be calculated.

This is for a pretty big app, with lots of migrations. The time to run makemigrations for all apps goes from 1 seconds to 77 seconds with one RenameModel migration.

A test where apps.state was replaced by temp_state = state.close(); apps = temp_state.apps resolved the issue, but I'm not completely sure of the side effects of doing that. (Or even better if it were possible to check immediately if 'apps' in state.__dict__ and skip the rendering completely.)

Attachments (1)

makemigrations.diff (950 bytes ) - added by vinay karanam 8 years ago.

Download all attachments as: .zip

Change History (12)

comment:1 by Simon Percivall, 8 years ago

Description: modified (diff)

comment:2 by Simon Charette, 8 years ago

This might be related to #24067.

Could you try disconnecting the django.contrib.contenttypes.management.inject_rename_contenttypes_operations receiver from the pre_migrate signals and see if it solves your issue. The receiver is connected in django.contrib.contenttypes.apps.ContentTypesConfig.ready().

comment:3 by Simon Charette, 8 years ago

Actually this might not be related at all as #24067 should only affects migrate (not makemigrations).

Could you try bisecting the commit that introduced this slow down? Did you notice it when upgrading from a specific version of Django to another?

comment:4 by Simon Percivall, 8 years ago

Well, no, it's the first RenameModel in our project, so I haven't seen it before.

What I do see, to be more specific, is that, timing each self.nodes[node].mutate_state(...) call in MigrationGraph.make_state, each iteration is ~0.01ms. If I add a RenameModel, it clocks in at ~1 second, and every mutate_state after that is >0.5 seconds, because, like I said, state.apps has been evaluated and set.

It looks to be the same in Django 1.8 (i.e. things don't get expensive until state.apps has been evaluated once, since most of the state operations are no-ops if not 'apps' in self.__dict__).

comment:5 by Markus Holtermann, 8 years ago

It might be related to https://github.com/django/django/blob/master/django/db/migrations/state.py#L53 but haven't tried it. If it is, I'm afraid there's no solution to it except for squashing the migrations in that app (possibly manually).

comment:6 by Simon Percivall, 8 years ago

A RemoveModel and CreateModel combo (which in my mind seems like they should have an equivalent effect on the state as a RenameModel) does not have the same problem.

comment:7 by Tim Graham, 8 years ago

Triage Stage: UnreviewedAccepted
Type: UncategorizedCleanup/optimization

In the event this can't be fixed, perhaps we can reclassify as a documentation issue.

by vinay karanam, 8 years ago

Attachment: makemigrations.diff added

comment:8 by vinay karanam, 8 years ago

I recently came across this issue and the above patch fixed it.

comment:9 by vinay karanam, 8 years ago

Has patch: set
Needs tests: set
Patch needs improvement: set

comment:10 by vinay karanam, 8 years ago

Version: 1.91.10

comment:11 by vinay karanam, 8 years ago

Resolution: duplicate
Status: newclosed

duplicate of #27279

Note: See TracTickets for help on using tickets.
Back to Top