Opened 9 years ago

Closed 9 years ago

#26273 closed Bug (invalid)

Module level queries in a project can crash migrations if the queried model schema changed.

Reported by: Sven Coenye Owned by: nobody
Component: Uncategorized Version: 1.9
Severity: Normal Keywords: classmethod queryset manage.py
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Starting with

class Graduation(models.Model):
    ceremony_date = models.DateField()
    ...

    @classmethod
    def current_graduation(cls):
        early = date.today() - timedelta(183)
        late = date.today() + timedelta(183)
        
        return cls.objects.get(ceremony_date__gte=early, ceremony_date__lte=late)

class Graduate(models.Model):
    conferral    = models.ForeignKey(Graduation)
    first_name   = models.CharField(max_length = 14)
    last_name    = models.CharField(max_length = 19)
    ...

and

class TestReport(ListView):
    queryset = Graduate.objects.filter(conferral=Graduation.current_graduation()) \
                               .order_by("last_name", "first_name")

    template_name       = "graduation/reports/applicants.html"
    context_object_name = "applicants"

The object is to automatically show the applicants for the next upcoming graduation. This works like a charm until the Graduation model is modified:

class Graduation(models.Model):
    ceremony_date = models.DateField()
    crash_test    = models.CharField(max_length=10)
    ...

    @classmethod
    def current_graduation(cls):
        early = date.today() - timedelta(183)
        late = date.today() + timedelta(183)
        
        return cls.objects.get(ceremony_date__gte=early, ceremony_date__lte=late)

At this point

python manage.py makemigrations

crashes (and so does pretty much any invocation besides asking for --help)

Full stack trace attached, but the cause appears to be that the entire application is loaded, triggering execution of the class method. This in turn ends up asking the database for a column that does not exist yet as the ORM generates a select including all field names.

Not defining queryset on the ListView and using get_queryset() fixes the problem, but as I originally got bitten by this when rolling a changeset out to production, I thought the condition is worth documenting.

Attachments (1)

django_manage_crash.log (6.1 KB ) - added by Sven Coenye 9 years ago.
Stack trace of manage.py crash

Download all attachments as: .zip

Change History (4)

by Sven Coenye, 9 years ago

Attachment: django_manage_crash.log added

Stack trace of manage.py crash

comment:1 by Simon Charette, 9 years ago

Summary: Use of @classmethod in queryset can crash manage.py if a model is changedModule level queries in a project can crash migrations if the queried model schema changed.

Hi scoenye,

From what I can see this has not much to do with migrations but is related to the fact you should never execute queries at module level.

This is related to #25454 and probably a large number of closed tickets.

I wonder if should simply prevent django.db from executing queries until django.apps.apps.ready or at least issue a RuntimeWarning.

We would have to go through deprecation but I'm pretty sure this would uncover a lot of existing application bugs and prevent future ones.

comment:2 by Sven Coenye, 9 years ago

Thank you. I did search here, the docs, and Google, but it looks like my keywords were not a good match for the other reported manifestations.

Perhaps adding an explicit note to the CBV documentation would help those of us who are still feeling their way through Django avoid this pitfall? As-is, it looks like the only reference to this issue is in the testing documentation.

comment:3 by Tim Graham, 9 years ago

Resolution: invalid
Status: newclosed

I started a discussion on django-developers based on Simon's proposal to add a warning or prohibit module level queries. Let's open a new ticket with action items based on the results.

Note: See TracTickets for help on using tickets.
Back to Top