Opened 19 months ago

Closed 19 months ago

Last modified 19 months ago

#34527 closed Bug (invalid)

Unexpected behavior with division in aggregation

Reported by: Egor R Owned by: nobody
Component: Database layer (models, ORM) Version: 3.2
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Egor R)

I came across an unexpected behavior (code taken from a real project, and the models are renamed):

django.__version__
# '3.2.18'

MyModel.objects.all().annotate(
    score=Sum("relatedmodel__points", output_field=FloatField(), filter=Q(relatedmodel__owner=user),)
    / Count("relatedmodel__id", filter=Q(relatedmodel__owner=user)),
).values_list("score", flat=True)
# <QuerySet [None, 10.0, 8.0, None, 9.0, None]>

MyModel.objects.all().annotate(
    score=Sum("relatedmodel__points", output_field=FloatField(), filter=Q(relatedmodel__owner=user),)
).values_list("score", flat=True)
# <QuerySet [None, 40.0, 35.0, None, 37.0, None]>

MyModel.objects.all().annotate(
    score=Count("relatedmodel__id", filter=Q(relatedmodel__owner=user),),
).values_list("score", flat=True)
# <QuerySet [0, 4, 4, 0, 4, 0]>

Since we're specifying output_field=FloatField() for Sum, I expected to get 10, 8.75 and 9.25 as the results of the first query, but I'm getting 10.0/8.0/9.0.
I looked into SQL code generated by the query - there's no casting of Sum to float there, so it's somewhat understandable why it is happening. But, shouldn't Django cast Sum to float in SQL?

Explicitly casting Count as FloatField works, though:

MyModel.objects.all().annotate(
    score=Sum("relatedmodel__points", output_field=FloatField(), filter=Q(relatedmodel__owner=user),)
    / Cast(Count("relatedmodel__id", filter=Q(relatedmodel__owner=user)), FloatField())
).values_list("score", flat=True)
# <QuerySet [None, 10.0, 8.75, None, 9.25, None]>

Relevant snippets of the models:

class MyModel(models.Model):
    ...

class RelatedModel(models.Model):
    points = models.PositiveSmallIntegerField(
        "description", default=10, null=True, blank=True
    )
    mymodel = models.ForeignKey("myapp.MyModel", on_delete=models.CASCADE)
    owner = models.ForeignKey(User, on_delete=models.CASCADE)
    ...

I created an MRE, will attach it to the ticket.
Tested on Postgres 12 (real project) and 15 (MRE).

PS Real project uses django-tenants, and I included it in the MRE as I tried to track down the bug. Can retest and create MRE without django-tenants if needed.

Attachments (1)

dj_agg_bug_clean.zip (9.3 KB ) - added by Egor R 19 months ago.
zip of the MRE

Download all attachments as: .zip

Change History (6)

by Egor R, 19 months ago

Attachment: dj_agg_bug_clean.zip added

zip of the MRE

comment:1 by Egor R, 19 months ago

Description: modified (diff)

comment:2 by David Sanders, 19 months ago

Hi,

Casting is required here, output_field just lets Django know what type the returned data is:

The output_field argument requires a model field instance, like IntegerField() or BooleanField(), into which Django will load the value after it’s retrieved from the database.

Ref: https://docs.djangoproject.com/en/4.2/ref/models/expressions/#aggregate-expressions

comment:3 by Egor R, 19 months ago

Thanks, got it. Not quite obvious, though.

Should I close the ticket?

comment:4 by Mariusz Felisiak, 19 months ago

Resolution: invalid
Status: newclosed

comment:5 by David Sanders, 19 months ago

Thanks, got it. Not quite obvious, though.
Should I close the ticket?

Looks like Mariusz beat me to it xD

If there's anything in the docs that you think could be improved we're always open to suggestions! Best place to start a conversation is on the Django Forum: https://code.djangoproject.com/wiki/DevelopersMailingList

Note: See TracTickets for help on using tickets.
Back to Top