Float-valued aggregations and annotations with filters fail with AttributeError

[Chuan-Zheng Lee]

When any float-valued aggregation or annotation (Avg, StdDev, Variance) is used with the filter= keyword argument, the following exception is raised:

AttributeError: 'WhereNode' object has no attribute 'output_field'

For example, these queries all fail with this error:

Speaker.objects.annotate(average=Avg('speakerscore__score', filter=Q(speakerscore__ghost=False)))
Speaker.objects.annotate(average=StdDev('speakerscore__score', filter=Q(speakerscore__ghost=False)))
Team.objects.annotate(average=Avg('debateteam__teamscore__score', filter=Q(debateteam__teamscore__forfeit=False)))

The error seems to be raised irrespective of what's in the database (e.g., it's raised even for an empty database). However, it doesn't affect aggregations like Sum, Max or Min that don't use NumericOutputFieldMixin. Also, queries that don't use the filter= keyword in the aggregation work fine.

The exception in question is raised from line 46 of django/db/models/functions/mixins.py, which looks for an output_field attribute of every element of self.get_source_expressions(), where self is the object containing NumericOutputFieldMixin, in this case Avg or some other subclass of Aggregate. But Aggregate.get_source_expressions() includes self.filter in its list, and (post-compilation) self.filter is a WhereNode, which doesn't have an output_field attribute.

This issue is new in version 2.2. Everything works fine in version 2.1 (where I believe NumericOutputFieldMixin didn't exist, or at least wasn't on the MRO for Avg, StdDev or Variance).

Minimal reproducible example

In a blank (or any) project, insert in models.py:

from django.db.models import Model, FloatField, BooleanField

class Book(Model):
    price = FloatField()
    fiction = BooleanField()

then run migrations and in python manage.py shell:

$ python manage.py shell
Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django22_output_field_bug.models import Book
>>> from django.db.models import Avg, Q
>>> Book.objects.annotate(average=Avg('price', filter=Q(fiction=True)))
Traceback (most recent call last):
[...]
  File "/[...]/django/db/models/functions/mixins.py", line 46, in <genexpr>
    if any(isinstance(s.output_field, DecimalField) for s in source_expressions):
AttributeError: 'WhereNode' object has no attribute 'output_field'
>>> Book.objects.aggregate(average=Avg('price', filter=Q(fiction=True)))
Traceback (most recent call last):
[...]
  File "/[...]/django/db/models/functions/mixins.py", line 46, in <genexpr>
    if any(isinstance(s.output_field, DecimalField) for s in source_expressions):
AttributeError: 'WhereNode' object has no attribute 'output_field'

[Chuan-Zheng Lee]

I couldn't tell you for sure because I haven't actually tried reproducing it on this commit or its parent, but if I had to take a guess as to which commit introduced the issue, I'd try a0b19a0f5b1731cf575546175034da53f5af5367, which introduced OutputFieldMixin, a precursor to NumericOutputFieldMixin with similar logic. I'll say something if I manage to give this a go. → see #comment:3

[Étienne Beaulé]

​PR

[Chuan-Zheng Lee]

Contrary to my earlier guess, the first commit with the issue is c690afb873cac8035a3cb3be7c597a5ff0e4b261, which modifies Avg to use the new logic. Its parent, 3d5e0f8394688d40036e27cfcfac295e6fe62269, works fine, because while it modified the mixin to its current form, it didn't touch Avg.

[Nick Pope]

Correct, this is a regression in ​c690afb which affects Avg(). StdDev() and Variance() are affected by ​e85afa5 and ​6d4efa8 respectively.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 4b6dfe1:

Fixed #30542 -- Fixed crash of numerical aggregations with filter.

Filters in annotations crashed when used with numerical-type
aggregations (i.e. Avg, StdDev, and Variance). This was caused as the
source expressions no not necessarily have an output_field (such as the
filter field), which lead to an AttributeError: 'WhereNode' object has
no attribute output_field.

Thanks to Chuan-Zheng Lee for the report.

Regression in c690afb873cac8035a3cb3be7c597a5ff0e4b261 and two following
commits.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 4e6f002:

[2.2.x] Fixed #30542 -- Fixed crash of numerical aggregations with filter.

Filters in annotations crashed when used with numerical-type
aggregations (i.e. Avg, StdDev, and Variance). This was caused as the
source expressions no not necessarily have an output_field (such as the
filter field), which lead to an AttributeError: 'WhereNode' object has
no attribute output_field.

Thanks to Chuan-Zheng Lee for the report.

Regression in c690afb873cac8035a3cb3be7c597a5ff0e4b261 and two following
commits.

Backport of 4b6dfe16226a81fea464ac5f77942f4d6ba266e8 from master.