Opened 4 years ago
Closed 4 years ago
#32663 closed Cleanup/optimization (wontfix)
Remove Error raising on annotation & distinct call
Reported by: | Yovel Cohen | Owned by: | nobody |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | 3.2 |
Severity: | Normal | Keywords: | SQLCompiler ORM Query |
Cc: | Triage Stage: | Unreviewed | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
if you try to call distinct on an annotated query, it sometimes works, sometimes get's ignore, and sometimes raises an error.
I had a query facing all three scenarios and solving the last one was the one that worked out.
the error being raised is:
NotImplementedError: annotate() + distinct(fields) is not implemented.
after looking at the source code for the exception at django.db.models.sql.compiler.SQLCompiler
row 594
if grouping: if distinct_fields: raise NotImplementedError('annotate() + distinct(fields) is not implemented.') order_by = order_by or self.connection.ops.force_no_ordering() result.append('GROUP BY %s' % ', '.join(grouping)) # rest of the as_sql() method
after just removing the distinct_fields condition:
if grouping: order_by = order_by or self.connection.ops.force_no_ordering() result.append('GROUP BY %s' % ', '.join(grouping)) # rest of the as_sql() method
it works, at least in the following ways I tried (annotations are just made up for sake of the example)
model_scores_latest_date_annotation = Max('model_scores__date') latest_score_annotation = Case(When(model_scores__date=F('latest_date'), then='model_scores__score') base_query_set = base_query_set = (Model.objects. filter(**filters). alias(latest_date=model_scores_latest_date_annotation). values(ID). annotate(latest_score=latest_score_annotation, latest_date=model_scores_latest_date_annotation) )
all the following distinct calls worked:
query_set = base_query_set.order_by('latest_date').distinct('id', 'latest_date') query_set = base_query_set.distinct('id') query_set = base_query_set.distinct('id', 'latest_date')
which makes me think that the as_sql method on SQLCompiler can handle more cases easily and this just fell beneath the cracks.
Change History (5)
comment:1 by , 4 years ago
Description: | modified (diff) |
---|
comment:2 by , 4 years ago
Description: | modified (diff) |
---|
comment:3 by , 4 years ago
Description: | modified (diff) |
---|
There's a valid reason why this error is raised as explained in these comment https://code.djangoproject.com/ticket/6422#comment:59; it's less about the validity of the generated SQL and more about the correctness of the returned results.
Given the rare cases where mixing aggregation and distinct is necessary and the complexity involved in getting it right I'm of opinion that we should keep disallowing it.