Admin changelist view defaults to `DISTINCT`, unusable on reasonably sized databases

[Henrique C. Alves]

The default behavior on the admin is not usable for reasonably sized tables (a couple thousand tuples is enough for >1s queries).

Right now, it always includes a DISTINCT clause if you add a M2M field on list_filter, even if you're not applying any filters. Use of DISTINCT is known to be painfully slow, and even more so without a WHERE clause, because it forces SQL databases into a full table scan. The behavior is not easily overridden because the logic is scattered on a couple ChangeList methods (​https://github.com/django/django/blob/master/django/contrib/admin/views/main.py#L299).

The only solution right now is not including a M2M field on list_filter and limiting functionality. It would be better if that behavior could be overridden (a use_distinct parameter on get_query_set?) or, optimally, moving this logic somewhere else. Ticket #15819 already includes a patch for that.

[Travis Swicegood]

Resolving this as a dupe #15819. Looks like that issue has all of the same points here and fixes this issue.

Feel free to re-open if I missed something in this ticket that's different than #15819.

[AlexMalek]

I think it is a different issue.
It is possible to alter the check that examines list_filter to determine if distinct() is needed based on if the field appears in the query/request but this does not address the deeper issues in #15819
For example in Django 1.5.2 I think the following changes solves the issue:

django/contrib/admin/util.py:
20c20
< def lookup_needs_distinct(opts, lookup_path, query_dict=None):
---
> def lookup_needs_distinct(opts, lookup_path):
29,31c29
<          not field.field.unique) and
<        (query_dict is None or (field_name in ",".join(query_dict.keys())))
<       ):
---
>          not field.field.unique)):

django/contrib/admin/views/main.py:
128c128
<                                                           field_path, lookup_params))
---
>                                                           field_path))

[AlexMalek]

Proposed patch w/ tests:

​https://github.com/amalek215/django/compare/ticket_18729

[Tim Graham]

I left comments for improvement on the PR. Please uncheck "Patch needs improvement" when you update it, thanks.

[Tim Graham]

​PR

[Tim Graham <timograham@…>]

In 8f76673:

Fixed #18729 -- Made admin changelist not use distinct() if a list_filter doesn't require it.