Opened 3 years ago

Closed 3 years ago

#32809 closed Bug (needsinfo)

Filtering with Q and OR gets duplicated entries

Reported by: Ismael Jerez Owned by: nobody
Component: Database layer (models, ORM) Version: 3.2
Severity: Normal Keywords: queryset, Q, filtering, OR, annotation, |
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Hi:

Django version: 3.2.4
Python version: 3.8

I cannot reproduce this bug in a simple example, so I can only show you screenshots from my Pycharm Debug Tool, I am sorry.

I have one queryset with 3 objects which contains annotations, CharField and ManyToManyField attributes. Then I filter by a query using Q like this:

(OR: ('created_date_dt_custom__icontains', 'a'), ('id__icontains', 'a'), ('title__unaccent__icontains', 'a'), ('created_date_dt_custom__icontains', 'a'), ('id__icontains', 'a'), ('title__unaccent__icontains', 'a'), ('promoters__in', <QuerySet [<User: admin>, <User: u********>]>))

And gets a queryset with 4 objects as a result, one of the objects is duplicated. If I delete the last filter ('promotersin', <QuerySet [<User: admin>, <User: u********>]>) it works fine, so the problem is on this filter or maybe the concatenation of filters with this kind.

I have no problem on Django version ~= 2.

Here are the screenshots of my debug tool showing the problem:

Thank you in advance, hope it is fixed soon,
Ismael.

Attachments (2)

ss1.png (13.6 KB ) - added by Ismael Jerez 3 years ago.
ss2.png (14.7 KB ) - added by Ismael Jerez 3 years ago.

Download all attachments as: .zip

Change History (6)

by Ismael Jerez, 3 years ago

Attachment: ss1.png added

by Ismael Jerez, 3 years ago

Attachment: ss2.png added

comment:1 by Mariusz Felisiak, 3 years ago

Resolution: needsinfo
Status: newclosed

Thanks for this report, unfortunately we will not be able to reproduce the issue or confirm that it's a regression without a queryset. It looks like a regression in c8b659430556dca0b2fe27cf2ea0f8290dbafecd but again it's hard to confirm without a queryset.

Can you provide models and filters? Can you confirm that it's a regression in 3.2.1?

comment:2 by Ismael Jerez, 3 years ago

Resolution: needsinfo
Status: closednew

Hi again:

I am sorry for not being able to reproduce this before. Here is a simple example:

1) models.py:

from django.contrib.auth.models import User
from django.db import models


class Example(models.Model):
    example_attr = models.CharField(max_length=100)
    users = models.ManyToManyField(User)

2) I created two User's instances from admin site:

  • username = 'admin'
  • username = 'other'

And two Group's instances from admin site:

  • name = 'Group1'
  • name = 'Group2'

Associate Group1 and Group2 to users 'admin' and 'other' from admin site too.

3) Now the code:

from example.models import Example
from django.db.models import Q
from django.contrib.auth.models import User, Group

example = Example.objects.create(example_attr='have a nice day')
user1 = User.objects.get(username='admin')
user2 = User.objects.get(username='other')
example.users.add(user1, user2)
filters = Q(example_attr__icontains='a') | Q(users__in=User.objects.filter(groups__in=[Group.objects.get(name='Group1').pk]))
print(Example.objects.count()) # This prints "1"
print(Example.objects.filter(filters).count()) # This prints "2" 

Hope this helps to reproduce the bug.

This bug is happening since Django 3.2.0 until 3.2.4.

Thanks in advance,
Ismael.

comment:3 by Ismael Jerez, 3 years ago

Forgot to mention that the bug is only happening when the second Q filter is used but not with simple 'icontains' filters concatenated by OR expressions.

Version 0, edited 3 years ago by Ismael Jerez (next)

comment:4 by Mariusz Felisiak, 3 years ago

Resolution: needsinfo
Status: newclosed

Thanks for extra details, however the described behavior is expected, documented, and has not been changed in Django 3.2. Django < 3.2 behaves the same.

Please reopen the ticket if you can debug your issue and provide way to reproduce a regression in Django 3.2

Note: See TracTickets for help on using tickets.
Back to Top