Filtering with Q and OR gets duplicated entries

[Ismael Jerez]

Hi:

Django version: 3.2.4
Python version: 3.8

I cannot reproduce this bug in a simple example, so I can only show you screenshots from my Pycharm Debug Tool, I am sorry.

I have one queryset with 3 objects which contains annotations, CharField and ManyToManyField attributes. Then I filter by a query using Q like this:

(OR: ('created_date_dt_custom__icontains', 'a'), ('id__icontains', 'a'), ('title__unaccent__icontains', 'a'), ('created_date_dt_custom__icontains', 'a'), ('id__icontains', 'a'), ('title__unaccent__icontains', 'a'), ('promoters__in', <QuerySet [<User: admin>, <User: u********>]>))

And gets a queryset with 4 objects as a result, one of the objects is duplicated. If I delete the last filter ('promotersin', <QuerySet [<User: admin>, <User: u********>]>) it works fine, so the problem is on this filter or maybe the concatenation of filters with this kind.

I have no problem on Django version ~= 2.

Here are the screenshots of my debug tool showing the problem:

• Screenshot of the code: ​https://i.ibb.co/Fnb53rn/ss1.png
• Screenshot of debug tool with variable values (Project object (19) duplicated): ​https://i.ibb.co/b34FwSM/ss2.png

Thank you in advance, hope it is fixed soon,
Ismael.

[Mariusz Felisiak]

Thanks for this report, unfortunately we will not be able to reproduce the issue or confirm that it's a regression without a queryset. It looks like a regression in c8b659430556dca0b2fe27cf2ea0f8290dbafecd but again it's hard to confirm without a queryset.

Can you provide models and filters? Can you confirm that it's a regression in 3.2.1?

[Ismael Jerez]

Hi again:

I am sorry for not being able to reproduce this before. Here is a simple example:

1) models.py:

from django.contrib.auth.models import User
from django.db import models


class Example(models.Model):
    example_attr = models.CharField(max_length=100)
    users = models.ManyToManyField(User)

2) I created two User's instances from admin site:

• username = 'admin'
• username = 'other'

And two Group's instances from admin site:

• name = 'Group1'
• name = 'Group2'

Associate Group1 and Group2 to users 'admin' and 'other' from admin site too.

3) Now the code:

from example.models import Example
from django.db.models import Q
from django.contrib.auth.models import User, Group

example = Example.objects.create(example_attr='have a nice day')
user1 = User.objects.get(username='admin')
user2 = User.objects.get(username='other')
example.users.add(user1, user2)
filters = Q(example_attr__icontains='a') | Q(users__in=User.objects.filter(groups__in=[Group.objects.get(name='Group1').pk]))
print(Example.objects.count()) # This prints "1"
print(Example.objects.filter(filters).count()) # This prints "2" 

Hope this helps to reproduce the bug.

This bug is happening since Django 3.2.0 until 3.2.4.

Thanks in advance,
Ismael.

[Ismael Jerez]

Forgot to mention that the bug is only happening when the second Q filter is used but not with simple 'icontains' filters concatenated by OR expressions.

[Mariusz Felisiak]

Thanks for extra details, however the described behavior is expected, documented, and has not been changed in Django 3.2. Django < 3.2 behaves the same.

Please reopen the ticket if you can debug your issue and provide way to reproduce a regression in Django 3.2