Queryset doesn't have a "not equal" filter operator

[jdetaeye]

Querysets can be filter for 'gt', 'gte', 'lt', 'lte', 'exact'.
The ability to filter for inequality / 'ne' is not in the list.

Using 'exclude' it is possible to negate a filter, but it's not as clear and clean as a direct filter. Including an inequality operator isn't hard and difficult, I believe.

In case this feature is accepted to be included in Django, feel free to assign it to me: I'm willing to take on creating the patch, its documentation and testing.

[jdetaeye]

Along the same lines, an 'isnotnull' operator could also come in handy...

[durdinator]

@jdetaeye: Doesn't field__isnull=False fulfil that role?

[jdetaeye]

@durdinator: it sure does. I was too quick with my additional comment...

[Dan Watson]

Along the same lines, if this enhancement is accepted (and I hope it is), there should probably be a "is distinct from" operator to do null-safe comparisons.

[James Bennett]

This is pretty much what exclude is for; there used to be an ne lookup type pre-magic-removal, but then exclude was introduced and covered both the original use case for ne plus many others. In the spirit of not needlessly offering multiple ways to do the same thing, I'm going to wontfix this.

[Greg Brown]

Unfortunately, while exclude is supposed to do everything a "not equal" operator would, it doesn't in practice - see ​http://code.djangoproject.com/ticket/5763 for details. This is technically a problem with exclude(), but adding a "not equal" operator would negate the problem.

[Greg Brown]

Sorry, I've just realised I can negate my problem in ​http://code.djangoproject.com/ticket/5763 using isnull instead, so ignore the above comment.

[Jacques Mattheij <j@…>]

In case somebody else runs in to this, non-emtpy text fields evaluate to > a blank field.

So you can do xxx_gt= to exclude rows with blank fields.

[Adrian Holovaty]

I'm reopening this. See ​http://groups.google.com/group/django-developers/browse_thread/thread/78767bab3f753ef0

[anonymous]

I would love to see this. Case in point:

        return self.filter(
            group__sites=Site.objects.get_current(),
            user=user
        ).exclude(
            group__status=GROUP_REMOVED,
            status=MEMBER_STATUS_BANNED,
            subscription=SUBSCRIPTION_NEVER
        )

Generates:

   SELECT ... 
   INNER JOIN ...
   WHERE (
      "groups_group_sites"."site_id" = 1  AND 
      "groups_membership"."user_id" = 17  AND NOT (
         "groups_membership"."status" = 99  AND 
         "groups_group"."status" = 3  AND (
            "groups_membership"."subscription" = 0  AND 
            "groups_membership"."subscription" IS NOT NULL
         )
      )
   ) ORDER BY "groups_membership"."status" DESC

Would like to see instead:

"groups_group_sites"."site_id" = 1  AND 
"groups_membership"."user_id" = 17  AND 
"groups_membership"."status" != 99  AND 
"groups_group"."status" != 3  AND 
"groups_membership"."subscription" != 0  AND 
"groups_membership"."subscription" IS NOT NULL

[Carl Meyer]

Based on discussion with Alex and Andrew at the sprint, closing this wontfix on the basis that it doesn't make sense to have __ne without any other negated queries, when we already have .exclude() to express negatives.

Alex promises that he really will fix the SQL generation to not be stupid with .exclude().

[asmoore82]

Apologies for the re-open - but I wasn't sure what would be worse, a re-open or a new ticket for the same old discussion...

I'm going to have to take the position that the absence of a __ne operator still
represents a functional hole in the querying API, even with consideration of exclude()

it doesn't make sense to have __ne without any other negated queries,

Ah but there are indeed other negated queries.

filter(__gte) and exclude(__lt) are almost equivalent.

Similarly, you can almost approximate __ne with a hokey Q(__lt) | Q(__gt) contraption.

But the real issue lies within that "almost" -- this is what could be cause for a separate ticket, but a quick and dirty __ne would head the issue off altogether ;). Chaining multiple filter() and exclude() calls on multi-valued relationships yields not-so-surprising but nonetheless undesired results. To quote from the Django docs:

Django has a consistent way of processing filter() and exclude() calls. Everything inside a single filter() call is applied simultaneously to filter out items matching all those requirements. Successive filter() calls further restrict the set of objects, but for multi-valued relations, they apply to any object linked to the primary model, not necessarily those objects that were selected by an earlier filter() call.

This consistency is a good thing but it combines with the lack of __ne to form a problem.

Say you have blogs with entries with tags and author_counts

If you want to get all entries that are tagged 'django' with more than 2 co-authors:

Entry.objects.filter(tag__name='django', author_count__gt=2)

It is similarly easy to get blogs with entries with the above criteria:

Blog.objects.filter(entry__tag__name='django', entry__author_count__gt=2)

But what if you want entries tagged 'django' that are not co-authored by 2:

Entry.objects.filter(tag__name='django').exclude(author_count=2)

But if you want the blogs with those entries, it can't easily be done:

Blog.objects.filter(entry__tag__name='django').exclude(entry__author_count=2)

is not the equivalent of the imaginary query:

Blog.objects.filter(entry__tag__name='django', entry__author_count__ne=2)

which can currently be approximated with:

Blog.objects.filter(Q(entry__author_count__lt=2) | Q(entry__author_count__gt=2), entry__tag__name='django')

You can also get the desired results with a .extra() call but let's not go there :P.

This brings us to the most surprising aspect, using the negation operator ~ on Q() objects that select on multi-valued relations is technically possible but can yield the undesired results on the unsuspecting, which actually runs sort of contrary to the Documentation quote above.

Bad Surprise!!:

Blog.objects.filter(~Q(entry__author_count=2), entry__tag__name='django')

^The more I look at this, the more I think it is cause for a ticket in its own right. The fix for this would be to smarten up the Q() objects so that a NOT operator on __gt becomes __lte, a NOT on __lt becomes __gte, and so on. But you will ultimately be missing the __ne and other NOT primitives to fall back on.

In other words, this ticket is a "could-go-either-way" blocker for ^that more important ticket. I'll do some research on that and make a ticket if it hasn't already been addressed.

Thanks to all who make Django awesome! I'm a database newbie and loving it!

~Adam sM

[Chris Beaven]

Thanks for your thoughts, Adam, but reopening a ticket closed by a core developer is a no-no (as mentioned in the ​contributing documentation).

Try bringing this up in the django-developers mailing list.

[German M. Bravo]

This adds ne and notin filters

[German M. Bravo]

django 1.4 compatible patch

[PhiR_42]

patch updated for 1.5, plus handles negative contains and icontains

[PhiR_42]

Just for the record, this is very important for a specific use case: when you want to add extra select columns from the joined tables. I have an ORM-based search form and using exclude will break in many cases because it uses more subqueries. Using filter(relatedfieldne = something) will work as expected while exclude(relatedfield = something) will generate a subquery and remove the possibility of selecting tables.

[Marc Tamlyn]

It is worth noting that in 1.7 this can be written as a custom lookup if you need it. I think it might even be the example in the docs.

[Anssi Kääriäinen]

Be warned - not using subqueries might actually produce incorrect results. Specifically, .exclude() guarantees that you get the complement of what you get with .filter(). There are cases where field__ne will not produce the complement of field__exact. This happens for example if the field has NULL values. Both __exact and __ne will miss such rows, as in SQL terms both 'val = NULL' and 'val != NULL' have unknown result.

As mjtamlyn noted a __ne lookup will be fairly straightforward to implement in 1.7. Check ​https://docs.djangoproject.com/en/dev/ref/models/custom-lookups/ for details.

[PhiR_42]

Thanks for that clarification.
In my case it seems the patch mitigates #14645 though, so I will probably take my chances in production.

[Amichai Schreiber]

Since this is an issue that keeps coming up for me, and I'm seeing it in discussions on the web as well as on this ticket, and its usefulness has been demonstrated by asmoore82 (and I can give more examples) --

is there an explanation why this shouldn't be implemented? Is there some consideration by which this ticket isn't a good idea?

[Tim Graham]

Please first read through the ticket history as well as the mailing list threads linked from the the ticket. If you disagree with the conclusions that have been reached so far, you are welcome to argue your case on the DevelopersMailingList.

[cpbotha]

Is there a way to work around the absence of the ne operator when one wants to do a get_or_create() or an update_or_create() where one wants to select records with a non-blank text field value?

In other words, how would I do for example Blog.objects.update_or_create(text_contents__ne='', other_field_indicating_sort=42, defaults={yet_another_field: True}) ?

[Simon Charette]

You should be able to use exclude for that.

Blog.objects.exclude(
    text_contents='',
).update_or_create(
    other_field_indicating_sort=42,
    defaults={yet_another_field: True},
)

[dennisvang]

The "... spirit of not needlessly offering multiple ways to do the same thing ..." makes a lot of sense to me.

However, there's this question on StackOverflow:

​How do I do a not equal in Django queryset filtering?

Current stats:

• 555246 views
• 859 upvotes
• bookmarked 179 times

Just to be clear, this is a really high view count: top 0.02% (top 5500 out of 23 million questions, according to ​https://data.stackexchange.com/stackoverflow)

Apparently, the alternatives to an __ne lookup are not that obvious to many people.

Perhaps it's worth re-opening this issue and, for example, implementing the ne from the ​custom lookup example?