Select_related filters objects with no related member

[natmaster@…]

In Django <=1.4 if you used select_related and the related object doesn't exist, it would be filled in as None, but the original object would still show up in the results.

In Django 1.5, not having a related object when using select_related results in the original object not existing when calling get() or indexing it.

Note that the SQL is still the proper LEFT OUTER JOIN. Furthermore, a count() will result in the proper expected number of objects.

[Russell Keith-Magee]

This is a concerning report, since it describes a backwards incompatibility. However, I can't reproduce the problem you describe.

Using:

class OwnedObject(models.Model):
    owner = models.ForeignKey(User, null=True, blank=True)
    name = models.CharField(max_length=100)

and

>>> user1 = MyUser.objects.get(email='user1@example.com')
>>> user2 = MyUser.objects.get(email='user2@example.com')
>>> obj1 = OwnedObject.objects.create(name='obj1', owner=user1)
>>> obj2 = OwnedObject.objects.create(name='obj2', owner=user2)
>>> obj3 = OwnedObject.objects.create(name='obj2')
>>> OwnedObject.objects.select_related('owner')

the last statement returns 3 objects, as expected, even though obj3.owner doesn't exist. This is using git revision ba50d3e05b.

Closing as worksforme; please reopen if you can provide specific instructions for reproducing the problem you describe.

[ntucker]

Isn't ba50d3e05b the 1.6.x line? AKA from the master branch?

Also, are you sure you inspected the object directly, and not just executed count()

[ntucker]

I think this might have been my mistake, let me investigate.

[ntucker]

It appears I was confusing this new behavior

​https://github.com/django/django/blob/stable/1.5.x/django/db/models/fields/related.py
line 278

if rel_obj is None:

raise self.related.model.DoesNotExist

With the object not being retrieved, because I was accessing the related object after getting the first object.

This doesn't appear to be documented here ​https://docs.djangoproject.com/en/dev/releases/1.5/#backwards-incompatible-changes-in-1-5 though, so that might need to change.

I am curious as to why this change even took place. None representing null makes more sense to me.

[ntucker]

Actually, this might be a bug still...because the behavior is different when select_related isn't used.

Tell me if you expect this behavior:
At the end of your example add

obj = OwnedObject.objects.select_related('owner').get(name=obj2, owner__isnull=True)
print(obj.owner)

That should throw DoesNotExist, which it does not do if you remove the select_related. Is this expected (and new in 1.5) behavior?

[Aymeric Augustin]

This ticket revolves around my fixes in the related object caching code.

I fixed a bunch of cases where adding select/prefetch_related didn't eliminate subsequent queries (specifically when there was no reverse related object) or changed the behavior (return None instead of DoesNotExist).

If I understand correctly the comment above there's still an instance of the second class of problems.

I'll take a look when I have access to a computer.

[Aymeric Augustin]

When a nullable foreign key is null, accessing the related object returns None. That's what you're doing in comment 5.

However, when you traverse a one-to-one relation backwards (eg. replace ForeignKey by OneToOneField in the models above and do user.ownedobject), a DoesNotExist exception is raised if no matching parent object is found. OneToOneField is asymmetrical in SQL and Django's APIs reflect it: the lack of a related object is expressed by None in the "forwards" direction and by DoesNotExist is the "backwards" direction.

In comment 4, you're quoting the code of SingleRelatedObjectDescriptor, which implements reverse lookup through one-to-one relations. (I know from experience it's very hard to follow which side of the relation the code is dealing with in this area.)

Before Django 1.5, adding select_related could change the behavior: see #13839. You were relying on this bug. By definition, bugfixes are backwards-incompatible — they change the behavior of Django — but we don't document them all in the release notes.

However, if another core dev thinks this one is sufficiently important to make an exception, I'll write a paragraph...

[ntucker]

If it is supposed to be throwing DoesNotExist in this case, shouldn't it do the same thing regardless of whether you use select_related or not? i.e., shouldn't it throw that exception when you don't use select_related and try to access the same relationship from that side?

[Aymeric Augustin]

Replying to ntucker:

If it is supposed to be throwing DoesNotExist in this case, shouldn't it do the same thing regardless of whether you use select_related or not?

Yes, the behavior must be the same with and without select_related in all cases (except for the number of queries).

I have reviewed your comments carefully and haven't found any evidence of a different behavior with and without select_related, on 1.5rc1 or master. If you see a different behavior, could you help me reproduce it:

• either by writing a test case (​examples),
• or by pasting a definition of models and a shell session exhibiting the problem, like Russell did in the first comment?

Thanks!

[ntucker]

Nvm, you're right. Thanks for looking at this! :)