Opened 12 years ago

Closed 12 years ago

Last modified 12 years ago

#19602 closed Bug (worksforme)

Select_related filters objects with no related member

Reported by: natmaster@… Owned by: Aymeric Augustin
Component: Database layer (models, ORM) Version: 1.5-beta-1
Severity: Release blocker Keywords: select_related
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

In Django <=1.4 if you used select_related and the related object doesn't exist, it would be filled in as None, but the original object would still show up in the results.

In Django 1.5, not having a related object when using select_related results in the original object not existing when calling get() or indexing it.

Note that the SQL is still the proper LEFT OUTER JOIN. Furthermore, a count() will result in the proper expected number of objects.

Change History (11)

comment:1 by Russell Keith-Magee, 12 years ago

Resolution: worksforme
Severity: NormalRelease blocker
Status: newclosed
Triage Stage: UnreviewedAccepted

This is a concerning report, since it describes a backwards incompatibility. However, I can't reproduce the problem you describe.

Using:

class OwnedObject(models.Model):
    owner = models.ForeignKey(User, null=True, blank=True)
    name = models.CharField(max_length=100)

and

>>> user1 = MyUser.objects.get(email='user1@example.com')
>>> user2 = MyUser.objects.get(email='user2@example.com')
>>> obj1 = OwnedObject.objects.create(name='obj1', owner=user1)
>>> obj2 = OwnedObject.objects.create(name='obj2', owner=user2)
>>> obj3 = OwnedObject.objects.create(name='obj2')
>>> OwnedObject.objects.select_related('owner')

the last statement returns 3 objects, as expected, even though obj3.owner doesn't exist. This is using git revision ba50d3e05b.

Closing as worksforme; please reopen if you can provide specific instructions for reproducing the problem you describe.

comment:2 by ntucker, 12 years ago

Isn't ba50d3e05b the 1.6.x line? AKA from the master branch?

Also, are you sure you inspected the object directly, and not just executed count()

comment:3 by ntucker, 12 years ago

I think this might have been my mistake, let me investigate.

comment:4 by ntucker, 12 years ago

Resolution: worksformeinvalid

It appears I was confusing this new behavior

https://github.com/django/django/blob/stable/1.5.x/django/db/models/fields/related.py
line 278

if rel_obj is None:

raise self.related.model.DoesNotExist

With the object not being retrieved, because I was accessing the related object after getting the first object.

This doesn't appear to be documented here https://docs.djangoproject.com/en/dev/releases/1.5/#backwards-incompatible-changes-in-1-5 though, so that might need to change.

I am curious as to why this change even took place. None representing null makes more sense to me.

comment:5 by ntucker, 12 years ago

Actually, this might be a bug still...because the behavior is different when select_related isn't used.

Tell me if you expect this behavior:
At the end of your example add

obj = OwnedObject.objects.select_related('owner').get(name=obj2, owner__isnull=True)
print(obj.owner)

That should throw DoesNotExist, which it does not do if you remove the select_related. Is this expected (and new in 1.5) behavior?

comment:6 by Aymeric Augustin, 12 years ago

Resolution: invalid
Status: closednew

This ticket revolves around my fixes in the related object caching code.

I fixed a bunch of cases where adding select/prefetch_related didn't eliminate subsequent queries (specifically when there was no reverse related object) or changed the behavior (return None instead of DoesNotExist).

If I understand correctly the comment above there's still an instance of the second class of problems.

I'll take a look when I have access to a computer.

comment:7 by Aymeric Augustin, 12 years ago

Owner: changed from nobody to Aymeric Augustin
Status: newassigned

comment:8 by Aymeric Augustin, 12 years ago

Resolution: worksforme
Status: assignedclosed

When a nullable foreign key is null, accessing the related object returns None. That's what you're doing in comment 5.

However, when you traverse a one-to-one relation backwards (eg. replace ForeignKey by OneToOneField in the models above and do user.ownedobject), a DoesNotExist exception is raised if no matching parent object is found. OneToOneField is asymmetrical in SQL and Django's APIs reflect it: the lack of a related object is expressed by None in the "forwards" direction and by DoesNotExist is the "backwards" direction.

In comment 4, you're quoting the code of SingleRelatedObjectDescriptor, which implements reverse lookup through one-to-one relations. (I know from experience it's very hard to follow which side of the relation the code is dealing with in this area.)

Before Django 1.5, adding select_related could change the behavior: see #13839. You were relying on this bug. By definition, bugfixes are backwards-incompatible — they change the behavior of Django — but we don't document them all in the release notes.

However, if another core dev thinks this one is sufficiently important to make an exception, I'll write a paragraph...

comment:9 by ntucker, 12 years ago

If it is supposed to be throwing DoesNotExist in this case, shouldn't it do the same thing regardless of whether you use select_related or not? i.e., shouldn't it throw that exception when you don't use select_related and try to access the same relationship from that side?

in reply to:  9 comment:10 by Aymeric Augustin, 12 years ago

Replying to ntucker:

If it is supposed to be throwing DoesNotExist in this case, shouldn't it do the same thing regardless of whether you use select_related or not?

Yes, the behavior must be the same with and without select_related in all cases (except for the number of queries).

I have reviewed your comments carefully and haven't found any evidence of a different behavior with and without select_related, on 1.5rc1 or master. If you see a different behavior, could you help me reproduce it:

  • either by writing a test case (examples),
  • or by pasting a definition of models and a shell session exhibiting the problem, like Russell did in the first comment?

Thanks!

comment:11 by ntucker, 12 years ago

Nvm, you're right. Thanks for looking at this! :)

Note: See TracTickets for help on using tickets.
Back to Top