admin::list_editable causes failure of reverse one to many lookup

[daniel oberski]

Python Version: 2.6.4
Django version: 1.1.1

Description

I have a model "Question" that has a foreign key referring to an "Item".

This means I can do, for example

    Item.objects.filter(question__in = some_list)

But as soon as I register a model admin with a list_editable for Item, the above line fails with a FieldError. When I comment the list_editable out, everything works as expected again.

Slightly more detailed description:

class Item(models.Model):
    name     = models.CharField(max_length=70)
    ...snip...

class ItemAdmin(admin.ModelAdmin):
    list_filter = ('study', )
    search_fields = ('name', 'admin' )
    list_display = ('name','admin','study','long_name','experiment', )
    list_editable = ('experiment','trait','method')

admin.site.register(Item, ItemAdmin) 

...snip...
class Question(models.Model):
    item     = models.ForeignKey(Item)
    language = models.ForeignKey(Language)
    ...snip...

Then in views.py:

lans = Language.objects.filter(coders = request.user)
...
items = Item.objects.filter(question__language__in = lans).distinct()

causes FieldError "Cannot resolve keyword 'question' into field. With the following traceback (line numbers are given after the colon)

/usr/local/lib/python2.6/dist-packages/Django-1.1.1-py2.6.egg/django/db/models/manager.py in filter:129

/usr/local/lib/python2.6/dist-packages/Django-1.1.1-py2.6.egg/django/db/models/query.py in filter:498

/usr/local/lib/python2.6/dist-packages/Django-1.1.1-py2.6.egg/django/db/models/query.py in _filter_or_exclude:516

/usr/local/lib/python2.6/dist-packages/Django-1.1.1-py2.6.egg/django/db/models/sql/query.py in add_q:1675

/usr/local/lib/python2.6/dist-packages/Django-1.1.1-py2.6.egg/django/db/models/sql/query.py in add_filter:1569

/usr/local/lib/python2.6/dist-packages/Django-1.1.1-py2.6.egg/django/db/models/sql/query.py in setup_joins:1737

[Karen Tracey]

Can't recreate with provided (incomplete) information. Mocking up apparently matching models filling in the blanks of fields/models that haven't been shown doesn't produce the described results:

(InteractiveConsole)
>>> from ttt.models import Item, Language
>>> from django.contrib.auth.models import User
>>> kmt = User.objects.get(username='kmt')
>>> lans = Language.objects.filter(coders=kmt)
>>> items = Item.objects.filter(question__language__in = lans).distinct()
>>> items
[<Item: Thing>]
>>>

Note the provided ItemAdmin is invalid and will raise an exception with DEBUG=True:

django.core.exceptions.ImproperlyConfigured: 'ItemAdmin.list_editable[1]' refers to 'trait' which is not defined in 'list_display'.

Turning DEBUG off allows that to pass without error, and even with that problem I do not see the results you describe, with either Django 1.1.1 or trunk r12433. Something that's been omitted from the description is apparently responsible for what you are seeing, but I have no idea what that is.

[daniel oberski]

OK, I created a clean new test project and app. The code below will give you a complete test case.

models.py looks like this:
(nothing left out this time)

from django.db import models
from django.contrib.auth.models import User
from django.contrib import admin


class Experiment(models.Model):
    name = models.CharField(max_length=70)

    def __unicode__(self):
      return self.name
admin.site.register(Experiment)


class Language(models.Model):
    name = models.CharField(max_length=100)
    coders = models.ManyToManyField(User)	
    def __unicode__(self):
      return self.name
    class Meta:
        ordering = ('name',)
class LanguageAdmin(admin.ModelAdmin):
    filter_horizontal = ('coders',)
admin.site.register(Language, LanguageAdmin)

   
class Item(models.Model):
    name     = models.CharField(max_length=70)
    admin    = models.CharField(max_length=10)
    long_name = models.CharField(max_length=70)
    admin_letter = models.CharField(max_length=1, blank=True, null=True)
    admin_number = models.IntegerField(blank=True, null=True)
    experiment = models.ForeignKey(Experiment, blank=True, null=True)

    def __unicode__(self):
        return self.name
    class Meta:
        ordering = ( 'admin_letter', 'admin_number', 'id')
class ItemAdmin(admin.ModelAdmin):
    search_fields = ('name', 'admin' )
    list_display = ('name','admin','long_name','experiment',)
# Bug in django.contrib.admin: causes failed keyword 'question' resolution for Item
    list_editable = ('experiment',)
admin.site.register(Item, ItemAdmin) 

 
class Question(models.Model):
    item     = models.ForeignKey(Item)
    language = models.ForeignKey(Language)
admin.site.register(Question)

Now if you go to the shell and do

from testproj.testapp.models import *
u = User.objects.all()[0]
lans = Language.objects.filter(coders = u)
Item.objects.filter(question__language__in = lans)

You get

FieldError: Cannot resolve keyword 'question' into field. Choices are: admin, admin_letter, admin_number, experiment, id, long_name, name

However, when I remove the line

    list_editable = ('experiment',)

From models.py then I get

In [4]: Item.objects.filter(question__language__in = lans)
Out[4]: [<Item: asdad>]

Just like you do.

I hope you will be able to reproduce the problem this time.

[Karen Tracey]

OK, now this looks like #11448.

Note putting that bug aside, you should not be putting admin definitions and registrations in your models.py file. In a production server setup it's quite possible your models file won't be loaded as early as it is with the dev server. A consequence of delayed loading, if you have your admin definitions there, will be that your admin site doesn't have all the models registered that you expect it to. The fix is to put all your admin definitions and registrations in an admin.py file and include a call to admin.autodiscover() in urls.py. A side-effect of following that practice will be that you won't encounter this bug.

[daniel oberski]

Replying to kmtracey:

OK, now this looks like #11448.

I believe you. However, that ticket says

I retract this ticket, I just found that this creates issues elsewhere as well, which cannot be solved this easily. A warning in the docs about not doing models.py-level queries or mixing forms and models would be appreciated though.

But as far as I am able to tell as a user, I am not doing any queries in models.py.

Note putting that bug aside, you should not be putting admin definitions and registrations in your models.py file. In a production server setup it's quite possible your models file won't be loaded as early as it is with the dev server. A consequence of delayed loading, if you have your admin definitions there, will be that your admin site doesn't have all the models registered that you expect it to. The fix is to put all your admin definitions and registrations in an admin.py file and include a call to admin.autodiscover() in urls.py. A side-effect of following that practice will be that you won't encounter this bug.

Thanks, I never got around to changing that since this is the new recommendation. Now I followed your advice.

After I move all admin related things to admin.py, the testcase I gave indeed works, but the bigger application I have still does not. Even though there is no other models.py. Presumably, somewhere, something happens (unrelated to the whole admin.py business) that triggers the bug.

So the current status of this bug for me is still that there is no fix and nobody is planning to make one...

[Karen Tracey]

The last comment in #11448 is not a particularly helpful one. The commenter did not say what exactly these additional problems are, nor if they were introduced by the proposed fix or merely other issues with the code that triggered the "cannot resolve" error. Without such specifics it seems premature to say the identified problem isn't worth fixing. And no one has made that decision -- the ticket is still open, despite that comment.

If moving admin defs out of models.py did not fix the problem for your larger project, then there's likely something else triggering the same situation in your larger project's models.py. There's another ticket linked to that one, that was closed as a dupe, showing another way to trigger the same error. It too could be solved by reordering things so that all models and thus all inter-model relationships were defined before any attempt to use the models.

So far as I recall no instance of this particular problem has been identified that could not be solved by simply changing the order of model/form definitions in the application. I'd expect, then, that you could do the same by some careful evaluation of what exactly is in your models.py file and moving code around to avoid using models before all inter-model relationships are defined. If in fact you have a case where more careful ordering of code can't fix the problem, that would be interesting to know, and could increase the priority of getting a fix for #11448 in. As it is now it's a problem with a known easy workaround and a vague reference to other additional problems that aren't easily fixed, so it's not exactly at the top of the priority list for fixing.

(BTW admin.autodiscover() was added prior to Django 1.0, which was released nearly 1.5 years ago. I would not have thought it still qualified as "new".)