Sitemap Index with Cache gives a NoReverseMatch exception

[JayKlehr]

I posted this in the Google Groups ( ​http://groups.google.com/group/django-users/browse_thread/thread/fc88c493b16623f/28bfe98a7b4f98c5 ) list but didn't get a response so I thought I'd create a ticket for it since it does seem like a bug to me.

Following the djangoproject.com Sitemap example ( source:djangoproject.com/django_website/urls.py ) and the documentation on Sitemaps ( ​http://www.djangoproject.com/documentation/sitemaps/ ) I'm trying to implement a caching Sitemap Index that will link out to other Sitemaps for my site (one for each main model basically). I had the Sitemap Index working with the separate sitemaps just fine until I decided I wanted to add caching in my urls.py (I am using the djangoproject.com urls.py as an example.)

Here's some snippets...

from django.conf.urls.defaults import *
from django.contrib.sitemaps import views as sitemap_views
from django.views.decorators.cache import cache_page

urlpatterns = patterns('',
    (r'^admin/', include('django.contrib.admin.urls')),
    (r'^sitemap.xml$', cache_page(sitemap_views.index, 60 * 60 * 24), {'sitemaps': my_sitemaps}),
    (r'^sitemap-(?P<section>.+).xml$', cache_page(sitemap_views.sitemap, 60 * 60 * 24), {'sitemaps': my_sitemaps}),
)

my_sitemaps is a dictionary of my sitemaps (built following the example in the Sitemap documentation). With this code I can access the individual sitemaps without trouble, but can't access the main sitemap.xml file (that's when I get the "NoReverseMatch at /sitemap.xml"
exception). However, the following code worked for all files:

from django.conf.urls.defaults import *

urlpatterns = patterns('',
    (r'^admin/', include('django.contrib.admin.urls')),
    (r'^sitemap.xml$', 'django.contrib.sitemaps.views.index', {'sitemaps': my_sitemaps}),
    (r'^sitemap-(?P<section>.+).xml$', 'django.contrib.sitemaps.views.sitemap, {'sitemaps': my_sitemaps}),
)

But I can't implement the cache using this method because you can't pass the string 'django.contrib.sitemaps.views.index' as the first parameter of cache_page since it's expecting a callable.

[Adrian Holovaty]

Duplicate of #2564.

[Chris Beaven]

Actually, that other ticket was user error, this ticket is the correct one.

[Chris Beaven]

So the problem here, is that the function is being decorated in the urlconf instead of the function being overridden in the view with the decorated version of itself.

The function != the decorated function, so of course it will give a NoReverseMatch. Unless we want to provide a way for decorators to pass through their decorated function to the reverse engine, I say this isn't a bug. But I'll let another developer look at this and decide that for sure.

[Chris Beaven]

Actually, thinking some more about this, this has been fixed by named url patterns (see [4901])

[stephanejais]

I can see how this can be considered fixed with named urls, although
I don't manage to have it work.

Here's my conf:

urlpatterns = patterns('',
    ...
    url(r'^sitemap.xml$', cache_page(sitemap_views.index, 86400), kwargs={'sitemaps': sitemaps}, name="django.contrib.sitemaps.views.index"),
    url(r'^sitemap-(?P<section>.+)\.xml$', cache_page(sitemap_views.sitemap, 86400), kwargs={'sitemaps': sitemaps}, name="django.contrib.sitemaps.views.sitemap"),
    
)  

Still getting the NoReverseMatch error.
The problem seems to be this line in the sitemap.views file:

sitemap_url = urlresolvers.reverse('django.contrib.sitemaps.views.sitemap', kwargs={'section': section})

until 'django.contrib.sitemaps.views.sitemap' is hardcoded there,
and since it resolves to a valid view function,
naming my sitemap pattern after the view name doesn't work.

If I change the name of my url pattern to "foobar", and the reverse call in the views file of the sitemap contrib like this:

sitemap_url = urlresolvers.reverse('foobar', kwargs={'section': section})

I get it to work.
Not very satisfying though.
Any suggestions ?

[Stavros Korokithakis]

I can confirm that this doesn't work. How about making a named view mandatory for the sitemap, such as "contrib.sitemap.index" and "contrib.sitemap.sitemap"?

[Stavros Korokithakis]

I have submitted a patch that checks for the normal view and falls back on a "contrib.sitemaps.sitemap" named view if there's an error. This fixes the problem and allows caching.

[Chris Beaven]

Yes, named urls didn't directly fix this ticket.

Stavros: hard coding an alternate url name isn't the best way to do it - here's a better patch (with docs). [side note: your patch also seemed to be back-the-front]

Even though contrib.sitemaps currently has no tests, some regression tests should probably be added showing this works. Want to have a go at these?

[Stavros Korokithakis]

That is indeed much better, for some reason I didn't think of it, even though I didn't like my solution myself.

My patch might have been off because I made it by hand (I've actually never tried to make one before).

Sure, I'll try to write some when I have some time (it should be soon). Thanks for the patch, I hope it gets implemented soon!

[Stavros Korokithakis]

Any idea where the regression tests should go? There are no preexisting tests.

Does this code even need tests? It's pretty straightforward (I'm pretty sure this will come back to bite me in the ass).

[Russell Keith-Magee]

Replying to stavros:

Any idea where the regression tests should go? There are no preexisting tests.

Then now is the time to write some.

Does this code even need tests? It's pretty straightforward (I'm pretty sure this will come back to bite me in the ass).

Yes, tests are required. Tests are not optional. You need to provide a very compelling reason that tests are either impractical or impossible before you get a pass on writing tests.

[Chris Beaven]

Replying to stavros:

Any idea where the regression tests should go?

Answering the initial question (Russ must have missed this after the second question threw him into a rage :)), put them in the contrib app itself. I'd suggest

/django/contrib/sitemaps/tests/__init__.py   <- put the test here
/django/contrib/sitemaps/tests/urls.py       <- your unit test will need a custom urlconf

At the bare minimum, just write some tests to prove the new behaviour works. But feel free to add additional tests of base sitemap functionality for extra brownie points.

[Russell Keith-Magee]

Replying to SmileyChris:

Replying to stavros:

Any idea where the regression tests should go?

Answering the initial question (Russ must have missed this after the second question threw him into a rage :)),

Hulk Angry! Crush patch with no tests! :-)

Apologies - you're right. I should have given some direction on test location.

Chris' feedback is pretty much on the money, with one modification. Rather than putting the tests directly into in init.py, init.py should be used as a staging module to load the tests from other locations. See contrib.auth for an example of the way to organize the tests.

[Stavros Korokithakis]

I tried and don't seem to be able to write the tests, as I don't know how to pass a request object or make reverse() work... Maybe someone can provide guidance on this?

[askfor]

Got the same problem here.
I was wondering why the cache_page decorator works for www.djangoproject.com according to the code on the trac (​http://code.djangoproject.com/browser/djangoproject.com/django_website/urls.py) ?
And there's also another problem. Since sitemap reverse the url by 'django.contrib.sitemaps.views.sitemap' , if there's a app also use sitemap, there will be a conflict.

[Stavros Korokithakis]

The patch fixes the issue, but it needs tests. Can anyone experienced write some so we can get this functionality?

[Jannis Leidel]

#13908 was a duplicate.

[Michael]

Updated patch + unittest

[Michael]

I've updated the patch to work with trunk and added one unittest

[Aymeric Augustin]

In [17408]:

Fixed #2713 -- Made the name of the sitemap view a parameter of the sitemap index view, in order to allow decorators. Also cleaned up code in the area and removed redundant comments from the tests.