WSGIRequest should set request.path to full uri path

[rmunn@…]

Following the instructions at ​http://wiki.dreamhost.com/index.php/Django, I got Django running under FastCGI. But the login form points to the wrong URL: instead of pointing to ​http://django.example.com/django-admin.fcgi/admin/, it points to ​http://django.example.com/admin/.

The reason is found in django/core/handlers/wsgi.py, where the WSGIRequest class's __init__() sets self.path equal to environ['PATH_INFO']. The request.path value eventually makes its way to the login template, which uses "app_path" as the form submission URL. In a CGI environment (including FastCGI), PATH_INFO contains the path referenced *after* the script name (here, "/admin/"). SCRIPT_NAME contains the path to the script (here, "/django-admin.fcgi". To construct a proper self-referential URL, you need to concatenate SCRIPT_NAME and PATH_INFO to obtain, in this case, "/django-admin.fcgi/admin/".

[hugo <gb@…>]

I think that's more a bug in the FCGI description. It needs a rewrite rule that rewrites the URLs matching /admin/ to /django-admin.fcgi/admin/ so that the right PATH_INFO is passed in. You might have more luck with ​lighttpd+FCGI or ​Apache+FCGI, depending on the server you use.

[rmunn@…]

I was looking through the code, and there are a lot of places that assume that request.path describes the entire URL. Changing that to a concatenation of SCRIPT_NAME and PATH_INFO would be quite tricky, because you'd need to distinguish between URL's destined for URL pattern matching (where you'd want to match against the PATH_INFO component only), and URL's destined for output in HTML (where you'd want to use the concatenation of SCRIPT_NAME and PATH_INFO). I'm starting to think that the complexity of the fix is not worth it when, as Hugo said, you can solve this one with a simple rewrite rule at the server level.

[rmunn@…]

Closing ticket as "wontfix" since I've concluded that this should be solved with a rewrite rule at the server level.

[Thomas Güttler]

According to ​http://www.ietf.org/rfc/rfc3875 (See 4.1.6.)

PATH_INFO of ​http://django.example.com/django-admin.fcgi/admin/ should bei /admin/

It is sad, that django does not handle his according to the RFC. If you cannot add a rewrite
rule, you can modify the PATH_INFO variable to include the path to the script. I did
this for django/core/servers/cgi.py (from Trac ticket). You need to modify your url.py, too.

[Chris Beaven]

Well if the RFC says it, we should be doing it.

It's not difficult. Thomas' suggestion is close, but rather than modify PATH_INFO, it's better to just change request.path to concatenate SCRIPT_NAME and PATH_INFO. This way, the Django url resolver can still (with a small tweak) use PATH_INFO for it's resolving so nothing breaks.

Related tickets: #1516, #2407

[Chris Beaven]

Note: in debugging this, I noticed that ModPythonRequest wasn't actually setting PATH_INFO to the correct value!

It is currently using .path_info which according to the mod_python docs is "What follows after the file name, but is before query args, if anything".

It "sounds" like the right thing, but it isn't. Since we're not actually using SCRIPT_NAME in ModPythonRequest, it should just be set to .uri instead. This is a pretty big claim and I'm no expert, so maybe I'm totally wrong in which case someone please show me the flaw in my simplistic logic :)

[jumo@…]

Replying to rmunn@pobox.com:

Closing ticket as "wontfix" since I've concluded that this should be solved with a rewrite rule at the server level.

this isn't a cool way to fix this bug, i seems indeed very stupid for django beginners wich had some hours of trouble caused by it.

[Chris Beaven]

The ticket has been reopened, so obviously the "wontfix" doesn't apply now.

[jmelesky]

try and accomodate request.path appropriately in mod_python and under wsgi, add request.full_path for script_name plus path

[jmelesky]

This patch was made against [6230] and has not been fully tested.

In order to accomodate mod_python, it has a new conf directive:

PythonOption django.root /base/django/uri

So, if, for example, you're trying to run django in location /blog, you'd have something like:

<Location /blog>

... blah blah ...
PythonOption django.root /blog

</Location>

I'll attach a doc patch after i do some more testing.

[jmelesky]

Oh, and this is being worked on as part of the Sprint14Sep

[jmelesky]

corrected and far more comprehensive patch, against [6292]

[jmelesky]

The prior patch was both limited and wrong. This patch corrects the bug and expands the expected behavior to include request.get_full_path(), as well as updating a number of internal references from request.path to request.full_path. It has been tested against mod_python, mod_wsgi on apache, and against mod_[fs]cgi on lighty.

I don't, however, know if this is enough. We may also need to update the behavior of {% url %}.

Also, though this is a big change, for existing mod_python installs it is not actually backwards-incompatible. Leaving out the PythonOption django.root should retain the original behavior.

[jmelesky]

another patch revision, against [6300], including docs and suggestions from Malcom

[jmelesky]

Another revision, against [6316]. This one actually passes the test suite.

[jmelesky]

There is one problem of note, here. If you're running, say, FastCGI, the SCRIPT_NAME gets populated back as 'blahblah.fcgi'. This wasn't an issue before. Now, though, redirects will point back to '/blahblah.fcgi/blah/blah' instead of '/blah/blah'. This will likely still work (unless the server is rewriting everything), but produces ugly and unexpected URLs.

Which points out a fundamental issue: SCRIPT_NAME + PATH_INFO is not the same thing as the URI that the user sees. I think more and more that these are related, but separate problems.

We end up with three distinct pieces of information:

1. The absolute URL prefix (Apache's <Location> directive, "/mysite")
2. The name of the callable thing (SCRIPT_NAME, "/user/django.fcgi")
3. The remainder of the path ("/blog/tags/Sprint14Sep")

The first two might be different due to server URL rewriting.

The question, then, is: Do we want to support all three pieces of information? Because if we do, my patch is insufficient. I'm happy to continue work on this if y'all want to move in that direction, but this needs more design decision.

[Brian Rosner]

#5576 marked as a duplicate.

[Jacob]

Patch needs review (which Malcolm says he'll do), but otherwise this looks great.

[Rob Hudson <treborhudson@…>]

I just got bit by this and wanted to verify what SmileyChris saw with WSGI vs modpython, that req.uri under mod_python is the path and not req.path_info. Looks like this patch will fix it in this small case. Thanks.

[cwebber@…]

This patch is great, but it doesn't actually set the path in the wsgi request. Maybe this works with mod_python, but in our scgi environment at my work I'm not seeing it take any effect.

It appears to me that in wsgi.py after self.full_path is set up that this should actually become self.path, correct?

IE, afaict for this to take any effect, WSGIRequest's init_ should look like this:

class WSGIRequest(http.HttpRequest):
    def __init__(self, environ):
        self.environ = environ
        self.path = force_unicode(environ.get('PATH_INFO', '/'))
        self.full_path = (force_unicode(environ.get('SCRIPT_NAME', ''))
                          + force_unicode(environ.get('PATH_INFO', '/')))
        self.META = environ
        self.META['PATH_INFO'] = self.path
        self.META['SCRIPT_NAME'] = force_unicode(environ.get('SCRIPT_NAME', ''))
        self.method = environ['REQUEST_METHOD'].upper()
        self.path = self.full_path  ### <-- the added line

[ruddick@…]

I applied these changes, and they mostly fixed the problem, but the links in the upper right corner of the admin page (Documentation, Change Password, Log out) still do not use the full path. I do not know enough to fix the problem, so I will leave it to your expertise.

[Marc Fargas]

Replying to jacob:

Patch needs review (which Malcolm says he'll do), but otherwise this looks great.

As Malcolm is currently "offline" I'm moving this back to Accepted to clean up the "RFC" list (Ready For Checkin")

[gsf@…]

Is #3414 a duplicate of this ticket?

[Malcolm Tredinnick]

I'm reviewing this in detail today and I'm going to change something fundamental in the interests of maintaining as much backwards-compatibility as possible. Instead of full_path and path, we'll have path (which is SCRIPT_NAME and PATH_INFO) and path_info. That means references to request.path will still be the right thing. This will reduce the amount of porting required almost everywhere (you have to fix your root URLConf once -- and even that's optional with mod_python -- and things will Just Work(tm)).

I want to work out something around the fact that some FastCGI implementations might be broken, according to jmelesky's comment 14. SCRIPT_NAME + PATH_INFO + QUERY_STRING should be the submitted URL; if it isn't, part of the web server pipeline is setting environment variables incorrectly and we'll have to work around that. That and some testing of the mod_python comments people have brought up are the blockers for me right now and I'm working on that this weekend. So this is back on the front burner.

[Richard Davies <richard.davies@…>]

I want to make sure that this thread is aware of one use case, which is unusually behaved, but should be supported.

I run Django with Lighttpd, using the error-handler-404 mechanism borrowed from the standard Rails config for this web server (​http://github.com/rails/rails/tree/master/railties/configs/lighttpd.conf)

When run in this manner, Lighttpd sets only REQUEST_URI and SCRIPT_NAME (to the handler!), but neither PATH_INFO nor QUERY_STRING (​http://trac.lighttpd.net/trac/wiki/FrequentlyAskedQuestions#Whatkindofenvironmentdoesserver.error-handler-404setup)

I have just submitted a patch against #3414 which builds both PATH_INFO and QUERY_STRING out of REQUEST_URI if they are missing in WSGIHandler._init_ . It looks like similar code is probably needed in the eventual patch generated from this thread?

[Malcolm Tredinnick]

(In [8015]) Changed/fixed the way Django handles SCRIPT_NAME and PATH_INFO (or
equivalents). Basically, URL resolving will only use the PATH_INFO and the
SCRIPT_NAME will be prepended by reverse() automatically. Allows for more
portable development and installation. Also exposes SCRIPT_NAME in the
HttpRequest instance.

There are a number of cases where things don't work completely transparently,
so mod_python and fastcgi users should read the relevant docs.

Fixed #285, #1516, #3414.

[anonymous]

maybe FORCE_SCRIPT_NAME = will help

[allo]

This is broken in 1.0 beta again: my admin-loginform now points to /mysite.fcgi/admin/, which is wrong, it should be /admin/.
My setup is a lighttpd with fastcgi and rewrite from /(.*) to /mysite.fcgi$1
it would be cool if this can be fixed soo.

[Richard Davies <richard.davies@…>]

Have you tried

FORCE_SCRIPT_NAME = ''

in your settings.py? That will almost certainly fix your issue - it fixed the same problem for me when updating to more recent Django.

I'm reclosing this as "worksforme". You can reopen if you still have the problem...

Replying to allo:

This is broken in 1.0 beta again: my admin-loginform now points to /mysite.fcgi/admin/, which is wrong, it should be /admin/.
My setup is a lighttpd with fastcgi and rewrite from /(.*) to /mysite.fcgi$1
it would be cool if this can be fixed soon.

[allo]

i tried FORCE_SCRIPT_NAME='/mysite.fcgi' and without /, because i thought it should filter the prefix. now it works, thanks.

[Malcolm Tredinnick]

Please don't reopen this ticket no matter what the problem is here. The root problem has been fixed. You are seeing something that is a consequence of the solution, but that's just needing a tweak to the solution (if it's not a config issue on your end). If that's the case, it's a new ticket, not this one.

[Jacob]

Milestone 1.0 alpha deleted