Apache MultiViews, mod_python, and certain URLs result in a 500 error

[Carl Meyer]

If Apache is configured with mod_negotiation and MultiViews on, a URL like /index may be automatically rewritten to /index.html. In the course of this internal URL rewrite, the SetEnv DJANGO_SETTINGS_MODULE directive in a Django/mod_python VirtualHost appears to be called twice, resulting in a duplicate entry in the mod_python req.subprocess_env table, which is a dict-like object but apparently supports multi-values. This duplicate entry causes a 500 error when Django's ModPythonHandler attempts to do os.environ.update(req.subprocess_env) , with the following traceback in the Apache log:

File "/usr/lib/python2.5/site-packages/mod_python/importer.py", line 1537, in HandlerDispatch
    default=default_handler, arg=req, silent=hlist.silent)
File "/usr/lib/python2.5/site-packages/mod_python/importer.py", line 1229, in _process_target
    result = _execute_target(config, req, object, arg)
File "/usr/lib/python2.5/site-packages/mod_python/importer.py", line 1128, in _execute_target
    result = object(arg)
File "/var/www/meyer/lib/py/django/core/handlers/modpython.py", line 226, in handler
    return ModPythonHandler()(req)
File "/var/www/meyer/lib/py/django/core/handlers/modpython.py", line 181, in __call__
    os.environ.update(req.subprocess_env)
File "/usr/lib/python2.5/os.py", line 488, in update
    self[k] = dict[k]
File "/usr/lib/python2.5/os.py", line 473, in __setitem__
    putenv(key, item)
TypeError: putenv() argument 2 must be string, not list

(The double-calling of SetEnv resulting in duplicate entries in subprocess_env is possibly a bug in Apache/mod_python. In any case it doesn't make much sense to use MultiViews on an Apache instance serving Django. Nonetheless, since subprocess_env is a data structure that supports multi-values, and multi-values can occur in it, Django needs to be more careful in calling os.environ.update() this way.)

Reproduced on Apache 2.2.6 (Debian Etch) and 2.2.8 (Ubuntu Hardy) with Django 1.0 and trunk.

[Graham Dumpleton]

The underlying problem is actually caused by a bug, or arguable poor behaviour, on the part of Apache itself and not mod_python.

​http://issues.apache.org/jira/browse/MODPYTHON-146

It isn't helped by mod_python not using a proper resource based mapping system for URLs, but instead trying to use Location blocks instead, which results in some resource based mappings being applied when they shouldn't be.

[Carl Meyer]

Regardless of the upstream issues, Django could be robust against this and prevent a really nasty, hard-to-debug crash, potentially with as simple a fix as catching TypeError. I can work on a fix for 1.0.1 if it's desired.

[Jacob]

I don't much like the idea of papering over obvious bugs in other software, but I'd like to see what the patch looks like first.

[Carl Meyer]

The justification for addressing this, as noted in the description, is that subprocess_env is a data structure capable of carrying multiple values for a given key, so it seems that Django should be able to handle multi-values in it. The fact that this particular occurrence of a multi-value results from a bug in Apache seems secondary.

That said, I no longer use mod_python, so I'm not likely to come up with a patch anytime soon, nor am I particularly bothered if this is wontfixed.

[Carl Meyer]

Wontfixing this edge-case bug, since mod_python support is deprecated anyway.