Opened 18 years ago

Closed 17 years ago

#2970 closed defect (fixed)

[patch] HttpResponse should treat headers case-insensitively

Reported by: Adrian Holovaty Owned by: Philippe Raoult
Component: HTTP handling Version:
Severity: normal Keywords: sprintsept14
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

(Submitted by Bryan O'Sullivan, who had problems with Trac's spam filter.)

The HttpResponse implementation uses a normal Python dict to store
headers. This makes it necessary to have every component that sets
headers in a response agree on the exact case to use for every header
name. I found this problem while trying to glue a WSGI app into a
Django app, only to discover that one believes that a header should be
named "content-type", while the other prefers "Content-Type". This
results in a HTTP response that contains both headers, leading to
undefined (and currently bad) results on the client side.

Probably the HttpResponse class should use a dict-like object that
preserves, but ignores, case, so that "Foo" and "fOo" will map to the
same item.

Attachments (5)

case_insensitive_dict.py (3.0 KB ) - added by Chris Beaven 18 years ago.
Here's a (2.3 compatible) case insensitive dict I whipped up
http_response_insensitive.patch (1.2 KB ) - added by Chris Beaven 18 years ago.
Here's the patch
http_response_insensitive.2.patch (1.2 KB ) - added by Chris Beaven 18 years ago.
more backwardsly-compatible
http_response_insensitive.3.patch (7.2 KB ) - added by Chris Beaven 18 years ago.
Tests and included CaseInsensitiveDict
2970.diff (1.4 KB ) - added by Philippe Raoult 17 years ago.
refactoring using a basic python dict with lowercase keys

Download all attachments as: .zip

Change History (18)

by Chris Beaven, 18 years ago

Attachment: case_insensitive_dict.py added

Here's a (2.3 compatible) case insensitive dict I whipped up

by Chris Beaven, 18 years ago

Here's the patch

comment:1 by Chris Beaven, 18 years ago

Summary: HttpResponse should treat headers case-insensitively[patch] HttpResponse should treat headers case-insensitively

Patch doesn't contain case_insensitive_dict.py, so you'll need to put that in django.utils yourself.

comment:2 by Bastian Kleineidam <calvin@…>, 18 years ago

The second part of your patch modifying has_header() is not backwards compatible, and could break existing applications if they used a plain dictionary for header values

If the patch is applied, this should be noted in the documentation.

by Chris Beaven, 18 years ago

more backwardsly-compatible

comment:3 by Chris Beaven, 18 years ago

Good point, Bastian. This patch should keep you happier :)

comment:4 by Jacob, 18 years ago

Looks like a great idea. Can we get some unit tests for both the new data structure and the HttpResponse behavior? Oh, and a revised patch that moved CaseInsensitiveDict into django.utils.datastructures would be nice.

by Chris Beaven, 18 years ago

Tests and included CaseInsensitiveDict

comment:5 by Chris Beaven, 18 years ago

Here you go, Jacob.

I haven't included any tests for the HttpResponse behaviour specifically. If you check out the code, it's just doing a basic property(get, set) to keep the identical behaviour as before. That, and I didn't know exactly what tests you'd want to run - I haven't changed any external behaviour apart from the case insensitivity which is being 100% done in the CaseInsensitiveDict.

comment:6 by Jacob, 18 years ago

Awesome; thanks. I'll check this in later tonight.

comment:7 by Chris Beaven, 18 years ago

Triage Stage: UnreviewedReady for checkin

comment:8 by Malcolm Tredinnick, 18 years ago

Owner: changed from Adrian Holovaty to Malcolm Tredinnick
Triage Stage: Ready for checkinDesign decision needed

There's a lot of processing overhead going on in the CaseInsensitiveDict class just to preserve the case of the original key. I can't think of a reason when this is really needed (we will document that the headers are case-insensitive so that people won't be surprised when headers.keys() returns all lower-case). Am I missing some obvious use-case beyond "it might be nice"?

If there isn't a strong reason for keeping the original key case, I'd like to replace CaseInsensitiveDict with something like this Python cookbook recipe (I'll use one of the versions from later in the comments, since they fix problems with the earlier version). That code just looks a bit easier to read.

I'll move this to "design decision needed" for the time being. Chris, if I'm missing a good reason why you have that functionality, feel free to point it out. I'm willing to admit to being dense sometimes. No need to go to all the trouble of updating the patch if we can just make the switch: I'll just drop in the replacement and some tests when I do the commit.

comment:9 by Chris Beaven, 18 years ago

You are right, preserving case it probably doesn't matter... in which case, yea there's probably a simpler way to do it. I was just blindly following Adrian's suggestion to "preserve":

Probably the HttpResponse? class should use a dict-like object that preserves, but ignores, case, so that "Foo" and "fOo" will map to the same item.

by Philippe Raoult, 17 years ago

Attachment: 2970.diff added

refactoring using a basic python dict with lowercase keys

comment:10 by Philippe Raoult, 17 years ago

Component: Core frameworkHTTP handling
Keywords: sprintsept14 added
Owner: changed from nobody to Philippe Raoult

The new patch has less overhead and has_header() should be much faster than before. http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2 says that field names are case-insensitive so there's no need at all to preserve case.

comment:11 by Jacob, 17 years ago

Resolution: fixed
Status: newclosed

(In [6212]) Fixed #2970: made HttpResponse headers case-insensitive. Thanks to SmileyChris for the original patch and PhiR for the final one.

comment:12 by calvin@…, 17 years ago

Resolution: fixed
Status: closedreopened

In my experience changing header names to lower-case will break some clients or servers (let alone bad programmed proxies).

To follow the reference, only key lookup must be case insensitive. Otherwise the application should preserve the original header value. So please consider reverting the __setitem__ change and the changed "Content-Type" case.

comment:13 by Malcolm Tredinnick, 17 years ago

Resolution: fixed
Status: reopenedclosed

(In [6546]) Changed the way we handle HTTP headers internally so that they appear
case-insensitive, but the original case is preserved for output. This increases the chances of working with non-compliant clients without changing the external interface. Fixed #2970.

Note: See TracTickets for help on using tickets.
Back to Top