Consider removing conditional_content_removal

[Aymeric Augustin]

It's the last survivor of the "response_fixes" and I think it should go away. Its origin is #5898.

Graham Dumpleton has described better than I would why frameworks should leave the task stripping the body of HEAD requests to servers: ​http://blog.dscpl.com.au/2009/10/wsgi-issues-with-http-head-requests.html (first five paragraphs).

We should check whether mod_wsgi, gunicorn and uwsgi properly strip the body of HEAD requests before proceeding. We should also move that behavior into runserver -- that is, on the other side of the WSGI.

[Akshesh Doshi]

I would like to work on this ticket. Can anyone please guide me exactly what changes would be required.

Thanks

[Tim Graham]

​PR

[Tim Graham]

I think the PR needs improvement as described in the ticket description, "mod_wsgi, gunicorn and uwsgi properly strip the body of HEAD requests before proceeding. We should also move that behavior into runserver."

[Tim Graham]

I observed that gunicorn and runserver (based on Python's server) don't send content for HEAD requests. I didn't check mod_wsgi or uwsgi since they are marginally more complicated to setup a test project. If we go ahead and remove this, one place where developers might see a difference would be in the test client. Do you think it's worth trying to keep it in that context? I lean toward simply removing it and reacting if anyone complains.

[Tim Graham]

Some data about the rest of the behavior of conditional_content_removal (discarding the content of responses with 1xx, 204, and 304 status codes):

Python's BaseHTTPRequestHandler (which runserver uses) also has ​discards the response content for 204 and 304 status codes. I observed the same behavior from gunicorn as well.

I also don't get any response data on HTTP 1xx responses regardless of whether or not conditional_content_removal is active.

[Claude Paroz]

I tested the HEAD behaviour of mod_wsgi without conditional_content_removal and confirms it strips the content.

If it's not too hard to simulate this behavior in the test client, I think we should do it at that level.

[Tim Graham]

Okay, here's a ​PR based on that suggestion.

[Tim Graham <timograham@…>]

In bb0b4b70:

Fixed #26052 -- Moved conditional_content_removal() processing to the test client.

[Nick Pope]

Tim,

I think that some documentation might need a little rewording after this change:

​https://github.com/django/django/blob/bb0b4b7/docs/topics/http/decorators.txt#L47-L52

[Tim Graham]

Thanks for pointing that out. I think we could replace "Django will automatically" with "Most web servers" or "Web servers should". Did you have anything else in mind?

[Tim Graham <timograham@…>]

In baf3ec2:

Refs #26052 -- Corrected a sentence for conditional_content_removal() removal.

[Ed Morley]

Python's BaseHTTPRequestHandler (which runserver uses) also has ​discards the response content for 204 and 304 status codes

Unfortunately the linked BaseHTTPRequestHandler handling appears to just be for error conditions (send_error()) so this change caused runserver.py to start returning a body for HTTP 200 HEAD requests.

I've filed #28054.