broken pipe issue

[vanderkerkoff]

Hello everyone

I've had a good day looking into this and can't work out what the f*ck is going on.

I've updated my django_src as of today, 31/05/07.

My sites are all running fine, but if I'm using Firefox 2.0.0.4 I'm getting the error below appear in my terminal. It doesn't happen with Safari, but it does screw AB up, only one request gets returned.

Can anyone shed some light on this issue, I'd be really grateful.

Traceback (most recent call last):
  File "/usr/local/lib/python2.5/site-packages/django/core/servers/basehttp.py", line 273, in run
    self.finish_response()
  File "/usr/local/lib/python2.5/site-packages/django/core/servers/basehttp.py", line 312, in finish_response
    self.write(data)
  File "/usr/local/lib/python2.5/site-packages/django/core/servers/basehttp.py", line 389, in write
    self.send_headers()
  File "/usr/local/lib/python2.5/site-packages/django/core/servers/basehttp.py", line 441, in send_headers
    self.send_preamble()
  File "/usr/local/lib/python2.5/site-packages/django/core/servers/basehttp.py", line 372, in send_preamble
    self._write('Date: %s\r\n' % time.asctime(time.gmtime(time.time())))
  File "/usr/local/lib/python2.5/socket.py", line 261, in write
    self.flush()
  File "/usr/local/lib/python2.5/socket.py", line 248, in flush
    self._sock.sendall(buffer)
error: (32, 'Broken pipe')

[Malcolm Tredinnick]

What is the minimum that has to be done to cause the problem? What are you doing in Firefox to cause the error to occur? There is nothing repeatable in this bug report, so it's a bit hard to proceed further.

[anonymous]

I confused myself yesterday by assuming that the development server runserver would behave in the same manner as lighttpd and fcgi when you throw AB at it, dumass as I am. Runserver can't handle AB tests, and if you think about it for one second you will realise this. I just assumed and then went to bad places from there.

I tracked ths issue down and it's nothing at all to do with Django

One of our designers had loaded webdesign into the tempalte that was being called in.

This webdesign randomises lorem ipsum output so that the desginers can imitate different types of text output, that was giving AB a real headache. Removed it and django runs like the wind again.

I sincerely apologise for posting this up here, it's the wrong place but I didn't know it at the time.

Hopefully it might help someone else at some point in time, eh Ed ;-)

[anonymous]

For searching purposes, AB is referring to Apache Server Benchmarking Tool. "AB" is too short to search for.

[anonymous]

I'm also getting the same of similar error from runserver.

Traceback (most recent call last):

File "/usr/local/lib/python2.4/site-packages/django/core/servers/basehttp.py", line 273, in run

self.finish_response()

File "/usr/local/lib/python2.4/site-packages/django/core/servers/basehttp.py", line 312, in finish_response

self.write(data)

File "/usr/local/lib/python2.4/site-packages/django/core/servers/basehttp.py", line 396, in write

self._write(data)

File "/usr/local/lib/python2.4/socket.py", line 256, in write

self.flush()

File "/usr/local/lib/python2.4/socket.py", line 243, in flush

self._sock.sendall(buffer)

error: (32, 'Broken pipe')

It seems to work fine on my development box, but when I try to test django on a production server (using the servers IP instead of localhost - manage.py runserver 192.168.*.* 8000). I've also noticed that {{ request.user.first_name }} doesn't work in my templates when on the production server - but it does work on my dev box...

[anonymous]

Also - Firefox and IE both pull the error. It doesn't seem to be limited to Firefox in my case. The only non-standard library I'm using is Sphene Community - ​http://sct.sphene.net/wiki/show/Start/

[berto]

I'm seeing this problem on my machine with Firefox 2.0.0.4. I can confirm it does not happen with Safari.

I tried this on a bare install, with the following INSTALLED_APPS:

    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.admin',

There are no additional middleware or template loaders either.

I can get the error by simply accessing the admin site.

[sandro@…]

I get this consitently if I add ThreadLocals middleware defined here:

​http://code.djangoproject.com/wiki/CookBookThreadlocalsAndUser

at this point browsing admin site outputs the error in the developement console.
Appearently there is no other problem. Sometimes I get the error only on the second hit
of the page

[anonymous]

I can confirm this does not happen the first time a page is accessed using the development server (e.g. when you make a change to the code and the server reloads). The exception occurs the second and any subsequent page request.

[eatmywake AT gmail DOT com]

Hi,
I was getting this error too (with Mozilla Firefox v2.0.0.5 and Mozilla Seamonkey v1.1.3 on Fedora7 served by Django v0.96, dev server), however what I found interesting was that I was able to replicate & eliminate (or decrease the frequency of the error, at least) in my app! I'll explain a little what I was up to.

I'm building a chat room app with Django and its AJAXed using YUI toolkit. Part of the app is a continuous loop, which polls the server for updates. This is simplifing things a bit, but the app has, lets say, a poll interval and a timeout period setting, so that if a request doesn't return from the server within the timeout period, the request is aborted.

What I found (and it kinda makes sense, with hindsight...) was that by making the timeout period much less than the poll interval, say half, the frequency of the error was much reduced. At the moment I'm not getting any errors. I think I may have been DoS-ing my own machine by sending too many requests, before the previous requests had completed!!

I don't know if this will help the ticket resolution, but the same error had me head scratching for a while.
Tom (Mayo, Ireland.)

Below is the exact error copied from my terminal;

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/django/core/servers/basehttp.py", line 273, in run
    self.finish_response()
  File "/usr/lib/python2.5/site-packages/django/core/servers/basehttp.py", line 312, in finish_response
    self.write(data)
  File "/usr/lib/python2.5/site-packages/django/core/servers/basehttp.py", line 391, in write
    self.send_headers()
  File "/usr/lib/python2.5/site-packages/django/core/servers/basehttp.py", line 443, in send_headers
    self.send_preamble()
  File "/usr/lib/python2.5/site-packages/django/core/servers/basehttp.py", line 373, in send_preamble
    'Date: %s\r\n' % time.asctime(time.gmtime(time.time()))
  File "/usr/lib/python2.5/socket.py", line 261, in write
    self.flush()
  File "/usr/lib/python2.5/socket.py", line 248, in flush
    self._sock.sendall(buffer)
error: (32, 'Broken pipe')

[anonymous]

I am getting exactly this same error. However, in my case the socket connection is coming from a JavaScript I'm running inside After Effects (the script is spawned by my Django app via AppleScript and After Effects defines a Socket object for its scripts). So, I don't think it's a browser issue. Same deal as above, the first time the view is called (in my case it's really just a function and not technically a "view" since it never sends an HttpResponse to the browser), the socket works perfectly; data is transmitted and received, and the sockets appear to be closed properly. The second time I hit this view, however, I get the same broken pipe error and trace in the Terminal as listed above and no socket is opened.

Here is my view code:

{{{def test_socket(request):

print 'TESTING SOCKET'

conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
if conn is None:

print 'Connection is invalid, bailing.'
return None

conn.bind((, 7021))
print 'SERVER: waiting for connection.'
conn.listen(5)
print 'SERVER: listening.'

try:

try:

newSocket, address = conn.accept()
print 'SERVER: connected from address', address

while True:

rawData = newSocket.recv(8192)
print 'SERVER: received data: "%s"' % rawData

if rawData is not None:

# newSocket.shutdown(1)
# We got the data, get out of the loop
break

finally:

if newSocket is not None:

newSocket.sendall('DONE')
newSocket.close()
print 'SERVER: data receive socket closed.'

finally:

if conn is not None:

conn.close()
print 'SERVER: main socket closed.'

print 'SOCKET TEST COMPLETED'

return None}}}

[anonymous]

Sorry, I guess to use WikiFormatting, the braces need to be on their own line, trying again with code sample...

def test_socket(request):
	print 'TESTING SOCKET'

	conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
	if conn is None:
		print 'Connection is invalid, bailing.'
		return None
		
	conn.bind(('', 7021))
	print 'SERVER: waiting for connection.'
	conn.listen(5)
	print 'SERVER: listening.'

	try:
		try:
			newSocket, address = conn.accept()
			print 'SERVER: connected from address', address
			
			while True:
				rawData = newSocket.recv(8192)
				print 'SERVER: received data: "%s"' % rawData

				if rawData is not None:
					# newSocket.shutdown(1)
					# We got the data, get out of the loop
					break
		finally:
			if newSocket is not None:
				newSocket.sendall('DONE')
				newSocket.close()
				print 'SERVER: data receive socket closed.' 
	finally:
		if conn is not None:
			conn.close()
			print 'SERVER: main socket closed.'

	print 'SOCKET TEST COMPLETED'
	
	return None

[Philippe Raoult]

This is common error which happens whenever your browser closes the connection while the dev server is still busy sending data. The best we could is to have a more explicit error message.

[trbs]

django.core.servers.basehttp saner broken pipe message

[trbs]

added a patch which results in a saner default message for the 'broken pipe' issue that happens whenever a browsers closes the connection early.

i found that this mostly/exclusively with '/favicon.ico' on my test project.

the patch changes the output from a lengthy traceback message to:

[31/May/2008 20:47:21] "GET /favicon.ico HTTP/1.1" 200 6335 Broken pipe

Since /favicon.ico is normally ignored by the logs messages it might make good sense to ignore the 'broken pipe' from this path too.
This is where the second patch comes in :) It's much cleaner as it uses the normal log_request routine. (and thus ignores the favicon path)

[anonymous]

I am still having problems with this broken pipe issue.
And I am using the newest django distribution 0.96.2.
When I try to access my development server with ​http://192.168.0.3:8000/myurl/ I get the error displayed. And the patches didn't solve anything...

[anonymous]

And I have installed both patches and if I fast click many times to reload my page I get the following error message:

Traceback (most recent call last):

File "/usr/local/lib/python2.5/site-packages/django/core/servers/basehttp.py", line 273, in run

self.finish_response()

File "/usr/local/lib/python2.5/site-packages/django/core/servers/basehttp.py", line 312, in finish_response

self.write(data)

File "/usr/local/lib/python2.5/site-packages/django/core/servers/basehttp.py", line 396, in write

self._write(data)

File "/usr/local/lib/python2.5/socket.py", line 262, in write

self.flush()

File "/usr/local/lib/python2.5/socket.py", line 249, in flush

self._sock.sendall(buffer)

error: (104, 'Connection reset by peer')

[trbs]

Dear anonymous,

There is nothing to 'fix' here in your application, your just experiencing a browser issue/quirk.

This is exactly the reason i voted for changing these kinds of tracebacks to less scary looking errors and hiding the traceback behind --traceback. (this's what the switch it's for no ? :)

The reason for this patch is not to 'fix' the broken pipe issue. As it's not really an issue. But there seems to be a lack of clear consensus on how serious this is.

According to many sources the 'Broken Pipe' is a normal browser quirk. For example, the browser reads from the socket and then decides that the image it's been reading apparently didn't change. The browser now this (forcefully) closes the connection because it does not need more data. The other end of this socket (the python runserver) now raises a socket exception telling the program that the client 'Broke the socket pipe'.

If this is really only a non-issue, aka browsers do this often and it's not serious, then i suggested that we at least change this to a slightly less scary looking error. The idea here is to turn an exception (which is programming lingo) into an user error.

Hopefully this explains the issue a bit better for people hitting it.

[Ramiro Morales]

One thing the multiple reporters don't make clear is if the development web server keeps running or if this is a non fatal error.

Maybe a FAQ entry could be added with a blurb similar to trbs explanation above or to how CherryPy people describe the issue:
​http://www.cherrypy.org/wiki/CherryPyBrokenPipe

[trbs]

That might be a good idea for both sides on this discussion ? :)

If we create a decent wiki page for this and put the wiki page in the exception raised by Django then it would be clear to people what the error is about.

[Loststylus]

Hm, but when i test on IE, i have to restart the development server periodically, because of the broken pipe error (the browser tries to get data, but gets stuck until a reload the server).

Sorry, for reopening, i just want someone to pay attention to the problem. I don't think that server restart is a good idea.

How should this be handled, anyway?

[James Bennett]

Take it to the django-developers list, please.

[imagei]

So here we are, 7 years later and this still happens. Google and StackOverflow point to this bug report but it seems a dead end now -- can someone point towards an active mailing list discussion or another, active ticket please?

I appreciate this is technically nobody's fault but Django happens to print the message so it gets the blame.

This is massively distracting during development when each time I reload a page I have to check whether it's Django printing its usual garbage or something *actually* exploded.

These strack traces all over the place are ugly and affect productivity so I hope it doesn't get swept under the rug.

File "/usr/lib/python2.7/SocketServer.py", line 582, in process_request_thread
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/local/lib/python2.7/site-packages/django/core/servers/basehttp.py", line 150, in _init_
super(WSGIRequestHandler, self)._init_(*args, kwargs)
File "/usr/lib/python2.7/SocketServer.py", line 640, in _init_
self.finish()
File "/usr/lib/python2.7/SocketServer.py", line 693, in finish
self.wfile.flush()
File "/usr/lib/python2.7/socket.py", line 303, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe

[Shai Berger]

I am not aware of any active discussion of this issue, but if you want to change the current resolution, please start one on the DevelopersMailingList.

[Matthew Somerville]

Just as an update - as requested, I started a thread on the mailing list which can be seen at ​https://groups.google.com/forum/#!topic/django-developers/_oC_PAxxnQI , and I have made a pull request which can be found at ​https://github.com/django/django/pull/3121 .

[Claude Paroz]

Worth a review...

[Tim Graham]

I am not really happy with that patch which copies the simple_server.WSGIRequestHandler.handle() method from Python's version in order to override it. The copied version is not in sync with the latest Python and I'd prefer not to be in a position where we'd have to copy changes from there to Django.

[Tim Graham <timograham@…>]

In 9253042d53d61b23b3f727a00c41e8089be73836:

Fixed #4444 - Made runserver suppress 'broken pipe' errors

One handler in WSGIServer, to catch the error when raised from
SocketServer.BaseServer's finish_request, and one in WSGIRequestHandler
(by creating a subclass of ServerHandler), to catch the error when
raised in wsgiref.handlers.BaseHandler's finish_response.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 28124e7b:

Refs #4444 -- Added tests for handling broken pipe errors in WSGIServer.