Selenium tests terminate with [Errno 10054]

[Kevin Christopher Henry]

When I run the test suite on Windows with the --selenium option, I consistently see errors like this:

Exception happened during processing of request from ('127.0.0.1', 51603)
Traceback (most recent call last):
  File "C:\Program Other\Python27\Lib\SocketServer.py", line 295, in _handle_request_noblock
    self.process_request(request, client_address)
  File "C:\Program Other\Python27\Lib\SocketServer.py", line 321, in process_request
    self.finish_request(request, client_address)
  File "C:\Program Other\Python27\Lib\SocketServer.py", line 334, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Django\django\django\core\servers\basehttp.py", line 126, in __init__
    super(WSGIRequestHandler, self).__init__(*args, **kwargs)
  File "C:\Program Other\Python27\Lib\SocketServer.py", line 649, in __init__
    self.handle()
  File "C:\Program Other\Python27\Lib\wsgiref\simple_server.py", line 116, in handle
    self.raw_requestline = self.rfile.readline()
  File "C:\Program Other\Python27\Lib\socket.py", line 447, in readline
    data = self._sock.recv(self._rbufsize)
error: [Errno 10054] An existing connection was forcibly closed by the remote host

Based on some ​anecdotal ​data, I tried putting a refresh() before the quit() in contrib.admin.tests.AdminSeleniumWebDriverTestCase._tearDownClassInternal() and that reliably solved the problem.

I have no idea why this works, but if it does it's probably worth doing since it should be harmless.

[Tim Graham]

There are some other places where we call selenium.quit() -- should those places be updated as well?

docs/topics/testing/overview.txt: cls.selenium.quit()
tests/view_tests/tests/test_i18n.py: cls.selenium.quit()

[Kevin Christopher Henry]

I think it's worth doing this wherever we're calling selenium.quit(), since it seems harmless and does solve the spurious failures. Here's a patch for it: ​https://github.com/django/django/pull/1806.

However, I hesitate to give this workaround canonical status by putting it in the documentation, since it's basically a bug in another project, could be fixed at any time, and adds a little extra noise and cognitive dissonance to the documentation. It only seems to pop up in some environments, and the workaround can be easily found with a quick internet search. On the other hand, for people developing reusable applications, it might be nice to warn them about this since it might affect them even if they aren't seeing the bug in their own test environments. Maybe the wiki is the right place for this?

[Tim Graham]

In 08c9ab5a0f564a3ac7803e6a97fae855f2e0524e:

Fixed #21227 -- Added workaround for selenium test failures

Added a refresh() before quit() in the selenium tests, since this
solves the problem of spurious test failures in some environments.

[Florian Apolloner]

@marfire That workaround slows down firefox pretty much and should Be unneeded; can you test if this sequence works better for you:

        if hasattr(cls, 'selenium'):
            cls.selenium.close()
        super(AdminSeleniumWebDriverTestCase, cls)._tearDownClassInternal()
        if hasattr(cls, 'selenium'):
            cls.selenium.quit()

[Florian Apolloner]

@marfire: Is this a hard failure or just output on the console? Cause essentially everything which is a subclass of Exception is captured and more or less ignored.

[Kevin Christopher Henry]

I'm out of town right now, but I'll try this next week and report back...

[Florian Apolloner <florian@…>]

In 7d0a5190328f277b97b1f55171e1fecb22a1e461:

Revert "Fixed #21227 -- Added workaround for selenium test failures"

This reverts commit 08c9ab5a0f564a3ac7803e6a97fae855f2e0524e.

[Florian Apolloner]

I just reverted it since it doubles (or more) the time for firefox tests. The real fix should go something like this: When we tear down set a flag that we are in a shutdown process and then ignore the error.

[tkhyn]

Hi,

An attempt to mitigate this issue is in attached patch above.

The only way to make it work is to Attempt to set server_thread.httpd.ignore_errors = True before calling WebDriver.quit() in the Selenium test case class's tearDownClass, like that:

class MySeleniumTestCase(LiveServerTestCase)

    [...]

    @classmethod
    def tearDownClass(cls):
        if hasattr(cls, 'server_thread'):
            # test if server_thread attribute is available (as there may have been an exception in setUpClass)
            # setting ignore_errors flag on WSGI server thread to avoid unwanted 10054
            cls.server_thread.httpd.ignore_errors = True
        cls.wd.quit() # cls.wd is the WebDriver instance
        super(LiveServerTestCase, cls).tearDownClass()

Could not find a simpler way to do it as the flag must be set before calling wd.quit(). Fully integrating it in Django would require a django-specific SeleniumTestCase class on top of LiveServerTestCase.

[Tim Graham]

#22252 was a duplicate.

[anonymous]

For anyone looking to repro this, here's a single test case that shows the error:

python runtests.py --selenium admin_widgets.tests.DateTimePickerSeleniumFirefoxTests

be aware that if this PR on logging gets accepted (​https://github.com/jmbowman/django/commit/c34c5860d4e842e62574657ec61a125bc91b6c0f) then the error will no longer be visible on stdout. If I get time I'll try and build some kind of specific failing test for just this issue...

[Tyeth Gundry]

This has recently bitten me when testing locally.

Following theoldtestinggoat.com TDD in python tutorial, and decided to switch to chrome instead of firefox, and the tests pass but with this error logged each time when in conjunction with a local django test server, if tested against a remote host the connection gets closed cleanly and no error is reported.

Adding the driver.refresh() call before close() alleviated my chrome-based woes, that and disabling gpu acceleration when calling chrome in headless mode.

[mrts]

I got rid of the ConnectionResetError: [WinError 10054] by creating ConnectionResetErrorSwallowingLiveServerThread as follows that swallows the error:

import socket
import errno

from django.core.servers.basehttp import ThreadedWSGIServer
from django.test.testcases import QuietWSGIRequestHandler, LiveServerThread


class ConnectionResetErrorSwallowingQuietWSGIRequestHandler(QuietWSGIRequestHandler):
    def handle_one_request(self):
        try:
            super().handle_one_request()
        except socket.error as err:
            if err.errno != errno.WSAECONNRESET:
                raise


class ConnectionResetErrorSwallowingLiveServerThread(LiveServerThread):
    def _create_server(self):
        return ThreadedWSGIServer((self.host, self.port), ConnectionResetErrorSwallowingQuietWSGIRequestHandler, allow_reuse_address=False)

It can be used it in tests that extend StaticLiveServerTestCase as follows:

class AdminSiteTestCase(StaticLiveServerTestCase):

    server_thread_class = ConnectionResetErrorSwallowingLiveServerThread

@timgraham, what do you think, could something like this be added to django.test.testcases?

[Mariusz Felisiak]

Fixed in 772eca0b0219f63129282c3596fc210951433c13 because WSGIServer now suppress ConnectionResetError.