Multiple LiveServerTestCase subclasses cannot reuse the same port

[Jeremy Bowman]

The fix for #26011 ("allow_reuse_address = 1 on WSGIServer can cause random test failures in LiveServerTestCase on Windows"), released with Django 1.10, seems to have had the unintended side effect of preventing a port from being reused for multiple subclasses of LiveServerTestCase in the same test suite. I maintain a couple of libraries (sbo-selenium and django-nose-qunit) which facilitate Selenium testing of Django applications, and while updating them to work with 1.10 I noticed that both of them now fail with OSError: [Errno 48] Address already in use after the first test succeeds. DJANGO_LIVE_TEST_SERVER_ADDRESS had been configured to use only a single port, and after updating it to use a range and inserting some debugging code, I noticed that subsequent test classes were needing to increment to ever higher port numbers in the range. (I'm seeing this behavior at least on Mac OS X El Capitan, haven't yet tested more widely.)

The underlying problem seems to be described well at ​https://brokenbad.com/address-reuse-in-pythons-socketserver/ :

Checking with netstat and ps, I found that although the process itself is no longer running, the socket is still listening on the port with status “TIME_WAIT”. Basically the OS waits for a while to make sure this connection has no remaining packets on the way.

I don't think we want to just revert the change that caused this, because it's true that we should not blindly ignore port conflicts with other processes. Maybe it can be changed to set allow_reuse_address=False only for the first LiveServerThread in the process (to avoid stomping on any already-running processes), then flip a variable which would set it to True for all subsequent instances so each test class can reuse the same port?

As a workaround (and the reason I suspect this hadn't already been noticed), you can just assign a large range of ports in DJANGO_LIVE_TEST_SERVER_ADDRESS and let the tests cycle through them. But this doesn't necessarily work in a large test suite with many subclasses of LiveServerTestCase, and there are cases where it's very useful for integration with other scripts to know deterministically which port will be used to run the server.

[Tim Graham]

As an aside, I'm not sure if the issue affects master due to #26791. It would be nice if you could also test on master and see if that fix for that ticket causes problems for your use case.

[Jeremy Bowman]

The auto-binding to port 0 does seem to fix the immediate problem (because it ignores the setting that was restricting it to a single specified port). I certainly hope that the OS would free up the already used ports before it started running out of available ones, so I don't expect much of a problem there. The last problem would be the inability to script the test to use a known port so other code can also interact with it; I vaguely remember having a use case for that before (something something running Selenium in a Docker container), but it's probably a bad thing to be depending on anyway. And it might have been just the IP address I needed, not the port.

So yeah, it looks like this may be fixed for the next major release, and would just be an issue for 1.10.x.

[Tim Graham]

If you could provide a patch, that would be great. I'm not sure if I could write one myself.

[Jeremy Bowman]

I started working on a patch, but a family emergency came up. I'll try to finish the patch (or determine that one isn't really feasible) within the next week.

[Tim Graham]

The time to fix this is running out since the last bug fix release for Django 1.10 is April 1. Since a patch hasn't been provided and no one else has reported it, I assume it isn't a high priority issue. Marking as "fixed" in the sense that the issue is fixed in 1.11.

[Jeremy Bowman]

Fine with me; this hasn't stayed at the top of my priorities since none of my production code is using 1.10 yet, and as you say an upgrade to 1.11 should fix it. Sorry for not staying on top of this; at least it seems like there aren't many other people having trouble with it either.

[David Muller]

Hello, thanks for reporting this bug Jeremy. We are trying to upgrade from Django 1.9.12 to Django 1.10.6 and having the same problem that Jeremy described -- each LiveServerTestCase seems to require a brand new port.

More concretely, our ability to specify one (and only) one live server address via the --liveserver option (e.g. python manage.py test --liveserver=localhost:8000) is now broken -- as soon as we try to instantiate our second LiveServerTestCase subclass we get an address already in use error.

Marking as "fixed" in the sense that the issue is fixed in 1.11.

Could you guys elaborate a little more on how this is fixed in 1.11? I'm a little lost on what changes in 1.11 to fix this..

[Jeremy Bowman]

The change is documented in the draft release notes at ​https://docs.djangoproject.com/en/1.11/releases/1.11/#liveservertestcase-binds-to-port-zero . Instead of specifying which port(s) the test server should serve at, it binds to port 0 and counts on the OS to assign a currently unused port. This gets around the need to specify known-available ports at all. You can still find out which port is actually in use from the test class via self.server_thread.port (which is included in self.live_server_url).

You mentioned specifying a particular port on the command line; if you need to know which port is being used in order to access it from a completely separate process (which really isn't what LiveServerTestCase was designed for), you could do something like assign that port to an environment variable in the test class setup and read it back from the other process. But the new mechanism doesn't allow you to always use the same predetermined port.

[David Muller]

Ah -- that makes sense now. Thanks for your help, Jeremy.