Performance improvements for HttpRequest.build_absolute_uri()

[gcbirzan]

In cases where a lot of URLs are generated, the performance of it is suboptimal.

• It calls self.get_host() several times. That function does a lot of work that isn't needed more than once.
• It calls urljoin() even on the trivial case of absolute paths (which is the most common case). urljoin is quite expensive as well.

My patch fixes these by using a cached_property for the scheme://host part and by just concatenating the current scheme/host with the location when the location is absolute, doesn't change the host, and doesn't contain '.' or '..'. The last can be implemented in there, but that will just complicate the code I think, for again, a not very common case.

All the tests pass, but they did even when I wasn't checking for '.' and '..', so I added a test for that too.

While the improvements might be minor for some use cases, an example of this being slow can be found in this ​DRF ticket.

I've made a ​PR with these changes.

[gcbirzan]

I've posted these in the PR comments, but, some benchmarks:

I ran this benchmark:

timeit.timeit("request.build_absolute_uri(location='///foo/bar/')", number=1000000, globals={'request': request})

The results were:

• With my fix: 4.474267777055502
• With my fix but with bits.path[0] == '/': 4.34382488578558
• My shortcut but no cached property: 9.473239112645388 (this is to simulate running this on different requests)
• No shortcut but with cached property: 12.506602805107832
• Original version: 17.600296460092068

There is a regression. Running without the cached property, to simulate calling it in different requests, on a path with '.':

timeit.timeit("request.build_absolute_uri(location='/foo/./bar/')", number=1000000, globals={'request': request}))

The results:

• My version, no cached property: 19.713809736073017
• Original version: 18.129451751708984

So, the extra checks do add some overhead, but it's for an uncommon case. I don't have any evidence, obviously, but I'm fairly confident that the overwhelming uses build_absolute_url() are with an absolute path that doesn't have '.' or '..' in it and that the result is coming from reverse.

[Keryn Knight]

Anecdata verifying the improvement, using the following code against Python 3.5.1 on OSX:

from django.test.client import RequestFactory
import timeit
request = RequestFactory().get('/')
timeit.timeit("request.build_absolute_uri(location='///foo/bar/')", number=1000000, globals={'request': request})
request = RequestFactory().get('/')
timeit.timeit("request.build_absolute_uri(location='/foo/./bar/')", number=1000000, globals={'request': request})

Master as of d60e8b856b49922deb85a168e48e56f16facd5df consistently yields 50.x-51.x seconds for each timeit usage.
PR yields 9 seconds for the ///foo/bar/ case, and 33 seconds for the less common /foo/./bar/ case.

[Tim Graham <timograham@…>]

In 5bf62825:

Fixed #28828 -- Improved performance of HttpRequest.build_absolute_uri().