Python 3.5 compatibility

[Tim Graham]

Python 3.5 is not scheduled for final until September 2015, but this is a tracking ticket for compatibility fixes for Django submitted in the meantime.

[Tim Graham <timograham@…>]

In b07aa52e8a8e4c7fdc7265f75ce2e7992e657ae9:

Added a dummy class for HTMLParserError; refs #23763.

[Tim Graham]

All the tests currently pass except for some ImportError exceptions being reported, likely due to ​http://bugs.python.org/issue7559.

======================================================================
ERROR: apps.failing_app (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/cpython/Lib/unittest/case.py", line 58, in testPartExecutor
    yield
  File "/home/tim/code/cpython/Lib/unittest/case.py", line 577, in run
    testMethod()
  File "/home/tim/code/cpython/Lib/unittest/loader.py", line 36, in testFailure
    raise exception
ImportError: Failed to import test module: apps.failing_app
Traceback (most recent call last):
  File "/home/tim/code/cpython/Lib/unittest/loader.py", line 451, in _find_test_path
    package = self._get_module_from_name(name)
  File "/home/tim/code/cpython/Lib/unittest/loader.py", line 358, in _get_module_from_name
    __import__(name)
  File "/home/tim/code/django/tests/apps/failing_app/__init__.py", line 1, in <module>
    raise ImportError("Oops")
ImportError: Oops


======================================================================
ERROR: migrations.faulty_migrations.import_error (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/cpython/Lib/unittest/case.py", line 58, in testPartExecutor
    yield
  File "/home/tim/code/cpython/Lib/unittest/case.py", line 577, in run
    testMethod()
  File "/home/tim/code/cpython/Lib/unittest/loader.py", line 36, in testFailure
    raise exception
ImportError: Failed to import test module: migrations.faulty_migrations.import_error
Traceback (most recent call last):
  File "/home/tim/code/cpython/Lib/unittest/loader.py", line 451, in _find_test_path
    package = self._get_module_from_name(name)
  File "/home/tim/code/cpython/Lib/unittest/loader.py", line 358, in _get_module_from_name
    __import__(name)
  File "/home/tim/code/django/tests/migrations/faulty_migrations/import_error/__init__.py", line 1, in <module>
    import fake_python_module  # NOQA
ImportError: No module named 'fake_python_module'

Also, some GIS related exceptions like the following if GIS dependencies aren't installed:

======================================================================
ERROR: django.contrib.gis.admin (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/cpython/Lib/unittest/case.py", line 58, in testPartExecutor
    yield
  File "/home/tim/code/cpython/Lib/unittest/case.py", line 577, in run
    testMethod()
  File "/home/tim/code/cpython/Lib/unittest/loader.py", line 36, in testFailure
    raise exception
ImportError: Failed to import test module: django.contrib.gis.admin
Traceback (most recent call last):
  File "/home/tim/code/cpython/Lib/unittest/loader.py", line 451, in _find_test_path
    package = self._get_module_from_name(name)
  File "/home/tim/code/cpython/Lib/unittest/loader.py", line 358, in _get_module_from_name
    __import__(name)
  File "/home/tim/code/django/django/contrib/gis/admin/__init__.py", line 7, in <module>
    from django.contrib.gis.admin.options import GeoModelAdmin      # NOQA
  File "/home/tim/code/django/django/contrib/gis/admin/options.py", line 2, in <module>
    from django.contrib.gis.admin.widgets import OpenLayersWidget
  File "/home/tim/code/django/django/contrib/gis/admin/widgets.py", line 9, in <module>
    from django.contrib.gis.geos import GEOSGeometry, GEOSException
ImportError: cannot import name 'GEOSGeometry'

[Tim Graham <timograham@…>]

In 42b5e4feeacf7cfa57867bf9fd5a6046de8c1cd3:

Fixed #23730 -- Moved support for SimpleCookie HIGHEST_PROTOCOL pickling to http.cookie.

This fix is necessary for Python 3.5 compatibility (refs #23763).

Thanks Berker Peksag for review.

[Tim Graham <timograham@…>]

In be1357e70983d4ad029a1ecdd05292f8be917a80:

Fixed a query failure on Python 3.5; refs #23763.

The failure was introduced in Django by
c7fd9b242d2d63406f1de6cc3204e35aaa025233 and the change in
Python 3.5 is ​https://hg.python.org/cpython/rev/a3c345ba3563.

[Tim Graham <timograham@…>]

In cb90d489da4247c1c7ced86a42a2d89488fec67c:

[1.8.x] Fixed a query failure on Python 3.5; refs #23763.

The failure was introduced in Django by
c7fd9b242d2d63406f1de6cc3204e35aaa025233 and the change in
Python 3.5 is ​https://hg.python.org/cpython/rev/a3c345ba3563.

Backport of be1357e70983d4ad029a1ecdd05292f8be917a80 from master

[Tim Graham <timograham@…>]

In 0386b97706052b88cd6fbbf777698810981cfeb6:

Fixed test_runner test failure on Python 3.5; refs #23763.

Python change is ​http://bugs.python.org/issue22032

[Tim Graham <timograham@…>]

In 242c9538c8d252f27581e2b192237301f94a2928:

[1.8.x] Fixed test_runner test failure on Python 3.5; refs #23763.

Python change is ​http://bugs.python.org/issue22032

Backport of 0386b97706052b88cd6fbbf777698810981cfeb6 from master

[Tim Graham <timograham@…>]

In 62df1834b8d60c106c8c16524b275b8a1f47ac3a:

Fixed expressions test on Python 3.5; refs #23763.

[Tim Graham <timograham@…>]

In fe770a6452919807bba59c125b02464290feeba1:

[1.8.x] Fixed expressions test on Python 3.5; refs #23763.

Backport of 62df1834b8d60c106c8c16524b275b8a1f47ac3a from master

[Tim Graham <timograham@…>]

In c0cc8f69e7abfa8578729031f97ae4b96c5cdafe:

Refactored tests that rely on an ImportError for Python 3.5 compatibility

A change in Python test discovery [1] causes the old packages that raised
an error to be discovered; now we use a common directory that's
ignored during discovery. Refs #23763.

[1] ​http://bugs.python.org/issue7559

[Tim Graham <timograham@…>]

In 3e24ab6f4ca6632c910c684fda3d0f937fade52a:

[1.8.x] Refactored tests that rely on an ImportError for Python 3.5 compatibility

A change in Python test discovery [1] causes the old packages that raised
an error to be discovered; now we use a common directory that's
ignored during discovery. Refs #23763.

[1] ​http://bugs.python.org/issue7559

Backport of c0cc8f69e7abfa8578729031f97ae4b96c5cdafe from master

[Tim Graham <timograham@…>]

In 4e59156c:

Fixed sessions test on Python 3.5; refs #23763.

SimpleCookie.repr() changed in
​https://hg.python.org/cpython/rev/88e1151e8e02

[Tim Graham <timograham@…>]

In 7b9f7b66:

[1.8.x] Fixed sessions test on Python 3.5; refs #23763.

SimpleCookie.repr() changed in
​https://hg.python.org/cpython/rev/88e1151e8e02

Backport of 4e59156c10d36eeb1e6fced4dff0f11157f68264 from master

[Tim Graham <timograham@…>]

In c2bc1ce:

Refs #23763 -- Silenced SimpleTestCase.assertRaisesMessage() deprecation warning on Python 3.5.

Deprecation warning was introduced in ​https://bugs.python.org/issue24134

[Tim Graham <timograham@…>]

In 7d97c57:

Refs #23763 -- Fixed SMTPServer Python 3.5 deprecation warning in mail test.

[Tim Graham <timograham@…>]

In e89c3a46:

Added backwards compatibility for assertRaisesMessage callable_obj param.

This was broken in c2bc1cefdcbbf074408f4a4cace88b315cf9d652 (refs #23763).

[Tim Graham <timograham@…>]

In 3f2de803:

Refs #23763 -- Fixed Python 3.5 PendingDeprecationWarning in LazyStream.

Fixed "PendingDeprecationWarning: generator 'LazyStream.read.<locals>.parts'
raised StopIteration" per PEP 0479.

[Barry Warsaw]

Any thoughts on the GEOSGeometry import error on Python 3 only? I think I've applied all the relevant patches here for Ubuntu's build of 1.7.9 for Python 2.7, 3.4, and 3.5, and this is the last failure I can't yet eliminate. Well, I can if I depend on libgeos-dev which gives you the right library, but I don't want to add that dependency because of cross-pocket issues (django-python is in main, but libgeos-dev is in universe, and that would require extra bureaucracy to change).

I've confirmed that HAS_SPACIAL_DB is False in both Python 2 and 3, and django.contrib.gis.tests doesn't get added to discovery_paths (in runtests.py), but yet those tests are still discovered.

Is there some patch not yet attached to this issue, or some other ticket or commit I should be looking at?

[Tim Graham]

I don't believe it's a known issue. It doesn't seem related to Python 3.5 compatibility, so please open a new ticket with more specifics, thanks!

[Tim Graham]

The test suite for master (Django 1.9) passes on Python 3.5.

[Tim Graham <timograham@…>]

In 2a36a9bb:

[1.8.x] Refs #23763 -- Fixed Python 3.5 PendingDeprecationWarning in LazyStream.

Fixed "PendingDeprecationWarning: generator 'LazyStream.read.<locals>.parts'
raised StopIteration" per PEP 0479.

Backport of 3f2de803181ca3c5526ec9d708b2098b8f683808 from master

[Tim Graham <timograham@…>]

In 95b1ae76:

[1.8.x] Refs #23763 -- Fixed SMTPServer Python 3.5 deprecation warning in mail test.

Backport of 7d97c5745e420e3edc22965a2c68c8c9d2eab374 from master