django-1.8.1 fails test with pypy - FAIL: test_serialize_datetime_safe (migrations.test_writer.WriterTests)

[Justin]

This happens with pypy-2.5.1 and python-dateutil-2.4.2

test_written_spooled_temp (files.tests.SpooledTempTests) ... ok

======================================================================
ERROR: test_serialize_datetime (migrations.test_writer.WriterTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/tests/migrations/test_writer.py", line 240, in test_serialize_datetime
    self.assertSerializedEqual(datetime.datetime.utcnow)
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/tests/migrations/test_writer.py", line 176, in assertSerializedEqual
    self.assertEqual(self.serialize_round_trip(value), value)
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/tests/migrations/test_writer.py", line 172, in serialize_round_trip
    string, imports = MigrationWriter.serialize(value)
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/django/db/migrations/writer.py", line 465, in serialize
    "topics/migrations/#migration-serializing" % (value, get_docs_version())
ValueError: Cannot serialize: <bound method type.utcnow of <class 'datetime.datetime'>>
There are some values Django cannot serialize into migration files.
For more, see https://docs.djangoproject.com/en/1.8/topics/migrations/#migration-serializing

======================================================================
ERROR: test_simple_migration (migrations.test_writer.WriterTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/tests/migrations/test_writer.py", line 425, in test_simple_migration
    output = writer.as_string()
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/django/db/migrations/writer.py", line 166, in as_string
    operation_string, operation_imports = OperationWriter(operation).serialize()
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/django/db/migrations/writer.py", line 124, in serialize
    _write(arg_name, arg_value)
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/django/db/migrations/writer.py", line 75, in _write
    arg_string, arg_imports = MigrationWriter.serialize(item)
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/django/db/migrations/writer.py", line 303, in serialize
    item_string, item_imports = cls.serialize(item)
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/django/db/migrations/writer.py", line 377, in serialize
    return cls.serialize_deconstructed(path, args, kwargs)
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/django/db/migrations/writer.py", line 268, in serialize_deconstructed
    arg_string, arg_imports = cls.serialize(arg)
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/django/db/migrations/writer.py", line 465, in serialize
    "topics/migrations/#migration-serializing" % (value, get_docs_version())
ValueError: Cannot serialize: <bound method type.utcnow of <class 'datetime.datetime'>>
There are some values Django cannot serialize into migration files.
For more, see https://docs.djangoproject.com/en/1.8/topics/migrations/#migration-serializing

======================================================================
FAIL: test_serialize_datetime_safe (migrations.test_writer.WriterTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/tests/migrations/test_writer.py", line 263, in test_serialize_datetime_safe
    ("datetime.date(2014, 3, 31)", {'import datetime'})
  File "/var/tmp/portage/dev-python/django-1.8.1/work/Django-1.8.1/tests/migrations/test_writer.py", line 179, in assertSerializedResultEqual
    self.assertEqual(MigrationWriter.serialize(value), target)
AssertionError: Tuples differ: (u'datetime.datetime.date(2014... != (u'datetime.date(2014, 3, 31)'...

First differing element 0:
datetime.datetime.date(2014, 3, 31)
datetime.date(2014, 3, 31)

- (u'datetime.datetime.date(2014, 3, 31)', set([u'import datetime']))
?    ---------

+ (u'datetime.date(2014, 3, 31)', set([u'import datetime']))

----------------------------------------------------------------------

[Alexander Schrijver]

I've discovered the problem, I think it's a bug in PyPy. But I don't know if there is any Python API specification where this is specified in.

The difference lies in the way the __repr__ is made for date, time and datetime classes.

In pypy the __repr__ is determined as follows:

date: ​https://bitbucket.org/pypy/pypy/src/5d676752ae6613d0b423f4c1028ac47e1571d5fa/lib_pypy/datetime.py?at=default#cl-790
time: ​https://bitbucket.org/pypy/pypy/src/5d676752ae6613d0b423f4c1028ac47e1571d5fa/lib_pypy/datetime.py?at=default#cl-1281
datetime: ​https://bitbucket.org/pypy/pypy/src/5d676752ae6613d0b423f4c1028ac47e1571d5fa/lib_pypy/datetime.py?at=default#cl-1692

While in cpython it is done like this:

date: ​https://github.com/python/cpython/blob/master/Modules/_datetimemodule.c#L2633
time: ​https://github.com/python/cpython/blob/master/Modules/_datetimemodule.c#L3581
datetime: ​https://github.com/python/cpython/blob/master/Modules/_datetimemodule.c#L4442

Which causes the date, datetime and time classes in datetime_safe module to return the wrong repr value.

e.g. in cpython 2 and 3 repr(datetime_safe.date(2015, 1, 1)) would return:

date(2015, 1, 1)

While in pypy this would return:

datetime.date(2015, 1, 1)

And the same would happen for datetime and time.

There are three solutions I can think of:

• Override the __repr__ function in datetime_safe.{date,time,datetime} which return the representation consistently for both PyPy and cpython.
• Make an hard "if PyPy" condition and then don't prepend the 'datetime' in that case (this would be the ugly solution imo)
• Fix it in PyPy.

[Markus Holtermann]

After a brief talk to Maciej Fijalkowski in #pypy on IRC, his response was "we can probably fix it". I documented the inconsistency in PyPy's issue tracker: ​https://bitbucket.org/pypy/pypy/issue/2062 . Hence I'm closing this issue here.

[Markus Holtermann]

Fix is committed to PyPy: ​https://bitbucket.org/pypy/pypy/commits/8c313d798d123b85636a20f1c5a28c4eb8b5162b