DeserializationError local variable 'pk' referenced before assignment (which hides real error)

[Lakin Wecker]

The first error is this:

Environment:


Request Method: GET
Request URL: http://localhost:8000/admin/artcollection/artobject/2298/history/734/

Django Version: 1.9.6
Python Version: 2.7.10
Installed Applications:
['saskartsboard.artcollection',
 'saskartsboard.options',
 'saskartsboard.descriptive_and_visual_dictionary',
 'grappelli',
 'versatileimagefield',
 'select2',
 'ckeditor',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'reversion']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']



Traceback:

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  149.                     response = self.process_exception_by_middleware(e, request)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  147.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapped_view
  149.                     response = view_func(request, *args, **kwargs)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  57.         response = view_func(request, *args, **kwargs)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/contrib/admin/sites.py" in inner
  244.             return view(request, *args, **kwargs)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/reversion/admin.py" in revision_view
  228.         return self.revisionform_view(request, version, self.revision_form_template or self._get_template_list("revision_form.html"), context)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/reversion/admin.py" in revisionform_view
  191.                 version.revision.revert(delete=True)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/reversion/models.py" in revert
  92.         safe_revert(version_set)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/reversion/models.py" in safe_revert
  31.                 version.revert()

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/reversion/models.py" in revert
  207.         self.object_version.save()

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/reversion/models.py" in object_version
  168.         return list(serializers.deserialize(self.format, data, ignorenonexistent=True))[0]

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/core/serializers/json.py" in Deserializer
  85.         six.reraise(DeserializationError, DeserializationError(e), sys.exc_info()[2])

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/core/serializers/json.py" in Deserializer
  79.         for obj in PythonDeserializer(objects, **options):

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/core/serializers/python.py" in Deserializer
  142.                     raise base.DeserializationError.WithData(e, d['model'], d.get('pk'), pk)

Exception Type: DeserializationError at /admin/artcollection/artobject/2298/history/734/
Exception Value: local variable 'pk' referenced before assignment

Which is caused by referring to the 'pk' variable on line 142, which is only set in the first iteration of the for loop on line 139. If you add a raise before line 142 you can see the real exception which is:

Environment:


Request Method: GET
Request URL: http://localhost:8000/admin/artcollection/artobject/2298/history/734/

Django Version: 1.9.6
Python Version: 2.7.10
Installed Applications:
['saskartsboard.artcollection',
 'saskartsboard.options',
 'saskartsboard.descriptive_and_visual_dictionary',
 'grappelli',
 'versatileimagefield',
 'select2',
 'ckeditor',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'reversion']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']



Traceback:

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  149.                     response = self.process_exception_by_middleware(e, request)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  147.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapped_view
  149.                     response = view_func(request, *args, **kwargs)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  57.         response = view_func(request, *args, **kwargs)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/contrib/admin/sites.py" in inner
  244.             return view(request, *args, **kwargs)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/reversion/admin.py" in revision_view
  228.         return self.revisionform_view(request, version, self.revision_form_template or self._get_template_list("revision_form.html"), context)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/reversion/admin.py" in revisionform_view
  191.                 version.revision.revert(delete=True)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/reversion/models.py" in revert
  92.         safe_revert(version_set)

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/reversion/models.py" in safe_revert
  31.                 version.revert()

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/reversion/models.py" in revert
  207.         self.object_version.save()

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/reversion/models.py" in object_version
  168.         return list(serializers.deserialize(self.format, data, ignorenonexistent=True))[0]

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/core/serializers/json.py" in Deserializer
  85.         six.reraise(DeserializationError, DeserializationError(e), sys.exc_info()[2])

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/core/serializers/json.py" in Deserializer
  79.         for obj in PythonDeserializer(objects, **options):

File "/media/mausoleum/Work/Development/saskartsboard-collections/env/local/lib/python2.7/site-packages/django/core/serializers/python.py" in Deserializer
  139.                     for pk in field_value:

Exception Type: DeserializationError at /admin/artcollection/artobject/2298/history/734/
Exception Value: 'NoneType' object is not iterable

Which is caused by not dealing with None values. I'm not exactly certain why the None value is set for this field value, but it is.

My best guess at this point is that this field used to be a ForeignKey but has been migrated migrated into a ManyToMany and when the object was first saved, the FK was None, but now the ManyToMany is empty.

If you believe this is an issue with django reversion, I will happily move it there, but I'm not so sure.

Changing line 139 to:

for pk in field_value or []:

Seems to get around the error.

[Tim Graham]

Could you try to put together a test case or minimal steps to reproduce that doesn't involve django-reversion?

[Lakin Wecker]

I'm not sure I fully understand how reversion is using the serialization so it would likely take me some time. My idea of a test case to try would be:

1. Create model with FK to another model.
2. Create instance of said model with None value.
3. Serialize said instance, save value.
4. Change field to M2M.
5. Migrate
6. Attempt to restore model from previously serialized data.

But I'm not sure when I'll find time to try this out. I also don't know how I would turn this into a unit test either.

[Tim Graham]

I guess it's probably a legitimate bug, but if someone cannot reproduce it upon investigation, feel free to close as "needsinfo".

[Baptiste Mispelon]

I managed to reproduce the described issue using the steps described in comment:2 (using dumpdata/loaddata).

With Python 3's chained exceptions it's a bit easier to see what happens but it wasn't too complicated or invasive to fix the issue properly so I think it's worth it.

I have a ​PR with some tests ready for review.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 738e9e61:

Fixed #26743 -- Fixed UnboundLocalError crash when deserializing m2m fields and value isn't iterable.