Indeterministic PicklingError

[etam]

There is something wrong with session:

[Thu Jul 08 19:15:38 2010] [error] [client 79.162.31.162] mod_wsgi (pid=3072): Exception occurred processing WSGI script '/home/www/shop/django.wsgi'., referer: http://shop.domain.com/accounts/checkout/?
[Thu Jul 08 19:15:38 2010] [error] [client 79.162.31.162] Traceback (most recent call last):, referer: http://shop.domain.com/accounts/checkout/?
[Thu Jul 08 19:15:38 2010] [error] [client 79.162.31.162]   File "/usr/lib/python2.5/site-packages/django/core/handlers/wsgi.py", line 245, in __call__, referer: http://shop.domain.com/accounts/checkout/?
[Thu Jul 08 19:15:38 2010] [error] [client 79.162.31.162]     response = middleware_method(request, response), referer: http://shop.domain.com/accounts/checkout/?
[Thu Jul 08 19:15:38 2010] [error] [client 79.162.31.162]   File "/usr/lib/python2.5/site-packages/django/contrib/sessions/middleware.py", line 36, in process_response, referer: http://shop.domain.com/accounts/checkout/?
[Thu Jul 08 19:15:38 2010] [error] [client 79.162.31.162]     request.session.save(), referer: http://shop.domain.com/accounts/checkout/?
[Thu Jul 08 19:15:38 2010] [error] [client 79.162.31.162]   File "/usr/lib/python2.5/site-packages/django/contrib/sessions/backends/db.py", line 57, in save, referer: http://shop.domain.com/accounts/checkout/?
[Thu Jul 08 19:15:38 2010] [error] [client 79.162.31.162]     session_data = self.encode(self._get_session(no_load=must_create)),, referer: http://shop.domain.com/accounts/checkout/?
[Thu Jul 08 19:15:38 2010] [error] [client 79.162.31.162]   File "/usr/lib/python2.5/site-packages/django/contrib/sessions/backends/base.py", line 88, in encode, referer: http://shop.domain.com/accounts/checkout/?
[Thu Jul 08 19:15:38 2010] [error] [client 79.162.31.162]     pickled = pickle.dumps(session_dict, pickle.HIGHEST_PROTOCOL), referer: http://shop.domain.com/accounts/checkout/?
[Thu Jul 08 19:15:38 2010] [error] [client 79.162.31.162] PicklingError: Can't pickle <class 'decimal.Decimal'>: it's not the same object as decimal.Decimal, referer: http://shop.domain.com/accounts/checkout/?

It happens sometimes, not always.

[etam]

There is similar problem (thanks to Dan Breslau from stackoverflow.com - ​http://stackoverflow.com/questions/3292383/django-mod-wsgi-picklingerror-while-saving-object):

>>> import cPickle
>>> class Foo(object):
...     pass
... 
>>> f = Foo()
>>> s = cPickle.dumps(f)
>>>
>>> # Redefine class Foo 
>>> class Foo(object):
...     pass
... 
>>> # Now attempt to pickle the same object that was created with the old Foo class
>>> s = cPickle.dumps(f)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
cPickle.PicklingError: Can't pickle <class '__main__.Foo'>: it's not the same object as __main__.Foo
>>>
>>> # Create an object with the new Foo class, and try to pickle it (this works)
>>> f2 = Foo()
>>> s = cPickle.dumps(f2)

[Adam Nelson]

This looks more like a wsgi issue - maybe post a ticket there?

[aweakley]

There seems to be more detail about this specific instance here: ​http://stackoverflow.com/questions/3292383/django-mod-wsgi-picklingerror-while-saving-object

This page indicates that pickle for certain types of object may be fragile with mod_wsgi: ​http://code.google.com/p/modwsgi/wiki/IssuesWithPickleModule

If the decimal module came to be reloaded somehow between the creation of the Decimal instance and the attempt to pickle it, then that might give this error:

>>> import decimal
>>> three = decimal.Decimal(3)
>>> cPickle.dumps(three)
"cdecimal\nDecimal\np1\n(S'3'\ntRp2\n."
>>> reload(decimal)
<module 'decimal' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/decimal.pyc'>
>>> cPickle.dumps(three)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
cPickle.PicklingError: Can't pickle <class 'decimal.Decimal'>: it's not the same object as decimal.Decimal

We need some more information about the server environment and wsgi setup before we could write a test case that could reproduce the problem.

[mariarchi]

=> it's related to mod_wsgi, not Django. Moreover, using pickling in your web application is usually a bad idea, because pickled files are easily corruptible and can cause security issues.