prepare_database_save in add_update_fields makes some custom fields be impossible.

[bear330]

Hi,

I wrote two types of custom field, ObjectField and ClassField.
ObjectField is used to store pickled object to database,
ClassField is used to store class (qualified name) to database.

All works fine before 1.0.4, but it broken after #10443.
This is because django introduce the prepare_database_save protocol.

This makes these two kinds of fields be impossible. This is also a issue for people who want to do the same thing (​http://www.djangosnippets.org/snippets/1694/).

I can do some hackish to django to make my ObjectField work. But for ClassField, there is no good way to do that.
I think the only and elegant way to fix that is to modify django's source. So I create this ticket.
This might not django's defect, but it will make things difficult.

I suggest that add some check in add_update_fields method (subqueries.py) to make it more flexible.

here is my ObjectField and ClassField implementation.

class ClassField(models.CharField):
    """
    Class field to store qualified class type to database.
    """

    __metaclass__ = SubfieldBase

    def __init__(self, *args, **kws):
        """
        Constructor.
        """
        kws['editable'] = False
        kws.setdefault('max_length', 256)
        super(ClassField, self).__init__(*args, **kws)

    def get_db_prep_save(self, value):
        if value and not isinstance(value, basestring):
            value = "%s.%s" % (value.__module__, value.__name__)

        return super(ClassField, self).get_db_prep_save(value)

    def to_python(self, value):
        try:
            value = classForName(super(ClassField, self).to_python(value))
        except exceptions.ValidationError:
            raise
        except:
            pass

        return value

    def get_db_prep_lookup(self, lookup_type, value):
        # We only handle 'exact' and 'in'. All others are errors.
        if lookup_type == 'exact':
            return [self.get_db_prep_save(value)]
        elif lookup_type == 'in':
            return [self.get_db_prep_save(v) for v in value]
        else:
            raise TypeError('Lookup type %r not supported.' % lookup_type)

    def get_internal_type(self):
        return 'CharField'

# If I try to update a model that have ClassField, it will raise error because line:
#            if hasattr(val, 'prepare_database_save'):
#                val = val.prepare_database_save(field)
#            else:
#                val = field.get_db_prep_save(val)
# I will got error because val is a Class type.

class ObjectField(models.TextField):
    """
    Object field to store object instance to database.
    """

    __metaclass__ = SubfieldBase

    def __init__(self, *args, **kws):
        """
        Constructor.
        """
        kws['editable'] = False
        super(ObjectField, self).__init__(*args, **kws)

    def _mock_prepare_database_save(self, this):
        """
        Dirty hack to django to make it work for update.
        Please see http://code.djangoproject.com/ticket/10443
        Mock original prepare_database_save to get_db_prep_save.

        @param this Please note, after mocking, this parameter is ObjectField,
                    self is value.
        @return get_db_prep_save.
        """
        return this.get_db_prep_save(self)

    def get_db_prep_save(self, value):
        # We must encode the data appropriately before passing it through in
        # a text field.
        # Please see: http://code.djangoproject.com/ticket/10398
        value = binascii.b2a_base64(pickle.dumps(value))
        return super(ObjectField, self).get_db_prep_save(value)

    def to_python(self, value):
        if not isinstance(value, basestring):
            # Dirty hack to django to make it work for update.
            # Please see http://code.djangoproject.com/ticket/10443
            if (hasattr(value, 'prepare_database_save') and
                value.__class__.prepare_database_save !=
                    self._mock_prepare_database_save.im_func):
                model = value.__class__
                model._prepare_database_save = model.prepare_database_save
                model.prepare_database_save = types.MethodType(
                    self._mock_prepare_database_save.im_func,
                    None, model)
            elif hasattr(value, '_prepare_database_save'):
                model = value.__class__
                model.prepare_database_save = model._prepare_database_save

            return value

        try:
            # Decode.
            value = pickle.loads(binascii.a2b_base64(value))
        except:
            pass

        return value

    def get_db_prep_lookup(self, lookup_type, value):
        # We only handle 'exact' and 'in'. All others are errors.
        if lookup_type == 'exact':
            return [self.get_db_prep_save(value)]
        elif lookup_type == 'in':
            return [self.get_db_prep_save(v) for v in value]
        else:
            raise TypeError('Lookup type %r not supported.' % lookup_type)

    def get_internal_type(self):
        return 'TextField'

[Alex Gaynor]

You have not described what it is about this change that makes your field impossible.

[bear330]

OK, for the ObjectField and ClassField, I can use it in models like this:

# This is just a example, no meaning.
class MyModel(models.Model):
    obj = ObjectField()
    clz = ClassField()

my = MyModel()
my.obj = User.object.all()[0] 
my.clz = User
my.save() # This is OK.

my.obj = User.object.all()[-1] # Change another user.
my.save() # Update it, but it will raise error because User (all models) has prepare_database_save attribute, so the code in add_update_fields method (subqueries.py):
# if hasattr(val, 'prepare_database_save'): 
#    val = val.prepare_database_save(field) 
# else: 
#    val = field.get_db_prep_save(val) 
#
# val (user object) will be changed and I can't do anything before that.
# So, I do some hackish in ObjectField above to make this work.

# But for ClassField, this is no good way to do that.
my.clz = Profile
my.save() # It will raise error, because hasattr(val, 'prepare_database_save') is True,
# it will call val = val.prepare_database_save(field) 
# then:
# TypeError: unbound method prepare_database_save() must be called with Profile instance as first argument.

This is why I said that is impossible do to this kind of custom field. Thanks!

[Russell Keith-Magee]

I'm going to mark this wontfix. I can see that you're having a problem, but you're in a corner case that is in direct conflict with something Django needs to do. Some difficulty is to be expected.

Django uses duck typing in a number of places in the query compilation system. Methods like as_sql, get_placeholder, and prepare_database_save are used to determine how to prepare a value for use in a query. This is how we are able to support GIS data types.

My suggestion to you would be to introduce a wrapper class. Just as GIS types are a wrapper for a blob of GIS geometries, you should be able to write a wrapper that can hold objects or classes. Override the set and get method on field to apply and remove these wrappers.

Yes, it will take some work, but like I said - you're in a corner case here.

[Sebastián Hernández]

So, how did you handled this wontfix situation? Did you implement a working ClassField?