Opened 10 years ago

Closed 8 years ago

#23222 closed Bug (wontfix)

Empty BinaryField != b'' on Python 2

Reported by: wkschwartz@… Owned by: Grzegorz Ślusarek
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords: py2
Cc: cmawebsite@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by William Schwartz)

Problem

In Python 2, at least under SQLite, the initial value for an empty binary field behaves inconsistently. The ORM thinks it's an empty bytes: b''. The database connection manager thinks it's a buffer. Now, the buffer evaluates to False and has zero length. So it'll mostly work. But not always -- and most importantly to me, not in my unit tests!

See http://stackoverflow.com/q/12625557/1286628

Note this was not a problem under Python 3.4.

Steps to Reproduce

Using Python 2.7.8, SQLite, and either Django 1.7rc2 or Django @ edcc75e5ac5b9dc2f174580e7adacd3be586f8bd (HEAD at the time of this writing; the error exists in both places)

  1. Make a new project and a new app and add the app to settings.py
  2. Fill in app/models.py as follows
from django.db import models                                                                                                         
class BinModel(models.Model):                                                                                                        
    data = models.BinaryField()                                                                                                      
  1. Run from the command line:
(venv) $ ./manage.py makemigrations app && ./manage.py migrate && ./manage.py shell
  1. Run from the resulting Python shell
>>> from app import models; m = models.BinModel(); m.save(); n = models.BinModel.objects.get()
>>> m.data
''
>>> m.data == b''
True
>>> n.data
<read-write buffer ptr 0x10eaa62b0, size 0 at 0x10eaa6270>
>>> n.data == b''
False
>>> bool(n.data)
False
>>> len(n.data)
0
>>> bytes(n.data)
''

Note that the same problem persisted when I had a default value for the field. There was no problem under Python 3.4.

Change History (9)

comment:1 by Collin Anderson, 10 years ago

Cc: cmawebsite@… added
Severity: Release blockerNormal
Version: 1.7-rc-2master

I'm seeing the same behavior on django 1.6.0 and stable/1.6.x, so I don't think this is a release blocker.

Last edited 10 years ago by Collin Anderson (previous) (diff)

comment:2 by Areski Belaid, 10 years ago

I dont think it's SQLite related, I have the same output using PostgreSQL

comment:3 by Collin Anderson, 10 years ago

Summary: Empty BinaryField != b'' in Python 2/SQLiteEmpty BinaryField != b'' on Python 2

comment:4 by Baptiste Mispelon, 10 years ago

Triage Stage: UnreviewedAccepted

Marking this as accepted per previous comments.

This seem like it might be caused by six using buffers on python 2 [1].

Thanks.

[1] https://github.com/django/django/blob/master/django/utils/six.py#L681

comment:5 by ppYang, 10 years ago

I'm not sure, if it's only a problem with empty BinaryField.

I try to add a test function in model_fields.tests.BinaryFieldTests, as below:

    def test_retrieve_compare_non_empty_field(self):
        dm = DataModel(data=b'test')
        self.assertTrue(dm.data == b'test')
        dm.save()
        dm = DataModel.objects.get(pk=dm.pk)
        self.assertTrue(dm.data == b'test')

On python2, the first assertion will be passed, and the second one will be failed.

As the data field is a buffer type after retrieving, we can't do a simple compare.

We can see the definition:

class BinaryField(Field):
    ...
    def get_db_prep_value(self, value, connection, prepared=False):
        value = super(BinaryField, self).get_db_prep_value(value, connection, prepared)
        if value is not None:
            return connection.Database.Binary(value)
        return value

I can't go further as I know little about six, and hope this can help:)

Last edited 10 years ago by ppYang (previous) (diff)

comment:6 by Grzegorz Ślusarek, 10 years ago

Owner: changed from nobody to Grzegorz Ślusarek
Status: newassigned

comment:7 by William Schwartz, 10 years ago

Description: modified (diff)

comment:8 by Tim Graham, 8 years ago

Keywords: py2 added

comment:9 by Tim Graham, 8 years ago

Resolution: wontfix
Status: assignedclosed

Closing due to the end of Python 2 support in master in a couple weeks.

Note: See TracTickets for help on using tickets.
Back to Top