select_related() changes type of DecimalField

[Carsten Fuchs]

As originally reported at ​http://groups.google.com/group/django-users/browse_thread/thread/f797684ef2670ed2:

Using Django 1.3 with Python 2.6.5 on Ubuntu 10.04 and mod_wsgi, with Oracle database, I've just experienced a case where the use of select_related() changes the result type of a DecimalField in a related object from decimal.Decimal to float (which in turn breaks my application).

In detail, please consider the following models (unrelated fields omitted for clarity):

class Code(models.Model):
    id        = models.BigIntegerField(primary_key=True)
    grenzwert = models.DecimalField(null=True, max_digits=5, decimal_places=2, blank=True)

class Erfasst(models.Model):
    id   = models.AutoField(primary_key=True)
    code = models.ForeignKey(Code, db_column='code')

And the following shell session:

lori@keep-surfing:~/Zeiterfassung$ python manage.py shell
Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from Zeiterfassung.Lori.models import *

>>> z = Erfasst.objects.get(id=2726498)
>>> z.code.grenzwert
Decimal('10.00')
>>> type(z.code.grenzwert)
<class 'decimal.Decimal'>
>>> z.code.grenzwert > Decimal(0)
True

>>> z = Erfasst.objects.select_related().get(id=2726498)
>>> z.code.grenzwert
10.0
>>> type(z.code.grenzwert)
<type 'float'>
>>> z.code.grenzwert > Decimal(0)
False

(The last line is the one that caused/causes the problems in my app...)

In the second part of the example with select_related(), I'd have expected that it returns type decimal.Decimal as in the first.

Python 2.7 seems to handle Decimal-float comparisons differently/better than 2.6, and the above would probably silently work with Python 2.7, but not 2.6.

[Luke Plant]

I can't reproduce this. I'm using Oracle 10g Express Edition, instantclient 11.2. I tested with Python 2.5 and Python 2.6, Django trunk, and got the following results:

In [1]: import decimal

In [2]: from ticket15766.models import *

In [3]: c = Code(grenzwert=decimal.Decimal('10.00'), id=1)

In [4]: c.save()

In [5]: e = Erfasst(code=c)

In [6]: e.save()

In [7]: Erfasst.objects.get(id=1).code.grenzwert
Out[7]: Decimal("10.00")

In [8]: Erfasst.objects.select_related().get(id=1).code.grenzwert
Out[8]: Decimal("10")

In [9]: type(Erfasst.objects.select_related().get(id=1).code.grenzwert)
Out[9]: <class 'decimal.Decimal'>

If you can provide enough info for us to reproduce it, please do so and re-open this ticket.

Also, I notice that in my results, the value Decimal("10") given with select_related() is strictly speaking different from Decimal("10.00"). I don't have enough experience with Decimal to know if this is a material difference in itself. If it is, then you could re-open on that basis too. But the info to track down your exact problem would probably be useful anyway.

[Carsten Fuchs]

Ok, sorry for not having provided more info earlier.

Using the attached models.py file, I can reproduce the issue with these steps:

carsten@thubi:~/Zeiterfassung$ python manage.py shell
Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)

>>> from ticket15766.models import *
>>> import decimal

>>> c = Code(text="test #15766", grenzwert=decimal.Decimal('10.00'))
>>> c.save()
>>> c.id     # No output, probably because it's a BigIntegerField instead of an AutoField.
>>> c = Code.objects.get(id=104)
>>> c.text
u'test #15766                                                 '

>>> m = Mitarbeiter(name="test", key="test2", code=c)
>>> m.save()
>>> m.id
7479L

>>> s = Status(status="test #15766", statustext="#15766")
>>> s.save()
>>> s.id
64L

>>> k = Kostenstelle(name="test #15766")
>>> k.save()
>>> k.id
445L

>>> e = Erfasst(code=c, key=m, status=s, realkst=k)
>>> e.save()
>>> e.id
2457639L

>>> Erfasst.objects.get(id=2457639).code.grenzwert
Decimal('10.00')
>>> type(Erfasst.objects.get(id=2457639).code.grenzwert)
<class 'decimal.Decimal'>

>>> Erfasst.objects.select_related().get(id=2457639).code.grenzwert
10.0
>>> type(Erfasst.objects.select_related().get(id=2457639).code.grenzwert)
<type 'float'>

Note that Code is referenced by foreign key both from Erfasst.code as well as from Erfasst.key.code, maybe that is the cause of the issue?

I'd be very happy if you had another look at this. :-)

[Carsten Fuchs]

The models.py file referred to in comment #2 to reproduce the issue.

[Luke Plant]

I can't get past the c.save() line:

With Oracle:

IntegrityError: ORA-01400: cannot insert NULL into ("TEST"."CODE"."ID")

Which is what it must do. It could only succeed if your database actually is allowing a NULL primary key for the field, which isn't what the model says. If we've got one mismatch between what the model says and what the database allows, we may have others, perhaps including the field showing the bug. I don't think we are intending to support the case where the model disagrees with the database about what the field is.

So, if you can produce a minimal project that shows this issue when you create a database from scratch using syncdb etc, please post it and re-open. Thanks!

So you've got something else going on here.

[Carsten Fuchs]

lukeplant, you are right:
I had to start my project with a legacy Oracle database, and used manage.py inspectdb to create the initial models.py, and never realized that it had a mix of database fields of type FLOAT and NUMBER that in models.py were both addressed with DecimalField(...).

Changing the database to use type NUMBER instead of FLOAT fixes my problem (even though the Oracle docs seem to indicate that FLOAT is "equivalent" to NUMBER...).

Please forgive me the false alarm, I'm very sorry for the trouble.
Many thanks for your patience and help!

[anonymous]

xcsdasdasd