Opened 9 years ago

Closed 9 years ago

Last modified 9 years ago

#26324 closed Bug (fixed)

DurationField stores certain values incorrectly in sqlite

Reported by: chausner Owned by: nobody
Component: Database layer (models, ORM) Version: 1.8
Severity: Release blocker Keywords: sqlite, DurationField
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by chausner)

When encoding certain timedelta values in a DurationField, saving the object to a sqlite database and then reading it again, DurationField returns None.

class Foo(models.Model):
    bar = models.DurationField()

Foo.objects.create(bar=timedelta(seconds=2.05))
Foo.objects.create(bar=timedelta(seconds=3.05))

print str(Foo.objects.all()[0].bar) # 'None'
print str(Foo.objects.all()[1].bar) # '3.05'

I can reproduce this bug with Django 1.8.5 and 1.9.3 using sqlite as database backend. It does not occur with MySQL, so it seems to be related to how sqlite stores bigints. It renders DurationField essentially useless with sqlite.

Attachments (1)

26324-test.diff (621 bytes ) - added by Tim Graham 9 years ago.

Download all attachments as: .zip

Change History (10)

comment:1 by chausner, 9 years ago

Description: modified (diff)

comment:2 by Tim Graham, 9 years ago

Severity: NormalRelease blocker
Triage Stage: UnreviewedAccepted
Version: 1.91.8

Problem seems to be in parse_duration(). It can't parse the resulting value and so converts it to None.

Accepting as a release blocker due to possible data corruption. Regression test attached.

by Tim Graham, 9 years ago

Attachment: 26324-test.diff added

comment:3 by chausner, 9 years ago

Not sure if it's of any help but this bug report here looks related: https://github.com/potatolondon/djangae/issues/512.

comment:4 by Neraste, 9 years ago

I am currently running into the very same problem (Django 1.8.6).

The get_db_prep_value method of DurationField class always returns a float to the database:

return value.total_seconds() * 1000000

But according to the doc, it has to be stored as a BigInt in SQLite database.

Analyzing Django debug logs shows the SQLite driver always receive float values. Some are harmless (eg: 119019000.0), others are harmfull (eg: 133592000.00000001). The last ones come from the funny side effects of floating point computation:

>>> 133.592 * 1000000
133592000.00000001

I think it can be fixed with:

return int(round(value.total_seconds() * 1000000))
Last edited 9 years ago by Neraste (previous) (diff)

comment:5 by Tim Graham, 9 years ago

Has patch: set

comment:6 by Simon Charette, 9 years ago

Triage Stage: AcceptedReady for checkin

comment:7 by Tim Graham <timograham@…>, 9 years ago

Resolution: fixed
Status: newclosed

In 4f0cd0fd:

Fixed #26324 -- Fixed DurationField with fractional seconds on SQLite.

comment:8 by Tim Graham <timograham@…>, 9 years ago

In 5b6c7512:

[1.9.x] Fixed #26324 -- Fixed DurationField with fractional seconds on SQLite.

Backport of 4f0cd0fd162122da96978b357ac9fc9534529410 from master

comment:9 by Tim Graham <timograham@…>, 9 years ago

In c7764ca3:

[1.8.x] Fixed #26324 -- Fixed DurationField with fractional seconds on SQLite.

Backport of 4f0cd0fd162122da96978b357ac9fc9534529410 from master

Note: See TracTickets for help on using tickets.
Back to Top