forms.DateTimeField() rendered with HiddenInput looses microseconds

[Thomas Güttler]

forms.DateTimeField() looses the microseconds if you render it with HiddenInput.

A patch and unittest are attached.

[Tobias McNulty]

if you just want this to work and you don't want to patch django you can use this:

​http://www.djangosnippets.org/snippets/1342/

[stryderjzw]

On a related note:

Rendering the DateTimeField as a hidden field, {{ field.as_hidden }}, will make the form invalid when submitting because the HiddenInput keeps the microseconds and the field validation does not.

[Russell Keith-Magee]

The proposed patch is extremely brittle - it only works providing you don't use a '.' in the your custom rendering of datetime. If you allow an input format of '%Y.%m.%d %H:%M:%S", the code will break hard.

If you dig into the code, it isn't just HiddenField that is affected by this - the default DateTimeField also loses milliseconds. Essentially, what is required is a way to parse milliseconds from a string. Python 2.6 adds %f to strftime/strptime, which will solve this problem - you can then allow "%Y-%m-%d %H:%M:%S.%f" as an input format. We may need to port strftime and strptime back from Python 2.6. Other options also welcome.

This problem:

1. Is Annoying
2. Has always existed in newforms, and probably in oldforms before that
3. Only affects a subset of the user community

Since the required fix will be quite big, and we're already overdue for v1.1, I'm going to drop this from the v1.1 roadmap. If someone wants to tackle porting strptime and strftime (and this includes resolving any licensing issues) and they can get a patch ready quickly, feel free to put this back on the v1.1 path.

[Tobias McNulty]

Slightly more acceptable approach, with a couple bug fixes:

class DateTimeWithUsecsField(forms.DateTimeField):
    def clean(self, value):
        if value and '.' in value: 
            value, usecs = value.rsplit('.', 1)
            usecs += '0'*(6-len(usecs)) # right pad with zeros if necessary
            try:
                usecs = int(usecs) 
            except ValueError: 
                raise ValidationError('Microseconds must be an integer') 
        else: 
            usecs = 0 
        cleaned_value = super(DateTimeWithUsecsField, self).clean(value)
        if cleaned_value:
            cleaned_value = cleaned_value.replace(microsecond=usecs)
        return cleaned_value

Also updated at ​http://www.djangosnippets.org/snippets/1342/

[bjunix]

The version of this ticket is 1.0. I guess this bug also affects trunk. Can someone confirm this and change the Version to SVN?

[Karen Tracey]

Replying to bjunix:

The version of this ticket is 1.0. I guess this bug also affects trunk. Can someone confirm this and change the Version to SVN?

If the version when reported was 1.0, and the ticket is still open, then likely the behavior is still present in trunk. (There's some chance an unrelated or unrealized-duplicate ticket changed the behavior, so it's only likely and not certain, but it seems unlikely that has happened in this particular case.)

Changing the version to SVN is unnecessary and actually unhelpful, for a couple of reasons. First, you lose the easy access to the information that the behavior has existed as far back as 1.0. Also, version being set to SVN is a possible indication of a regression: behavior that did not exist in previous released versions but exists now in trunk. Those tickets are particularly important to resolve before a release, so setting version to SVN for behavior that has existed since forever is counter-productive for the release team.

[bjunix]

Replying to kmtracey:
My bad. Thanks for the detailed explanation.

[Thomas Güttler]

I uploaded a new version which works with python2.5 and 2.6. I checked both versions with "./runtests.py --settings test_sqlite forms".

If you use Python2.6 nothing is special, since datetime.datetime.strptime can handle %f. If you use py2.5, the %f must be
at the end of the format string.

[Thomas Güttler]

Can someone please test this with Python 2.4?

​http://groups.google.de/group/django-developers/browse_frm/thread/d9e0d06cedeb1024

[Luke Plant]

There are lots of errors with Python 2.4, which all seem to boil down to this:

AttributeError: type object 'datetime.datetime' has no attribute 'strptime' 

[Karen Tracey]

See r13078 for how to replace strptime with something that works on 2.4.

[Thomas Güttler]

Updated patch, new: sys.version_info[:2]

[patchhammer]

datetime-microseconds-py24.patch fails to apply cleanly on to trunk

[dmclain]

sackcloth and ashes for bringing the hammer. Applies cleanly and tests run under 2.6 and 2.5

[Michael Thornhill]

Modified patch to apply to more date formats and added extra test cases to reflect this.
patch will apply with patch -p1 < ticket9459.patch
Tested with python 2.5 and python 2.7

[Andrew Godwin]

In [16347]:

Fixed #9459: forms.HiddenInput and DateTime field loses microseconds. Thanks to mt.

[Thomas Güttler]

Thank you mt and andrewgodwin for getting this patch into SVN.

[Claude Paroz]

In [17870]:

Removed pre-2.6 compatibility code in date-based form fields. Refs #9459.

[Thomas Güttler]

Unfortunately there is still a microseconds bug if L10N is active: #16888