Opened 8 years ago

Closed 8 years ago

#27475 closed Bug (fixed)

Bug in admin with date_hierarchy and Daylight savings

Reported by: Hugo Tácito Owned by: nobody
Component: contrib.admin Version: dev
Severity: Normal Keywords:
Cc: desecho@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When I have a date_hierarchy in admin and I select a range with a record with the day equal the day of a daylight savings I got an error of NonExistentTimeError.

I discovered that indeed 2016-10-16 00:00:00 does not exist in my Timezone ('America/Sao_Paulo') because that was when Daylight Savings started.

So when I have a record with that date in my database, it will brake the date_hierarchy feature.

Attachments (1)

djtest.zip (14.3 KB ) - added by Anton Samarchyan 8 years ago.

Download all attachments as: .zip

Change History (12)

comment:1 by Tim Graham, 8 years ago

I'm not sure what the expected behavior is. What could Django do if you have bad data?

comment:2 by Hugo Tácito, 8 years ago

Hello Tim,

I'm sorry for my lack of information and clarity, but even if I do have an existing datetime the problem still occur.

For example I have a User with date_joined equals 2016-10-16 12:17:44.270221-03 and when I select a date_hierarchy filter, that still raise the same exception of:

Environment:


Request Method: GET
Request URL: http://localhost:8000/admin/auth/user/

Django Version: 1.9.9
Python Version: 2.7.12
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'daylight']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']


Template error:
In template /Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/contrib/admin/templates/admin/change_list.html, error at line 73
   2016-10-16 00:00:00   63 :       {% endif %}
   64 :     {% endblock %}
   65 :     {% if cl.formset.errors %}
   66 :         <p class="errornote">
   67 :         {% if cl.formset.total_error_count == 1 %}{% trans "Please correct the error below." %}{% else %}{% trans "Please correct the errors below." %}{% endif %}
   68 :         </p>
   69 :         {{ cl.formset.non_form_errors }}
   70 :     {% endif %}
   71 :     <div class="module{% if cl.has_filters %} filtered{% endif %}" id="changelist">
   72 :       {% block search %}{% search_form cl %}{% endblock %}
   73 :       {% block date_hierarchy %} {% date_hierarchy cl %} {% endblock %}
   74 : 
   75 :       {% block filters %}
   76 :         {% if cl.has_filters %}
   77 :           <div id="changelist-filter">
   78 :             <h2>{% trans 'Filter' %}</h2>
   79 :             {% for spec in cl.filter_specs %}{% admin_list_filter cl spec %}{% endfor %}
   80 :           </div>
   81 :         {% endif %}
   82 :       {% endblock %}
   83 : 


Traceback:

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  174.                     response = self.process_exception_by_middleware(e, request)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  172.                     response = response.render()

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/response.py" in render
  160.             self.content = self.rendered_content

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/response.py" in rendered_content
  137.         content = template.render(context, self._request)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/backends/django.py" in render
  95.             return self.template.render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in render
  206.                     return self._render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in _render
  197.         return self.nodelist.render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in render
  992.                 bit = node.render_annotated(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in render_annotated
  959.             return self.render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  173.         return compiled_parent._render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in _render
  197.         return self.nodelist.render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in render
  992.                 bit = node.render_annotated(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in render_annotated
  959.             return self.render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  173.         return compiled_parent._render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in _render
  197.         return self.nodelist.render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in render
  992.                 bit = node.render_annotated(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in render_annotated
  959.             return self.render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  69.                 result = block.nodelist.render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in render
  992.                 bit = node.render_annotated(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in render_annotated
  959.             return self.render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  69.                 result = block.nodelist.render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in render
  992.                 bit = node.render_annotated(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py" in render_annotated
  959.             return self.render(context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/library.py" in render
  223.         _dict = self.func(*resolved_args, **resolved_kwargs)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/contrib/admin/templatetags/admin_list.py" in date_hierarchy
  380.                 } for day in days]

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/db/models/query.py" in __iter__
  258.         self._fetch_all()

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/db/models/query.py" in _fetch_all
  1074.             self._result_cache = list(self.iterator())

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/db/models/query.py" in __iter__
  158.         for row in compiler.results_iter():

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in results_iter
  808.                     row = self.apply_converters(row, converters)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in apply_converters
  792.                 value = converter(value, expression, self.connection, self.query.context)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/db/models/expressions.py" in convert_value
  950.             value = timezone.make_aware(value, self.tzinfo)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/utils/timezone.py" in make_aware
  364.         return timezone.localize(value, is_dst=is_dst)

File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/pytz/tzinfo.py" in localize
  327.                 raise NonExistentTimeError(dt)

Exception Type: NonExistentTimeError at /admin/auth/user/
Exception Value: 2016-10-16 00:00:00

Even with an existing datetime.

The admin I did to test:

class UserAdmin(admin.ModelAdmin):
    date_hierarchy = 'date_joined'
    list_display = ('email', 'first_name', 'last_name', 'date_joined')
    list_filter = ('is_staff', 'is_superuser')


admin.site.unregister(User)
admin.site.register(User, UserAdmin)

The internationalization settings:

LANGUAGE_CODE = 'pt-br'

TIME_ZONE = 'America/Sao_Paulo'

USE_I18N = True

USE_L10N = True

USE_TZ = True

With TIME_ZONE = 'UTC' the problem doesn't occur.

Edit:
I've did this in a new clean project, with just this object at the User model.

Last edited 8 years ago by Hugo Tácito (previous) (diff)

comment:3 by Hugo Tácito, 8 years ago

This also happens if we set a datetime to 2015-10-18. For example: 2015-10-18 10:45:29.270221-03
In 2015 at 10-18 was started the daylight savings at Brazil too.

https://www.timeanddate.com/time/change/brazil/sao-paulo?year=2015

comment:4 by Tim Graham, 8 years ago

Any idea what Django should do to resolve this?

comment:5 by Hugo Tácito, 8 years ago

Hello Tim,

It seems that the datetime truncation is too hard in the date_hierarchy templatetag.

Changing this line:
https://github.com/django/django/blob/master/django/contrib/admin/templatetags/admin_list.py#L385

to:
days = getattr(days, dates_or_datetimes)(field_name, 'second')

Probably will fix it.
I can also create a patch if you want.

comment:6 by Aymeric Augustin, 8 years ago

Triage Stage: UnreviewedAccepted

The problem seems to stem from the fact that Django wants dates, but fetches them as datetimes, and applies the converter for datetimes, which causes an exception in that case.

It isn't possible to represent "2016-10-16 in local time in São Paulo" as datetime.datetime(2016, 10, 16, tzinfo=pytz.timezone('America/Sao_Paulo')) because that value doesn't exist. The only proper way to represent it is datetime.date(2016, 10, 16).

I believe the resolution should be along the lines of making the year / month / day truncation expressions cast their result to a DateField instead of a DateTimeField. Reducing the problem to a lower-level test case may help.

It seems possible that the same problem could occur with the hour truncation expression if some time zones have their DST between 1:30 and 2:30. I'm not sure what we could do about that; it seems much harder to fix.

by Anton Samarchyan, 8 years ago

Attachment: djtest.zip added

comment:7 by Anton Samarchyan, 8 years ago

Cc: desecho@… added
Version: 1.10master

I've attached a django project to reproduce the problem. Admin credentials - admin/AiNeuw3uOhvei7da
To reproduce, open /admin/app/test/?datemonth=10&dateyear=2016

Version 0, edited 8 years ago by Anton Samarchyan (next)

comment:8 by Anton Samarchyan, 8 years ago

Has patch: set

Added PR

comment:9 by Tim Graham, 8 years ago

Patch needs improvement: set

comment:10 by Anton Samarchyan, 8 years ago

Patch needs improvement: unset

comment:11 by Tim Graham <timograham@…>, 8 years ago

Resolution: fixed
Status: newclosed

In e88d2df:

Fixed #27475 -- Fixed NonExistentTimeError crash in ModelAdmin.date_hierarchy.

Note: See TracTickets for help on using tickets.
Back to Top