Admin date_hierarchy filter by month displays an extra day at timezone boundary.

[Lavrenov Ivan]

When I authorized by user with not-UTC timezone, like America/Los_Angeles , and open filter by date in month, I see one extra day, that follows to the first day of the previous month

[Lavrenov Ivan]

representation of bug

[Tim Graham]

Could you be more specific about how to reproduce this? I thought you meant TIME_ZONE = 'Los-Angeles/America' but I get "Incorrect timezone setting". Which database are you using?

[Lavrenov Ivan]

The timezone is America/Los_Angeles, sorry for inaccuracy.
I'm using Mysql database. I think it is something сonnected with converstion to local time. I have a record, which in UTC in the 1st September, but in local time it is still in the 31 August.

P.S. Values stored in UTC format

[Lavrenov Ivan]

Replying to Tim Graham:

Could you be more specific about how to reproduce this? I thought you meant TIME_ZONE = 'Los-Angeles/America' but I get "Incorrect timezone setting". Which database are you using?

The timezone is America/Los_Angeles, sorry for inaccuracy.
I'm using Mysql database. I think it is something сonnected with converstion to local time. I have a record, which in UTC in the 1st September, but in local time it is still in the 31 August.
P.S. Values stored in UTC format

[Carlton Gibson]

Yes. I can reproduce this.

1. Have a DateTimeField in date_hierarchy (and optionally `list_display)
2. Create an object in the DB for midnight 1st Sep (UTC).
3. Set project TIME_ZONE = 'America/Los_Angeles'

• Test object is show in list at Aug 31 5PM.
• Date hierarchy choice is created for 1st September
• Link for that is incorrect: ?<field>__day=1&<field>__month=8&<field>__year=2018, i.e. Aug 1st.

Presumably error is in ​`date_hierarchy()` template tag function.

[Alexander Holmbäck]

​PR

[Carlton Gibson]

Duplicate reported in #30749

[yab]

Two distinct date_hierarchy bugs : one relate to TZ effects and one related to DST effects :

#29724 (TZ matter)
When year and month lookup are chosen, if you have an event late in last day of the month that is first day of next month in UTC, queryset.dates method gives a date_hierarchy choice for next month 1st.
It seems it can be solved modifiyng admin/contrib/templatetags/admin_list.py this way, using queryset.filter and lists that may be very slow :
Needs importing:

from django.utils.timezone import make_naive
from django.conf import settings

and after "elif year_lookup and month_lookup:":

		elif year_lookup and month_lookup:
			### This is a way to avoid using queryset.dates fucntion, which was giving naive datetimes with problems when the monthrange is different between utc and localtime.

			month_filter=field_name+'__month'
			year_filter=field_name+'__year'
			dates_or_datetimes = cl.model.objects.filter(**{year_filter:year_lookup, month_filter:month_lookup}).values_list(field_name, flat=True).distinct()

			days = dates_or_datetimes
			if isinstance(dates_or_datetimes[0], datetime.datetime) and settings.USE_TZ:
				day_list = []
				days = []
				for day in dates_or_datetimes:
					if make_naive(day).day not in day_list:
						day_list.append(make_naive(day).day)
						days.append(make_naive(day))
			### 

###30749 (DST matter)
If the month selected by date_hierarchy is the month of the DST, the objects of the last day of the month does not appear in the changelist.
Changing the way "to_date" is set after elif month line 165 solves my problem: Instead of just adding 32 days with timedelta, I use make_aware with tzinfo=None. That way the to_date is set to the correct date and time.

				elif month:
					### The tzinfo changes between first and last day of the month when DST applies.  That's why we need to make_aware to_date separately from from_date
					to_date = make_aware((from_date + timedelta(days=32)).replace(day=1,tzinfo=None))

[Carlton Gibson]

OK, thanks for the follow-up yab. TBH I'm still inclined to view these as the same issue: incorrect collection of items at time-period boundaries (demonstrated with DST and TZ examples). However... one-ticket/two-tickets is not so important.

More is that, we need to make sure we have test cases for both these types of example, and then we can make sure any fix addresses both of those.

Super. 👍

[yab]

OK, my proposal passes existing tests. Not sure how to create specific test cases for that purpose. My proposal is above and there : ​https://github.com/yab228/django/blob/master/django/contrib/admin/templatetags/admin_list.py

[Carlton Gibson]

yab: please make a pull request with your suggested changes. (See the ​contributing guide.)

For tests, please add a test case with the example(s) here. Presumably they should fail with the exisiting code, showing the bug, and then pass with your patch applied.

[yab]

The tests I made at home were OK.
But several of the automatic same tests on github (in djangoci) were not OK. Especially test_date_hierarchy_timezone_dst.
Thoses automatic tests in many environments are too hard for a newbie like me. Too much time that I have not. I am giving up.

One last thing before I vanish : do the django programers plan to modify the hole behaviour of date_hierarchy ? I read it should stick more to result_list... If you can "just" tell me where I can see the discussions about that, I'm interested in reading. It seems to me date_hierarchy has to have two branches, on in case of datetimes+TZ_USE, one in other cases.

Thanks, anyway, to all the django-project team.

[Carlton Gibson]

#30922 was also a duplicate related.

[Hasan Ramezani]

​PR

[Mariusz Felisiak <felisiak.mariusz@…>]

In 53b6a466:

Refs #29724 -- Added is_dst parameter to QuerySet.datetimes().

Thanks Simon Charette for the review and Mariusz Felisiak for tests.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 55cdf6c5:

Fixed #29724 -- Fixed timezone handling in ModelAdmin.date_hierarchy queries.

Thanks Alexander Holmbäck for the initial patch.