[patch] Timezone bug in development server using a Windows environment

[ils_rules@…]

Django seems to screw up the python time settings. I put TIME_ZONE = 'Europe/Amsterdam' in my current project. The datetime.datetime.now() then returns a different time that if i just run python and run a datetime.datetime.now() (which returns the CORRECT time in Europe/Amsterdam). The difference is exactly one hour.

This difference is VERY annoying, because I can not do synchornization, different imports, etc. :(

[Malcolm Tredinnick]

Since all Django is doing here is setting the TZ environment variable, this is strange; it should just work. If you are on a Unix-based system (Linux / Solaris / Mac OSX / ...), could you please supply the results of the following. The first and third (at least) should probably work on Windows, too. Not sure how to change the environment-local timezone for Windows, though. Everywhere there is "..." below, please supply the value your system shows.

python
>>> import datetime, os, time
>>> os.environ.get('TZ')
...
>>> datetime.datetime.now()
...
>>> time.localtime()
...
>>> time.gmtime()
...

TZ=Europe/Amsterdam python
>>> import datetime
>>> datetime.datetime.now()
...

and finally

./manage.py shell
>>> import datetime, os, time
>>> os.environ['TZ']
...
>>> datetime.datetime.now()
...
>>> time.localtime()
...
>>> time.gmtime()
...

In theory, if your machine is set to Europe/Amsterdam time, these should all return the same things for now() and the latter two should return "Europe/Amsterdam" for the timezone, whilst the first one may or may not return a value for the TZ environment variable. The first test will also show what your machine currently things is the difference between its local timezone and UTC.

[ils_rules@…]

Ok this is getting stranger every minute.. I'm running on a windows box, with time settings set to Europe/Amsterdam.

Local time on box is 11:19

running on the 'runserver' in django:

datetime.now(): 2006-07-10 10:19:05.234000

time.localtime(): (2006, 7, 10, 10, 19, 5, 0, 191, 1)

time.gztime(): (2006, 7, 10, 9, 19, 5, 0, 191, 0)

running in apache:

datetime.now(): 2006-07-10 11:20:03.406000

time.localtime(): (2006, 7, 10, 11, 20, 3, 0, 191, 1)

time.gztime(): (2006, 7, 10, 9, 20, 3, 0, 191, 0)

so there is something wrong with the built-in server ??

[dan]

I'm having the same problem. I'm in Ann Arbor, Michigan. I tried setting the time zone code to both Detroit and Chicago in the settings file. And it just gives me GMT. This is on Windows ME using the test server. But it works correctly on Linux/Apache.

[dan]

One more thing. I'm using SQLite with Windows and the test server. And PostgreSQL with Linux/Apache.

[Malcolm Tredinnick]

Arg! Windows! :-(

This is not going to be possible to solve in a way that isn't a little bit painful. Timezone strings on Windows are a mess. The database isn't as complete as the strings used on Unix variants and there are differences in the common areas. On Linux, it is possible, if you set the wrong string to always get the "non daylight-savings" version and it looks like the same thing is possible on Windows, too.

This link and the associated thread explains the pain with trying to make this portable: ​http://mail.python.org/pipermail/python-dev/2003-March/034062.html

Messing with the environment via TZ in Windows is probably a doomed exercise. We really need somebody with Windows expertise to write a win32 patch that does The Right Thing(tm) here. None of the main Django developers seem to use Windows, so this is going to be fixed by somebody who is comfortable on the platform scratching their own itch and submitting a patch.

[django@…]

This problem occurs on MYSQL SQLite3 and postgresql on Windows. the TZ stuff works fine until you import django.db
Once that happens the TZ for the process is set to UTC. Why? no clue, but it is something that django is doing and is not DB specific. It is windows specific.

Have not had time to track down what the %#^%%$@ django is doing on windows to cause this but it has caused some major problems for us as it throws off matplotlib (used to generate dynamic graphs) for our date based graphing. As this is effecting the C runtime of the process this even causes problems for SciPy.

This bug needs to be highlited in the documentation! I have talked to 5 people on teh irc channel who were effected by this.
The solution has been to have a special settings variable that the other code uses to skew data (sometimes the querey result data, sometimes lie about the TZ to matplotlib), and set it special if we are windows.

[Chris Beaven]

Fix for windows timezone problems with development server

[Chris Beaven]

Ok, patch attached disables the settings.TIME_ZONE functionality in Windows, since it doesn't work anyway. As people have reported (and I can confirm) this causes the built-in development server to treat all times as UTC no matter what TIME_ZONE is set to.

On a side note, time.tzset() should probably be set after the os.environ['TZ'] = self.TIME_ZONE line (according to Python documentation) but I didn't want to change anything more in this patch than getting this Windows bug fixed.

[Chris Beaven]

To clarify, this doesn't disable settings.TIME_ZONE in Windows, it just doesn't use it for time and datetime modules (because it didn't work anyway, and caused problems with the development server).

It is still required for the django.db stuff.

[Chris Beaven]

Ok, in a further bout of weirdness (and further testing), it's not even removing the os.environ['TZ'] = self.TIME_ZONE which fixes this problem. Seemingly all that is required to fix it in windows is to import time. Not actually use it - just import it.

Weird.

[Chris Beaven]

Ok, here's my findings so far:

• It's not just in the development server! I just had this happen on my Windows server (running Apache & MySQL).
• On my development machine (while running Apache & PostgreSQL) I did not encounter the same problem. But it IS a problem with Python.
• I made a testtime view which just consisted of return HttpResponse(str(datetime.now()))
  • First call to that, time was correct (UTC+13)
  • Called a database method
  • Second call to testtime view was incorrect (UTC)
• I tried just using import time and it didn't fix it
• I tried the patch attached to this ticket and it fixed it

[angelo@…]

hey, just checked to be sure if i have the newest version, but yeah. this patch didnt get into svn so far, so i add my note.

i had the same problem on my windows dev env, and the patch helped. thanks alot.

[Chris Beaven]

Accepted because it's a confirmed bug.

In fact, I'm tempted to mark as ready for checkin since it's not going to have any side effects on other platforms, and Windows just plain doesn't work with os.environ['TZ'].

Probably needs some sort of documentation addition to explain that Windows people should always just set settings.TIME_ZONE to their local time unless they want problems.

[Marc Fargas <telenieko@…>]

Docs for this ticket, see comments below.

[Marc Fargas <telenieko@…>]

Docs added, does two things:

• Adds a note to the settings.py template about that, I didn't like the idea of placing a "windows specific" warning there, but as people can get in trouble with this on windows maybe we can let this one line be there (unless we hope that newcomers will google to reach the settings documentation and read the note there).
• Adds a note on the corresponding settings.txt section.

As always, I leave the stage untouched for triage review :)

[Chris Beaven]

Thanks Marc. Let's see what the core have to say about this... I say it's a rather common use-case so the Windows note is justified. However personally I'm not sure about having documentation reference a ticket.

[Marc Fargas <telenieko@…>]

I didn't want to reference the ticket also, but option 2 was to create a "windows.txt" titled "Windows specific issues" which for the newcomer could be translated to "Oh hell, this thing called Django has trouble with windows!!" but it's an option, maybe if there arise more Windows specific issues we'll have to create a windows.txt file hehe.

[mir@…]

just as a note, #2625 is probably a duplicate of this.

[Malcolm Tredinnick]

One question before I commit this: how sensitive is the solution to where the "import time" statement is? I would like to have it just before or just after "import os" in order to keep things in a logical order (system imports all together).

I have no problem with referencing a bug in the docs, by the way. The bug report doesn't contain crucial information -- just backup supporting evidence. So I'll leave that reference there; you can still read the docs and make sense of them without referring to the bug report.

[Marc Fargas <telenieko@…>]

Changing status due to last comment by mtredinnick. Could be ready for checking when it has an answer ;)

[Chris Beaven]

import moved, docs included

[Chris Beaven]

I tested the new patch, still works fine.

[Malcolm Tredinnick]

When applying this and reading everything over, the docs read better without the ticket reference, so I removed it (it's in the code, though, so we can track back why we're doing this funky stuff). Sorry for the change-of-mind, guys.

[Marc Fargas <telenieko@…>]

Replying to SmileyChris:

I tested the new patch, still works fine.

SmileyChris, mtredinnick asked whether the "import time" statement must be where it is or if it can be placed just above/below the import os line, can you check if the import statement can be moved?

[Malcolm Tredinnick]

(In [4487]) Fixed #2315 -- added work around for Windows timezone setting (i.e. we can't do
it). This will work until somebody wants to write some full Win32 timezone changing code for us. Thanks to Marc Fargas and SmileyChris for the combined patch.

[Malcolm Tredinnick]

Replying to Marc Fargas <telenieko@telenieko.com>:

Replying to SmileyChris:

I tested the new patch, still works fine.

SmileyChris, mtredinnick asked whether the "import time" statement must be where it is or if it can be placed just above/below the import os line, can you check if the import statement can be moved?

Just to clarify, Chris posted a new patch (the one he tested) with the import statement moved to a better place. All is good.

[Marc Fargas <telenieko@…>]

Forget about my last comment, patch already checked-in! ;)

Replying to Marc Fargas <telenieko@telenieko.com>:

Replying to SmileyChris:

I tested the new patch, still works fine.

SmileyChris, mtredinnick asked whether the "import time" statement must be where it is or if it can be placed just above/below the import os line, can you check if the import statement can be moved?

[Marc Fargas <telenieko@…>]

Replying to mtredinnick:

Replying to Marc Fargas <telenieko@telenieko.com>:

Replying to SmileyChris:

I tested the new patch, still works fine.

SmileyChris, mtredinnick asked whether the "import time" statement must be where it is or if it can be placed just above/below the import os line, can you check if the import statement can be moved?

Just to clarify, Chris posted a new patch (the one he tested) with the import statement moved to a better place. All is good.

Sorry, didn't spot the new upload ;)

[chad@…]

doc tweak: postgres' options don't always correlate

[anonymous]

I'm seeing similar problems on FreeBSD 6. Turns out that FreeBSD doesn't recognize 'US/Eastern' as an option for TIME_ZONE. I've attached a patch that clarifies this in the settings.py docs.

[Malcolm Tredinnick]

(In [4678]) Fixed #2315 -- Clarified that the available PostgreSQL timezone options may
provide more options than are strictly available. This is probably the best we
can do for such a varied area of standardisation.

[Malcolm Tredinnick]

I did not apply the docs patch attached here, because /etc/localtime is a binary file and asking people to reverse engineer /usr/share/zoneinfo/ is a bit of a hurdle. We can't really include a full tutorial on setting timezone options, so I've included a warning about the slight variability instead.