daemonize.py: unbuffered text IO error (python3)

[alex01@…]

When trying to run fcgi server I get "ValueError: can't have unbuffered text I/O"

My python version and django version:

$ python --version
Python 3.2.3

>>> django.get_version()
'1.7.dev20130726184538'

Can't start server:
(virtenv02)jsmith@thinkpad01:~/virtenv02/mysite$ python manage.py runfcgi host=127.0.0.1 port=8100
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/jsmith/virtenv02/django-trunk/django/core/management/__init__.py", line 397, in execute_from_command_line
    utility.execute()
  File "/home/jsmith/virtenv02/django-trunk/django/core/management/__init__.py", line 390, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/jsmith/virtenv02/django-trunk/django/core/management/base.py", line 240, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/jsmith/virtenv02/django-trunk/django/core/management/base.py", line 283, in execute
    output = self.handle(*args, **options)
  File "/home/jsmith/virtenv02/django-trunk/django/core/management/commands/runfcgi.py", line 22, in handle
    runfastcgi(args)
  File "/home/jsmith/virtenv02/django-trunk/django/core/servers/fastcgi.py", line 176, in runfastcgi
    become_daemon(our_home_dir=options["workdir"], **daemon_kwargs)
  File "/home/jsmith/virtenv02/django-trunk/django/utils/daemonize.py", line 28, in become_daemon
    so = open(out_log, 'a+', 0)
ValueError: can't have unbuffered text I/O

Server starts successfully and works if I give it daemonize=False option:
(virtenv02)jsmith@thinkpad01:~/virtenv02/mysite$ python manage.py runfcgi host=127.0.0.1 port=8100 daemonize=False

As far as I understand, this issue is specific to python3 and is related to changes in open function introduced in pep-3116

Alex

[Claude Paroz]

Unfortunately, this is old untested code. I wonder if the solution would be to simply forget the 0 argument (unbuffered) of the open call. Maybe someone more knowledgeable about buffered/unbuffered output may chime in to enlighten us?

[Aymeric Augustin]

Note that FastCGI is deprecated in 1.7.

[dk+django@…]

Replying to aaugustin:

Note that FastCGI is deprecated in 1.7.

Interesting decision, why so? Cannot spot any rationale in the 1.7 release notes. Pointers?
However, the error is not really fcgi-related, any code using django/utils/daemonize.py will have the same issue in Py3k.
Log files are text files, and you cannot open them in unbuffered mode.

[dk+django@…]

Replying to claudep:

Maybe someone more knowledgeable about buffered/unbuffered output may chime in to enlighten us?

Change it to

open( out_log, 'a+', 1)

This buffers up to the next EOL, which makes sense for log files. (Same for err_log)

As per ​Python3 docs:

buffering is an optional integer used to set the buffering policy. Pass 0 to switch buffering off (only allowed in binary mode), 1 to select line buffering (only usable in text mode)

Looking at ​open() in Python2, line buffering should also work:

• If something goes wrong while writing to out_log, you get an exception
• I haven't yet seen traceback failing in the middle of a line, so it is probably also safe for err_log

Hope this helps.

[Tim Graham]

Ticket for FastCGI deprecation is #20766.

[Aymeric Augustin <aymeric.augustin@…>]

In 5836a5771f2aefca83349b111f4191d6485af1d5:

Fixed #20815 -- Don't enforce unbuffered I/O on Python 3.

No test because this code is already deprecated (part of FastCGI support).

[Aymeric Augustin <aymeric.augustin@…>]

In 211ff288a03ba4623399e222d15a33075e958f0a:

[1.7.x] Fixed #20815 -- Don't enforce unbuffered I/O on Python 3.

No test because this code is already deprecated (part of FastCGI support).

Backport of 5836a577 from master