problems with streaming files uplod with Django 1.0

[Faheem Mitha]

Hi,

I just upgraded to Django 1.0, and I've just finished fixing all my
unit tests. However, streaming file uploads are still broken. The
behaviour I'm seeing is as follows:

When I start the upload of the file, a file which looks like
eg. /tmp/tmpGjWpee.upload appears almost immediately, but does not
change in size thereafter. The size is either empty or 16M - 16M most
of the time. I haven't seen any other values appear. The file upload
completes some of the time, regardless, but is very slow. For very
large files, like 1g, it took so long I stopped it.

I've seen this behavior on two independent installations of 1.0, with
the same project/application. I have a different installation with a
pre-1.0 snapshot with a patch from #2070 applied, and that works fine
with an earlier version of the same model/application, with the
temporary file in /tmp appearing and growing in size as the upload
progresses, as expected.

I've created a small test project which shows the behavior, which I
attach. This was used on Debian with a source installation of Django
1.0, and Apache 2 with the mod_wsgi module.

The apache config appears
below. This is on Debian etch, with the following installed.

faheem@wsgi:/var/django/testproj/testapp$ dpkg -l | grep apache
ii  apache2                         2.2.3-4+etch5                            Next generation, scalable, extendable web se
ii  apache2-mpm-prefork             2.2.3-4+etch5                            Traditional model for Apache HTTPD 2.1
ii  apache2-utils                   2.2.3-4+etch5                            utility programs for webservers
ii  apache2.2-common                2.2.3-4+etch5                            Next generation, scalable, extendable web se
ii  libapache2-mod-python           3.2.10-4                                 Apache 2 module that embeds Python within th
ii  libapache2-mod-wsgi             2.0-1~bpo40+1                            Python WSGI adapter module for Apache

Sincerely, Faheem Mitha.

*****************************************************************
# Testproj config.
Alias /media_testproj/ /var/django/media_testproj/
<Directory /var/django/media_testproj>
Order deny,allow
Allow from all
</Directory>

WSGIScriptAlias /testproj /var/django/testproj/apache/django.wsgi
<Directory /var/django/testproj/apache>
Order deny,allow
Allow from all
</Directory>
******************************************************************

[Faheem Mitha]

tarball of test project to reproduce problem

[Ramiro Morales]

Re-formatted Description field. Next time please remember to use the Preview button.

[Faheem Mitha]

I thought updates to the ticket would automatically be sent to the submitter, but
this doesn't appear to be the case. So, CCing myself.

[Jacob]

I've never seen this behavior myself, and I can't replicate it with the attached test project. I have to wonder if it's something specific to your server. It's been some months: is this still going on with 1.0.2? How about trunk?

[Faheem Mitha]

Hi Jacob,

Thanks for the response, and apologies for the slow reply - I'm currently visiting India. I'll try to test with more recent versions of Django when I get a chance. As regards something specific to my server, the only thing I can think of is that I use linux vservers for testing. I run apache + django inside the vserver and contact it from the host. This enables me to run different setups on the same machine. However, this setup worked fine till now, so I can't imagine why it should now be a problem. I'll also try testing without the use of a vserver.

[Faheem Mitha]

Update. Back from India, and tested my setup in a normal server, and it appears to behave normally.
So, looks like this is a vserver issue. What is the attitude towards the project wrt running django inside a virtual machine?

Regards, Faheem Mitha.

[Adam Nelson]

Django should work in a VM, but by the same token, a VM shouldn't act any differently than a real machine at this level. If this is replicable now on a VM, feel free to reopen.