Opened 15 years ago

Closed 14 years ago

Last modified 14 years ago

#12955 closed (fixed)

Large file (bigger than 2GB) upload

Reported by: iscarface Owned by: Ivan Sagalaev
Component: File uploads/storage Version: 1.1
Severity: Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When i'm uploading more than 2GB file i have an error:

OverflowError at /admin/films/film/add/

long int too large to convert to int

Request Method: POST

Request URL: ​http://192.168.1.10/admin/films/film/add/[[BR]]

Exception Type: OverflowError

Exception Value:
long int too large to convert to int

Exception Location: /usr/lib/python2.5/site-packages/django/db/models/fields/files.py in _get_size, line 76

Python Executable: /usr/bin/python

Python Version: 2.5.2

Python Path: ['/var/www/', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages/PIL', '/var/lib/python-support/python2.5']

Server time: Втр, 23 Фев 2010 17:16:19 +0200

Django version = (1, 2, 0, 'alpha', 0)

Attachments (1)

django-12955.diff​ (758 bytes ) - added by Sebastian Wiedenroth 15 years ago.
following isagalaevs suggestion this patch fixed the problem for us

Download all attachments as: .zip

Change History (8)

comment:1 by Russell Keith-Magee, 15 years ago

Triage Stage: Unreviewed → Accepted

comment:2 by scarface, 15 years ago

Traceback:

Environment:

Request Method: POST
Request URL: ​http://213.111.127.198/admin/films/testfile/add/
Django Version: 1.2 pre-alpha
Python Version: 2.5.2
Installed Applications:
['django.contrib.auth',

'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.admin',
'adm_manage.apps.core',
'adm_manage.apps.tags',
'adm_manage.apps.rating',
'adm_manage.apps.films',
'adm_manage.apps.serials',
'djapian']

Installed Middleware:
('django.middleware.common.CommonMiddleware',

'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware')

Traceback:
File "/usr/lib/python2.5/site-packages/django/core/handlers/base.py" in get_response

  1. response = callback(request, *callback_args, callback_kwargs)

File "/usr/lib/python2.5/site-packages/django/contrib/admin/options.py" in wrapper

  1. return self.admin_site.admin_view(view)(*args, kwargs)

File "/usr/lib/python2.5/site-packages/django/utils/decorators.py" in call

  1. return self.decorator(self.func)(*args, kwargs)

File "/usr/lib/python2.5/site-packages/django/utils/decorators.py" in _wrapped_view

  1. response = view_func(request, *args, kwargs)

File "/usr/lib/python2.5/site-packages/django/utils/decorators.py" in call

  1. return self.decorator(self.func)(*args, kwargs)

File "/usr/lib/python2.5/site-packages/django/views/decorators/cache.py" in _wrapped_view_func

  1. response = view_func(request, *args, kwargs)

File "/usr/lib/python2.5/site-packages/django/contrib/admin/sites.py" in inner

  1. return view(request, *args, kwargs)

File "/usr/lib/python2.5/site-packages/django/utils/decorators.py" in _wrapped_view

  1. response = view_func(request, *args, kwargs)

File "/usr/lib/python2.5/site-packages/django/db/transaction.py" in _commit_on_success

  1. res = func(*args, kw)

File "/usr/lib/python2.5/site-packages/django/contrib/admin/options.py" in add_view

  1. self.save_model(request, new_object, form, change=False)

File "/usr/lib/python2.5/site-packages/django/contrib/admin/options.py" in save_model

  1. obj.save()

File "/usr/lib/python2.5/site-packages/django/db/models/base.py" in save

  1. self.save_base(using=using, force_insert=force_insert, force_update=force_update)

File "/usr/lib/python2.5/site-packages/django/db/models/base.py" in save_base

  1. for f in meta.local_fields if not isinstance(f, AutoField)]

File "/usr/lib/python2.5/site-packages/django/db/models/fields/files.py" in pre_save

  1. file.save(file.name, file, save=False)

File "/usr/lib/python2.5/site-packages/django/db/models/fields/files.py" in save

  1. self.name = self.storage.save(name, content)

File "/usr/lib/python2.5/site-packages/django/core/files/storage.py" in save

  1. name = self._save(name, content)

File "/usr/lib/python2.5/site-packages/django/core/files/storage.py" in _save

  1. for chunk in content.chunks():

File "/usr/lib/python2.5/site-packages/django/core/files/base.py" in chunks

  1. counter = self.size

File "/usr/lib/python2.5/site-packages/django/db/models/fields/files.py" in _get_size

  1. return len(self.file)

Exception Type: OverflowError at /admin/films/testfile/add/
Exception Value: long int too large to convert to int
Environment:

Request Method: POST
Request URL: ​http://213.111.127.198/admin/films/testfile/add/
Django Version: 1.2 pre-alpha
Python Version: 2.5.2
Installed Applications:
['django.contrib.auth',

'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.admin',
'adm_manage.apps.core',
'adm_manage.apps.tags',
'adm_manage.apps.rating',
'adm_manage.apps.films',
'adm_manage.apps.serials',
'djapian']

Installed Middleware:
('django.middleware.common.CommonMiddleware',

'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware')

Traceback:
File "/usr/lib/python2.5/site-packages/django/core/handlers/base.py" in get_response

  1. response = callback(request, *callback_args, callback_kwargs)

File "/usr/lib/python2.5/site-packages/django/contrib/admin/options.py" in wrapper

  1. return self.admin_site.admin_view(view)(*args, kwargs)

File "/usr/lib/python2.5/site-packages/django/utils/decorators.py" in call

  1. return self.decorator(self.func)(*args, kwargs)

File "/usr/lib/python2.5/site-packages/django/utils/decorators.py" in _wrapped_view

  1. response = view_func(request, *args, kwargs)

File "/usr/lib/python2.5/site-packages/django/utils/decorators.py" in call

  1. return self.decorator(self.func)(*args, kwargs)

File "/usr/lib/python2.5/site-packages/django/views/decorators/cache.py" in _wrapped_view_func

  1. response = view_func(request, *args, kwargs)

File "/usr/lib/python2.5/site-packages/django/contrib/admin/sites.py" in inner

  1. return view(request, *args, kwargs)

File "/usr/lib/python2.5/site-packages/django/utils/decorators.py" in _wrapped_view

  1. response = view_func(request, *args, kwargs)

File "/usr/lib/python2.5/site-packages/django/db/transaction.py" in _commit_on_success

  1. res = func(*args, kw)

File "/usr/lib/python2.5/site-packages/django/contrib/admin/options.py" in add_view

  1. self.save_model(request, new_object, form, change=False)

File "/usr/lib/python2.5/site-packages/django/contrib/admin/options.py" in save_model

  1. obj.save()

File "/usr/lib/python2.5/site-packages/django/db/models/base.py" in save

  1. self.save_base(using=using, force_insert=force_insert, force_update=force_update)

File "/usr/lib/python2.5/site-packages/django/db/models/base.py" in save_base

  1. for f in meta.local_fields if not isinstance(f, AutoField)]

File "/usr/lib/python2.5/site-packages/django/db/models/fields/files.py" in pre_save

  1. file.save(file.name, file, save=False)

File "/usr/lib/python2.5/site-packages/django/db/models/fields/files.py" in save

  1. self.name = self.storage.save(name, content)

File "/usr/lib/python2.5/site-packages/django/core/files/storage.py" in save

  1. name = self._save(name, content)

File "/usr/lib/python2.5/site-packages/django/core/files/storage.py" in _save

  1. for chunk in content.chunks():

File "/usr/lib/python2.5/site-packages/django/core/files/base.py" in chunks

  1. counter = self.size

File "/usr/lib/python2.5/site-packages/django/db/models/fields/files.py" in _get_size

  1. return len(self.file)

Exception Type: OverflowError at /admin/films/testfile/add/
Exception Value: long int too large to convert to int

comment:3 by Ivan Sagalaev, 15 years ago

The problem is caused by using len(self.file) because len() tries to typecast the result of {{len()}} to an actual int. Which on 32-bit machines is < 2G. I think the fix might be as simple as using directly {{.size}} instead of {{len()}} everywhere...

comment:4 by Ivan Sagalaev, 15 years ago

Owner: changed from nobody to Ivan Sagalaev

by Sebastian Wiedenroth, 15 years ago

Attachment: django-12955.diff​ added

following isagalaevs suggestion this patch fixed the problem for us

comment:5 by Adam Nelson, 15 years ago

Has patch: set
Needs tests: set
Triage Stage: Accepted → Ready for checkin

This looks good and the patch is even more pythonic. Shouldn't there be a test for filesize above 2GB or would that be onerous?

comment:6 by Jannis Leidel, 14 years ago

Resolution: → fixed
Status: new → closed

(In [14878]) Fixed #12955 -- Use the size attribute of the file instead of len() to support uploads larger than 2GB. Thanks, swiedenroth and isagalaev.

comment:7 by Jannis Leidel, 14 years ago

(In [14903]) [1.2.X] Fixed #12955 -- Use the size attribute of the file instead of len() to support uploads larger than 2GB. Thanks, swiedenroth and isagalaev.

Backport from trunk (r14878).

Note: See TracTickets for help on using tickets.
Back to Top