File Streaming in Django (Sending large files through Django)

I am planning to build something involving serving large files over HTTP. Putting the files in a public accessible directory is not an option since I need to monitor the users who download the file. Users will need to login and certain groups will have access to certain files. In PHP, I would have no problem in implementing this but I am new to Python and Django so I had to look around. There will be very few users in this app – the most important requirement: the server will need to say alive when people download the big files (let’s say 200-500MB).

So I searched for “File Streaming in Django / Python” (not the exact keywords, but you get the idea). And I found this.

Since I use this blog like a “notebook” so that I can go back to it in case I forget things, I am going to copy-paste the entire code here. There is a bonus; the snippet includes “zipping a file” then sending it to the stream.

This snippet demonstrates how you can send a file (or file-like object) through Django without having to load the whole thing into memory. The FileWrapper will turn the file-like object into an iterator for chunks of 8KB.

This is a full working example. Start a new app, save this snippet as views.py, and add the views to your URLconf. The send_file view will serve the source code of this file as a plaintext document, and the send_zipfile view will generate a Zip file with 10 copies of it.

Use this solution for dynamic content only, or if you need password protection with Django’s user accounts. Remember that you should serve static files directly through your web server, not through Django:

http://www.djangoproject.com/documentation/modpython/#serving-media-files

import os, tempfile, zipfile
from django.http import HttpResponse
from django.core.servers.basehttp import FileWrapper
 
def send_file(request):
    """                                                                         
    Send a file through Django without loading the whole file into              
    memory at once. The FileWrapper will turn the file object into an           
    iterator for chunks of 8KB.                                                 
    """
    filename = __file__ # Select your file here.                                
    wrapper = FileWrapper(file(filename))
    response = HttpResponse(wrapper, content_type='text/plain')
    response['Content-Length'] = os.path.getsize(filename)
    return response
 
def send_zipfile(request):
    """                                                                         
    Create a ZIP file on disk and transmit it in chunks of 8KB,                 
    without loading the whole file into memory. A similar approach can          
    be used for large dynamic PDF files.                                        
    """
    temp = tempfile.TemporaryFile()
    archive = zipfile.ZipFile(temp, 'w', zipfile.ZIP_DEFLATED)
    for index in range(10):
        filename = __file__ # Select your files here.                           
        archive.write(filename, 'file%d.txt' % index)
    archive.close()
    wrapper = FileWrapper(temp)
    response = HttpResponse(wrapper, content_type='application/zip')
    response['Content-Disposition'] = 'attachment; filename=test.zip'
    response['Content-Length'] = temp.tell()
    temp.seek(0)
    return response

A comment from the site:

jdriscoll (on October 9, 2007):
For people on Windows you’ll need to specify “read binary” mode for anything other than a text file:

wrapper = FileWrapper(file(filename), “rb”)

Also, I chatted for a few minutes in IRC (#django) asking for suggestions/recommendations from other people. “andym recommends you use X-Sendfile”. So, after I implement the code above, I will look into X-Sendfile – that will probably in another post.

The source is found here: http://djangosnippets.org/snippets/365/. (Thanks!)

Any comments/suggestions, post them below. I would especially love to read other recommendations on how you would go about sending big files through Django/Python.