Lincoln Loop

On Static Media and Django

We all know not to serve static media (images, CSS, Javascript, etc.) in production directly from Django. Thankfully, Django gives us some nice settings like MEDIA_URL and MEDIA_ROOT to make serving them a lot less painless. Lately, however, I’ve come to realize that these settings shouldn’t really apply to all static media.

Not All Static Media Is Created Equal

Static media really comes in two flavors. Media that is part of your site like your stylesheets and user generated media or files that are uploaded to the site once it is live. We don’t want to serve all this media from the same place for a couple of reasons:

1. Our source checkouts shouldn’t be crufted up by stuff our users are uploading
2. User generated content and source code should live in two different places on the filesystem

We could fix the first problem with some .gitignore, but that doesn’t solve the filesystem issue.

Solution: Segregate Your Static Files

The answer is to segregate your static files. Django already does this with ADMIN_MEDIA_PREFIX. We use two additional settings in our Django projects, STATIC_URL and STATIC_ROOT. This lets us put user generated content in MEDIA_ROOT outside our project while still serving STATIC_ROOT from inside our source repo.

Implementation

We use Nginx as a front-end proxy to most of our sites and add this stanza to serve all our static media:

location /s/ {
    alias   /var/www/domainname.com/;
}

Then we add three directories to /var/www/domainname.com/:

• admin, a symlink to django/contrib/admin/media
• static, a symlink pointing inside our project repo respectively
• media, a new folder that stores all the user generated content.

Our project settings file looks like this now:

MEDIA_ROOT = '/var/www/domainname.com/media/'
MEDIA_URL = '/s/media/'

STATIC_ROOT = '/path/to/project_src/static/'
STATIC_URL = '/s/static/'

ADMIN_MEDIA_PREFIX = '/s/admin/'

and we slap the following code into our urls.py to serve media during development:

from django.conf import settings
if getattr(settings, 'LOCAL_DEV', False):
    urlpatterns += patterns('django.views.static',
        (r'^%s(?P<path>.*)' % settings.STATIC_URL, 'serve',
         {'document_root': settings.STATIC_ROOT}),
        (r'^%s(?P</path><path>.*)' % settings.MEDIA_URL, 'serve',
         {'document_root': settings.MEDIA_ROOT}),
    )
</path>

This has worked very well for us, how do you tackle the issue (or is it not an issue to you)?

Comments

• November 14, 2008 at 9:54 a.m. #
  Peter Baumgartner piped up:

  Looks like some of my code got BeautifulSoup’ed. Both the regexes in the urls.py are r'^%s(?P<path>.*)'

• November 14, 2008 at 10:08 a.m. #
  Andreas responded:

  First of all i use a separate domain for static/media, preferable 2 because it speeds up your site (default settings in all browsers is to have max 2 concurrent connections to the same domain) something like static1.foo.com static2.foo.com which resolves to the same ip. You dont want more than 2 domains for statics because domain resolving on many domains slows down your site (you can read about this in steves sounder (creator of yslow)s book High Performence Web Sites.

  My preferred setup then looks like this. one or multiple apache instances with mod_wsgi for the django app, and then one or multiple nginx instances for the statics depening on how large the site is, and one nginx acting as a proxy infront of the whole setup. If its a small site i serve the statics from the same nginx which acts as proxy.

  And for separating user generated statics from the sites i do something like this:

  
location / { root /home/foo/site/media;
}

  location /static { root /home/foo/upload;
}


• November 14, 2008 at 12:27 p.m. #
  Brandon Konkle commented:

  Great post! I love your method here, and I’m going to start using it going forward. Thanks so much for sharing such a useful best practice!

• December 1, 2008 at 4:40 p.m. #
  Mikkel Høgh commented:

  I know this is a bit old, but I noticed a problem with the code above, since I tried to use it in my own app.

  I had to set up my patterns like this:

  urlpatterns += patterns('django.views.static’, (r’^%s(?P.*)’ % settings.STATIC_URL[1:], 'serve’, {'document_root’: settings.STATIC_ROOT}), (r’^%s(?P.*)’ % settings.MEDIA_URL[1:], 'serve’, {'document_root’: settings.MEDIA_ROOT}), )

  With the [1:] slice on STATIC_URL and MEDIA_URL, because otherwise the initial slash in those URLs will cause the URLs not to work.

  I’ve included it in:
  http://github.com/mikl/djime/commit/874829443423aae83a2f0af1d4b7c39cfde8f37d – thanks :)

• December 9, 2008 at 12:52 p.m. #
  Nikolaj mentioned:

  @Andreas – I am a bit skeptical of the max 2 concurrent connections thing. I have observed safari 3 and firefox load multiple files simultaneously, and generally from a browser maker’s perspective this seems like it would make the browser feel slower and less snappy (which is not what they are motivated to do).

  I looked at the oreilly website for the book you mention, and i don’t see that seperation in the “core 14 priniciples”, but I do see “Reduce DNS Lookups”, which would make me lean towards centralize everything on one domain.

  However, I am always very interested in making our sites load faster, so if you have any references about this specifically, I am intersted in it, so in the off chance you hit this thread again, please post some!

• December 13, 2008 at 10:56 a.m. #
  Filip Salomonsson commented:

  The Google Browser Security Handbook has some numbers on concurrent connections in various browsers:

  http://code.google.com/p/browsersec/wiki/Part2#Simultaneous_connection_limits

• January 29, 2009 at 4:17 p.m. #
  Erik Allik commented:

  How do you handle reusable apps (i.e. apps that multiple projects are using) that provide their own static media? Do you symlink to the app media folder from the project media folder? And each app always uses STATIC_URL instead of MEDIA_URL? If that’s the case, how do you handle apps written by someone else that also happen to provide static files?

• February 4, 2009 at 3:35 p.m. #
  Peter Baumgartner responded:

  @Erik
  Good question and one I don’t have a great answer for yet. Things I’ve done in the past:

  1. Give the reusable app its own APP_MEDIA_URL and use that, falling back on MEDIA_URL if it is not defined in settings. Drawback: requires context processor
  2. Rewrite the URLs with the webserver or frontend proxy. Drawback: maintenance, additional layer of complexity
  3. Symlink it into the project MEDIA_URL path and feel slightly dirty.

Got something to say?