A client recently asked me why all the Django projects we create have a setup.py in the root of the project. Lots of projects get by just fine without one, so why do I use it? The explanation turned out a bit long-winded, so I thought I’d turn it into a blog post here.
What is setup.py?
According to Python’s official docs:
The setup script is the centre of all activity in building, distributing, and installing modules using the Distutils. The main purpose of the setup script is to describe your module distribution to the Distutils, so that the various commands that operate on your modules do the right thing.
Distutils is part of the Python standard library and is extended by tools you’re probably more familiar with like setuptools, pip, and pipenv. Every package you download off of the Python Package Index (PyPI) has a setup.py. It is used by pipenv/pip/setuptools/distutils to figure out where the Python module is in your project, what dependencies it needs, what scripts it needs to install, and more.
Why?
So why should a Django project use setup.py?
Django is Just Python
While very early versions of Django eschewed many Python best-practices, this was universally seen as a “bad thing” (read some history about the “magic removal branch”). Great steps have been made to make Django work just like any other Python code. Ask a core developer and they’ll tell you, “Django is just Python.”
The reason Django is a parenthentical in the title of this post is because the answer doesn’t have anything to do with Django. One reason we use setup.py is because it is the Python standard and Django projects should be following the Python standards.
Your Project Is (or Should Be) a Python Module
The last line in the The Zen of Python says:
Namespaces are one honking great idea – let’s do more of those!
Python modules are namespaces and your project should define one (and only one) top-level namespace which it uses. If you used Django’s startproject command to create your project, it created that top-level module for you. Any more Python modules you create (including Django apps) should live within that module/namespace.
There’s a few reasons for this:
- You reduce the risk of name clashes. If every app in your project uses the top-level namespace, you’re likely to run into a conflict with some third-party library eventually.
- It simplifies logging configuration. Rather than having to configure a logger for every module in your project, you just configure one.
- This is circular reasoning, but it makes it easier to install your project using
setup.py. Hopefully this post convinces you that is important.
You Want to Build, Distribute, and Install Your Project
Distributing Python code is not just done via PyPI. Even if you have no intention of uploading your project to PyPI, you almost certainly plan on distributing and installing it somewhere other than your local development machine. The act of running your locally developed code on a server qualifies as “building, distributing, and installing” a Python module. As per the Python docs quoted above, the setup script is the center of all that activity.
How?
Adding a setup.py to your project is trivial in most cases. The simplest example looks like this:
python
#!/usr/bin/env python
from setuptools import setup, find_packages
setup(name='myproject',
version='1.0',
packages=find_packages())
The directory structure (as created by startproject) would be:
setup.py
myproject/__init__.py
myproject/... < All your Python files in this directory
The setup function supports many other arguments, but this is good enough to get you started for our purposes.
What You Get
Consistent Imports
With your setup.py in place you can run pip install -e . and myproject is now a module on your Python path that can be imported and used by Python. Passing the -e (also accepted by pipenv means it is an “editable” install and changes you make in your directory will be reflected in the Python module (they are one and the same).
You may be saying, “My Django project works fine without this, why should I bother?” Chances are that you’re depending on a feature of Python that inserts the current directory onto the Python path. That is a fragile dependency and one that is likely to burn you at some point in deployment. To test this, simply try the following in your project’s activated virtualenv:
$ cd ~/path/to/myproject
$ python -c "import myproject"
$ cd # move to home directory
$ python -c "import myproject"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named myproject
I, and probably most developers, would prefer that the same imports are available to the Python interpreter no matter what directory you are in when it runs.
💡 Note: If you ever want to do non-editable installs, look into the zip_safe and package_data arguments to setup ensure non-Python files like templates and static assets are included in your installation.
Bonus: manage.py on the Path
I typically add an additional scripts argument to setup so it looks like this,
python
setup(name='myproject',
version='1.0',
packages=find_packages(),
scripts=['manage.py'])
With this in place, manage.py will be on the $PATH of the virtualenv, allowing you to run management commands from any directory like you would any other shell command, for example:
bash
$ manage.py runserver
An even nicer perk of this is that you can call it directly without activating your virtualenv and it will use the virtualenv’s Python, ensuring all the correct modules are available. In that case, you’d need to call it by the full path of the script that setuptools installs:
bash
$ /path/to/virtualenv/bin/manage.py migrate --noinput
This is awesome for running one-off tasks as part of a configuration management system or cronjob/Systemd timer. Without this in place, it’s common to see ugly one-liners like this:
bash
# don't do this
$ . /path/to/virtualenv/bin/activate; cd /path/to/myproject; python manage.py migrate --noinput
To wrap things up, adding a setup.py file to your project makes it easy to distribute and install using the standard Python tooling. It helps you avoid some unexpected pitfalls around module importing and can bring some added benefits, like easier execution of scripts outside your local environemnt.
I’ve found this is a surprisingly polarizing topic in the Python/Django world. If you have strong feelings about including (or excluding) setup.py in your project, I’d love to hear them.
