Filtering Results from Coverage.py

Coverage.py makes it easy to see how much of your code is covered by your test suite. It can be configured to spit out reports in XML (for consumption by a continuous integration server like Jenkins), in HTML (for human viewing), or simply dumping out plain text.

Getting started with coverage and Django is easy. First make sure you have the latest version of coverage (3.5 at time of writing) installed by running pip install -U coverage, then run your test suite and print a report:

coverage run manage.py test
coverage report

This is all well and good, but you’ll quickly notice that coverage reports on every file it touches including third-party libraries, the Django framework, and even Python itself. Often, you only want to see the coverage report for your code. Coverage provides a few options to narrow the scope of the report. They are:

--omit=PAT1,PAT2,...  Omit files when their filename matches one of these
                      patterns. Usually needs quoting on the command line.
--include=PAT1,PAT2,...
                      Include files only when their filename path matches
                      one of these patterns.  Usually needs quoting on the
                      command line.

You’ll notice that these take file paths instead of python module names. Running coverage as follows will only report coverage on files in my_project.

coverage run manage.py test
coverage report --include="/path/to/my_project/*"

This works well, but what if you want to deliver a portable test script including coverage? You won’t always know the path to the project when it is installed on other people’s computers. With a little Python sorcery, we can figure out the filepath at runtime:

coverage run manage.py test
coverage report --include="`python -c \"import os, my_project; print os.path.dirname(my_project.__file__)\"`/*"

How about checking coverage on multiple modules? We can do that too with a small shell script:

TEST_MODULES="my_project my_lib your_lib"
#get comma-separated file paths for module names
MOD_PATHS=""
for mod in `echo $TEST_MODULES`
do
    MOD_PATHS="$MOD_PATHS,`python -c \"import os, $mod; print os.path.dirname($mod.__file__)\"`/*"
done
#remove preceding comma
MOD_PATHS=`echo $MOD_PATHS|cut -c2-`

coverage run manage.py test
coverage report --include=$MOD_PATHS

A script like this can be bundled with your project so everyone on your team can easily test and see the coverage. Have it output XML and it is ready for use in a continuous integration system (see coverage help and coverage help report for more options). It’s worth noting that the techniques described here are not Django-specific and can be used with any Python code. Simply replace coverage run manage.py test with your test script, eg. coverage run tests.py.

Peter Baumgartner