Resetting Your South Migrations

Why?

A common question from South users is how to reset the migrations back to an initial state. The reasons for doing this are often related to the number of migrations growing steadily over the life of the project. You’ve probably noticed that tests and parts of your workflow are bogging down while waiting for a bunch of migrations to run. In practice it is rare that you’ll have a need to roll back to an early version of a model that is potentially dozens of migrations back in history. Therefore, the idea of resetting your South migrations is worth examining when deciding how to manage your project history long-term.

An important consideration is whether or not you control all environments where your app will be deployed. For example, if you are maintaining or planning to release a public open source project, resetting migrations is rarely going to be a good idea.

Trade-offs

There are varying opinions about the value of maintaining a full migration history for a project. My take on it is that the most value comes from maintaining a full history if you’ve also:

been diligent about writing backwards migrations.
have a close knit team where most/all users are comfortable with South.
have continuous integration that tests the full forwards and backwards migration series.

On our projects we typically interact with mixed groups of developers and technical people. A sweet spot for us with South is to not place an emphasis on maintaining backwards migrations and to use it as a team tool to keep people who are most frequently involved with the project an easy to way to stay current with schema changes. This keeps South-related workflow down to two commands (migrate and schemamigration) which is makes it easier for more developers at all skill levels to feel comfortable interacting in the workflow. If your team is new to South, you might not have anyone super-comfortable with how South works, and a common pitfall is burning a bunch of time trying to fix some aspect of your migrations that you’ll never end up benefitting from anyway. If you’re new to South, in the early stages of a new project, and are banging your head against some change you don’t understand, it might be prudent to just reset your migrations and move on. (You’ll get ‘em next time tiger)

I’m not endorsing resetting migrations as a general rule, but many South users will decide that this is an acceptable thing to do. For what it’s worth, across a number of sizable projects I don’t remember any occasion where I needed to roll back more than a few migrations. Most of the time this is just to fix an oops locally on something I’ve just changed. That said, I always make sure I have a plan for getting back to the current schema/data state before applying any operations.

In my opinion the biggest loss for removing migration history is that regression testing is probably going to be more complicated. That’s why we tie South resets to releases. Typically sometime prior to a Release Candidate we do a reset, and another one if necessary once the release is tagged if any last-minute migrations were added. That way each release starts with a fresh migration history. If there is a situation where regression testing requires testing against prior versions, you can pull the old migrations out from revision control history and patch together a complete migration history without much fuss.

Doing the Reset

If you have more than a few apps, I recommend generating a list for yourself for later use. You can get this in a few ways:

manage.py syncdb

The apps managed by South will listed at the end of the output for syncdb. Alternatively you could get them from the South migration history table:

select distinct(app_label) from south_migrationhistory;

Now, the first step is to remove the South migration history from the database:

#this should work for MySQL and PostgreSQL
echo 'truncate south_migrationhistory' | manage.py dbshell
#
# Or, more generally:
#
echo 'delete from south_migrationhistory' | manage.py dbshell

Next you’ll remove all of the migrations directories. The following is an example of doing this in a project managed in a Git repository::

find . -type d -name migrations -exec git rm -r '{}' \;
git commit -m "resetting migrations"
find . -type d -name migrations -exec rm -rf '{}' \;

If South sees migration directories in your apps it won’t allow the command we need to run next. Git doesn’t remove the directories so I delete them. Now we’re ready to create initial migrations for all the apps. You do this via:

manage.py convert_to_south appname

If you made a list of apps that were managed by South, you can use some editor/shell love to generate a snippet of code to automate calling convert_to_south for all your apps that need to be managed by South. Finally, when you push your changes with the reset migrations, you’ll let your team members know that when they update they’ll need to take a special step::

echo 'truncate south_migrationhistory' | manage.py dbshell
manage.py migrate --fake

This will allow your coworkers to be in the new reset state while maintaining their current data. The key is the --fake parameter to migrate, which tells South to mark any unapplied migrations as ‘done’ but not actually run them. At this point everyone should be able to resume the normal schemamigration/@migrate@ workflow. Note that this process is the same as what is normally done when apps are converted to South, with the additional step of clearing out South’s migration history. See Convert an App to South for more information about the convert_to_south command. Given how often I see this question asked, it seems worth investigating if adding first-class support for a reset operation in South is a convenience worth having. Make sure to visit #django-south on IRC if you have questions – I’m usually lurking during PDT working hours; alternatively drop a note in the mailing list where south author Andrew Godwin is always responsive.

Why?

Trade-offs

Doing the Reset

Brian Luft