Lincoln Loop

Reusable Django Apps and Forking

One of the things that drew me towards Django was the idea of being able to create reusable applications that would sit on my PYTHONPATH instead of copied across multiple sites. Coming from WordPress, the constant security updates that required me to revisit old projects began to drive me mad.

Trouble in Paradise

With some real world Django experience under my belt, I find myself re-using apps all the time, but not how I originally expected. A fair amount of my client work comes from building content management systems, so I started out building a generic app like flatpages but more extendable and a blogging app. I dropped them in my PYTHONPATH and started adding them to INSTALLED_APPS on my projects.

Over time, they evolved and improved, but they started to handcuff me. I started thinking things like, “This would be a great feature for Project X, but it would break Project Y.” I was worrying about backwards compatibility for one-off sites that were already done and paid for. When I started going back into those projects to make them handle new versions of my application, I realized I was right back where I started with WordPress.

Use the Fork

Since then, I’ve started forking many of my apps for individual projects. While at first I thought I was losing all the benefits of reusable apps, I quickly realized they were still there. I can still get a site up and running rapidly by reusing open source and personal code libraries/applications, but now I have the added benefit of being able to customize them to my heart’s content without worrying about past projects. Things that improve the app go into the main repository for reuse in the next project.

Move On

But what about those once great projects that are now running on “legacy” code? Well… I’ve learned to walk away from them. DHH had a great post recently, All code will eventually go stale. It is true and it is OK. It means you’re learning and improving your craft. Unless I have some amazing new feature for the project or changes are requested by the client, I don’t touch it. If I do need to revisit it, a simple diff usually gets it up to date very quickly. I fix what I need to and leave the rest.

For apps that are polished and have a stable API, PYTHONPATH is often the best place for them. Don’t let it handcuff you though. If you need/want to make changes that don’t make sense in the global context, fork off a copy into your project and move on.

Disagree with me or have a different approach? Leave a comment, I’d love to hear about it.

Comments

• April 4, 2008 at 11:31 a.m. #
  Djoumé commented:

  What about creating a new branch when you need to fork an application? It will then be easier to exchange patches back and forth between the generic application and the customized one.

• April 4, 2008 at 12:16 p.m. #
  Miles chimed in with:

  That’s exactly the same conclusion I came to. I started building re-usable apps for a bunch of clients that were very similar (academic departments), but each department wanted small changes, so I ended up installing site-specific versions of the apps. I roll good, general-purpose changes into a master copy of the app, which I use to start new projects. It’s been the smoothest way to get the benefits of both.

• April 4, 2008 at 1 p.m. #
  Chris added:

  Very good ideas, and makes a lot of sense for some things I’ve been working on. I think for these “generic” apps, using version control is ideal. I’m thinking that in the app’s trunk would be the standard, generic version. Then create a branch for the customized version. The apps could then be included in the custom project via svn:externals. And having branches would allow me to make major bug fixes, if they arrise, in trunk and roll them into the branches.

• April 4, 2008 at 1:53 p.m. #
  Peter Baumgartner chimed in with:

  I like the idea of branching these out of a main repository. I need to brush up on my subversion commands, I haven’t used branches or external repositories much in the past. Maybe this would be a good time to learn git (or mercurial, or bazaar).

• April 4, 2008 at 4 p.m. #
  Nick added:

  Thank you for very useful topic. Could you please provide some kind of code examples or may be pseudo-code examples to describe these 2 situations when it is needed to split solid application into 2 separate or not to split. What is the main reason for splitting?

  For example I going to write cms. Do I need to develop these applications separately: authorization, categories management etc ?

• April 5, 2008 at 1:26 a.m. #
  Nicola Larosa chimed in with:

  The post is spot-on, and a good DVCS is really going to help you in that.

  My choice the second one you mention: Mercurial.

  It saddens me a bit when I see some Pythonista going to the wasteful complexity of git, and forgoing the jewel that Mercurial is.

  Bazaar, yes, it’s workable, but not as well thought out. Its only advantage over Mercurial is its ability to push changes to Subversion repositories: but it’s not needed for your own work, and it’s going to be added to Mercurial soon anyway.

  Some more details:

  Mercurial 1.0 released
  http://lwn.net/Articles/274823/

  Also important, the book is a joy to read:

  Distributed revision control with Mercurial
  http://hgbook.red-bean.com/

• April 5, 2008 at 7:01 a.m. #
  Luke Plant chimed in with:

  Peter: You are right, ditch Subversion for this type of thing. You can easily import svn to git/bzr/hg, and the latter tools make adhoc branching child’s play, while with Subversion it’s the work of devils (you have to use additional tools, and you really have to decide on your merge strategy beforehand, and good luck if you choose wrong).

• April 5, 2008 at 7:01 p.m. #
  coulix piped up:

  Sometimes when an app is well done and thought it gets quite easy to tweak without touching the actual app code. For example registration view methods accept templates, classes and other flexible joints as parameters.

  The more we separate from the generic track,the more writting your own branch become a need.

• April 6, 2008 at 8:15 a.m. #
  Peter Baumgartner mentioned:

  @coulix, I agree with you that when possible, a generic app that handles most use cases is best, but it just doesn’t always work out that way.

  1. Writing apps like that to cover all the bases takes time and typically isn’t worth it for a one-off site. As I build on my main repository, the app usually works towards being a more generic solution.
  2. You can’t be everything to everyone. Somebody might want a random field in their blog that no one else needs. I’d rather fork and give them exactly what they need rather than coming up with a generic work-around that is a mediocre solution for everyone.
• April 7, 2008 at 8:37 a.m. #
  kevin mentioned:

  @peter, I completely agree with you when you are working with someone else’s project. BUT when you’re building your own project I do think you can follow some of the django best practices and create a somewhat easily extensible api.

  James Bennett’s pdf from pycon gets into some of the high level details. here’s the link: http://www.b-list.org/weblog/2008/mar/15/slides/

  I do think model inheritance will play a role in taking this a step further, and then combining james’s notes – you pretty much have a lot of flexibility.

• April 7, 2008 at 9:08 a.m. #
  Peter Baumgartner mentioned:

  Don’t get me wrong here guys. I’m all for best practices, generic applications and extensible APIs.

  My point here is that frameworks like Django are about rapid application development. If you find yourself spending significant time on APIs that you may use in the future, I think you are wasting time. Read It’s a problem when it’s a problem for a better description of what I’m getting at.

  While I always keep in mind preparations for the future, I develop for the now. Tackling obstacles is best done when you know what they are not when you are speculating before you’ve even started. Sure, I take a step back and look at the big picture, but I don’t let it stop me from moving forward. I see a lot of projects get stalled in the planning phase because of “what ifs…” IMO, you’re better off to develop using concrete requirements and then get into a “Release early, release often” cycle.

  I have yet to encounter a situation where I’ve had to significantly backtrack because of this technique. Well designed code (the kind that Django basically forces you into) should allow you to change paths and correct as you go with little difficulty.

• April 7, 2008 at 9:22 a.m. #
  kevin commented:

  @peter, a valid response indeed. I don’t disagree with you when it comes to building out projects for clients. don’t waste time over thinking the “maybe” or “wouldn’t this be great” ideas, aka developer scope creep (where a developer creates his/her own unrequired “requirements” and thereby extends the development cycle and introduces greater complexity).

  but when it comes to building an app for the open source community, I do think, if it lends itself to “customization” opportunities then the developer should plan ahead to make it extensible where they can.

  even still, I think we’d be foolish to expect whatever app is delivered to be completely free from “forking” options. if you try hard enough you’ll find a reason to “fork” any app. the best I can do…is try to reduce where applicable.

• April 7, 2008 at 9:27 a.m. #
  Peter Baumgartner commented:

  Kevin, we’re in agreement :)

• May 21, 2008 at 11:51 a.m. #
  Milan Andric mentioned:

  Thanks for putting this in words, some validation that it makes perfect sense. Unfortunately now I have to learn git, not in the mood.

• August 22, 2008 at 1:09 p.m. #
  Henrique added:

  It should be possible, to some extent, to import a base application inside your application, and then just add specific custom bits required for the current project.

Got something to say?

Name
Email
Website
Comment