I've been working quite a bit lately on streamlining Lincoln Loop's standard deployment systems. One thorn we've always had is how to handle application configuration.
In the past, we would have our configuration management system write the configuration out to a JSON file at a known location on the filesystem. The application would read the JSON and set the necessary variables accordingly. This accomplished a few goals:
- Deployments didn't require any sort of modification to the code from the upstream repository.
- Production secrets could be encrypted and stored safely away from the code.
- Unlike environment variables, the data could have proper types (booleans, lists, etc.)
- It avoided using environment variables altogether, which can be problematic from a security perspective in many scenarios.
That being said, the setup had some downfalls as well.
- The configuration variables needed were not well documented. You had to read through the code to understand how and where they were used.
- We were maintaining a separate Django settings module for local and deployed environments since the file wasn't used locally.
- The configuration file was not friendly for other services like Heroku or Docker where environment variables are typically used.
Basically, I want to use our Python projects like I would expect to use any other software. Install the software, create a configuration file, and run.
With Goodconf, you create a class which defines all the configuration values your application expects to receive. They can have default values and help text which can be used to generate documentation.
from goodconf import GoodConf, Value class MyConf(GoodConf): "Configuration for My App" DEBUG = Value(default=False, help="Toggle debugging.") DATABASE_URL = Value( default='postgres://localhost:5432/mydb', help="Database connection.") SECRET_KEY = Value( initial=lambda: base64.b64encode(os.urandom(60)).decode(), help="Used for cryptographic signing. " "https://docs.djangoproject.com/en/2.0/ref/settings/#secret-key")
You can then instantiate the class like so:
config = MyConf( file_env_var="MYAPP_CONF", default_files=["/etc/myapp/myapp.yml", "myapp.yml"])
This lets you define the default location for configuration files (
$(pwd)/myapp.yml if that doesn't exist) and also an environment variable that can be used to load a file from a different location.
There is also a Django helper which lets you do
manage.py --config=/path/to/config.yml ....
With the configuration defined, you simply need to load it and start using it. Loading the config will try to read from the configuration files and fallback on environment variables (cast to the proper Python type) if none are found.
config.load() SECRET_KEY = config.SECRET_KEY ...
My favorite feature of Goodconf is the output you can generate from a config class:
config.generate_yaml()boilerplate YAML config with help text as comments
config.generate_json()boilerplate JSON config
config.generate_markdown()Documentation in Markdown format
This lets you quickly generate a config file for local development and documentation to drop into a
saltdash/__init__.pydefines the configuration
saltdash/settings/__init__.pyshows how to use it to define Django settings
[options.entry_points]) shows how to install scripts for the management command and configuration generator
Try it Out
We're using Goodconf in production now and happy with the results. We hope you'll try it out on your own projects. If you do, please give us feedback here or in GitHub.