Patterns of Use#
The rest of the documentation explains how Configurator works in abstract, while the sections below provide concrete examples of how it can be used in various applications.
Layered config files#
A common pattern is to have a system-wide configuration file, overlaid with an optional user-specific config file. For example:
/etc/my_app.yml
data_path: /var/wherever
logging_level: warning
foo_enabled: false
/home/some_user/.my_app.yml
logging_level: debug
foo_enabled: true
This could be loaded with a function such as this:
from configurator import Config
def load_config():
base = Config.from_path('/etc/my_app.yml')
user = Config.from_path('~/.my_app.yml', optional=True)
return base + user
Using the two example config files would result in this config:
>>> load_config()
configurator.config.Config(
{'data_path': '/var/wherever',
'foo_enabled': True,
'logging_level': 'debug'}
)
Config file that extends another config file#
With this pattern
, config files use a
key to explicitly specify another config file that they extend. For example:
base.yml
data_path: /var/wherever
logging_level: warning
foo_enabled: false
my_app.yml
extends: base.yml
logging_level: debug
foo_enabled: true
This could be loaded with a function such as this:
from configurator import Config
from configurator.patterns import load_with_extends
Using the two example config files would result in this config:
>>> load_with_extends('my_app.yml', key='extends')
configurator.config.Config(
{'data_path': '/var/wherever',
'foo_enabled': True,
'logging_level': 'debug'}
)
Config files that include other config files#
Another common pattern is to have an application-wide configuration file that includes sections of configuration from files to be found in a particular directory. For example:
/etc/myapp.yml
logging_level: warning
/etc/myapp.d/site1.yaml
domain: site1.example.com
root: /var/sites/site1
/etc/myapp.d/site2.yaml
domain: site2.example.com
root: ~someuser/site2
This could be loaded with a function such as this:
from configurator import Config, source, target
from glob import glob
def load_config():
config = Config({'sites': []})
config.merge(Config.from_path('/etc/myapp.yml'))
for path in glob('/etc/myapp.d/*.y*ml'):
config.merge(Config.from_path(path), mapping={source: target['sites'].append()})
return config
Using the example config files above would result in this config:
>>> load_config()
configurator.config.Config(
{'logging_level': 'warning',
'sites': [{'domain': 'site1.example.com',
'root': '/var/sites/site1'},
{'domain': 'site2.example.com',
'root': '~someuser/site2'}]}
)
Config file overlaid with environment variables#
Environment variables provide a way to inject configuration into an application. This can often be to override configuration from a file but doesn’t easily fit the schema of a config file. Environment variables are also hindered by the fact that they only natively able to have string values.
The mapping process Configurator offers can help with both of these problems. For example:
myapp.yml
enabled: false
threads: 1
The environment variables below can be mapped into the config file above.
>>> os.environ.get('MYAPP_ENABLED')
'True'
>>> os.environ.get('MYAPP_THREADS')
'13'
This could be done with a function such as this:
from configurator import Config, convert
from ast import literal_eval
import os
def load_config():
config = Config.from_path('myapp.yml')
config.merge(os.environ, mapping={
convert('MYAPP_ENABLED', literal_eval): 'enabled',
convert('MYAPP_THREADS', int): 'threads',
})
return config
Using the example config files above would result in this config:
>>> load_config()
configurator.config.Config({'enabled': True, 'threads': 13})
Config extracted from many environment variables#
If you have configuration that is spread across many environment
variables that share a common naming pattern, the Config.from_env
class method can provide a succinct way to extract these.
For example, the following environment variables:
>>> os.environ.get('MYAPP_POSTGRES_HOST')
'some-host'
>>> os.environ.get('MYAPP_POSTGRES_PORT')
'5432'
>>> os.environ.get('MYAPP_REDIS_HOST')
'other-host'
>>> os.environ.get('MYAPP_REDIS_PORT')
'6379'
A function such as the following could be used to load the configuration:
from configurator import Config, convert
from ast import literal_eval
import os
def load_config():
return Config.from_env(
prefix={'MYAPP_POSTGRES_': 'postgres',
'MYAPP_REDIS_': 'redis'},
types={'_PORT': int}
)
Using the example environment above would result in this config:
>>> load_config()
configurator.config.Config(
{'postgres': {'host': 'some-host', 'port': 5432},
'redis': {'host': 'other-host', 'port': 6379}}
)
Config file with command line overrides#
Many applications allow you to specify the config file on the command line as well as options that override some of the file based configuration.
For example, command line arguments could be parsed by a function such as this:
from argparse import ArgumentParser, FileType
def parse_args():
parser = ArgumentParser()
parser.add_argument('config', type=FileType('r'))
parser.add_argument('--verbose', action='store_true')
parser.add_argument('--threads', type=int)
return parser.parse_args()
These arguments can be merged into the config they specify with a function such as thing:
from configurator import Config, convert, if_supplied
def verbose_to_level(verbose):
if verbose:
return 'debug'
def load_config(args):
config = Config.from_stream(args.config)
config.merge(args, mapping={
convert('verbose', verbose_to_level): 'log_level',
if_supplied('threads'): 'threads',
})
return config
So, given these command line arguments:
>>> sys.argv
['myapp.py', 'myapp.yaml', '--verbose']
Along with a config file such as this:
myapp.yaml
log_level: warning
threads: 1
The two functions above would produce the following config:
>>> args = parse_args()
>>> load_config(args)
configurator.config.Config({'log_level': 'debug', 'threads': 1})
Application and framework configuration in the same file#
It can make sense for an application and the framework it’s built with to make use of the same config file, particularly when combined with layered config files, as described above. This can allow all applications on a system to share a basic default config while providing overrides to that configuration along with their own configuration in an application-specific config file.
What makes this work is keeping the application and framework configuration in separate top-level namespaces. For example:
myapp.yml
# framework configuration:
logging:
console_level: false
file_level: warning
# application configuration, containing within one top-level key:
my_app:
enabled: True
threads: 1
Configuring the framework and application then becomes dispatching the top-level config sections appropriately:
from configurator import Config
class MyApp:
def __init__(self, enabled, threads):
self.enabled, self.threads = enabled, threads
def build_app(config_path):
config = Config.from_path(config_path)
app_config = config.my_app
app = MyApp(**app_config.data)
del config.my_app
return configure_framework(app, **config.data)
Combining the above function and configuration file might result in:
>>> build_app('myapp.yml')
TheFramework running MyApp({'enabled': True, 'threads': 1})
logging: {'console_level': False, 'file_level': 'warning'}>
Global configuration object#
For applications where there is no sensible path for passing a configuration
object to the various parts that may need to access it, it can make sense to have a global
Config
that has configuration pushed on to it at a different time to its creation.
You may instantiate the Config
in a module global scope, potentially with
some defaults:
from configurator import Config
config = Config({'default_deny': True})
You may then have a web layer that uses the common pattern of decorated functions to map URLs to the code that renders them, but that also need access to configuration information:
@app.view('/')
def root(request):
db = connect(config.db_url)
if config.default_deny and not db.query(Roles).filter_by(user=request.user):
raise HttpForbidden()
...
That same web layer may also have a hook or event that lets you configure the application during startup:
@app.configurer
def configure():
config.push(Config.from_path('myapp.yml'))
Now, when testing, you can have a fixture that pushes configuration data suitable for use during automated tests:
@pytest.fixture()
def configured():
with config.push({'db_url': 'postgresql://localhost/test'}):
yield config