Add narrative docs for app configuration

docs/glossary.rst

@@ -0,0 +1,72 @@
+.. _glossary:
+
+Glossary
+========
+
+.. glossary::
+   :sorted:
+
+   app
+     Depending on context, may refer to the software application
+     overall, or the :term:`app handler`.
+
+   app database
+     The main database used by the :term:`app`.  There is normally
+     just one database (for simple apps) which uses PostgreSQL for the
+     backend.
+
+   app handler
+     Python object representing the core of the :term:`app`.  There is
+     normally just one "global" app handler, which is an instance of
+     :class:`~wuttjamaican.app.AppHandler`.
+
+   app name
+     The code-friendly name for the :term:`app`
+     (e.g. ``wutta_poser``).  This is available on the :term:`config
+     object` within Python as
+     :attr:`~wuttjamaican.conf.WuttaConfig.appname`.
+
+     See also the human-friendly :term:`app title`.
+
+   app title
+     The human-friendly name for the :term:`app` (e.g. "Wutta Poser").
+
+     See also the code-friendly :term:`app name`.
+
+   command
+     A top-level command line interface for the app.  Note that
+     top-level commands don't really "do" anything per se, and are
+     mostly a way to group :term:`subcommands<subcommand>`.  See also
+     :class:`~wuttjamaican.commands.base.Command`.
+
+   config
+     Depending on context, may refer to any of: :term:`config file`,
+     :term:`config object`, :term:`config setting`.  See also
+     :doc:`narr/config/index`.
+
+   config file
+     A file which contains :term:`config settings<config setting>`.
+     See also :doc:`narr/config/files`.
+
+   config object
+     Python object representing the full set of :term:`config
+     settings<config setting>` for the :term:`app`.  Usually it gets
+     some of the settings from :term:`config files<config file>`, but
+     it may also get some from the :term:`settings table`.  See also
+     :doc:`narr/config/object`.
+
+   config setting
+     The value of a setting as obtained from a :term:`config object`.
+     Depending on context, sometimes this refers specifically to
+     values obtained from the :term:`settings table` as opposed to
+     :term:`config file`.  See also :doc:`narr/config/settings`.
+
+   settings table
+     Table in the :term:`app database` which is used to store
+     :term:`config settings<config setting>`.
+
+   subcommand
+     A top-level :term:`command` may expose one or more subcommands,
+     for the overall command line interface.  Subcommands are the real
+     workhorse; each can perform a different function.  See also
+     :class:`~wuttjamaican.commands.base.Subcommand`.

docs/index.rst

@@ -2,62 +2,28 @@
 WuttJamaican
 ============
 
-aka. Whatcha Makin
+This package provides a "base layer" for custom apps.
-
-This package provides a "base layer" of sorts, for apps built with it.
 
 It mostly is a distillation of certain patterns developed within the
-Rattail Project, which are deemed generally useful.  (At least,
+`Rattail Project`_, which are deemed generally useful.  (At least,
 according to the author.)  It roughly corresponds to the "base layer"
-as described in the Rattail Manual (see :doc:`rattail-manual:base/index`).
+as described in the Rattail Manual (see
+:doc:`rattail-manual:base/index`).
+
+.. _Rattail Project: https://rattailproject.org/
 
 Good documentation and 100% test coverage are priorities for this project.
 
-Much remains to be done, and it may be slow going since I'll be trying
+Rattail is still the main use case so far, and will be refactored
-to incorporate this package into the main Rattail package along the
+along the way to incorporate what this package has to offer.
-way.  So we'll see where this goes...
-
-Main points of focus so far are the configuration and command line
-interfaces.
 
 
-Basic Usage
+Features
------------
+--------
 
-Install with:
+* flexible configuration, using config files and/or DB settings table
-
+* flexible command line interface, with arbitrary top-level and
-.. code-block:: sh
+  subcommands
-
-   pip install wuttjamaican
-
-Create a config file, e.g. ``my.conf``:
-
-.. code-block:: ini
-
-   [foo]
-   bar = A
-   baz = 2
-   feature = true
-   words = the quick brown fox
-
-In your app, load the config and reference its values as needed::
-
-   from wuttjamaican.conf import make_config
-
-   config = make_config('/path/to/my.conf')
-
-   # this call..                        ..returns this value
-
-   config.get('foo.bar')                # 'A'
-
-   config.get('foo.baz')                # '2'
-   config.get_int('foo.baz')            # 2
-
-   config.get('foo.feature')            # 'true'
-   config.get_bool('foo.feature')       # True
-
-   config.get('foo.words')              # 'the quick brown fox'
-   config.get_list('foo.words')         # ['the', 'quick', 'brown', 'fox']
 
 
 Contents
@@ -66,6 +32,8 @@ Contents
 .. toctree::
    :maxdepth: 3
 
+   glossary
+   narr/index
    api/index
 
 

docs/narr/config/files.rst

@@ -0,0 +1,202 @@
+
+Config Files
+============
+
+A :term:`config file` is just a text file with :term:`config
+settings<config setting>`.
+
+
+Basic Syntax
+------------
+
+Currently only INI-style syntax is supported.  Under the hood a
+:class:`~python:configparser.ConfigParser` instance is used to read
+the files.
+
+There is no "type hinting" within the config file itself, although you
+can ask the config object to interpret values according to a specific
+type.  See also :ref:`reading-config-settings`.
+
+The basic syntax looks like this:
+
+.. code-block:: ini
+
+   [myapp]
+   foo = A
+   bar = 2
+   feature = true
+   words = the,quick,brown,fox,"did something unusual"
+   paths =
+       /path/to/first/folder
+       "/path/to/folder with spaces"
+       /another/one   /and/another
+
+   [more]
+   things = go here
+
+Note that ``words`` and ``paths`` show 2 ways of defining lists, for
+use with :meth:`~wuttjamaican.conf.WuttaConfig.get_list()`.  This
+splits the value by whitespace as well as commas; quotation marks may
+be used to avoid unwanted splits.
+
+
+Specifying via Command Line
+---------------------------
+
+All :term:`commands<command>` accept the ``-c`` or ``--config`` params:
+
+.. code-block:: sh
+
+   wutta --config=myapp.conf
+
+   wutta -c first.conf -c second.conf
+
+
+Specifying via Environment Variable
+-----------------------------------
+
+Probably most useful for command line scripts etc.  Note that if the
+command line itself specifies ``-c`` or ``--config`` then the
+environment variables are ignored.
+
+.. code-block:: sh
+
+   WUTTA_CONFIG_FILES=myapp.conf
+
+   WUTTA_CONFIG_FILES=first.conf:second.conf
+
+The env variable name used will depend on the :term:`app name`.
+
+
+Specifying via Python
+---------------------
+
+Pass the files directly to :func:`~wuttjamaican.conf.make_config()`::
+
+   make_config('myapp.conf')
+
+   make_config(['first.conf', 'second.conf'])
+
+
+File Priority
+-------------
+
+If multiple config files are used then the sequence will matter in
+terms of value lookup.  Effectively, whenever
+:meth:`~wuttjamaican.conf.WuttaConfig.get()` is called on the config
+object, each file will be searched until a value is found.
+
+For example let's say you have 3 config files:
+
+* ``app.conf`` ("most specific to the app")
+* ``machine.conf`` ("less specific to the app")
+* ``site.conf`` ("least specific to the app")
+
+To ensure that sequence you must specify the files in that order (*),
+e.g. via command line:
+
+.. code-block:: sh
+
+   wutta -c app.conf -c machine.conf -c site.conf
+
+or via Python::
+
+   config = make_config(['app.conf', 'machine.conf', 'site.conf'])
+
+(*) Actually that isn't always true, but for now let's pretend.
+
+That way, if both ``app.conf`` and ``site.conf`` have a particular
+setting defined, the value from ``app.conf`` will "win" and the value
+from ``site.conf`` is simply ignored.
+
+The sequence of files actually read into the config object may be
+confirmed by inspecting either
+:attr:`~wuttjamaican.conf.WuttaConfig.files_read` or (for typical
+setups) the log file.
+
+
+Including More Files
+--------------------
+
+When :func:`~wuttjamaican.conf.make_config()` is called, it first
+determines the set of config files based on caller params etc.  It
+then gives that set of files to the
+:class:`~wuttjamaican.conf.WuttaConfig` constructor.
+
+But when these files are actually read into the config object, they
+can in turn "include" (or "require") additional files.
+
+For example let's again say you have these 3 config files:
+
+* ``app.conf``
+* ``machine.conf``
+* ``site.conf``
+
+In the previous section we mentioned you could request all 3 files in
+the correct order:
+
+.. code-block:: sh
+
+   wutta -c app.conf -c machine.conf -c site.conf
+
+But another, usually better way is to add config settings such as:
+
+in ``app.conf``
+
+.. code-block:: ini
+
+   [wutta.config]
+   include = %(here)s/machine.conf
+
+in ``machine.conf``
+
+.. code-block:: ini
+
+   [wutta.config]
+   include = %(here)s/site.conf
+
+And then you need only specify the main file when running the app:
+
+.. code-block:: sh
+
+   wutta -c app.conf
+
+or via Python::
+
+   make_config('app.conf')
+
+Examples above show the ``include`` syntax but ``require`` is similar:
+
+.. code-block:: ini
+
+   [wutta.config]
+   require = /path/to/otherfile.conf
+
+If an "included" file is missing it will be skipped, but if a
+"required" file is missing an error will be raised.
+
+
+Default Locations
+-----------------
+
+If no config files were specified via any method, then some default
+file paths may be tried as fallback.
+
+The actual paths used for defaults will vary based on :term:`app name`
+and other details such as operating system.  But as a simple (and
+incomplete) example, with app name of ``wutta`` running on Linux,
+default paths would include things like:
+
+* ``~/.wutta.conf``
+* ``/usr/local/etc/wutta.conf``
+* ``/etc/wutta.conf``
+
+While it is hoped that some may find this feature useful, it is
+perhaps better to be explicit about which config files you want the
+app to use.
+
+Custom apps may also wish to devise ways to override the logic
+responsible for choosing default paths.
+
+For more details see :func:`~wuttjamaican.conf.get_config_paths()` and
+:func:`~wuttjamaican.conf.generic_default_files()`.

docs/narr/config/index.rst

@@ -0,0 +1,11 @@
+
+Configuration
+=============
+
+.. toctree::
+   :maxdepth: 2
+
+   overview
+   settings
+   object
+   files

docs/narr/config/object.rst

@@ -0,0 +1,56 @@
+
+Config Object
+=============
+
+The app has a global :term:`config object` to track its settings.
+This object is an instance of :class:`~wuttjamaican.conf.WuttaConfig`
+and is usually available as e.g. ``self.config`` within code.
+
+
+Creating the Config Object
+--------------------------
+
+All apps create the config object by calling
+:func:`~wuttjamaican.conf.make_config()` during startup.  The desired
+config files may be specified directly via call params, or indirectly
+via environment variables.  (See also :doc:`files`.)
+
+In some cases, notably the :term:`command` line interface, there is
+already code in place to handle the ``make_config()`` call, and you
+must specify the config files in another way - command line parameters
+in this case.
+
+One-off scripts should create the config object before doing anything
+else.  To be safe this should happen before other modules are
+imported::
+
+   from wuttjamaican.conf import make_config
+
+   config = make_config()
+
+   from otherlib import foo
+
+   foo(config)
+
+
+Creating the App Handler
+------------------------
+
+The config object is also responsible for creating the :term:`app handler`.
+
+Whereas the process of creating the config object is "stable" and
+"always" produces an object of the same class, the app handler is more
+likely to vary.  So while there is a default
+:class:`~wuttjamaican.app.AppHandler` provided, it is expected that
+some apps will want to override that.
+
+The relationship between config object and app handler may be thought
+of as "one-to-one" since each app will have a global config object as
+well as a global app handler.  But the config object does come first,
+to solve the "chicken-vs-egg" problem::
+
+   from wuttjamaican.conf import make_config
+
+   config = make_config()
+
+   app = config.get_app()

docs/narr/config/overview.rst

@@ -0,0 +1,36 @@
+
+Overview
+========
+
+The app uses a global :term:`config object` to keep track of its
+:term:`config settings<config setting>`.  See also :doc:`object`.
+
+The app must call :func:`~wuttjamaican.conf.make_config()` during
+startup to obtain the config object.
+
+Values come (mostly) from :term:`config files<config file>` and/or the
+:term:`settings table`.  See also :doc:`settings`.
+
+Values are always strings in their raw format, as returned by
+:meth:`~wuttjamaican.conf.WuttaConfig.get()`.  But the config object
+also has methods to coerce values to various types, e.g.:
+
+* :meth:`~wuttjamaican.conf.WuttaConfig.get_bool()`
+* :meth:`~wuttjamaican.conf.WuttaConfig.get_int()`
+* :meth:`~wuttjamaican.conf.WuttaConfig.get_list()`
+
+The config object is also responsible for creating the :term:`app
+handler`::
+
+   from wuttjamaican.conf import make_config
+
+   config = make_config()
+   app = config.get_app()
+
+   if config.get_bool('foo.bar'):
+       print('YES for foo.bar')
+   else:
+       print('NO for foo.bar')
+
+   with app.short_session() as session:
+       print(session.bind)

docs/narr/config/settings.rst

@@ -0,0 +1,104 @@
+
+Config Settings
+===============
+
+The app uses :term:`config settings<config setting>` to control its
+behavior at runtime.
+
+The term "config setting" may be thought of as a combination of these
+terms:
+
+* :term:`config file`
+* :term:`settings table`
+
+It really refers to the **value** of such a config setting, when you
+get right down to it.  The app uses a :term:`config object` to keep
+track of its config settings.
+
+
+.. _reading-config-settings:
+
+Reading Values via Python
+-------------------------
+
+Call the config object's :meth:`~wuttjamaican.conf.WuttaConfig.get()`
+method to retrieve a value based on the setting name.
+
+Note that raw values are always strings.  The config object has other
+methods if you want to interpret the value as a particular type::
+
+   from wuttjamaican.conf import make_config
+
+   config = make_config()
+
+   config.get('foo.bar')
+   config.get_int('foo.baz')
+   config.get_bool('foo.feature')
+   config.get_list('foo.words')
+
+See :class:`~wuttjamaican.conf.WuttaConfig` for full details.
+
+
+.. _where-config-settings-come-from:
+
+Where Values Come From
+----------------------
+
+Config settings usually come from either a :term:`config file` or a
+:term:`settings table`.  The :term:`config object` is ultimately
+responsible for sorting out which value to return.
+
+Technically the app may also specify some fallback/default values; for
+sake of this discussion we'll treat those as if they come from config
+file.
+
+All apps are expected to use config file(s), but not all will have a
+settings table.  The config file(s) may specify whether a settings
+table should be used.
+
+There are only 2 config settings which control this behavior.  For a
+typical example which enables both:
+
+.. code-block:: ini
+
+   [wutta.config]
+   usedb = true
+   preferdb = true
+
+   [wutta.db]
+   default.url = sqlite://
+
+Note that to use a settings table you must of course define a DB
+connection.
+
+So the ``usedb`` and ``preferdb`` flags may be set to accomplish any
+of these scenarios:
+
+* enable both - settings table is checked first, config files used as
+  fallback
+* enable ``usedb`` but not ``preferdb`` - config files are checked
+  first, settings table used as fallback
+* disable ``usedb`` - config files only; do not use settings table
+
+Most apps will want to enable both flags so that when the settings
+table is updated, it will immediately affect app behavior regardless
+of what values are in the config files.
+
+The values for these flags is available at runtime as:
+
+* :attr:`~wuttjamaican.conf.WuttaConfig.usedb`
+* :attr:`~wuttjamaican.conf.WuttaConfig.preferdb`
+
+Regardless of what the "normal" behavior is for the config object (per
+those flags), you can explcitly request other behavior by passing
+similar flags to the config object's
+:meth:`~wuttjamaican.conf.WuttaConfig.get()` method::
+
+   config.get('foo.bar', usedb=True, preferdb=True)
+
+   config.get('foo.baz', usedb=False)
+
+Some of the "core" settings in the framework are fetched with
+``usedb=False`` so they will never be read from the settings table.
+Canonical example of this would be the setting(s) which defines the DB
+connection itself.

docs/narr/index.rst

@@ -0,0 +1,9 @@
+
+Documentation
+=============
+
+.. toctree::
+   :maxdepth: 2
+
+   install/index
+   config/index

docs/narr/install/index.rst

@@ -0,0 +1,10 @@
+
+Installation
+============
+
+Read on for setup instructions etc.
+
+.. toctree::
+   :maxdepth: 2
+
+   quickstart

docs/narr/install/quickstart.rst

@@ -0,0 +1,49 @@
+
+Quick Start
+===========
+
+Install with:
+
+.. code-block:: sh
+
+   pip install wuttjamaican
+
+Create a config file, e.g. ``my.conf``:
+
+.. code-block:: ini
+
+   [foo]
+   bar = A
+   baz = 2
+   feature = true
+   words = the quick brown fox
+
+In your app, load the config and reference its values as needed::
+
+   from wuttjamaican.conf import make_config
+
+   config = make_config('/path/to/my.conf')
+
+   # this call..                        ..returns this value
+
+   config.get('foo.bar')                # 'A'
+
+   config.get('foo.baz')                # '2'
+   config.get_int('foo.baz')            # 2
+
+   config.get('foo.feature')            # 'true'
+   config.get_bool('foo.feature')       # True
+
+   config.get('foo.words')              # 'the quick brown fox'
+   config.get_list('foo.words')         # ['the', 'quick', 'brown', 'fox']
+
+For more info see:
+
+* :func:`~wuttjamaican.conf.make_config()`
+* :class:`~wuttjamaican.conf.WuttaConfig` and especially
+  :meth:`~wuttjamaican.conf.WuttaConfig.get()`
+
+You can also define your own command line interface; see:
+
+* :class:`~wuttjamaican.commands.base.Command`
+* :class:`~wuttjamaican.commands.base.Subcommand`

src/wuttjamaican/conf.py

@@ -131,6 +131,38 @@ class WuttaConfig:
        These are listed in the same order as they were read.  This
        sequence also reflects priority for value lookups, i.e. the
        first file with the value wins.
+
+    .. attribute:: usedb
+
+       Whether the :term:`settings table` should be searched for
+       config settings.  This is ``False`` by default but may be
+       enabled via config file:
+
+       .. code-block:: ini
+
+          [wutta.config]
+          usedb = true
+
+       See also :ref:`where-config-settings-come-from`.
+
+    .. attribute:: preferdb
+
+       Whether the :term:`settings table` should be preferred over
+       :term:`config files<config file>` when looking for config
+       settings.  This is ``False`` by default, and in any case is
+       ignored unless :attr:`usedb` is ``True``.
+
+       Most apps will want to enable this flag so that when the
+       settings table is updated, it will immediately affect app
+       behavior regardless of what values are in the config files.
+
+       .. code-block:: ini
+
+          [wutta.config]
+          usedb = true
+          preferdb = true
+
+       See also :ref:`where-config-settings-come-from`.
     """
 
     def __init__(