docs: improve docs relating to app db, setup thereof

docs/conf.py

@@ -29,6 +29,7 @@ templates_path = ['_templates']
 exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
 
 intersphinx_mapping = {
+    'alembic': ('https://alembic.sqlalchemy.org/en/latest/', None),
     'mako': ('https://docs.makotemplates.org/en/latest/', None),
     'packaging': ('https://packaging.python.org/en/latest/', None),
     'python': ('https://docs.python.org/3/', None),

docs/glossary.rst

@@ -18,7 +18,10 @@ Glossary
      The main :term:`database` used by the :term:`app`.  There is
      normally just one database (for simple apps) which uses
      PostgreSQL for the backend.  The app database contains the
-     :term:`settings table`.
+     :term:`settings table` as well as :term:`data models <data
+     model>`.
+
+     For more info see :doc:`narr/db/app`.
 
    app dir
      Folder containing app-specific config files, log files, etc.
@@ -112,6 +115,10 @@ Glossary
    data model
      Usually, a Python class which maps to a :term:`database` table.
 
+     The :term:`app` (assuming it has an :term:`app database`) will
+     have an "official" set of data models, represented as the
+     :term:`app model`.
+
    database
      Generally refers to a relational database which may be queried
      using SQL.  More specifically, one supported by `SQLAlchemy`_.
@@ -119,7 +126,7 @@ Glossary
      .. _SQLAlchemy: https://www.sqlalchemy.org
 
      Most :term:`apps<app>` will have at least one :term:`app
-     database`.
+     database`.  See also :doc:`narr/db/index`.
 
    db session
      The "session" is a SQLAlchemy abstraction for an open database

docs/index.rst

@@ -53,10 +53,10 @@ Contents
    glossary
    narr/install/index
    narr/config/index
+   narr/db/index
    narr/cli/index
    narr/handlers/index
    narr/providers/index
-   narr/db/index
 
 .. toctree::
    :maxdepth: 1

docs/narr/db/app.rst

@@ -2,20 +2,105 @@
 App Database
 ============
 
-The :term:`app database` is used at minimum to contain the
+The :term:`app database` is used at minimum to store the
-:term:`settings table`.
+:term:`settings table`, but usually also has tables for other
+:term:`data models <data model>` used by the app.
 
-There is not yet support within WuttJamaican for the creation or setup
+If you *only* want a settings table but do not need the app database
-of the app database.  So for now you're on your own with that.
+to serve any other purpose, see :doc:`/narr/config/table`.
-
-See also :doc:`/narr/config/table`.
-
-Note that while any database supported by SQLAlchemy may be used, docs
-will generally assume PostgreSQL is being used.
 
 
-Configuring the Connection
+.. _create-appdb:
---------------------------
+
+Create the Database
+-------------------
+
+There is not currently any tooling in WuttJamaican to *create* the
+database (unless using a SQLite file, which may happen automatically).
+
+PostgreSQL is the recommended backend for production, as it is the
+only one with rigorous usage thus far.  MySQL should also work though
+and you're free to experiment.  Theoretically anything supported by
+SQLAlchemy should work; see :doc:`sqlalchemy:dialects/index`.
+
+You may need to install additional Python and/or OS packages to
+support your desired backend.
+
+PostgreSQL
+~~~~~~~~~~
+
+Install APT packages if needed:
+
+.. code-block:: sh
+
+   sudo apt install postgresql libpq-dev
+
+Install Python packages (to :term:`virtual environment`) if needed:
+
+.. code-block:: sh
+
+   pip install psycopg2
+
+Make a new DB user ("myuser") if needed:
+
+.. code-block:: sh
+
+   sudo -u postgres createuser myuser
+
+And if so, also set the password:
+
+.. code-block:: sh
+
+   sudo -u postgres psql -c "ALTER USER myuser PASSWORD 'mypassword'"
+
+And finally create the DB ("myappdb" owned by "myuser"):
+
+.. code-block:: sh
+
+   sudo -u postgres createdb -O myuser myappdb
+
+MySQL
+~~~~~
+
+Install APT packages if needed:
+
+.. code-block:: sh
+
+   sudo apt install default-mysql-server
+
+Install Python packages (to :term:`virtual environment`) if needed:
+
+.. code-block:: sh
+
+   pip install mysql-connector-python
+
+Make a new DB user ("myuser") if needed:
+
+.. code-block:: sh
+
+   sudo mysql -e "CREATE USER myuser@localhost"
+
+And if so, also set the password:
+
+.. code-block:: sh
+
+   sudo mysql -e "ALTER USER myuser@localhost  IDENTIFIED BY 'mypassword'"
+
+Create the DB ("myappdb"):
+
+.. code-block:: sh
+
+   sudo mysqladmin create myappdb
+
+And grant all perms (to "myuser" for "myappdb"):
+
+.. code-block:: sh
+
+   sudo mysql -e "GRANT ALL ON myappdb.* TO myuser@localhost"
+
+
+Configure the Connection
+------------------------
 
 Once you have a database ready, add to your :term:`config file` the
 details, for example:
@@ -23,7 +108,77 @@ details, for example:
 .. code-block:: ini
 
    [wutta.db]
-   default.url = postgresql://wutta:wuttapass@localhost/wuttadb
+
+   # postgres
+   default.url = postgresql://myuser:mypassword@localhost/myappdb
+
+   # mysql
+   default.url = mysql+mysqlconnector://myuser:mypassword@localhost/myappdb
+
+You also most likely want to prefer settings from the DB over those
+found in the config file(s).  See also
+:ref:`where-config-settings-come-from` but the gist is, you should add
+this config:
+
+.. code-block:: ini
+
+   [wutta.config]
+   usedb = true
+   preferdb = true
+
+
+Install the Schema
+------------------
+
+So far there is not a tool to "create all tables" for the :term:`app
+model` in one step per se.  Rather, we use Alembic to "apply all
+migrations" to get to the latest schema.  (The end result is the
+same.)
+
+See also the :doc:`Alembic docs <alembic:index>`, but our process is
+fairly simple.
+
+First add some Alembic settings to your :term:`config file`:
+
+.. code-block:: ini
+
+   [alembic]
+   script_location = wuttjamaican.db:alembic
+   version_locations = wuttjamaican.db:alembic/versions
+
+Usually the ``script_location`` shown above will work fine, but the
+``version_locations`` may vary depending on which packages contribute
+to your overall app model.
+
+For instance a Poser app which also uses :doc:`Wutta-Continuum
+<wutta-continuum:index>` may specify this instead:
+
+.. code-block:: ini
+
+   [alembic]
+   script_location = wuttjamaican.db:alembic
+   version_locations = wutta_continuum.db:alembic/versions poser.db:alembic/versions wuttjamaican.db:alembic/versions
+
+Note that is really specifying 3 different packages, and the sequence matters (*):
+
+* ``wutta_continuum.db:alembic/versions``
+* ``poser.db:alembic/versions``
+* ``wuttjamaican.db:alembic/versions``
+
+(*) While it does seem to matter, this is not yet fully understood.
+You may need to experiment.
+
+In any case once you've added the Alembic settings you can migrate schema:
+
+.. code-block:: sh
+
+   alembic -c /path/to/my.conf upgrade heads
+
+If you have multiple packages for schema (as shown above) and you get
+errors here, you may need to try a different package sequence in
+config.
+
+But if the migration went okay then you now have a complete app database.
 
 
 Multiple Databases
@@ -39,11 +194,23 @@ Using that example, the host config might look like:
 .. code-block:: ini
 
    [wutta.db]
-   # nb. host itself is referred to as 'default'
+   # nb. the localhost ("host") node is default
    keys = default, store001, store002, store003
 
-   default.url = postgresql://wutta:wuttapos@localhost/wutta-host
+   default.url = postgresql://wutta:wuttapass@localhost/wutta-host
 
-   store001.url = postgresql://wutta:wuttapos@store001/wutta-store
+   store001.url = postgresql://wutta:wuttapass@store001/wutta-store
-   store002.url = postgresql://wutta:wuttapos@store002/wutta-store
+   store002.url = postgresql://wutta:wuttapass@store002/wutta-store
-   store003.url = postgresql://wutta:wuttapos@store003/wutta-store
+   store003.url = postgresql://wutta:wuttapass@store003/wutta-store
+
+And to be thorough, each store config might look like:
+
+.. code-block:: ini
+
+   [wutta.db]
+   # nb. the localhost ("store") node is default
+   keys = default, host
+
+   default.url = postgresql://wutta:wuttapass@localhost/wutta-store
+
+   host.url = postgresql://wutta:wuttapass@host-server/wutta-host

docs/narr/db/index.rst

@@ -2,9 +2,21 @@
 Databases
 =========
 
-.. toctree::
+Most :term:`apps <app>` based on WuttJamaican will have an :term:`app
-   :maxdepth: 2
+database`.  This may be used to store :term:`config settings <config
+setting>` but usually, lots of other things.
+
+Each app can declare its :term:`app model` which is essentially the
+list of tables, each mapped to a Python class via SQLAlchemy ORM.  The
+default app model is :mod:`wuttjamaican.db.model`.
+
+But of course any other :term:`database(s) <database>` may be
+involved, for integration purposes etc.  So there are some
+conveniences around that too.
+
+
+.. toctree::
+   :maxdepth: 3
 
-   overview
    app
    other

docs/narr/db/overview.rst

@@ -1,13 +0,0 @@
-
-Overview
-========
-
-A :term:`database` is usually involved for "serious"
-:term:`apps<app>`.
-
-In particular an :term:`app database` may be needed to store
-:term:`config settings<config setting>` and usually, lots of other
-things.  See :doc:`app`.
-
-But it is possible to interact with many databases, not just the app
-database.  See :doc:`other`.

docs/narr/install/pkg.rst

@@ -8,9 +8,15 @@ To install the :term:`package` into your :term:`virtual environment`:
 
    pip install WuttJamaican
 
-Please also see
+Note that the above is only for basic config/app system and CLI.  If
-:doc:`packaging:guides/installing-using-pip-and-virtual-environments`
+you also want an :term:`app database` then add the 'db' extra:
-in upstream docs.
+
+.. code-block:: sh
+
+   pip install WuttJamaican[db]
+
+For more general info see
+:doc:`packaging:guides/installing-using-pip-and-virtual-environments`.
 
 
 Sanity Check

docs/narr/install/prereqs.rst

@@ -8,11 +8,7 @@ Wuttjamaican requires Python, and optionally a database of some sort.
 Python
 ------
 
-Currently at least Python 3.6 is required, however:
+Currently at least Python 3.8 is required.
-
-As of writing only Python 3.8 and newer are supported by the official
-Python team, so that is strongly recommended.  It is likely that will
-soon become the minimum requirement for WuttJamaican as well.
 
 Also note, Python 3.11 is the newest version being tested so far.
 
@@ -22,15 +18,8 @@ See also https://endoflife.date/python
 Database
 --------
 
-There is not yet much logic in WuttJamaican which pertains to the
+If you need an :term:`app database` then you will also need the
-:term:`app database` so we will not document much about that here
+backend and/or libraries to support that.
-either.
 
-For now just know that in a production environment, PostgreSQL is
+The recommendation is to use PostgreSQL but this is further documented
-recommended for the DB backend.  So install that if you want to be
+elsewhere; see :ref:`create-appdb`.
-certain of a good experience.
-
-But technically speaking, anything supported by `SQLAlchemy`_ should
-work.  See also :doc:`/narr/config/table`.
-
-.. _SQLAlchemy: https://www.sqlalchemy.org

docs/narr/install/quickstart.rst

@@ -30,7 +30,8 @@ Make and activate a new :term:`virtual environment` for your project:
    python3 -m venv poser
    source poser/bin/activate
 
-Make a new e.g. ``poser`` database in PostgreSQL (or MySQL).
+Make a new e.g. ``poser`` database in PostgreSQL (or MySQL).  Nothing
+special here but for instructions see :ref:`create-appdb`.
 
 Install and run `cookiecutter <https://cookiecutter.readthedocs.io/>`_
 with `wuttaweb template
@@ -75,7 +76,7 @@ the package with:
 
 .. code-block:: sh
 
-   pip install wuttjamaican
+   pip install WuttJamaican[db]
 
 Create a :term:`config file`, e.g. ``my.conf``:
 
@@ -125,6 +126,7 @@ For more info see:
 * :func:`~wuttjamaican.conf.make_config()`
 * :class:`~wuttjamaican.conf.WuttaConfig` and especially
   :meth:`~wuttjamaican.conf.WuttaConfig.get()`
+* :class:`~wuttjamaican.app.AppHandler`
 
 
 .. _db-setup:
@@ -134,20 +136,9 @@ Database Setup
 
 You should already have the package installed (see previous section).
 
-You also must install some package(s) for the particular database
-backend you wish to use.  PostgreSQL is recommended although MySQL
-etc. should also work.  For instance:
-
-.. code-block:: sh
-
-   # postgres
-   pip install psycopg2
-
-   # mysql
-   pip install mysql-connector-python
-
 Next you must create the database, as well as any user account needed,
-within the DB backend.
+within the DB backend.  This is pretty routine but for instructions
+see :ref:`create-appdb`.
 
 Now add the DB info to your :term:`config file` (e.g. ``my.conf`` as
 shown above).  Contents for this will look something like (using
@@ -163,8 +154,6 @@ shown above).  Contents for this will look something like (using
    # mysql
    default.url = mysql+mysqlconnector://USERNAME:PASSWORD@localhost/poserdb
 
-See :doc:`/narr/db/app` for more about that.
-
 You also must add some Alembic config, needed for DB schema
 migrations:
 
@@ -178,8 +167,17 @@ With config file updated you can run the Alembic command to migrate schema:
 
 .. code-block:: sh
 
-   cd /path/to/env
+   alembic -c /path/to/my.conf upgrade heads
-   bin/alembic -c /path/to/my.conf upgrade heads
 
 Now you should have all the tables required for a WuttJamaican
 :term:`app database`.
+
+If you wish to store :term:`config settings <config setting>` in the
+DB, don't forget to add to your config file (see also
+:ref:`where-config-settings-come-from`):
+
+.. code-block:: ini
+
+   [wutta.config]
+   usedb = true
+   preferdb = true