wutta/wuttjamaican
3
0
Fork 0
Code Activity

docs: improve docs relating to app db, setup thereof

Browse source
This commit is contained in:
Lance Edgar 2024-12-08 12:19:42 -06:00
parent d70df11be6
commit 45ea5c5bdc
9 changed files with 241 additions and 74 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
docs/conf.py
View file

@ -29,6 +29,7 @@ templates_path = ['_templates']
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
intersphinx_mapping = {
'alembic': ('https://alembic.sqlalchemy.org/en/latest/', None),
'mako': ('https://docs.makotemplates.org/en/latest/', None),
'packaging': ('https://packaging.python.org/en/latest/', None),
'python': ('https://docs.python.org/3/', None),

11
docs/glossary.rst
View file

@ -18,7 +18,10 @@ Glossary
The main :term:`database` used by the :term:`app`. There is
normally just one database (for simple apps) which uses
PostgreSQL for the backend. The app database contains the
:term:`settings table`.
:term:`settings table` as well as :term:`data models <data
model>`.
For more info see :doc:`narr/db/app`.
app dir
Folder containing app-specific config files, log files, etc.
@ -112,6 +115,10 @@ Glossary
data model
Usually, a Python class which maps to a :term:`database` table.
The :term:`app` (assuming it has an :term:`app database`) will
have an "official" set of data models, represented as the
:term:`app model`.
database
Generally refers to a relational database which may be queried
using SQL. More specifically, one supported by `SQLAlchemy`_.
@ -119,7 +126,7 @@ Glossary
.. _SQLAlchemy: https://www.sqlalchemy.org
Most :term:`apps<app>` will have at least one :term:`app
database`.
database`. See also :doc:`narr/db/index`.
db session
The "session" is a SQLAlchemy abstraction for an open database

2
docs/index.rst
View file

@ -53,10 +53,10 @@ Contents
glossary
narr/install/index
narr/config/index
narr/db/index
narr/cli/index
narr/handlers/index
narr/providers/index
narr/db/index
.. toctree::
:maxdepth: 1

201
docs/narr/db/app.rst
View file

@ -2,20 +2,105 @@
App Database
============
The :term:`app database` is used at minimum to contain the
:term:`settings table`.
The :term:`app database` is used at minimum to store the
:term:`settings table`, but usually also has tables for other
:term:`data models <data model>` used by the app.
There is not yet support within WuttJamaican for the creation or setup
of the app database. So for now you're on your own with that.
See also :doc:`/narr/config/table`.
Note that while any database supported by SQLAlchemy may be used, docs
will generally assume PostgreSQL is being used.
If you *only* want a settings table but do not need the app database
to serve any other purpose, see :doc:`/narr/config/table`.
Configuring the Connection
--------------------------
.. _create-appdb:
Create the Database
-------------------
There is not currently any tooling in WuttJamaican to *create* the
database (unless using a SQLite file, which may happen automatically).
PostgreSQL is the recommended backend for production, as it is the
only one with rigorous usage thus far. MySQL should also work though
and you're free to experiment. Theoretically anything supported by
SQLAlchemy should work; see :doc:`sqlalchemy:dialects/index`.
You may need to install additional Python and/or OS packages to
support your desired backend.
PostgreSQL
~~~~~~~~~~
Install APT packages if needed:
.. code-block:: sh
sudo apt install postgresql libpq-dev
Install Python packages (to :term:`virtual environment`) if needed:
.. code-block:: sh
pip install psycopg2
Make a new DB user ("myuser") if needed:
.. code-block:: sh
sudo -u postgres createuser myuser
And if so, also set the password:
.. code-block:: sh
sudo -u postgres psql -c "ALTER USER myuser PASSWORD 'mypassword'"
And finally create the DB ("myappdb" owned by "myuser"):
.. code-block:: sh
sudo -u postgres createdb -O myuser myappdb
MySQL
~~~~~
Install APT packages if needed:
.. code-block:: sh
sudo apt install default-mysql-server
Install Python packages (to :term:`virtual environment`) if needed:
.. code-block:: sh
pip install mysql-connector-python
Make a new DB user ("myuser") if needed:
.. code-block:: sh
sudo mysql -e "CREATE USER myuser@localhost"
And if so, also set the password:
.. code-block:: sh
sudo mysql -e "ALTER USER myuser@localhost IDENTIFIED BY 'mypassword'"
Create the DB ("myappdb"):
.. code-block:: sh
sudo mysqladmin create myappdb
And grant all perms (to "myuser" for "myappdb"):
.. code-block:: sh
sudo mysql -e "GRANT ALL ON myappdb.* TO myuser@localhost"
Configure the Connection
------------------------
Once you have a database ready, add to your :term:`config file` the
details, for example:
@ -23,7 +108,77 @@ details, for example:
.. code-block:: ini
[wutta.db]
default.url = postgresql://wutta:wuttapass@localhost/wuttadb
# postgres
default.url = postgresql://myuser:mypassword@localhost/myappdb
# mysql
default.url = mysql+mysqlconnector://myuser:mypassword@localhost/myappdb
You also most likely want to prefer settings from the DB over those
found in the config file(s). See also
:ref:`where-config-settings-come-from` but the gist is, you should add
this config:
.. code-block:: ini
[wutta.config]
usedb = true
preferdb = true
Install the Schema
------------------
So far there is not a tool to "create all tables" for the :term:`app
model` in one step per se. Rather, we use Alembic to "apply all
migrations" to get to the latest schema. (The end result is the
same.)
See also the :doc:`Alembic docs <alembic:index>`, but our process is
fairly simple.
First add some Alembic settings to your :term:`config file`:
.. code-block:: ini
[alembic]
script_location = wuttjamaican.db:alembic
version_locations = wuttjamaican.db:alembic/versions
Usually the ``script_location`` shown above will work fine, but the
``version_locations`` may vary depending on which packages contribute
to your overall app model.
For instance a Poser app which also uses :doc:`Wutta-Continuum
<wutta-continuum:index>` may specify this instead:
.. code-block:: ini
[alembic]
script_location = wuttjamaican.db:alembic
version_locations = wutta_continuum.db:alembic/versions poser.db:alembic/versions wuttjamaican.db:alembic/versions
Note that is really specifying 3 different packages, and the sequence matters (*):
* ``wutta_continuum.db:alembic/versions``
* ``poser.db:alembic/versions``
* ``wuttjamaican.db:alembic/versions``
(*) While it does seem to matter, this is not yet fully understood.
You may need to experiment.
In any case once you've added the Alembic settings you can migrate schema:
.. code-block:: sh
alembic -c /path/to/my.conf upgrade heads
If you have multiple packages for schema (as shown above) and you get
errors here, you may need to try a different package sequence in
config.
But if the migration went okay then you now have a complete app database.
Multiple Databases
@ -39,11 +194,23 @@ Using that example, the host config might look like:
.. code-block:: ini
[wutta.db]
# nb. host itself is referred to as 'default'
# nb. the localhost ("host") node is default
keys = default, store001, store002, store003
default.url = postgresql://wutta:wuttapos@localhost/wutta-host
default.url = postgresql://wutta:wuttapass@localhost/wutta-host
store001.url = postgresql://wutta:wuttapos@store001/wutta-store
store002.url = postgresql://wutta:wuttapos@store002/wutta-store
store003.url = postgresql://wutta:wuttapos@store003/wutta-store
store001.url = postgresql://wutta:wuttapass@store001/wutta-store
store002.url = postgresql://wutta:wuttapass@store002/wutta-store
store003.url = postgresql://wutta:wuttapass@store003/wutta-store
And to be thorough, each store config might look like:
.. code-block:: ini
[wutta.db]
# nb. the localhost ("store") node is default
keys = default, host
default.url = postgresql://wutta:wuttapass@localhost/wutta-store
host.url = postgresql://wutta:wuttapass@host-server/wutta-host

18
docs/narr/db/index.rst
View file

@ -2,9 +2,21 @@
Databases
=========
.. toctree::
:maxdepth: 2
Most :term:`apps <app>` based on WuttJamaican will have an :term:`app
database`. This may be used to store :term:`config settings <config
setting>` but usually, lots of other things.
Each app can declare its :term:`app model` which is essentially the
list of tables, each mapped to a Python class via SQLAlchemy ORM. The
default app model is :mod:`wuttjamaican.db.model`.
But of course any other :term:`database(s) <database>` may be
involved, for integration purposes etc. So there are some
conveniences around that too.
.. toctree::
:maxdepth: 3
overview
app
other

13
docs/narr/db/overview.rst
View file

@ -1,13 +0,0 @@
Overview
========
A :term:`database` is usually involved for "serious"
:term:`apps<app>`.
In particular an :term:`app database` may be needed to store
:term:`config settings<config setting>` and usually, lots of other
things. See :doc:`app`.
But it is possible to interact with many databases, not just the app
database. See :doc:`other`.

12
docs/narr/install/pkg.rst
View file

@ -8,9 +8,15 @@ To install the :term:`package` into your :term:`virtual environment`:
pip install WuttJamaican
Please also see
:doc:`packaging:guides/installing-using-pip-and-virtual-environments`
in upstream docs.
Note that the above is only for basic config/app system and CLI. If
you also want an :term:`app database` then add the 'db' extra:
.. code-block:: sh
pip install WuttJamaican[db]
For more general info see
:doc:`packaging:guides/installing-using-pip-and-virtual-environments`.
Sanity Check

21
docs/narr/install/prereqs.rst
View file

@ -8,11 +8,7 @@ Wuttjamaican requires Python, and optionally a database of some sort.
Python
------
Currently at least Python 3.6 is required, however:
As of writing only Python 3.8 and newer are supported by the official
Python team, so that is strongly recommended. It is likely that will
soon become the minimum requirement for WuttJamaican as well.
Currently at least Python 3.8 is required.
Also note, Python 3.11 is the newest version being tested so far.
@ -22,15 +18,8 @@ See also https://endoflife.date/python
Database
--------
There is not yet much logic in WuttJamaican which pertains to the
:term:`app database` so we will not document much about that here
either.
If you need an :term:`app database` then you will also need the
backend and/or libraries to support that.
For now just know that in a production environment, PostgreSQL is
recommended for the DB backend. So install that if you want to be
certain of a good experience.
But technically speaking, anything supported by `SQLAlchemy`_ should
work. See also :doc:`/narr/config/table`.
.. _SQLAlchemy: https://www.sqlalchemy.org
The recommendation is to use PostgreSQL but this is further documented
elsewhere; see :ref:`create-appdb`.

36
docs/narr/install/quickstart.rst
View file

@ -30,7 +30,8 @@ Make and activate a new :term:`virtual environment` for your project:
python3 -m venv poser
source poser/bin/activate
Make a new e.g. ``poser`` database in PostgreSQL (or MySQL).
Make a new e.g. ``poser`` database in PostgreSQL (or MySQL). Nothing
special here but for instructions see :ref:`create-appdb`.
Install and run `cookiecutter <https://cookiecutter.readthedocs.io/>`_
with `wuttaweb template
@ -75,7 +76,7 @@ the package with:
.. code-block:: sh
pip install wuttjamaican
pip install WuttJamaican[db]
Create a :term:`config file`, e.g. ``my.conf``:
@ -125,6 +126,7 @@ For more info see:
* :func:`~wuttjamaican.conf.make_config()`
* :class:`~wuttjamaican.conf.WuttaConfig` and especially
:meth:`~wuttjamaican.conf.WuttaConfig.get()`
* :class:`~wuttjamaican.app.AppHandler`
.. _db-setup:
@ -134,20 +136,9 @@ Database Setup
You should already have the package installed (see previous section).
You also must install some package(s) for the particular database
backend you wish to use. PostgreSQL is recommended although MySQL
etc. should also work. For instance:
.. code-block:: sh
# postgres
pip install psycopg2
# mysql
pip install mysql-connector-python
Next you must create the database, as well as any user account needed,
within the DB backend.
within the DB backend. This is pretty routine but for instructions
see :ref:`create-appdb`.
Now add the DB info to your :term:`config file` (e.g. ``my.conf`` as
shown above). Contents for this will look something like (using
@ -163,8 +154,6 @@ shown above). Contents for this will look something like (using
# mysql
default.url = mysql+mysqlconnector://USERNAME:PASSWORD@localhost/poserdb
See :doc:`/narr/db/app` for more about that.
You also must add some Alembic config, needed for DB schema
migrations:
@ -178,8 +167,17 @@ With config file updated you can run the Alembic command to migrate schema:
.. code-block:: sh
cd /path/to/env
bin/alembic -c /path/to/my.conf upgrade heads
alembic -c /path/to/my.conf upgrade heads
Now you should have all the tables required for a WuttJamaican
:term:`app database`.
If you wish to store :term:`config settings <config setting>` in the
DB, don't forget to add to your config file (see also
:ref:`where-config-settings-come-from`):
.. code-block:: ini
[wutta.config]
usedb = true
preferdb = true