More docs! w/ some emphasis on "reports" section

docs/base/cron.rst

@@ -0,0 +1,11 @@
+
+Scheduled Tasks
+===============
+
+TODO
+
+
+Luigi
+-----
+
+TODO

docs/base/email.rst

@@ -1,53 +0,0 @@
-
-Sending Email
-=============
-
-TODO
-
-
-Postfix Setup
--------------
-
-TODO
-
-
-Types of Emails
----------------
-
-TODO
-
-
-Configuring SMTP
-----------------
-
-TODO
-
-
-Configuring Recipients etc.
----------------------------
-
-TODO
-
-
-Customizing Templates
----------------------
-
-TODO
-
-
-How to programmatically send email
-----------------------------------
-
-TODO
-
-
-Nod to email via error logging
-------------------------------
-
-TODO
-
-
-Handling Email Bounces
-----------------------
-
-TODO

docs/base/email/bounces.rst

@@ -0,0 +1,6 @@
+
+========================
+ Handling Email Bounces
+========================
+
+TODO

docs/base/email/code.rst

@@ -0,0 +1,5 @@
+
+Programmatically Sending Email
+------------------------------
+
+TODO

docs/base/email/config.rst

@@ -0,0 +1,16 @@
+
+Rattail Configuration
+=====================
+
+TODO
+
+Configuring SMTP
+----------------
+
+TODO
+
+
+Configuring Recipients etc.
+---------------------------
+
+TODO

docs/base/email/dev.rst

@@ -0,0 +1,8 @@
+
+----------------------------
+ Development Considerations
+----------------------------
+
+TODO
+
+* aka. how not to send unwanted/test emails to staff or public!

docs/base/email/handler.rst

@@ -0,0 +1,6 @@
+
+===============
+ Email Handler
+===============
+
+TODO

docs/base/email/index.rst

@@ -0,0 +1,18 @@
+
+Sending Email
+=============
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   overview
+   postfix
+   system
+   logging
+   config
+   templates
+   code
+   handler
+   dev
+   bounces

docs/base/email/logging.rst

@@ -0,0 +1,6 @@
+
+================
+Logging to Email
+================
+
+TODO

docs/base/email/overview.rst

@@ -0,0 +1,24 @@
+
+==========
+ Overview
+==========
+
+This manual is primarily concerned with 3 "types" of email:
+
+* emails sent by the underlying system
+* emails sent via Python logging mechanism
+* emails sent explicitly by a Poser/Rattail app
+
+The first, "emails sent by the underlying system" refers primarily to emails
+sent by the ``cron`` daemon.  See :doc:`system` for more about those.
+
+The second, "emails sent via Python logging mechanism" is covered in
+:doc:`logging`.
+
+The third, "emails sent explicitly by a Poser/Rattail app" is the primary
+motivation behind this section of the manual.  In particular see the sections
+about :doc:`config`, :doc:`templates` and :doc:`code`.
+
+Finally, if you're sending email to the public, it may be helpful to detect
+bounces and e.g. flag an address as invalid if it triggers a bounce.  See
+:doc:`bounces` for more on that.

docs/base/email/postfix.rst

@@ -0,0 +1,5 @@
+
+Postfix Setup
+=============
+
+TODO

docs/base/email/system.rst

@@ -0,0 +1,5 @@
+
+System Emails
+=============
+
+TODO

docs/base/email/templates.rst

@@ -0,0 +1,6 @@
+
+=======================
+ Customizing Templates
+=======================
+
+TODO

docs/base/handlers.rst

@@ -3,3 +3,5 @@ App Handlers
 ============
 
 TODO
+
+* Report Handler

docs/base/index.rst

@@ -13,7 +13,9 @@ Base Layer
    config/index
    commands
    scripts
-   email
+   email/index
+   cron
    supervisor
    filemon
+   upgrades
    handlers

docs/base/upgrades.rst

@@ -0,0 +1,6 @@
+
+==============
+ App Upgrades
+==============
+
+TODO

docs/conf.py

@@ -33,8 +33,13 @@ author = 'Lance Edgar'
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
+    'sphinx.ext.intersphinx',
 ]
 
+intersphinx_mapping = {
+    'rattail': ('https://rattailproject.org/docs/rattail/', None),
+}
+
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 

docs/data/batch/custom.rst

@@ -0,0 +1,6 @@
+
+=====================
+Adding a Custom Batch
+=====================
+
+TODO

docs/data/batch/handlers.rst

@@ -0,0 +1,6 @@
+
+==============
+Batch Handlers
+==============
+
+TODO

docs/data/batch/index.rst

@@ -0,0 +1,13 @@
+
+Data Batch Processing
+=====================
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   overview
+   schema
+   handlers
+   native
+   custom

docs/data/batch/native.rst

@@ -0,0 +1,6 @@
+
+==================
+Native Batch Types
+==================
+
+TODO

docs/data/batch/overview.rst

@@ -0,0 +1,6 @@
+
+==========
+ Overview
+==========
+
+TODO

docs/data/batch/schema.rst

@@ -0,0 +1,6 @@
+
+==================
+Batch Table Schema
+==================
+
+TODO

docs/data/batches.rst

@@ -1,5 +0,0 @@
-
-Data Batch Processing
-=====================
-
-TODO

docs/data/datasync.rst

@@ -1,5 +0,0 @@
-
-Data Synchronization
-====================
-
-TODO

docs/data/index.rst

@@ -10,9 +10,11 @@ Data Layer
    trainwreck
    other
    importing
-   datasync
-   batches
+   sync
    versioning
+   batch/index
+   reports/index
+   problems
    multinode
    tempmon
    tasks

docs/data/problems.rst

@@ -0,0 +1,5 @@
+
+Problem Reports
+===============
+
+TODO

docs/data/reports/autoemail.rst

@@ -0,0 +1,6 @@
+
+====================
+Auto-Emailed Reports
+====================
+
+TODO

docs/data/reports/custom.rst

@@ -0,0 +1,81 @@
+
+========================
+ Adding a Custom Report
+========================
+
+We'll start with a simple report which shows all departments and the number of
+products for each.
+
+Each type of report has a "key" which uniquely identifies it.  For this example
+we'll use ``poser_dept_prod_counts`` for the key.  Note that it's a good idea
+to always use an app-specific prefix (e.g. ``poser_``) in the key.
+
+Typically you will create a new Python module which is named according to the
+key.  Here we'll create ``~/src/poser/poser/reports/dept_prod_counts.py``, and
+in it we'll define the report::
+
+   from sqlalchemy import orm
+
+   from rattail.reporting import ExcelReport
+
+
+   class DepartmentProductCounts(ExcelReport):
+       """
+       Shows all departments and the number of products for each.
+       """
+       type_key = 'poser_dept_prod_counts'
+       name = "Department Product Counts"
+       output_fields = [
+           'department_number',
+           'department_name',
+           'product_count',
+       ]
+
+       def make_data(self, session, params, progress=None, **kwargs):
+           model = self.model
+
+           # fetch all departments, with product lists pre-loaded
+           departments = session.query(model.Department)\
+                                .order_by(model.Department.number)\
+                                .options(orm.joinedload(model.Department.products))\
+                                .all()
+           rows = []
+
+           def add_row(department, i):
+               rows.append({
+                   'department_number': department.number,
+                   'department_name': department.name,
+                   'product_count': len(department.products),
+               })
+
+           self.progress_loop(add_row, departments, progress,
+                              message="Fetching data for report")
+           return rows
+
+Then you must register your new report type, to make it available to the app.
+This is done within your project's ``setup.py`` file, for instance::
+
+   setup(
+       name = "Poser",
+
+       # ...
+
+       entry_points = {
+
+           'rattail.reports': [
+               'poser_dept_prod_counts = poser.reports.dept_prod_counts:DepartmentProductCounts',
+           ],
+       },
+   )
+
+Once you've added that you must re-install the app to your virtual environment,
+for instance:
+
+.. code-block:: sh
+
+   cd /srv/envs/poser
+   source bin/activate
+   pip install -e ~/src/poser
+
+At this point the report should be available for running within the app.  See
+:doc:`generate` for more info.

docs/data/reports/generate.rst

@@ -0,0 +1,6 @@
+
+===================
+Generating a Report
+===================
+
+TODO

docs/data/reports/handler.rst

@@ -0,0 +1,19 @@
+
+==============
+Report Handler
+==============
+
+Each app will have exactly one "report handler" which is reponsible for the
+overall logic of running and emailing reports etc.  It also determines which
+report types are available to be ran.
+
+Rattail provides a default report handler so unless you have custom
+requirements, you will not need to define your own report handler.  See the
+:class:`~rattail:rattail.reporting.handlers.ReportHandler` class for more
+details.
+
+Programmatically you can access your report handler like so::
+
+   # assuming you already have config
+   app = config.get_app()
+   handler = app.get_report_handler()

docs/data/reports/index.rst

@@ -0,0 +1,13 @@
+
+Reports
+=======
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   overview
+   handler
+   generate
+   autoemail
+   custom

docs/data/reports/overview.rst

@@ -0,0 +1,28 @@
+
+==========
+ Overview
+==========
+
+A report is conceptually just what you think it is, probably.  But there are a
+few related concepts which we'll try to briefly outline here.
+
+* report type
+* report handler
+* report output
+
+First a "report type" refers to the logic for running a specific type of
+report.  For example, "Daily Sales" or "Slow Movers" might be report types.
+This defines what sort of data is included on the report, how it is displayed
+etc.  See :doc:`custom` for how to create a new report type.
+
+Next, a "report handler" is responsible for overall logic used to orchestrate
+the running of all reports for the app.  See :doc:`handler` for more on that.
+
+Finally "report output" refers to the result of running a report.  Generally
+speaking whenever a report runs, the details are saved to the DB for
+convenience and historical record.  See :doc:`generate` for more on that.
+
+It's perhaps worth noting, that in Rattail-speak "an email is not a report" and
+vice versa.  Meaning, a report is a *generated document* and if sent via email,
+it will be an attachment as opposed to being part of the message body.  See
+also :doc:`autoemail`.

docs/data/sync.rst

@@ -0,0 +1,5 @@
+
+Data Real-Time Sync
+===================
+
+TODO

docs/index.rst

@@ -18,10 +18,15 @@ Poser/Rattail app:
 * :doc:`base/index` - general app install, command line, plus filemon etc.
 * :doc:`data/index` - when a DB or data import/export come into it
 * :doc:`web/index` - all things web
-* :doc:`luigi/index` - using Luigi for better overnight automation
-* :doc:`deploy/index` - deals with production deployment
+* :doc:`monitor/index` - keeping an eye out for production issues
+* :doc:`deploy/index` - deploying a production system
 * :doc:`backup/index` - backing up the production system
 
+.. note::
+   Please understand, this manual is a work in progress.  We finally have an
+   "authoritative home base" for this documentation, but it may take some time
+   to fill in all the blanks.
+
 Please see `https://rattailproject.org/docs/rattail-manual/
 <https://rattailproject.org/docs/rattail-manual/>`_ for the latest version of
 this document.
@@ -35,7 +40,7 @@ this document.
    base/index
    data/index
    web/index
-   luigi/index
+   monitor/index
    deploy/index
    backup/index
 

docs/monitor/index.rst

@@ -1,6 +1,6 @@
 
-Luigi Layer
-===========
+Monitoring Layer
+================
 
 
 .. toctree::
@@ -8,3 +8,4 @@ Luigi Layer
    :caption: Contents:
 
    overview
+   shinken

docs/monitor/shinken.rst

@@ -0,0 +1,5 @@
+
+Shinken
+=======
+
+TODO