docs: add some narrative docs to explain basic concepts

still needs a lot of work i'm sure..gotta start somewhere

docs/narr/custom/command.rst

@@ -0,0 +1,9 @@
+
+Define Command
+==============
+
+Now that you have defined the import handler plus any importers
+required, you'll want to define a command line interface to use it.
+
+This section is here for completeness but the process is described
+elsewhere; see :doc:`/narr/cli/custom`.

docs/narr/custom/conventions.rst

@@ -0,0 +1,90 @@
+
+Conventions
+===========
+
+Below are recommended conventions for structuring and naming the files
+in your project relating to import/export.
+
+The intention for these rules is that they are "intuitive" based on
+the fact that all data flows from source to target and therefore can
+be thought of as "importing" in virtually all cases.
+
+But there are a lot of edge cases out there so YMMV.
+
+
+"The Rules"
+-----------
+
+There are exceptions to these of course, but in general:
+
+* regarding how to think about these conventions:
+
+  * always look at it from target's perspective
+
+  * always look at it as an *import*, not export
+
+* "final" logic is always a combo of:
+
+  * "base" logic for how target data read/write happens generally
+
+  * "specific" logic for how that happens using a particular data source
+
+* targets each get their own subpackage within project
+
+  * and within that, also an ``importing`` (nested) subpackage
+
+    * and within *that* is where the files live, referenced next
+
+  * target ``model.py`` should contain ``ToTarget`` importer base class
+
+    * also may have misc. per-model base classes, e.g. ``WidgetImporter``
+
+    * also may have ``ToTargetHandler`` base class if applicable
+
+  * sources each get their own module, named after the source
+
+    * should contain the "final" handler class, e.g. ``FromSourceToTarget``
+
+    * also contains "final" importer classes needed by handler (e.g. ``WidgetImporter``)
+
+
+Example
+-------
+
+That's a lot of rules so let's see it.  Here we assume a Wutta-based
+app named Poser and it integrates with a Foo API in the cloud.  Data
+should flow both ways so we will be thinking of this as:
+
+* **Foo → Poser import**
+* **Poser → Foo export**
+
+Here is the suggested file layout:
+
+.. code-block:: none
+
+   poser/
+   ├── foo/
+   │   ├── __init__.py
+   │   ├── api.py
+   │   └── importing/
+   │       ├── __init__.py
+   │       ├── model.py
+   │       └── poser.py
+   └── importing/
+       ├── __init__.py
+       ├── foo.py
+       └── model.py
+
+And the module breakdown:
+
+* ``poser.foo.api`` has e.g. ``FooAPI`` interface logic
+
+**Foo → Poser import** (aka. "Poser imports from Foo")
+
+* ``poser.importing.model`` has ``ToPoserHandler``, ``ToPoser`` and per-model base importers
+* ``poser.importing.foo`` has ``FromFooToPoser`` plus final importers
+
+**Poser → Foo export** (aka. "Foo imports from Poser")
+
+* ``poser.foo.importing.model`` has ``ToFooHandler``, ``ToFoo`` and per-model base importer
+* ``poser.foo.importing.poser`` has ``FromPoserToFoo`` plus final importers

docs/narr/custom/handler.rst

@@ -0,0 +1,93 @@
+
+Define Import Handler
+=====================
+
+The obvious step here is to define a new :term:`import handler`, which
+ultimately inherits from
+:class:`~wuttasync.importing.handlers.ImportHandler`.  But the choice
+of which class(es) *specifically* to inherit from, is a bit more
+complicated.
+
+
+Choose the Base Class(es)
+-------------------------
+
+If all else fails, or to get started simply, you can always just
+inherit from :class:`~wuttasync.importing.handlers.ImportHandler`
+directly as the only base class.  You'll have to define any methods
+needed to implement desired behavior.
+
+However depending on your particular source and/or target, there may
+be existing base classes defined somewhere from which you can inherit.
+This may save you some effort, and/or is just a good idea to share
+code where possible.
+
+Keep in mind your import handler can inherit from multiple base
+classes, and often will - one base for the source side, and another
+for the target side.  For instance::
+
+   from wuttasync.importing import FromFileHandler, ToWuttaHandler
+
+   class FromExcelToPoser(FromFileHandler, ToWuttaHandler):
+       """
+       Handler for Excel file → Poser app DB
+       """
+
+You generally will still need to define/override some methods to
+customize behavior.
+
+All built-in base classes live under :mod:`wuttasync.importing`.
+
+
+.. _register-importer:
+
+Register Importer(s)
+--------------------
+
+If nothing else, most custom handlers must override
+:meth:`~wuttasync.importing.handlers.ImportHandler.define_importers()`
+to "register" importer(s) as appropriate.  There are two primary goals
+here:
+
+* add "new" (totally custom) importers
+* override "existing" importers (inherited from base class)
+
+Obviously for this to actually work the importer(s) must exist in
+code; see :doc:`importer`.
+
+As an example let's say there's a ``FromFooToWutta`` handler which
+defines a ``Widget`` importer.
+
+And let's say you want to customize that, by tweaking slightly the
+logic for ``WigdetImporter`` and adding a new ``SprocketImporter``::
+
+   from somewhere_else import (FromFooToWutta, ToWutta,
+                               WidgetImporter as WidgetImporterBase)
+
+   class FromFooToPoser(FromFooToWutta):
+       """
+       Handler for Foo -> Poser
+       """
+
+       def define_importers(self):
+
+           # base class defines the initial set
+           importers = super().define_importers()
+
+           # override widget importer
+           importers['Widget'] = WidgetImporter
+
+           # add sprocket importer
+           importers['Sprocket'] = SprocketImporter
+
+           return importers
+
+   class SprocketImporter(ToWutta):
+       """
+       Sprocket importer for Foo -> Poser
+       """
+
+   class WidgetImporter(WidgetImporterBase):
+       """
+       Widget importer for Foo -> Poser
+       """

docs/narr/custom/importer.rst

@@ -0,0 +1,149 @@
+
+Define Importer(s)
+==================
+
+Here we'll describe how to make a custom :term:`importer/exporter
+<importer>`, which can process a given :term:`data model`.
+
+..
+   The example will assume a **Foo → Poser import** for the ``Widget``
+   :term:`data model`.
+
+
+Choose the Base Class(es)
+-------------------------
+
+As with the :term:`import handler`, the importer "usually" will have
+two base classes: one for the target side and another for the source.
+
+The base class for target side is generally more fleshed out, with
+logic to read/write data for the given target model.  Whereas the base
+class for the source side could just be a stub.  In the latter case,
+one might choose to skip it and inherit only from the target base
+class.
+
+In any case the final importer class you define can override any/all
+logic from either base class if needed.
+
+
+Example: Foo → Poser import
+---------------------------
+
+Here we'll assume a Wutta-based app named "Poser" which will be
+importing "Widget" data from the "Foo API" cloud service.
+
+In this case we will inherit from a base class for the target side,
+which already knows how to talk to the :term:`app database` via
+SQLAlchemy ORM.
+
+But for the source side, there is no existing base class for the Foo
+API service, since that is just made-up - so we will also define our
+own base class for that::
+
+   from wuttasync.importing import Importer, ToWutta
+
+   # nb. this is not real of course, but an example
+   from poser.foo.api import FooAPI
+
+   class FromFoo(Importer):
+      """
+      Base class for importers using Foo API as source
+      """
+
+      def setup(self):
+          """
+          Establish connection to Foo API
+          """
+          self.foo_api = FooAPI(self.config)
+
+   class WidgetImporter(FromFoo, ToWutta):
+      """
+      Widget importer for Foo -> Poser
+      """
+
+      def get_source_objects(self):
+          """
+          Fetch all "raw" widgets from Foo API
+          """
+          # nb. also not real, just example
+          return self.foo_api.get_widgets()
+
+      def normalize_source_object(self, widget):
+          """
+          Convert the "raw" widget we receive from Foo API, to a
+          "normalized" dict with data for all fields which are part of
+          the processing request.
+          """
+          return {
+              'id': widget.id,
+              'name': widget.name,
+          }
+
+
+Example: Poser → Foo export
+---------------------------
+
+In the previous scenario we imported data from Foo to Poser, and here
+we'll do the reverse, exporting from Poser to Foo.
+
+As of writing the base class logic for exporting from Wutta :term:`app
+database` does not yet exist.  And the Foo API is just made-up so
+we'll add one-off base classes for both sides::
+
+   from wuttasync.importing import Importer
+
+   class FromWutta(Importer):
+      """
+      Base class for importers using Wutta DB as source
+      """
+
+   class ToFoo(Importer):
+      """
+      Base class for exporters targeting Foo API
+      """
+
+   class WidgetImporter(FromWutta, ToFoo):
+      """
+      Widget exporter for Poser -> Foo
+      """
+
+      def get_source_objects(self):
+         """
+         Fetch all widgets from the Poser app DB.
+
+         (see note below regarding the db session)
+         """
+         model = self.app.model
+         return self.source_session.query(model.Widget).all()
+
+      def normalize_source_object(self, widget):
+          """
+          Convert the "raw" widget from Poser app (ORM) to a
+          "normalized" dict with data for all fields which are part of
+          the processing request.
+          """
+          return {
+              'id': widget.id,
+              'name': widget.name,
+          }
+
+Note that the ``get_source_objects()`` method shown above makes use of
+a ``source_session`` attribute - where did that come from?
+
+This is actually not part of the importer proper, but rather this
+attribute is set by the :term:`import handler`.  And that will ony
+happen if the importer is being invoked by a handler which supports
+it.  So none of that is shown here, but FYI.
+
+(And again, that logic isn't written yet, but there will "soon" be a
+``FromSqlalchemyHandler`` class defined which implements this.)
+
+
+Regster with Import Handler
+---------------------------
+
+After you define the importer/exporter class (as shown above) you also
+must "register" it within the import/export handler.
+
+This section is here for completeness but the process is described
+elsewhere; see :ref:`register-importer`.

docs/narr/custom/index.rst

@@ -0,0 +1,21 @@
+
+Custom Import/Export
+====================
+
+This section explains what's required to make your own import/export
+tasks.
+
+See also :doc:`/narr/concepts` for some terminology etc.
+
+..
+   The examples throughout the sections below will often involve a
+   theoretical **Foo → Poser** import, where Poser is a typical
+   Wutta-based app and Foo is some API in the cloud.
+
+.. toctree::
+   :maxdepth: 2
+
+   conventions
+   handler
+   importer
+   command