Commit graph

15 commits

Author SHA1 Message Date
Lance Edgar 2f7fa3430a Make server more tolerant of database restart
note that a retry is *not* attempted within a given "check readings" run.
rather, server will consider that full readings check to have failed, if any
part of it fails.

but then we keep track of type/amount of some (database connection) failures,
and will suppress logging the full error for first 3 attempts.  in practice
this lets us recover from simple database restarts, and if database becomes
truly unavailable we'll hear about it shortly.

any other type of error is immediately logged on first failure.
2018-10-07 18:16:18 -05:00
Lance Edgar b4fa6a17c5 Add "status since" to template context for email alerts 2018-10-06 20:17:46 -05:00
Lance Edgar 76e40063ee Tweak server logic for checking client readings
do not check readings for "archived" clients.  do not consider the client
"offline" unless it has *no* current probe readings.  previously we were
assuming offline if any probe readings were missing, even if some were found.
2018-10-06 18:09:02 -05:00
Lance Edgar 018a9dcb08 Don't let server mark client as offline until readings fail 3 times in a row
previously we were only letting this fail once, if that
2018-09-28 11:57:27 -05:00
Lance Edgar 5df3379995 Send email alert when tempmon server marks a client as offline
courtesy of Cole Chaney <cole@mamajeansmarket.com>
2018-02-07 17:48:07 -06:00
Lance Edgar 8a1551e0f5 Send first alert "immediately" if critical temp status
i.e. only wait for "first email" delay if *not* critical

courtesy of Cole Chaney <cole@mamajeansmarket.com>
2018-02-07 17:47:48 -06:00
Lance Edgar 90e431c617 Switch license to GPL v3 (no longer Affero)
refs #2
2017-07-06 23:38:50 -05:00
Lance Edgar 396b3739f3 Fix bug when marking client as offline from server loop
..i think?
2017-06-01 17:37:27 -05:00
Lance Edgar c4b371cedd Refactor main server loop a bit, to add basic retry w/ error logging
hopefully this lets us get past a simple Postgres restart..
2017-06-01 17:16:31 -05:00
Lance Edgar 4e11748b45 Add configurable delay per client; improve try/catch 2017-02-09 17:59:06 -06:00
Lance Edgar e6f3448acc Hopefully fix alert logic when status becomes good
No need to send alert for good status, if previous status never actually
triggered an alert.
2016-12-11 10:46:53 -06:00
Lance Edgar de1543018a DRY up some server alert logic a bit 2016-12-10 23:04:42 -06:00
Lance Edgar 646f09a413 Add mail templates, plus initial status alert delay for probes
i.e. let the temp get "proper bad" (for some minutes) before alerting
2016-12-10 12:40:50 -06:00
Lance Edgar fc7ec58303 Fix import bug in server daemon 2016-12-05 20:59:05 -06:00
Lance Edgar 7550e63940 Initial commit 2016-12-05 19:06:34 -06:00