Suggestion on best way to handle server restarts

Posted on
Thu Jan 26, 2017 10:08 pm
spiv offline
Posts: 190
Joined: Jul 15, 2014

Suggestion on best way to handle server restarts

I would like to fix the edge condition of schedules being skipped because the Indigo server is offline and then restarted.

I noticed that I was doing some system maintenance/upgrading and so Indigo was not running when my daily "Outside lighting" schedule usually runs. That schedule uses a condition of "Sunset + 15 minutes".

I noticed that I can set a trigger of type "Indigo Startup" that will run when the server is started.

I'm wondering what the best strategy is to have this trigger execute any applicable scheduled activities that may have been missed. Because this is a trigger and not a schedule, it has "condition" and "actions", but the condition syntax is not the same as the "schedule" tab. Thus, it is not trivial to replace a missed schedule as the "conditions" syntax doesn't directly allow for "If Sunset + 15 minutes or later" type of conditional test.

I know I can get close to that by just using the time or system variables, but I was wondering how other people are doing this?

I also don't like the idea of replicating existing scheduled tasks as triggers. Philosophically it violates the "DRY" programming as I would have to remember to change something in two places.

Does Indigo have a "run skipped schedules" capability or does that even make sense to consider a feature like that?

What is the best/simplest way to achieve this? Obviously, this occurs rarely, but for completeness I would like to handle this situation.

Posted on
Fri Jan 27, 2017 12:10 pm
jay (support) offline
Site Admin
User avatar
Posts: 18199
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Suggestion on best way to handle server restarts

Let me take a step back here and ask you a more philosophical question: how often do you run across an instance such that trying to do this "catch up" in an automated way is important?

When I first started using Indigo many (many) years ago, I had the same question. Turns out, I have some missed schedule happen maybe twice a year that I have to manually run later. So the tradeoff of trying to solve this problem vs the time it takes to manually fix the result of the schedule not running appropriately is a no brainer for me... ;)

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Posted on
Fri Jan 27, 2017 1:37 pm
spiv offline
Posts: 190
Joined: Jul 15, 2014

Re: Suggestion on best way to handle server restarts

Philosophically I might agree with you when it comes to my own installation.

However, recently with the big rain storms, both my own and my client's homes had power failures and we were both away when this happened. So although the occurrence is rare, it does happen.

For myself, I could remotely connect and manually reset devices after the power returned, but my clients do not have the expertise to do that. Also, there is the theoretical situation where power is restored, but Internet access might still be down.

From a marketing position/selling point, I heavily promote to clients that Indigo is a professional-level solution versus the DIY and consumer-grade products which are more toys, very limited, insecure, and highly dependent upon Internet connectivity. So professional-level capability like fully automated operation and intelligent recovery after unplanned power outages is an important part of that message (and helps justify the big cost difference).

Thus, I know I can setup some startup triggers manually and provide those benefits to my clients, but just looking to see what others have done or the system itself provides.

Thanks!

Posted on
Fri Jan 27, 2017 2:21 pm
GlennNZ offline
User avatar
Posts: 1555
Joined: Dec 07, 2014
Location: Central Coast, Australia

Re: Suggestion on best way to handle server restarts

Hi - thanks for the thread

I had the same thoughts - as had a few cases of missed schedules when power out - or I was fiddling with the server (latter more common cause!). And have a major set of outside lights that should turn off - but if outage stay on all night (which is fine for me to turn off but more of issue when away). (I have added a couple of extra offs through the night just in case)

Wouldn't it be a matter of registering the power outage (time/date) if Indigo server had UPS probably would need UPS hook to give that data. Issue with that is indigo may be running but during power-cut not much else is so any schedules wouldn't have any effect, hmmm. From a non-UPS or when power lost if there is no easier recorded time of last system up - Maybe reading the last log date/time entry?

And then on server-startup have a plugin run check against any schedules from outage time to present time that may not have run? (Then run them) (and probably have a limit on how many hours backwards need to run)

Obviously wouldn't deal with triggers that are time based - but running any missed schedules would certainly resolve some of my outage issues. My other major issues would be timers... running down to turn things off which obviously won't respond during outage. Probably would also need to pause any running timers in event of grid-loss and then restart them once restored.

Something to think about and maybe actively work on Plugin if can find time.

Glenn

Posted on
Sat Jan 28, 2017 12:00 pm
RogueProeliator offline
User avatar
Posts: 2501
Joined: Nov 13, 2012
Location: Baton Rouge, LA

Re: Suggestion on best way to handle server restarts

I've thought about this as well -- we have very frequent thunderstorms here and lose power multiple times during the year; generally these are short but there definitely can be some missed schedules. One reason I haven't bothered with this is the complexity that sometimes I wouldn't want a schedule to run. Example: power goes out right before dark and isn't restored until after we have gone to bed. Thus two light-related schedules would have potentially missed and be re-run - turning on lights near dark and putting the house to bed. These could conflict and, depending upon the timing of their execution, cause an unwanted situation where in the outside lights might be on at evening power instead of dimmed for night. Not the end of the world, but don't want lights potentially shining towards neighbors house full blast at midnight either.

Posted on
Sat Jan 28, 2017 2:49 pm
jay (support) offline
Site Admin
User avatar
Posts: 18199
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Suggestion on best way to handle server restarts

When you really start to think through the scenarios, it's really a significant undertaking to try to rewind schedules to figure out what the state should be after an outage. And since schedules tend to be regularly repetitive (every 24 hours or so) things tend to fix themselves given the right amount of time.

spiv wrote:
From a marketing position/selling point, I heavily promote to clients that Indigo is a professional-level solution versus the DIY and consumer-grade products which are more toys, very limited, insecure, and highly dependent upon Internet connectivity. So professional-level capability like fully automated operation and intelligent recovery after unplanned power outages is an important part of that message (and helps justify the big cost difference).


I don't believe professionally installed systems do much more in this scenario to be honest. But, the bottom line is that Indigo is a DIY system and is very significantly cheaper because of it. If it's missing some of these more esoteric features, that would be why. It doesn't effect our ability to market the system because expectations are somewhat lower.

We aren't a professionally-installed level system by choice.

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Posted on
Sat Jan 28, 2017 11:23 pm
GlennNZ offline
User avatar
Posts: 1555
Joined: Dec 07, 2014
Location: Central Coast, Australia

Suggestion on best way to handle server restarts

Hi

Complexity aside have put together a simple catchUP plugin that can save and load schedule states and survive a power outage. The aim being to save current schedule state when UPS turns on and then load schedule when AC power back on. The plugin creates these two actions load and save. They can then be called once power off or restored depending on other factors, time of day etc.

It then runs any missed schedules because of outage one after another with some pauses to enable catch-up.
(can select schedules always to exclude)

Obviously may cause light on light off until caught up.

Would be nice to run all the missed schedules through, see where devices were and then update them when power on - but don't believe is possible. And wouldn't deal with schedules that affect things other than device states.

Seems to work as designed, but want to also add save/restore for timers/pesters - which outages also impact for me (with a lot of countdown timers that then end without a received turn-off action as power out to devices) Aim is to also save and restore states and time of each running timer - working on this now.

Glenn


Sent from my iPhone using Tapatalk

Page 1 of 1

Who is online

Users browsing this forum: No registered users and 2 guests