Detect Hung Plugin and Restart it? I'm using the TED 5000

Posted on
Sun Aug 30, 2020 2:38 pm
sumocomputers offline
Posts: 267
Joined: Jun 23, 2008

Detect Hung Plugin and Restart it? I'm using the TED 5000

I currently have the TED 5000 Home Energy Monitoring plugin, and it often hangs. Unfortunately it looks to be an abandoned plugin.

Since this second by second data is important to my Grafana/InfuxDB, I can't afford to have it stay hung until I happen to notice it.

So I have a script that runs every hour to restart the plugin:

Code: Select all
plugin = indigo.server.getPlugin("com.ssi.indigoplugin.TED5000")
if plugin.isEnabled():
 plugin.restart()


Problem with that is that it takes it quite awhile to restart, so I am losing data for maybe a minute every hour, but also for some reason when that operation is happening, other unrelated Indigo triggers fail to work, enough that I see the effects maybe 2-3 times a week. I am not sure why that would be, but there is a definite correlation. I am on a 2018 Mac Mini, so I don't think it is a resource issue.

I am wondering if there is a way to tell if a plugin has hung and issue a restart it only then? Ideally I would just run this checker every 1 or 5 minutes, if there is such a thing.

Here is the TED 5000 Plugin:
viewforum.php?f=143

Posted on
Thu Dec 10, 2020 7:59 pm
whmoorejr offline
User avatar
Posts: 762
Joined: Jan 15, 2013
Location: Houston, TX

Re: Detect Hung Plugin and Restart it? I'm using the TED 500

Maybe not directly... but if this is a whole house energy meter, then the current power consumption is never zero. So you could set a trigger on the energy consumption that if it drops to zero, then run your script.

Just guessing, I don't have a TED 5000, but I have energy meters. Let me know if that was helpful.

Bill
My Plugin: My People

Posted on
Thu Dec 10, 2020 10:23 pm
sumocomputers offline
Posts: 267
Joined: Jun 23, 2008

Re: Detect Hung Plugin and Restart it? I'm using the TED 500

Unfortunately when the plug-in stops, the electricity rate read from the device is frozen, and continues to read that frozen value until the plug-in is restarted.

Posted on
Fri Dec 11, 2020 7:46 pm
whmoorejr offline
User avatar
Posts: 762
Joined: Jan 15, 2013
Location: Houston, TX

Re: Detect Hung Plugin and Restart it? I'm using the TED 500

I wonder if there is another way to get the data from the TED into indigo without the TED Plugin (since it's no longer maintained). Depending on the communication protocol of TED, I would guess either MQTT plugin if it's sending MQTT, or the Cynical Network plugin if it's sending TCP. Alternatively, maybe a script can pull the data if it's scriptable.... after a little googling, I found this: http://svn.navi.cx/misc/trunk/python/ted.py

Bill
My Plugin: My People

Posted on
Sat Dec 12, 2020 8:17 am
sumocomputers offline
Posts: 267
Joined: Jun 23, 2008

Re: Detect Hung Plugin and Restart it? I'm using the TED 500

Thanks for the info. The version I have is a 6000 that has an ethernet interface, so the Python script you found looks like it was written for an older USB based model.

I will look into the Cynical Network Plugin.

What really bums me out is that the TED Plugin pretty much does it all (including hanging, lol!).

If I actually knew what the problem was and knew how to write code, I am assuming I could deconstruct the plugin and fix it (the author doesn't have it on Github).

Posted on
Sat Dec 12, 2020 6:17 pm
sumocomputers offline
Posts: 267
Joined: Jun 23, 2008

Re: Detect Hung Plugin and Restart it? I'm using the TED 500

I am looking at the API here: http://files.theenergydetective.com/doc ... I-R330.pdf

I am going to see if I can just write some Python script(s) that run every 1 or 2 seconds on a schedule, which should get the data I want and then write them to a variable.

Posted on
Sat Dec 12, 2020 6:27 pm
whmoorejr offline
User avatar
Posts: 762
Joined: Jan 15, 2013
Location: Houston, TX

Re: Detect Hung Plugin and Restart it? I'm using the TED 500

To give you a leg up on scripting.... you can look directly at the TED Plugin to get a rough idea of how it's coded....

1st, disable the plugin.

Indigo 7.4/Plugins. --> Right-Click on the plugin and select "Show Package Contents", Then "Contents" --> "Server Plugin" --> plugin.py. Make a copy of that file on your desktop to mess and tinker with.

Bill
My Plugin: My People

Posted on
Sat Dec 12, 2020 6:32 pm
sumocomputers offline
Posts: 267
Joined: Jun 23, 2008

Re: Detect Hung Plugin and Restart it? I'm using the TED 500

Thanks for the heads up, and I started doing that earlier.

I have come up with a basic script that calls the TED API locally and returns values I want in XML, but as less than a newbie, I have been struggling to parse that data so I can insert it into a Indigo Variable. In particular I am interested in the "Now" value. This is what I came up with so far:

Code: Select all
# -*- coding: utf-8 -*-
# import indigo
import requests
import math
import urllib
import xml.etree.ElementTree as ET

#TED Dashboard GET
response = requests.get('http://192.168.20.90/api/DashData.xml')
requestURL = 'http://192.168.20.90/api/DashData.xml'
root = ET.fromstring(response.content)

print(response.content)
for child in root:
    print(child.tag, child.attrib)
print root


#Indigo Variable Update
# indigo.variable.updateValue(368223276, u"{0:.0f}".format(ted_response_01))  # Indigo Variable ID
# indigo.variable.updateValue(368223276, (ted_response_01))  # Indigo Variable ID


Which Returns:

Code: Select all
<DashData>
    <Now>623</Now>
    <TDY>23519</TDY>
    <MTD>120557</MTD>
    <Avg>27816</Avg>
    <Proj>834491</Proj>
    <Voltage>1218</Voltage>
    <Phase>131</Phase>
</DashData>

('Now', {})
('TDY', {})
('MTD', {})
('Avg', {})
('Proj', {})
('Voltage', {})
('Phase', {})

<Element 'DashData' at 0x10e936c90>

Posted on
Sat Dec 12, 2020 8:06 pm
whmoorejr offline
User avatar
Posts: 762
Joined: Jan 15, 2013
Location: Houston, TX

Re: Detect Hung Plugin and Restart it? I'm using the TED 500


Bill
My Plugin: My People

Posted on
Sat Dec 12, 2020 9:31 pm
sumocomputers offline
Posts: 267
Joined: Jun 23, 2008

Re: Detect Hung Plugin and Restart it? I'm using the TED 500

Unfortunately the GhostXML doesn't seem to be working quite right for me, not to mention it only lets me poll every 15 seconds (I need every 1 second).

However, I finally got a script that works (yes I realize it is very hacky):

Code: Select all
import indigo
import requests
import math
import urllib
import re
import xml.etree.ElementTree as ET

#TED Dashboard GET
response = requests.get('http://192.168.20.90/api/DashData.xml')
root = ET.fromstring(response.content)

now = response.content.split('\n')[1]
now = re.sub("<Now>","",now)
energyNow = float(now.split('<')[0])
energyNow = energyNow/1000

#Indigo Variable Update
indigo.variable.updateValue(108813449, u"{0:.3f}".format(energyNow))

Posted on
Sun Dec 13, 2020 10:06 pm
whmoorejr offline
User avatar
Posts: 762
Joined: Jan 15, 2013
Location: Houston, TX

Re: Detect Hung Plugin and Restart it? I'm using the TED 500

If it works, it works. If you run it that frequently, I'd save it as an external script. Indigo doesn't like to run to many scrips at the same time....

I had another thought.... I was just reading through "How to write a plugin" stuff, and it talks about when a plugin hangs, it should log an error. When your TED plugin gets stuck, does it just freeze, or does it throw an error in the log?

I ask, because you can trigger based on a log error. https://wiki.indigodomo.com/doku.php?id=indigo_7_documentation:plugins:sql_logger. I don't use this plugin to log anything, I just use it for it's triggers that watch the log file for some of my old insteon devices that are flakey. If back patio light (which turns off automatically at 10pm) doesn't acknowledge the "Turn Off", then a trigger asks the light to turn off again every 5 minutes. After the 6th attempt, it sends me a pushover notification.

Anyway, if the plugin is throwing an error, you might be able to use that SQL plugin to trigger off of the error log and use a restart plugin action.

Bill
My Plugin: My People

Posted on
Mon Dec 14, 2020 3:02 pm
sumocomputers offline
Posts: 267
Joined: Jun 23, 2008

Re: Detect Hung Plugin and Restart it? I'm using the TED 500

That was the rub dating back months (years?) ago: It wouldn't log any issue, even with advanced debug logging turned on, it simply hung with the last good data frozen.

I am pretty sure it worked pretty much flawlessly until Indigo 7.0, and by that point I think the plugin was mostly abandoned. Problems started with 7.0.

Unfortunately I have no clue what is actually happening with the plugin, but I do know if the plugin is disabled and re-enabled, it works again for a time.

Posted on
Mon Dec 14, 2020 5:09 pm
ryanbuckner offline
Posts: 1075
Joined: Oct 08, 2011
Location: Northern Virginia

Re: Detect Hung Plugin and Restart it? I'm using the TED 500

What about resetting a timer each time the data is updated and triggering your restart if a certain period of time goes by without hearing from TED? Worst case scenario is that it's reset without needing it?

Posted on
Tue Dec 15, 2020 6:01 pm
sumocomputers offline
Posts: 267
Joined: Jun 23, 2008

Re: Detect Hung Plugin and Restart it? I'm using the TED 500

A timer is a great idea. I think I will give that a go.

In the meantime, I I turned on debug logging and just going to let it run and see if maybe I can find something I missed previously.

Page 1 of 1

Who is online

Users browsing this forum: No registered users and 7 guests

cron