Matplotlib Plugin for Indigo 7 - New!

Posted on
Mon Sep 04, 2017 12:04 pm
DaveL17 offline
User avatar
Posts: 5900
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Matplotlib Plugin for Indigo 7 - New!

@Busta999: Sounds like you may have found a solution, but I'll offer another. Go to the plugin menu and select Advanced Settings. From there, enable Custom Line Segments.

Now go back to your charting device and you will find new controls added to allow you to configure Custom Line Segments. These are easy to add. The settings are entered in sets (or Tuples in Python parlance) that include the constant value to chart and the color. Like this:

(45, "#323232"), (15, "#323232")
or
(45, "blue"), (15, "blue")

For example, on my weather charts, I add a freeze warning line: (32, "blue")

If, however, your optimal temperature changes over time (seasonal, etc.), I'd recommend charting. a variable value.
Attachments
Screen Shot 2017-09-04 at 1.02.29 PM.png
Screen Shot 2017-09-04 at 1.02.29 PM.png (179.42 KiB) Viewed 2350 times

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Mon Sep 04, 2017 12:23 pm
Busta999 offline
User avatar
Posts: 704
Joined: Mar 30, 2016
Location: Wales UK

Re: Matplotlib Plugin for Indigo 7 - New!

DaveL17 wrote:
@Busta999: Sounds like you may have found a solution, but I'll offer another. Go to the plugin menu and select Advanced Settings. From there, enable Custom Line Segments.

Now go back to your charting device and you will find new controls added to allow you to configure Custom Line Segments. These are easy to add. The settings are entered in sets (or Tuples in Python parlance) that include the constant value to chart and the color. Like this:

(45, "#323232"), (15, "#323232")
or
(45, "blue"), (15, "blue")

For example, on my weather charts, I add a freeze warning line: (32, "blue")

If, however, your optimal temperature changes over time (seasonal, etc.), I'd recommend charting. a variable value.


Thanks.

The variable path works best for me.

Basically creating icon graphs that turn increasing Red if things are not good and increasingly green if good, plus it shows the trend over the last three hours.

These will go onto a Control Page for an overview, I’ll link them to detail graphs.

Having a lot of fun with your plugin, many thanks!

Mike


Sent from my iPad using Tapatalk

Posted on
Mon Sep 04, 2017 12:32 pm
DaveL17 offline
User avatar
Posts: 5900
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Matplotlib Plugin for Indigo 7 - New!

Glad you're enjoying the plugin.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Mon Sep 18, 2017 9:37 am
Busta999 offline
User avatar
Posts: 704
Joined: Mar 30, 2016
Location: Wales UK

Re: Matplotlib Plugin for Indigo 7 - New!

Is there anyway to have CSV Engines with different refresh intervals?

i.e. most things I want to get a general trend with over a three day period or so and an interval of every 15 minutes is fine.

Other things I would like to have a 5 minute interval for for the last 4 hours or so, so more data for a shorter period.

Not an issue just a thought

Posted on
Mon Sep 18, 2017 6:53 pm
DaveL17 offline
User avatar
Posts: 5900
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Matplotlib Plugin for Indigo 7 - New!

Busta999 wrote:
Is there anyway to have CSV Engines with different refresh intervals?

i.e. most things I want to get a general trend with over a three day period or so and an interval of every 15 minutes is fine.

Other things I would like to have a 5 minute interval for for the last 4 hours or so, so more data for a shorter period.

Not an issue just a thought

Although it's possible, It's a bit of a non-trivial update. I'll add it to the TODO list (which is getting quite long for this plugin :D )

Another approach would be to create an Indigo schedule that runs a Python script and then you can set whatever refresh frequency you like. This example uses an Indigo variable value and it could be easily adapted to read a device state value instead.
Code: Select all
import os


data_name = 'Some Data'  # The header name for your data
full_path = '/Users/username/Documents/Some Data.csv'  # Copy the path from the plugin configuration settings
target_lines = 300  # max number of lines in your data file
value = indigo.variable[1401389208].value  # where the observation comes from

# If the file doesn't exist, create it.
if not os.path.isfile(full_path):
    with open(full_path, 'w') as f:
        f.write('Timestamp, {0}\n'.format(data_name))

# Open the  file in read-only mode and count the number of lines.
with open(full_path, 'r') as f:
    lines = f.readlines()
    orig_num_lines = sum(1 for _ in lines)

# Write the file (retaining the header line and the last target_lines).
if orig_num_lines > target_lines:
    with open(full_path, 'w') as f:
        f.writelines(lines[0:1])
        f.writelines(lines[(orig_num_lines - target_lines) + 1: orig_num_lines])

# Write the latest observation.
with open(full_path, 'a') as f:
    f.write('{0},{1}\n'.format(indigo.server.getTime(), value))

There is probably a way to do this more elegantly....but it works.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Tue Sep 19, 2017 1:41 am
Busta999 offline
User avatar
Posts: 704
Joined: Mar 30, 2016
Location: Wales UK

Re: Matplotlib Plugin for Indigo 7 - New!

Thanks, but the last time I did any scripting/programming was 20 years ago and I still haven’t got my head around Python ;-) This is incredibly low priority. Just a thought. Very much appreciate the plugin, I am using it a lot .


Sent from my iPad using Tapatalk

Posted on
Tue Sep 19, 2017 5:22 am
DaveL17 offline
User avatar
Posts: 5900
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Matplotlib Plugin for Indigo 7 - New!

Glad that you're finding the plugin useful.

There's no need for you to do any scripting, just follow these steps.

1. Create a new Indigo Schedule and set it to run every 5 minutes.
2. On the Actions tab, select Server Actions --> Script and File Actions --> Execute Script.
3. Make sure to select Embedded Python (it should already be selected)
4. Copy the script above into the scripting window.

Change the following lines at the top of the script:
data_name = 'Some Data' # The header name for your data
full_path = '/Users/username/Documents/Some Data.csv' # Copy the path from the plugin configuration settings
target_lines = 300 # max number of lines in your data file
value = indigo.variable[1401389208].value # where the observation comes from


data_name is a descriptive name of your data, like "Living Room Temperature"
full_path is from the plugin settings -- where your data are stored
target_lines is the number of observations you want to save (in this case 5 min x 4 hours is 48).
value is where the data to save comes from: if it's a variable, just change the ID to match the ID of your variable. If it's a device, change the line to something like this (be sure to change the ID and state name to match yours).

Code: Select all
indigo.devices[65317564].states["Power_value"]

You can get the reference to the device state by right-clicking on the device state in the Indigo Devices window and selecting "Copy Python Reference".

The whole thing should take less than 5 minutes to set up. :D

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Tue Sep 19, 2017 8:48 am
Busta999 offline
User avatar
Posts: 704
Joined: Mar 30, 2016
Location: Wales UK

Re: Matplotlib Plugin for Indigo 7 - New!

Thanks, I’ll try that.


Sent from my iPad using Tapatalk

Posted on
Fri Oct 27, 2017 9:32 am
Busta999 offline
User avatar
Posts: 704
Joined: Mar 30, 2016
Location: Wales UK

Re: Matplotlib Plugin for Indigo 7 - New!

Dave

Any idea, why , after, standardising naming conventions for Temperature Sensors I am seeing weird stuff in the CSV files for Matplotlib?

i.e.

Code: Select all
2017-10-27 14:50:08.274000,22.0
2017-10-27 14:50:08.411000,NaN
2017-10-27 15:05:15.766000,22.3
2017-10-27 15:05:15.915000,NaN
2017-10-27 15:20:23.093000,22.1
2017-10-27 15:20:23.236000,NaN
2017-10-27 15:35:30.553000,21.6
2017-10-27 15:35:30.701000,NaN
2017-10-27 15:50:38.567000,21.1
2017-10-27 15:50:38.731000,NaN
2017-10-27 16:05:46.025000,21.0
2017-10-27 16:05:46.172000,NaN
2017-10-27 16:20:53.319000,21.0
2017-10-27 16:20:53.452000,NaN
2017-10-27 16:23:27.847000,20.9
2017-10-27 16:23:28.004000,NaN


It appears to be blocking the graphs from being drawn

The three Temp Sensors it is having this issue with are all Hue, the other Hue sensors - no issues.

Any clues - the NaN is intermittent.

Looking at the device it appears to be reporting normally.

Posted on
Fri Oct 27, 2017 9:36 am
Busta999 offline
User avatar
Posts: 704
Joined: Mar 30, 2016
Location: Wales UK

Re: Matplotlib Plugin for Indigo 7 - New!

Actually seeing more int he logs - Hold off for now - let me try to get to bottom of it.

Posted on
Fri Oct 27, 2017 10:11 am
Busta999 offline
User avatar
Posts: 704
Joined: Mar 30, 2016
Location: Wales UK

Re: Matplotlib Plugin for Indigo 7 - New!

mmmm not getting too far:-

Code: Select all
Matplotlib Critical Error       The settings for CSV Engine data element 'Sunroom' are not valid: [dev: 373567006, state/value: sensorValue]
   Matplotlib Critical Error       The settings for CSV Engine data element 'Closet (Hue)' are not valid: [dev: 384021673, state/value: sensorValue]
   Matplotlib Critical Error       The settings for CSV Engine data element 'Temp - Utility (Hue)' are not valid: [dev: 1604470955, state/value: sensorValue]


Except I have deleted these three CSV Engine data elements!

The three listed above are NOT in the Item List of the Configure CSV Engine dialog.....

MatPlotLib is still creating csv files for these three deleted elements - having shutdown and restarted matplotlib how do I get it to stop processing elements that are deleted ?

Good news is that I recreated the elements - with slightly different names and it is saving the temp sensor readings correctly to the csvs.....

So at least I can get graphs going again as a work around - but it would be tidy if I could get Matplotlib from saving NaN to csv files that it shouldn't.

Also Odd that these three dodgy ones record NaN and the three good ones, using exactly the same Temp Sensor devices are recording the temps ok.

Something amis here I think.

Posted on
Fri Oct 27, 2017 10:20 am
DaveL17 offline
User avatar
Posts: 5900
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Matplotlib Plugin for Indigo 7 - New!

Hmmm. I'll have to take a deeper look at this after work.

As far as I recall, the only time that the CSV engine should be writing 'NaN' to the file is if the state value to be written is the string 'None' or an actual Nonetype value. Matplotlib really doesn't like values that aren't numbers so I insert NaN in place of those. NaN is short for "Not a Number" (which you probably already know).

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Fri Oct 27, 2017 8:09 pm
DaveL17 offline
User avatar
Posts: 5900
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Matplotlib Plugin for Indigo 7 - New!

I've been working on this for a little while now and I haven't been able to recreate your experience. I've added and deleted CSV devices and the plugin (for me) is working as I would expect. But as I was working through the possibilities, I wrote a quick script to analyze your log entries above and there's something interesting that I can't quite explain. I took your data and added a third column to calculate the time difference from row to row and found something interesting:

Code: Select all
================================================================================
Oct 27, 2017, 8:49:33 PM
untitled text 12
--------------------------------------------------------------------------------
                 timestamp  value            diff
0  2017-10-27 14:50:08.274   22.0             NaT
1  2017-10-27 14:50:08.411    NaN 00:00:00.137000
2  2017-10-27 15:05:15.766   22.3 00:15:07.355000
3  2017-10-27 15:05:15.915    NaN 00:00:00.149000
4  2017-10-27 15:20:23.093   22.1 00:15:07.178000
5  2017-10-27 15:20:23.236    NaN 00:00:00.143000
6  2017-10-27 15:35:30.553   21.6 00:15:07.317000
7  2017-10-27 15:35:30.701    NaN 00:00:00.148000
8  2017-10-27 15:50:38.567   21.1 00:15:07.866000
9  2017-10-27 15:50:38.731    NaN 00:00:00.164000
10 2017-10-27 16:05:46.025   21.0 00:15:07.294000
11 2017-10-27 16:05:46.172    NaN 00:00:00.147000
12 2017-10-27 16:20:53.319   21.0 00:15:07.147000
13 2017-10-27 16:20:53.452    NaN 00:00:00.133000
14 2017-10-27 16:23:27.847   20.9 00:02:34.395000
15 2017-10-27 16:23:28.004    NaN 00:00:00.157000

In addition to the fact that the NaNs occur every other row, look at the time differences. Aside from the somewhat anachronistic 2:34 update (which could be explained by a plugin reload), the updates alternate between 15 minute and 0.15 minute updates. It's as if two CSV engine entries are writing to the same file.

Are you certain that there aren't two entries with the same name in the list?

Do me a solid and run the following script against your server and post back with the result. (If you're concerned about the contents, feel free to send me a PM).

Code: Select all
dev = indigo.devices[445253253]  # replace this ID number with your CSV engine ID
indigo.server.log(str(dev))

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Sat Oct 28, 2017 6:19 am
Busta999 offline
User avatar
Posts: 704
Joined: Mar 30, 2016
Location: Wales UK

Re: Matplotlib Plugin for Indigo 7 - New!

I have worked out what I did and why it was soooooo stoooopid :oops:

let's NEVER talk about this again :oops:

Posted on
Sat Oct 28, 2017 6:51 am
DaveL17 offline
User avatar
Posts: 5900
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Matplotlib Plugin for Indigo 7 - New!

Busta999 wrote:
I have worked out what I did and why it was soooooo stoooopid :oops:

let's NEVER talk about this again :oops:

FIXED is never a bad thing. :D

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Who is online

Users browsing this forum: No registered users and 4 guests