Plotting a variable

Posted on
Tue Oct 08, 2019 5:24 am
Busta999 offline
User avatar
Posts: 714
Joined: Mar 30, 2016
Location: Wales UK

Plotting a variable

Just an inconsistency I cannot understand.

It has changed from how it used to operate.

I am creating a graph that shows trend over a three hour period, part of a whole file of small (more icons) that show trending information around the house.

Chart Mini - Trend Molly Bedroom.png
Chart Mini - Trend Molly Bedroom.png (1.39 KiB) Viewed 3743 times


I have a variable that I have set as the baseline (red fill) I would like a bedroom to operate at, (actually a temp I would prefer it did not go over rather than the low)

The readings from the Thermostat in the room show the last three hours of readings.
The readings from the (static) TargetTemp Variable only have the last 23 minutes of readings.

How do I get the Variable to give me 3 hours of data instead of just the last 23 minutes?

As far as I can tell I have set it to collect the variable every 900 seconds (15 mins) and 12 readings would be 3 hours.

Indigo - MatPlotLib - Variable.jpeg
Indigo - MatPlotLib - Variable.jpeg (167.71 KiB) Viewed 3743 times


Any suggestions on how to force a variable read every 15 mins (900 s) instead of the current 60 odd?

Here is the Variable readings created by above settings.

Code: Select all
Timestamp,value
2019-10-08 11:47:47.302810,19
2019-10-08 11:48:57.172034,19
2019-10-08 11:50:07.310476,19
2019-10-08 11:51:32.563689,19
2019-10-08 11:52:46.242134,19
2019-10-08 11:53:57.359369,19
2019-10-08 11:55:08.123609,19
2019-10-08 11:56:18.308898,19
2019-10-08 11:57:27.382639,19
2019-10-08 11:58:35.763288,19
2019-10-08 11:59:49.547685,19
2019-10-08 12:00:00.952983,19
2019-10-08 12:01:14.714210,19
2019-10-08 12:02:25.960104,19
2019-10-08 12:03:37.181905,19
2019-10-08 12:04:48.536864,19
2019-10-08 12:06:01.272319,19
2019-10-08 12:07:11.461174,19
2019-10-08 12:08:21.625802,19
2019-10-08 12:09:34.602851,19
2019-10-08 12:10:44.205600,19

Posted on
Tue Oct 08, 2019 6:19 am
DaveL17 offline
User avatar
Posts: 6742
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Plotting a variable

Hmmm. Apart from a weird bug, I can't think of a reason that they would be different otherwise.

Are you using the same CSV engine device for both data elements?

Any chance you have another device writing to this same data file? (In other words, more than one CSV device writing to the same file.)

Any chance you have another script or Indigo process writing data to this file?

I don't recall making any significant changes to the CSV Engine part of the code, so I'm not sure what might be causing this. We'll get to the bottom of it.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Tue Oct 08, 2019 6:24 am
DaveL17 offline
User avatar
Posts: 6742
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Plotting a variable

Also, it's highly recommended to always consolidate CSV Engine devices of like duration. In other words, put all 3 hour (12/3/900) elements together in one CSV Engine device.

For example, I have one CSV Engine device that writes data every 15 minutes and saves 72 hours of data (and all elements that use this timing are in that one device).

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Tue Oct 08, 2019 6:24 am
Busta999 offline
User avatar
Posts: 714
Joined: Mar 30, 2016
Location: Wales UK

Re: Plotting a variable

I just figured you had a windy day and tweaked variable management :-)

Those Great Lakes do cause a distraction :-)

I once looked at buying a house up at Crystal Lake. Changed my mind and moved to Melbourne instead....

I'm just going out - will check out when I get back later today.

Thanks

Posted on
Tue Oct 08, 2019 6:31 am
DaveL17 offline
User avatar
Posts: 6742
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Plotting a variable

Nah. I've lived in the Chicago area my whole life and a bit of wind isn't going to even register.

Crystal Lake, IL vs. Melbourne. The advantage of Crystal Lake is that there's a whole lot less wildlife that's going to kill you. The advantage of Melbourne (one of many) is that you can go to the beach on Christmas. :D

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Tue Oct 08, 2019 9:51 am
Busta999 offline
User avatar
Posts: 714
Joined: Mar 30, 2016
Location: Wales UK

Re: Plotting a variable

1st Christmas in Melbourne, did a BBQ and swam in the pool on Christmas Day - it was cold brrrrrrrrr but had to be done :-)

We took photos and looked happy :-) then emailed them to relatives :-)

I run 4 chart engines,

once a day for a year - tracking battery consumption in battery devices Hue and ZWave

Every minute for two days recording motion around the house

Every 15 mins for 3 hours showing trends in heating around the house

Every 15 mins for 48 hours to track heating/hot water and room temps around the house.

Looks like I had a few doubled up there.

Will run for a day or so and see if everything is better now I have removed duplicated engines on the same devices/variables.

Hopefully that will sort it out.

No idea how they came to be duplicated.

Posted on
Tue Oct 08, 2019 6:54 pm
DaveL17 offline
User avatar
Posts: 6742
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Plotting a variable

Please do let me know how it goes.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Sat Oct 12, 2019 6:11 am
Busta999 offline
User avatar
Posts: 714
Joined: Mar 30, 2016
Location: Wales UK

Re: Plotting a variable

Dave,

Thanks

I had a concerted effort to clean up the Charts and the engines.

I have learnt over time to do this in the absence of other guidance.

Created 4 Engines.

Long - Updates every 24hrs - tracks Battery consumption over time
1Min - Updates every 60 Seconds - tracks Motion around the house - Unlikely it will remain active for too long - determines trends.
Main - Tracks every 15 mins for 48 hours gives me graphing for activity around the house, temp etc for understanding how Heating and temperature are operating. This is the Main Engine.
Mini - Tracks every 15 mins for 3 hours - this allows trending data for Key rooms around the house - allows users to see that a room is trending warmer or cooler, the Graph tends to be very narrowly focused upon the targeted temp for that room - le Lounge target (when in use) is 21C so the graph only draws between 20 and 22C making changes very obvious - in reality with Autologs TRV Plugin every room stays BANG ON target Temp - ok maybe .3 -.5C over target.

Then as I create each element to track I name it beginning with 1Min - or Main - or Long- and Mini - so there is no longer confusion over which element is performing which task.

The 24hr one I have running on a schedule.

Hopefully that avoids the mess I was in going forward.

Graphs are very important to me as they help me to determine patterns/norms, that allow me to create Triggers for out of expected behaviour and start Alerts and Alarms.

So thank you Dave for enabling me to do this, very much appreciated.

Mike

Posted on
Sat Oct 12, 2019 9:39 am
DaveL17 offline
User avatar
Posts: 6742
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Plotting a variable

Busta999 wrote:
Dave,

Thanks

I had a concerted effort to clean up the Charts and the engines.

I have learnt over time to do this in the absence of other guidance.

Created 4 Engines.

Long - Updates every 24hrs - tracks Battery consumption over time
1Min - Updates every 60 Seconds - tracks Motion around the house - Unlikely it will remain active for too long - determines trends.
Main - Tracks every 15 mins for 48 hours gives me graphing for activity around the house, temp etc for understanding how Heating and temperature are operating. This is the Main Engine.
Mini - Tracks every 15 mins for 3 hours - this allows trending data for Key rooms around the house - allows users to see that a room is trending warmer or cooler, the Graph tends to be very narrowly focused upon the targeted temp for that room - le Lounge target (when in use) is 21C so the graph only draws between 20 and 22C making changes very obvious - in reality with Autologs TRV Plugin every room stays BANG ON target Temp - ok maybe .3 -.5C over target.

Then as I create each element to track I name it beginning with 1Min - or Main - or Long- and Mini - so there is no longer confusion over which element is performing which task.

The 24hr one I have running on a schedule.

Hopefully that avoids the mess I was in going forward.

Graphs are very important to me as they help me to determine patterns/norms, that allow me to create Triggers for out of expected behaviour and start Alerts and Alarms.

So thank you Dave for enabling me to do this, very much appreciated.

Mike

Hopefully, this will work for you and there will be no additional trouble with the CSV Engines. Mine has been running stably for quite some time, but that doesn't mean that this will be the case for all installations (differences in Indigo versions, OSX versions, hardware, etc.)

I have learnt over time to do this in the absence of other guidance.

There is a plugin wiki (located here) that should help. If there are things that you think would be helpful to include there, please let me know. There hasn't been much call for that (I don't know how often people look at the wiki) so any feedback would be welcome.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Sat Oct 12, 2019 9:44 am
Busta999 offline
User avatar
Posts: 714
Joined: Mar 30, 2016
Location: Wales UK

Re: Plotting a variable

There is a plugin wiki (located here) that should help. If there are things that you think would be helpful to include there, please let me know. There hasn't been much call for that (I don't know how often people look at the wiki) so any feedback would be welcome.


Will do and return to you :-)

Again many thanks for your efforts here - very much appreciated.

Posted on
Sat Oct 12, 2019 9:47 am
DaveL17 offline
User avatar
Posts: 6742
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Plotting a variable

Thank you. Glad to hear that you find the plugin useful!

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Thu Oct 17, 2019 3:55 am
Busta999 offline
User avatar
Posts: 714
Joined: Mar 30, 2016
Location: Wales UK

Re: Plotting a variable

Dave,

Thanks, I had read all the docs before, most comprehensive.

The only real gap I see are best practice examples, naming conventions for engines/charts and items that are tracked.

Would have saved me a couple of times had I had advanced knowledge.

But I suspect that I am one of the people who use this a great deal and the audience does not warrant any additions to your documentation.

On a separate issue I am seeing MatPlotLib freeze indefinitely until after a restart of the Plugin.

I am running 7.4 and 0.8.26 of the plugin and 10.12.6 macOS.

I have setup a Restart MatPlotLib plugin every 4 hours so not an issue.

Looking at the logs I could not see any issue s with MatPlotlib not the normal complaints - just no updates to CSV files, some, but not all graphs were getting updated but with no further data.

If you want I can crank up the logs and information level and turn off the reset to get deeper into it, but with the 4 hour reset I have an easy work around.

Posted on
Thu Oct 17, 2019 6:07 am
DaveL17 offline
User avatar
Posts: 6742
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Plotting a variable

Busta999 wrote:
The only real gap I see are best practice examples, naming conventions for engines/charts and items that are tracked.

Would have saved me a couple of times had I had advanced knowledge.

But I suspect that I am one of the people who use this a great deal and the audience does not warrant any additions to your documentation.

I'm working on improving the documentation now, so I'll add some more information in there on best practices for CSV Engine devices. Thanks for the feedback.

On a separate issue I am seeing MatPlotLib freeze indefinitely until after a restart of the Plugin.

I am running 7.4 and 0.8.26 of the plugin and 10.12.6 macOS.

I have setup a Restart MatPlotLib plugin every 4 hours so not an issue.

Looking at the logs I could not see any issue s with MatPlotlib not the normal complaints - just no updates to CSV files, some, but not all graphs were getting updated but with no further data.

If you want I can crank up the logs and information level and turn off the reset to get deeper into it, but with the 4 hour reset I have an easy work around.

This one is a little trickier, I think. Matplotlib is not known for managing memory well (later versions are better I'm told) so the plugin tries to manage this in a couple of different ways. The plugin runs each chart device in its own process and--after charting--destroys the chart object and the process that it was running in. This causes the plugin to run a little bit more slowly than I would like because it has to establish this framework for each chart every time it's updated. Otherwise, I know the plugin will always eventually crash (for everyone).

Secondly, the plugin checks to see if things need to be refreshed every 15 seconds. If the entire amount of work that you're asking the plugin to do takes more than 15 seconds, the new tasks should be added to a queue that will process them in turn. However, if this is a constant condition, there may be a log jam developing that causes the plugin to never be able to catch up. I suspect that this may be what's happening for you. My install has been running without any problems for months at a time (I get a daily email of all errors logged on my production server, and don't get any that reference the plugin).

It would be helpful to see a debug log to see if there's anything there that might provide a clue. But if the plugin is running out of resources in the thread that Indigo assigns it to, I can't guarantee that it will.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Thu Oct 17, 2019 6:17 am
Busta999 offline
User avatar
Posts: 714
Joined: Mar 30, 2016
Location: Wales UK

Re: Plotting a variable

Dave

Ok, if I’m an edge case - I am absolutely fine auto resting the plugin every 4 hours - not an issue at all :-)

If I can help anyway let me know and I’ll get it done.

So if you want me to diag and record it no problem, but if I’m the only one - I’m mire than happy with my workaround.

I have no desire at all to increase your workload :-)


Sent from my iPhone using Tapatalk

Posted on
Thu Oct 17, 2019 9:29 am
DaveL17 offline
User avatar
Posts: 6742
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Plotting a variable

No worries at all about workload. I can't always guarantee that I can get something done right away, but it's always good to have insight into how the plugin is working for everyone.

If you can find some interesting log details that would help isolate your issue, please do share them.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Who is online

Users browsing this forum: No registered users and 3 guests