INDIGOplotD -- a plot tool for indigo

[richo]

Hi kw123,

I have started to use your Plot plugin and I'm really enjoying the initial results. I'm still struggling with all the settings but finally have managed to build the charts not only from the SQL database but also from external files. Below are a few examples.

I'm also working on a tool that will show the health of a z-wave mesh. In that attached chart i.ex. is the result of device pinging.
I'd like to recreate the map of a network and for that reason I would need to create a scattered chart. Is there any possibility to do it with your Plugin?

Keep doing a great work.
Ryszard

[kw123]

The default is a timeline. xvy. You could also do polar coordinates. The last one is a file where the first col is x and n other columns y that can be set as points or lines. You could use that. Let me think about some other options.


Sent from my iPhone using Tapatalk

[richo]

Unfortunatelly there are different numbers of neighbors for each device so I can't use typical Y columns. Scattered chart with 2 axis would be optimal.


Sent using Tapatalk

[kw123]

what would you like to plot?

dots / lines?

I see your graph is x=device# and y=ping time I guess

What would you like to add?

[kw123]

you could try making one plot transparent and overlay onto another
use "background = Transparent" in plot

you then only see the "dots" and lines. Then set the x, y tics to blank and make the text color same as color of other plot.

it should look something like the attached..

[jtodd]

https://www.dropbox.com/s/qhgv51zhkc13c ... 5.zip?dl=0 v 3-7-25 not officially released..

Karl -
Great! I'll give it a try. BUT: this unpacks to be 3-7-24 - did the right file get added to the .zip?

JT

[kw123]

new version released with all the time shift options and one fix for MATPLOT

http://forums.indigodomo.com/viewtopic. ... 811#p90811

[jtodd]

https://www.dropbox.com/s/qhgv51zhkc13c ... 5.zip?dl=0 v 3-7-25 not officially released..

JT-- you can now:

shift the whole plot window by x days
shift a single line by x days
set continuos mode to have the right y axis = now instead of shifting at midnight

Excellent! It works with the few graphs I've built on version 3-7-26. Much easier to see what's going on with energy use, and I'll move it to temperature, etc. Thanks!

Now... next questions (there is never a feature that doesn't require a feature, right?):

1) Could it be possible for Line B in the single line definition to be time-shifted in the same fashion? This would allow me to offset a line that shows the delta (positive or negative) in real numbers between yesterday's usage at noon, and today's usage at noon for example. As it stands with the new version, I can see that there are gaps between the lines indicating that they are different, but it's still some squinting to figure out how much the difference is.

2) This is a bit more difficult, and may just be in the "you're asking for too much" category... How about date ranges for lines and line B options? In other words, I'd like to create a line that is not just yesterday's data, but represents the average (or max, or min) values of a sample over N months. To make this more concrete, I'll use an example that is a 1 day plot and combine some lines and features:

- Line 1 shows my energy usage for today
- Line 2 shows my energy usage from yesterday
- Line 3 contains the AVERAGE energy usage for every day from the last 3 months
- Line 4 contains the MAXIMUM energy usage (smoothed) used at that point in time for any day in the last 3 months
- Line 5 contains the MINIMUM energy usage (smoothed) used at that point in time for any day in the last 3 months
- Line 6 (offset, using the right Y axis which goes from 90-110 but text shows -10, -5, 0, 5, 10) shows the positive or negative percentage of energy usage of today compared to the AVERAGE energy usage for the past 3 months (which is A/B and then multiply by 100)

Seems like this would require two new functions to be applied to a line or a "B" line:
range function: max, avg, min
range dates: [number of days the user wants to go back in history]



So if you're feeling REALLY adventurous, instead of just a pulldown menu of numbers, the range dates could get more sophisticated to allow lots more flexibility. I usually want to see comparisons that are more useful, so comparing my energy graph of a Monday versus a Sunday is always going to be skewed and is less useful. What I'd like to look at is today's data versus what a typical Monday is. Or maybe I want to exclude a few dates like where I'm spending all day welding in the garage so it doesn't bash my "normal" maximums. Or maybe I'm looking to compare my weekly running cost for energy with the same comparable few days from last week...

Possible syntax for range dates:

Code: Select all

  7 = go back one week and select every day
  7:1 = go back one week, but exclude yesterday (6 days of selection)
  30|sat = every Saturday for the last 30 days
  30|sat&30|sun:mon = every Saturday, Sunday, and Monday for the last 30 days
  30&!mon = every day for the last month but exclude Mondays
  30|DAYOFWEEK = special string to denote whatever day it is when the query is performed (sat, sun, mon, etc) and select every day of that type in the last month 
  60&!MONTH = every day in the last 60 days that isn't in this month (useful for cost calculations)
  30&90:60 =the last thirty days, plus the month that was three months ago, skipping the month that was two months ago
  365&!jul = last year, excluding July
  20150101:20150119 = fixed date range from the 1st to the 19th of Jan, 2015
  20140101:20141231|sat = every saturday for all of 2014
  20140101:20141231|!sun = every day in 2014 except for Sundays
  20140101:20141231|sat&!20140705 = every Saturday in 2014 except for July 5
  20140101:20141231&!20140601:20140631 = every day in 2014 except for the entire month of June 2014
  13:7|sun:DAYOFWEEK = every day in the last week up until the same weekday (useful for same-interval cost comparisons)

Enough for one evening...

[kw123]

JT,
that will just take 1 hour..

some comments:

currently it just collects 5 minute binned data for the last 8 days. If we want to do what you propose for the last 3 months... it will create a LOOOOOT of data.
some + and - / with shifted data-line B should be possible..

Let me think about is a little bit and see what we can do

I believe the data averages... summations etc should go into the data collection part not into the display part. Putting TOOO much math into the plot part will make it more complex or we need to generate too much general data to draw from -- if we don t know what we want to plot we need to prepare all kinds of data .
We need to preselect what should be tracked. Then we can do formulas that deliver lines that can be plotted..

the idea about specific weekdays make sense, not just yesterday or 5 days ago..

but thanks for your excitement, makes me feel good about the plugin.

Karl

[jtodd]

JT,
that will just take 1 hour..

"Just a simple matter of programming."

currently it just collects 5 minute binned data for the last 8 days. If we want to do what you propose for the last 3 months... it will create a LOOOOOT of data.
some + and - / with shifted data-line B should be possible..

Yep, some quick back of the envelope numbers get into the multi-gigabyte database size. On the other hand, half-terabyte drives are now something like $40USD, I think, and even SSD drives are getting pretty darn cheap.

Let me think about is a little bit and see what we can do

I believe the data averages... summations etc should go into the data collection part not into the display part. Putting TOOO much math into the plot part will make it more complex or we need to generate too much general data to draw from -- if we don t know what we want to plot we need to prepare all kinds of data .
We need to preselect what should be tracked. Then we can do formulas that deliver lines that can be plotted..

I think that's reasonable, but I don't see a way around keeping a lot of data.

This is a perennial problem with graphing or data analysis. I work in the DevOps space, and this has been a problem for a long time for large scale systems. Maybe take a look at some of these apps for some ideas - lots of people have hit this problem before. I considered just writing a tiny plugin for Indigo that fed data from all possible values in Indigo (variables, devices, etc.) into collectd and hence to graphite/whisper and then using graphite (and rearview and/or grafana on top of it) to graph. But that would have meant I maintained it. ugh.

http://graphite.wikidot.com/whisper or http://graphite.readthedocs.org/en/latest/whisper.html
http://graphite.readthedocs.org/en/late ... tions.html - interesting math on series!
http://graphite.readthedocs.org/en/latest/#
http://graphite.readthedocs.org/en/latest/tools.html
https://github.com/livingsocial/rearview - derived series construction

JT

[richo]

you could try making one plot transparent and overlay onto another
use "background = Transparent" in plot

you then only see the "dots" and lines. Then set the x, y tics to blank and make the text color same as color of other plot.

it should look something like the attached..

Thanks for the suggestion however I want to create a chart that would look like in the attached image. On both X and Y axis are node numbers that are "seen" by each of the device's. I'm afraid that the proposed method wouldn't work for so many series...
I have also attached an example csv file with raw data.

[kw123]

How about the size of the dot at the xy point that is prop to the number in the cell?


Sent from my iPhone using Tapatalk

[kw123]

richo: something like this.. (used some random data)

we could change size, color or symbol depending on "z" data

input file would be
x;y;z
x;y;z
...

[richo]

Looking closer to what I'm interested in.

In that case size of a dot isn't important as I'm just looking at the crossing of device ID's. In general however it's a good idea.

If I understand you correctly I.ex for device with ID=3 and neighbore nodes 12, 14, 34, 55
and device ID=5 and neighbore nodes 3, 14, 22 the input file would look like that:

3, 12
3, 14
3, 34
3, 55
5,3
5,14
5,22

Is it correct ?

[kw123]

yes, you can do this now. then use "DOT." or "DOTo" as line type