INDIGOplotD -- a plot tool for indigo

A Plugin to create Plots from INDIGO data and other sources
CrazyFin
Posts: 381
Joined: Thu Jan 08, 2015 4:09 pm
Location: Stockholm, SWEDEN

Re: INDIGOplotD -- a plot tool for indigo

Post by CrazyFin »

you would expect that the KWH does not go up fast from 500KWH!! if it would your electricity bill would be enormous!! I would expect to see an increase of ~ 10-20 a day (=10.5-21 SEK a DAY = 300-600/Month) which would hardly be visible. YOU NEED THE DELTA BETWEEN END and BEGINNING, that is your consumption.
Ok, I think you misunderstood me? It is not the jumps in kWh I need to see quickly, the energy graph (kWh) is fine to update slower.

In the wintertime I usually have approx 2000-4000 kWh usage per month and in the summer time around 900kWh. Lets assume that I use 3000 kWh in January, that is approx 96 kWh per day = 96 * 1.05 SEK = 100.80 SEK per day. On top of that I have the fixed monthly cost of 465 SEK so I tried to use the Offset = 465 to start the month at 465 SEK so that the plot would start every month on 465 SEK level and then keep on adding up the energy cost (approx 100 SEK per day in the wintertime on to the plot. I have also set the right scale for the cost to start at 465 SEK and to go up to max 3000 SEK.

The question was if the 465 offset would also be multiplied by the number I have set as a multiplier (if anything else than 1).

Do I also really need to use the delta between the end and beginning since I have configured the plugin to do a reset every month for the energy consumption? (as my screenshot in my previous post shows)

In order to see the ups and down use Watt or KW (=power=Volt*Amps)
Yes fully understand and I have a separate plot for that on the minute graph showing power for the total as well the power for each of the three clamps. It is this graph I would like to be able to update more often than once per minute but no matter how I try to set lower than 60 seconds on the SQL batch import (all data) it always reverts back to 60 seconds. (Screenshots in my prev post.)
(I will of course have to update the URL that points to the graph more often but that was easily done in the control page creator in Indigo)

The current cost mode does not have an offset, could add that, here a quick way:
use eConsumption and e-cost = 1.05 flat and pick your reset periods. i.e. day week month
That will give you the delta between the value at the end - value at the beginning of the reset period, i.e. how much consumption in a period (day, week, month, year or free period .. lets leave that for the next step)
THEN in the plot menu add 465 offset to it. I believe that should do it.
Yes this is exactly how I have set up as you can see in my screenshots in my previous post.
as for the multiplier and offset: there are 2 place you can do it:
in the config menu it is
value = (value +offset) * multiple.
Ah! So this answers my question from above! :-) Then it is not good to use this method since my accumulated kWh value for the month, for example 50kWh, would be added with 465 and then multiplied with the multiplier (in this case 1) giving a total number of 465 kWh when then would be multiplied with my energy cost of 1.05 SEK/kWh. The result would be a number that is way too high... :-)

in the plot/line menu it is
value = value*mult + offset
you can use both together or independently
first comes the device must/offset . that stores it into the files indigoplotd/data/day.dat hour.dat .. minute.dat
then in plot it reads these values and applies the factor offset to it while plotting.
Ok I´ll play around with both methods described and see what effects I get.

It looks like the graph is now updating (i.e. importing new SQL data properly) since this morning the graph was showing data for the night. Cool! :lol:
autolog
Posts: 4077
Joined: Tue Sep 10, 2013 3:07 am
Location: West Sussex, UK
Contact:

Re: INDIGOplotD -- a plot tool for indigo

Post by autolog »

kw123 wrote:aotolog:

could you try in a terminal:

/Library/PostgreSQL/bin/psql indigo_history postgres -t -A -F " " -c "SELECT id,ts,anotherproperty from device_history_1697069696 WHERE id > 1 LIMIT 2 ;"


==> output:
2 2015-01-12 17:01:16.600928


replace anotherProperty with e.g. onOffState of temperatureInput1 ... depending on your device
and replace 1697069696 with your device ID ..

this assumes that the db is indigo_history in the indigo log directory if not we need to add the path to it.


if we get this done, its a small change to make it work..

karl
Hi Karl,
Somewhat belatedly (due to the fact that I had retired for the evening) and probably not needed now :wink: , here is the result:

Code: Select all

/Library/PostgreSQL/bin/psql indigo_history postgres -t -A -F " " -c "SELECT id,ts,temperature from device_history_503785618 WHERE id > 1 LIMIT 2 ;"
2 2014-03-08 17:10:05.712869 18.7
3 2014-03-08 17:10:05.720922 18.7
I guess you have already progressed beyond this using your laptop :)
User avatar
kw123
Posts: 8705
Joined: Sun May 12, 2013 4:44 pm
Location: Dallas, TX
Contact:

Re: INDIGOplotD -- a plot tool for indigo

Post by kw123 »

I guess that seems to work.. I should have the code ready later today .. need to test the menu input section between sqlite and psql and how to enter the "command string"
/Library/PostgreSQL/bin/psql indigo_history postgres -t -A -F " " -c easily by the user..
User avatar
kw123
Posts: 8705
Joined: Sun May 12, 2013 4:44 pm
Location: Dallas, TX
Contact:

Re: INDIGOplotD -- a plot tool for indigo

Post by kw123 »

CrazyFin,

I suggest you do NOT reset the "acumEnergy every month. It is easier to do this afterwards in the software. If you don't reset you could get energy per year etc. or per billing period...

INDIGOplotD tries to be smart and detect resets, but sometime at least with my aeonlab device the accumEnergy has a number that is less than the number before. I put in 10% tolerance for that, but if it finds frequent reset, I am not sure if it is smart enough.

Karl
CrazyFin
Posts: 381
Joined: Thu Jan 08, 2015 4:09 pm
Location: Stockholm, SWEDEN

Re: INDIGOplotD -- a plot tool for indigo

Post by CrazyFin »

Ok I think you misunderstood me again or am I misunderstanding you? :oops:
I am NOT talking about resetting the accumulated kWh-number in my Aeon Labs HEM G2 meter, I am talking about the reset setting in IndigoPlotD when configuring the plugin.

Look at the settings in this screenshot for line number 3, are these ok or is it this reset you mean that I should NOT use:
IndigoPlotD - Plotting setting for Energy consumption COST.tiff
IndigoPlotD - Plotting setting for Energy consumption COST.tiff (174.73 KiB) Viewed 4746 times

Then suddenly in your post you mention that I should do like this:
"YOU NEED THE DELTA BETWEEN END and BEGINNING, that is your consumption.
The current cost mode does not have an offset, could add that, here a quick way:
use eConsumption and e-cost = 1.05 flat and pick your reset periods. i.e. day week month
That will give you the delta between the value at the end - value at the beginning of the reset period, i.e. how much consumption in a period (day, week, month, year or free period .. lets leave that for the next step)"
So here you say that I should use a reset period? :?
As you can see above in my screenshot I have chosen Reset period each month.

Look here on my 2nd screenshot for the electricity consumption cost settings. Are these ok? So this means that I have set my energy cost is 1.05 SEK per kWh.
IndigoPlotD - Plotting setting for Energy consumption COST - 2.tiff
IndigoPlotD - Plotting setting for Energy consumption COST - 2.tiff (253 KiB) Viewed 4746 times
Then I use these curve plot settings for the actual device to plot accumulated kWh for the month on a minute basis for the last 2 days and to plot the accumulated monthly energy consumption cost on the right axle starting with low right Y scale at 400 SEK and I also use an offset of 456 SEK which is my "admin cost" to my grid operator.
IndigoPlotD - Plotting setting for curves - 1.tiff
IndigoPlotD - Plotting setting for curves - 1.tiff (251.98 KiB) Viewed 4746 times
IndigoPlotD - Plotting setting for curves - 2.tiff
IndigoPlotD - Plotting setting for curves - 2.tiff (214.15 KiB) Viewed 4746 times
So lets say my accumulated kWh for the month is 220 kWh (which I see in Indigo GUI and as you can see in the chart below) and using a cost of 1.05 SEK/kWh + offset of 456 SEK I should get a cost graph that starts at 456 SEK and increases over the month with 1.05 SEK per used kWh up to 456 SEK + 220 * 1.05 = 687 SEK. However my graph is just a straight line at 456 SEK:
Accumulated kWh-minute-S1.png
Accumulated kWh-minute-S1.png (39.37 KiB) Viewed 4746 times
User avatar
kw123
Posts: 8705
Joined: Sun May 12, 2013 4:44 pm
Location: Dallas, TX
Contact:

Re: INDIGOplotD -- a plot tool for indigo

Post by kw123 »

sorry to have misunderstood you:

use Consumption NOT Average, see first screen shot (2 different examples: eConsumption +reset weekly , cost=1 and reset daily cost defined below in the cost section)

then set reset each day/week/month . for the data you like to generate (the plots show reset every day)
attached an example for the minute and day plot for #of days =2 and #of days=5 respectively

In my example I have a dynamic cost situation where the cost varies during time of day and weekday (yellow curve), blue histogram is KWH, green is Watt

hope that helps

karl
Attachments
Screen Shot 2015-01-13 at 4.11.37 PM.png
Screen Shot 2015-01-13 at 4.11.37 PM.png (56.86 KiB) Viewed 4742 times
test3-minute-S1.png
test3-minute-S1.png (45.9 KiB) Viewed 4742 times
test3-hour-S1.png
test3-hour-S1.png (55.77 KiB) Viewed 4742 times
CrazyFin
Posts: 381
Joined: Thu Jan 08, 2015 4:09 pm
Location: Stockholm, SWEDEN

Re: INDIGOplotD -- a plot tool for indigo

Post by CrazyFin »

Aaaah now it gets more and more clearer! :D

You wrote:
use Consumption NOT Average
So if I just want to plot a kWh graph (not cost graph) that resets daily (or whatever reset period I choose), I use the setting "Reset each day, cost = 1"?

And for the cost plots I choose "Reset each day" if I want to plot these both in the same graph window?

I really like your graph examples above! Exactly what I am looking for. Wonderful! :lol:
autolog
Posts: 4077
Joined: Tue Sep 10, 2013 3:07 am
Location: West Sussex, UK
Contact:

Re: INDIGOplotD -- a plot tool for indigo

Post by autolog »

Hi Karl,
I have downloaded and installed v3.7.20.

I have setup a plot for a thermostat and am getting an error as follows:

Code: Select all

  INDIGOplotD                      clearSqlData resetting sql import for Thermostat-10-temperature- Col#=1 will retry ... error message is:psql: warning: extra command-line argument " SELECT id, ts, temperature from device_history_503785618 WHERE  ID > 0 LIMIT 10000000;" ignored

  INDIGOplotD                      clearSqlData resetting sql import for Thermostat-10-temperatureRemote- Col#=2 will retry ... error message is:psql: warning: extra command-line argument " SELECT id, ts, temperatureRemote from device_history_503785618 WHERE  ID > 0 LIMIT 10000000;" ignored

  INDIGOplotD                      clearSqlData resetting sql import for Thermostat-10-heatSetpoint- Col#=3 will retry ... error message is:psql: warning: extra command-line argument " SELECT id, ts, heatSetpoint from device_history_503785618 WHERE  ID > 0 LIMIT 10000000;" ignored
Could be something I am doing because I launched straight into the setup without reading the PDF (It's late evening here) :)

I will look at it again tomorrow (probably afternoon)

In case it helps, the sqlcmd.sh file looks like this (userid masked out):

Code: Select all

cd '/Library/Application Support/Perceptive Automation/Indigo 6/Logs' 
date +"%H:%M:%S          Thermostat-10-temperature" >>/Users/<USERID>/Documents/INDIGOplotD/sql/sqlcmd.log 

/Library/PostgreSQL/bin/psql indigo_history postgres -t -A -F ' ' " SELECT id, ts, temperature from device_history_503785618 WHERE  ID > 0 LIMIT 10000000;"  2>/Users/<USERID>/Documents/INDIGOplotD/sql/503785618-temperature.error | awk 'NF>2 && !/data unavailable/ {print}'  > /Users/<USERID>/Documents/INDIGOplotD/sql/503785618-temperature.sqlOut

/usr/bin/python2.7  '/Library/Application Support/Perceptive Automation/Indigo 6/Plugins/INDIGOplotD.indigoPlugin/Contents/Server Plugin/fixSQLoutput.py'  /Users/<USERID>/Documents/INDIGOplotD/sql/ 503785618-temperature.sqlout 503785618-temperature sqlFix

ls -l -T  /Users/<USERID>/Documents/INDIGOplotD/sql/503785618-temperature | awk '{print $8 " "  $5 "      "  $10}' >>/Users/<USERID>/Documents/INDIGOplotD/sql/sqlcmd.log 
echo finished > /Users/<USERID>/Documents/INDIGOplotD/sql/503785618-temperature.done 
date +"%H:%M:%S          Thermostat-10-temperatureRemote" >>/Users/<USERID>/Documents/INDIGOplotD/sql/sqlcmd.log 

/Library/PostgreSQL/bin/psql indigo_history postgres -t -A -F ' ' " SELECT id, ts, temperatureRemote from device_history_503785618 WHERE  ID > 0 LIMIT 10000000;"  2>/Users/<USERID>/Documents/INDIGOplotD/sql/503785618-temperatureRemote.error | awk 'NF>2 && !/data unavailable/ {print}'  > /Users/<USERID>/Documents/INDIGOplotD/sql/503785618-temperatureRemote.sqlOut

/usr/bin/python2.7  '/Library/Application Support/Perceptive Automation/Indigo 6/Plugins/INDIGOplotD.indigoPlugin/Contents/Server Plugin/fixSQLoutput.py'  /Users/<USERID>/Documents/INDIGOplotD/sql/ 503785618-temperatureRemote.sqlout 503785618-temperatureRemote sqlFix

ls -l -T  /Users/<USERID>/Documents/INDIGOplotD/sql/503785618-temperatureRemote | awk '{print $8 " "  $5 "      "  $10}' >>/Users/<USERID>/Documents/INDIGOplotD/sql/sqlcmd.log 
echo finished > /Users/<USERID>/Documents/INDIGOplotD/sql/503785618-temperatureRemote.done 
date +"%H:%M:%S          Thermostat-10-heatSetpoint" >>/Users/<USERID>/Documents/INDIGOplotD/sql/sqlcmd.log 

/Library/PostgreSQL/bin/psql indigo_history postgres -t -A -F ' ' " SELECT id, ts, heatSetpoint from device_history_503785618 WHERE  ID > 0 LIMIT 10000000;"  2>/Users/<USERID>/Documents/INDIGOplotD/sql/503785618-heatSetpoint.error | awk 'NF>2 && !/data unavailable/ {print}'  > /Users/<USERID>/Documents/INDIGOplotD/sql/503785618-heatSetpoint.sqlOut

/usr/bin/python2.7  '/Library/Application Support/Perceptive Automation/Indigo 6/Plugins/INDIGOplotD.indigoPlugin/Contents/Server Plugin/fixSQLoutput.py'  /Users/<USERID>/Documents/INDIGOplotD/sql/ 503785618-heatSetpoint.sqlout 503785618-heatSetpoint sqlFix

ls -l -T  /Users/<USERID>/Documents/INDIGOplotD/sql/503785618-heatSetpoint | awk '{print $8 " "  $5 "      "  $10}' >>/Users/<USERID>/Documents/INDIGOplotD/sql/sqlcmd.log 
echo finished > /Users/<USERID>/Documents/INDIGOplotD/sql/503785618-heatSetpoint.done 
echo finished > /Users/<USERID>/Documents/INDIGOplotD/sql/sqlcmd.done 
Thanks again for doing the change to enable PostgreSQL :)
User avatar
kw123
Posts: 8705
Joined: Sun May 12, 2013 4:44 pm
Location: Dallas, TX
Contact:

Re: INDIGOplotD -- a plot tool for indigo

Post by kw123 »

autolog, "-c" was missing the psql statement.

fixed here:
http://forums.indigodomo.com/viewtopic. ... 431#p90431


could you try again.

Karl
autolog
Posts: 4077
Joined: Tue Sep 10, 2013 3:07 am
Location: West Sussex, UK
Contact:

Re: INDIGOplotD -- a plot tool for indigo

Post by autolog »

I am not getting any errors now but blank plot pages :?

However, the good news is that the data files have been created with data extracted from psql and the three data files are sitting in the sql folder.

So, next step is for me to read the pdf manual and set up the plots correctly :wink:

That will definitely be tomorrow as I am now closing down for this evening :)

I'll let you know how I get on.

Thanks again for fixing this - it's much appreciated. :)
User avatar
kw123
Posts: 8705
Joined: Sun May 12, 2013 4:44 pm
Location: Dallas, TX
Contact:

Re: INDIGOplotD -- a plot tool for indigo

Post by kw123 »

Try miniplot for your first plot. It should take you 1 minute to create your first plot.


Sent from my iPhone using Tapatalk
palantir
Posts: 73
Joined: Wed Sep 24, 2014 4:28 am

Re: INDIGOplotD -- a plot tool for indigo

Post by palantir »

Karl -

I have successfully changed from SQL Logger to Postgresql, yesterday. To my positive surprise, IndigoPlotD retained the logging data that was still sitting on the SQL Logger database (I guess IndigoPlotD keeps that in its own cache) and continued plotting my graphs without any interrupt in data. I neither had to make any changes to any of my (variable or device) configurations, nor to any of the plot definition.

Attached is a sample plot showing a graph for my living room for the last two days. The switch to Postgresql was made on Tuesday night. As you can see, data is displayed without interruptions.

Thanks for this amazing plugin! In addition to showing cool graphs, the plots have helped me tremendously with identifying coding errors in my own heating logic (ie "that graph looks weird, why is that - let me review my code - ah, I get it, what was I thinking! let's fix that!!!" :) ). Again, these were my own coding errors. IndigoPlotD was just plotting what was actually happening!

Ronald
Attachments
Screen Shot 2015-01-14 at 14.50.41.png
Screen Shot 2015-01-14 at 14.50.41.png (61.13 KiB) Viewed 4664 times
autolog
Posts: 4077
Joined: Tue Sep 10, 2013 3:07 am
Location: West Sussex, UK
Contact:

Re: INDIGOplotD -- a plot tool for indigo

Post by autolog »

Still getting blank plots :cry:

In trying to track down what the issue is, I looked at what I think is the source datafile e.g.:

Code: Select all

525749084-setpointHeat
525749084-temperatureInput1
Both these files only have one entry per day - maybe that is correct?

Also, when I started IndigoPlotD, I get these messages:

Code: Select all

  INDIGOplotD                     sql importfile file not created yet, resetting import parameters /Users/<USERID>/Documents/INDIGOplotD/sql/525749084-temperatureInput1
  INDIGOplotD                     sql importfile file not created yet, resetting import parameters /Users/<USERID>/Documents/INDIGOplotD/sql/525749084-setpointHeat
  INDIGOplotD                     sql commands launched 2015-01-14 17:17:48.643542  waiting for SQL tasks to end to read data into INDIGOplotD pid=25047
  INDIGOplotD                     Reading SQL files,filtering,convting Time Stamp   Device----- State name-------------------------------  records read/imported/  LastID/  F size--Reject: ValueRng/  Number/ TimeStp/   2Days/   col#
  INDIGOplotD                     Converted                         0:00:00.048650 Stella-Z10-setpointHeat-average-                                 309/       1/    9861/   10452--rejects:       0/       0/       0/       0/     2
  INDIGOplotD                     Converted                         0:00:00.101419 Stella-Z10-temperatureInput1-average-                            309/       1/    9861/   11079--rejects:       0/       0/       0/       0/     1
  INDIGOplotD                     SQL import            2015-01-14 17:17:49.809839 ... finished, total number of records imported:                    2
  INDIGOplotD                      printing data to formatted file
  INDIGOplotD                     indigoCommand: redoParameters
MiniPlot doesn't produce anything either.

If I use Valentina Studio to query the table for the device history and extract temperatureInput1 - it is finding 9,848 records :?

I tried a complete re-install but IndigoPlotD seems to remember some previous plots. I deleted the Documents/IndigoPlotD folder (and removed it from trash), disabled the plugin, deleted it and then restarted the server. Is there anything else I should delete to get to a clean slate?

I am sure it is something I am doing wrong - I am quite keen to get this working if possible :)
User avatar
kw123
Posts: 8705
Joined: Sun May 12, 2013 4:44 pm
Location: Dallas, TX
Contact:

Re: INDIGOplotD -- a plot tool for indigo

Post by kw123 »

the logfile states that essentially it found just one entry per device/state

could you email karl wachs at me the following:

the log file output of
- print/log device
- print/log plot
- the file when you do " print formatted ... to file" in documents/indigoplotd/data ...formatted. there should be 3 of those files.

Karl
User avatar
kw123
Posts: 8705
Joined: Sun May 12, 2013 4:44 pm
Location: Dallas, TX
Contact:

Re: INDIGOplotD -- a plot tool for indigo

Post by kw123 »

Autolog,
this version should fix the PSQL issue:
http://forums.indigodomo.com/viewtopic. ... 565#p90565
Post Reply

Return to “INDIGOplotD”