Page 6 of 7

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Tue May 26, 2020 7:38 pm
by kw123
posted V7.20.71

added a check for psql command string: if it is too short or does not contain "psql" it will be set (w warning) to :
/Applications/Postgres.app/Contents/Versions/latest/bin/psql indigo_history -U postgres

This could remove some of the sql import issues. If the postgres statement does not contain a psql statement.. it will likely not work . :o

if your implementation works right now, no change for you.

No functional change in this release

Karl

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Thu Jul 30, 2020 3:38 pm
by welchgregusa
This is awesome—thank you for developing and sharing it.

I'm so close to making it work, but I'm running into an issue. (macOS 10.15.6, Indigo 7.4.1, plugin 7.20.71).

I had the plugin configured for simple and then some more complex plots, and then started seeing "Error in plugin execution runConcurrentThread" errors as in https://forums.indigodomo.com/viewtopic.php?f=165&t=20037. Per that thread, I unloaded the plugin, removed the prefs, re-loaded, and re-configured.

Now, when I enable the plugin, in the Indigo log I'm seeing:

Enabling plugin "INDIGOplotD 7.20.71"
Starting plugin "INDIGOplotD 7.20.71" (pid 28345)
INDIGOplotD initializing ...
INDIGOplotD path To files: =================
INDIGOplotD indigo /Library/Application Support/Perceptive Automation/
INDIGOplotD installFolder /Library/Application Support/Perceptive Automation/Indigo 7.4/
INDIGOplotD plugin.py /Users/home/Documents/Perceptive Automation/Indigo 7.4/Plugins/INDIGOplotD.indigoPlugin/Contents/Server Plugin/
INDIGOplotD Plugin params /Library/Application Support/Perceptive Automation/Indigo 7.4/Preferences/Plugins/com.karlwachs.INDIGOplotD/
INDIGOplotD (testing logger; see >/Library/Application Support/Perceptive Automation/Indigo 7.4/Logs/com.karlwachs.INDIGOplotD/plugin.log< for detailed logging
INDIGOplotD Error !!!!INFO ONLY!!!! logger enabled for ERROR !!!!INFO ONLY!!!!
INDIGOplotD Critical Error !!!!INFO ONLY!!!! logger enabled for CRITICAL !!!!INFO ONLY!!!!
INDIGOplotD check /Library/Application Support/Perceptive Automation/Indigo 7.4/Logs/com.karlwachs.INDIGOplotD/plugin.log <<<< for detailed logging
INDIGOplotD Plugin short Name INDIGOplotD
INDIGOplotD my PID 28345
INDIGOplotD set params 4 indigo V 7.4
Started plugin "INDIGOplotD 7.20.71"
INDIGOplotD initializing ... ; debuglevel=[]
INDIGOplotD number of days per bin category are: [8, 38, 390] for [days, hours, minutes] data
INDIGOplotD using '/usr/bin/python2.7' for utily programs
INDIGOplotD testing if variable enableTimeTracking_INDIGOplotD is == on/off/print-option to enable/end/print time tracking of all functions and methods (option:'',calls,cumtime,pcalls,time)
INDIGOplotD SQLMode: batch2Days; GNUPLOT/MATPLOT: mat; GNUplotVersion= ; PLOT-Directory= /Library/Application Support/Perceptive Automation/Indigo 7.4/Preferences/Plugins/com.karlwachs.INDIGOplotD/
INDIGOplotD command: reLoad data from SQL for last 2 days started
INDIGOplotD checking py-restore files


Then in /Library/Application Support/Perceptive Automation/Indigo 7.4/Logs/com.karlwachs.INDIGOplotD/plugin.log I am seeing:

30 09:13:19 !!!!INFO ONLY!!!! logger enabled for THREADDEBUG !!!!INFO ONLY!!!!
30 09:13:19 !!!!INFO ONLY!!!! logger enabled for DEBUG !!!!INFO ONLY!!!!
!!!!INFO ONLY!!!! logger enabled for INFO !!!!INFO ONLY!!!!
09:13:19 !!!!INFO ONLY!!!! logger enabled for WARNING !!!!INFO ONLY!!!!
2020-07-30 09:13:19.808 ERROR Plugin.__init__ !!!!INFO ONLY!!!! logger enabled for ERROR !!!!INFO ONLY!!!!
2020-07-30 09:13:19.808 CRITICAL Plugin.__init__ !!!!INFO ONLY!!!! logger enabled for CRITICAL !!!!INFO ONLY!!!!
09:13:19 myLogSet setting parameters -- logFileActive= ; logFile= /Library/Application Support/Perceptive Automation/Indigo 7.4/Preferences/Plugins/com.karlwachs.INDIGOplotD/plugin.log; debugLevel= []
09:13:19 initializing ... ; debuglevel=[]
09:13:19 number of days per bin category are: [8, 38, 390] for [days, hours, minutes] data
09:13:19 using '/usr/bin/python2.7' for utily programs
09:13:19 gnuplot is not installed
09:13:19 SQLMode: batch; GNUPLOT/MATPLOT: mat; GNUplotVersion= ; PLOT-Directory= /Library/Application Support/Perceptive Automation/Indigo 7.4/Preferences/Plugins/com.karlwachs.INDIGOplotD/
09:13:20 dataversion 2 dataOffsetInTimeDataNumbers:5
read file /Library/Application Support/Perceptive Automation/Indigo 7.4/Preferences/Plugins/com.karlwachs.INDIGOplotD/data/minute.dat lines: 1728 ok
read file /Library/Application Support/Perceptive Automation/Indigo 7.4/Preferences/Plugins/com.karlwachs.INDIGOplotD/data/hour.dat lines: 864 ok
read file /Library/Application Support/Perceptive Automation/Indigo 7.4/Preferences/Plugins/com.karlwachs.INDIGOplotD/data/day.dat lines: 388 ok
09:13:20 command: reLoad SQL started
09:13:20 sql logger used 906.9 secs CPU so far
indigoMPplot INDIGO matplot started Version 6.8; pid:5193; at: 2020-07-30 09:13:21.689554
09:13:25 checking py-restore files
09:13:25 initializing ...2
09:13:27 Reading SQL files,filtering,converting
09:13:27 Elapsed time Device----- State name------------------------------- records read/imported/ LastID/ F size--Rej: ValueRng/ Number/ TimeSeq/timeWind/ Data/ col#
09:13:27 0:00:00.110708 Multisensor, EXT-11, CR Win 4, Luminance-sensorValue-average 240/ 240/ 1200/ 6270-rejec: 0/ 0/ 0/ 0/ 0/ 9
09:13:27 0:00:00.218827 Multisensor, EXT-07, CR Door, Luminance-sensorValue-average 329/ 324/ 1351/ 8719-rejec: 5/ 0/ 0/ 0/ 0/ 5
09:13:28 0:00:00.330514 Multisensor, EXT-09, CR Win 2, Luminance-sensorValue-average 357/ 357/ 993/ 9057-rejec: 0/ 0/ 0/ 0/ 0/ 7
09:13:28 0:00:00.442041 Multisensor, EXT-08, CR Win 1, Luminance-sensorValue-average 363/ 358/ 1402/ 9669-rejec: 5/ 0/ 0/ 0/ 0/ 6
09:13:28 0:00:00.552294 Multisensor, EXT-10, CR Win 3, Luminance-sensorValue-average 425/ 425/ 1449/ 11171-rejec: 0/ 0/ 0/ 0/ 0/ 8
09:13:28 0:00:00.666471 Multisensor, EXT-06, FR Win, Luminance-sensorValue-average 470/ 470/ 1510/ 12399-rejec: 0/ 0/ 0/ 0/ 0/ 4
09:13:28 0:00:00.778519 Multisensor, EXT-05, FR Door, Luminance-sensorValue-average 470/ 470/ 1499/ 12478-rejec: 0/ 0/ 0/ 0/ 0/ 3
09:13:28 0:00:00.890734 Multisensor, EXT-03, LR Door, Luminance-sensorValue-average 474/ 474/ 1768/ 12523-rejec: 0/ 0/ 0/ 0/ 0/ 2
09:13:28 0:00:01.001046 Multisensor, EXT-04, KIT Win, Luminance-sensorValue-average 478/ 478/ 1778/ 12640-rejec: 0/ 0/ 0/ 0/ 0/ 1
09:13:28 SQL imp. ... finished, total number of records read/imported: 3606/ 3596
09:14:47 main loop stopped
30 09:15:06 !!!!INFO ONLY!!!! logger enabled for THREADDEBUG !!!!INFO ONLY!!!!
30 09:15:06 !!!!INFO ONLY!!!! logger enabled for DEBUG !!!!INFO ONLY!!!!
!!!!INFO ONLY!!!! logger enabled for INFO !!!!INFO ONLY!!!!
09:15:06 !!!!INFO ONLY!!!! logger enabled for WARNING !!!!INFO ONLY!!!!


which repeats.

Any ideas what might be messed up?

Thanks.

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Thu Jul 30, 2020 10:24 pm
by kw123
Will check tomorrow

If you could, switch on all debugging in config and run one more time


Sent from my iPhone using Tapatalk

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Fri Jul 31, 2020 6:20 am
by welchgregusa
Great—thank you.

Strange—this morning I did not notice any errors in the Indigo log (since last night), and the plots appear to be updating....

However, when I reload (or unload/load) the plugin I still see the initial ERROR and CRITICAL ERROR, which concerned me at first, but now I see the message says "INFO ONLY" so perhaps it's fine?

Anyway...I enabled all of the debugging (although the "special" flag did not seem to persist through reloads), unloaded, reloaded, and put the log file here:

https://www.dropbox.com/s/m3whnyobgoiuqli/plugin_welch_20200731_081100.log?dl=0

The latest unload/load was right around/at 08:11:00.

Please let me know if there is anything else I can do to help, if indeed you do see an issue. (I'm not even sure now there is an issue....)

I can't thank you enough for making this plugin available. For the longest time I had been writing data to tab-delimited files, copying, importing, etc. This is clean and beautiful.

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Fri Jul 31, 2020 6:31 am
by kw123
The initial messages “error ... info only “. just show which debug levels are enabled. And which do not show - most of them show in plugin. Log
The indigo log only shows warnings and higher.

[edit] looking the logfile, all looks fine



Sent from my iPhone using Tapatalk

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Tue Aug 04, 2020 7:12 am
by welchgregusa
Thank you. I've now got something more "serious" in that I seem unable to add new data sources. I had added several, then when I went to add another, it seems to work fine (I was careful w/ data entry, and clicking CONFIRM), when I go back to add yet another the previous one is not there. From that point on I tried adding secret data sources (different devices) and each time it appeared to work fine at first, but then when I checked the source was not actually listed. I tried unloading the plugin and deleting all of the preferences (folder, etc.) to stat fresh, but once again after adding my first/single data source, I was unable to add any more.

I've tried turning on debugging in the plugin, and watching the log file, and I do see something interesting:

08:59:58 myLogSet setting parameters -- logFileActive= ; logFile= /Library/Application Support/Perceptive Automation/Indigo 7.4/Preferences/Plugins/com.karlwachs.INDIGOplotD/plugin.log; debugLevel= []
09:01:18 Reading SQL files,filtering,converting
09:01:18 Elapsed time Device----- State name------------------------------- records read/imported/ LastID/ F size--Rej: ValueRng/ Number/ TimeSeq/timeWind/ Data/ col#
09:01:43 buttonConfirmDeviceStatesCALLBACK: devNo 2 not found in DEVICE
09:01:43 buttonConfirmDeviceStatesCALLBACK: devices defined= defined DEVICE# are:1 - Weather, Orientation, CR Door; 0 - None;
09:01:43 buttonConfirmDeviceStatesCALLBACK: make sure you CONFIRM select a DEVICE first


So the first device (Weather, Orientation, CR Door) appears to be OK, but something's up w/ the second one I added (Weather, Orientation, KIT-FR). The first (present) and second (missing) are Cynical Weather orientation devices. I've tried adding dimmers and other "normal" (not plugin) devices, but I can't seem to add anything beyond the first one.

Thank you in advance for any ideas.

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Tue Aug 04, 2020 7:50 am
by kw123
could you post how you select the second device / state?

are you taking about adding several states of a single device or new device/state to be added?

Karl

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Tue Aug 04, 2020 2:58 pm
by welchgregusa
I'm trying to add multiple devices, each with single states. This is in Indigo 7.4.1 Pro, macOS 10.15.6, on a Mac Mini (2018).

Here are the steps I'm using.

    1. Plugins -> INDIGOplotD -> Define / select data sources, needed in PLOTS

    2. With "Expert Mode ON / OFF" checked, under section 1.1, I'm selecting ">>>> Pick new Device/Variable" from the "Select existing or new Device/Variable..." popup. (I see only the one/first device that I added, and then the ">>>> Pick new..." choice.)

    3. From the "Pick new Device/Variable" popup I'm selecting the next similar device to the first one—an Orientation device from the Cynical Weather plugin. (I've tried other "normal" devices such as dimmers too.)

    4. I'm clicking "CONFIRM" in the button immediately below ("After selection click CONFIRM"). Section 1.2 ("Select States of .... to be tracked.") then becomes enabled.

    5. I am choosing the "index" state from {"index," "height," "azimuth"}.

    6. I am setting the "Ignore if ... LESS than" and "... GREATER than" limits to 0 and 100 respectively (the index is normalized between 0 and 100).

    7. I give it a short nick-Name, e.g., "SI_KIT_FR" (for the sun index for the kitchen and family room).

    8. I scroll to the bottom of the window and click the "CONFIRM" button ("After selecting properties click...... CONFIRM"). Section 1.2 becomes grayed out.

    9. Before I even click the "Close" button, if I go back to the top and try and repeat step 2 above, the new device I added (in steps 3–8) is not listed. As in step 2, I see only the one/first device that I added, and then the ">>>> Pick new..." choice. The new one is not there.

    10. I click the "Close" button.

    11. I repeat steps 1 and 2 above, and again in the "Select existing or new Device/Variable..." popup I see only the one/first device that I added, and then the ">>>> Pick new..." choice.
I hope that helps....

Thank you.

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Tue Aug 04, 2020 3:09 pm
by kw123
can you do these steps agin w debug (2) and (3) on (in config)?
and post the plugin.log output

Karl

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Tue Aug 04, 2020 3:34 pm
by welchgregusa
I did as you asked—see attached for the settings. Does it bother you that the "How should data be retrieve...." field appears unset? I set it, click Save and when I come back later, the "How should data be retrieve...." field again appears unset. Is that normal?

After that things get strange.

I already had a tail -f running for the plugin log file, and when I looked at that terminal window (before repeating the steps as you asked) I saw

home:com.karlwachs.INDIGOplotD home$ tail -f plugin.log
08:56:21 theCol:1; updatesALL:True; sqlDynamic:batch-resetTo-batch2Days; sqlHistListStatus:[0, 50]
08:56:21 Convert no records found, try later 1
08:56:21 Convert no records found, try later 2
08:56:21 redolineDataSource called from : cleanData
08:56:21 redolineDataSource: listOfSelectedDataColumnsAndDevPropName[(0, 'None'), (1, u'SI_CR_Door')] calledfrom :cleanData
08:56:21 clearSqlData sqlupdates: [0, 0]
08:56:21 SQL imp. 2020-08-04 08:56:21.161700 ... sqlColListStatus: [0, 0]
2020-08-04 08:56:22.926 ERROR Plugin.checkSQLdata Line '9284' has error='None'
08:56:22 matplot stopped PID: 47774
08:56:22 main loop stopped


Note that the log files is in /Users/home/Documents/Perceptive Automation/Indigo 7.4/Logs/com.karlwachs.INDIGOplotD/plugin.log (under my Documents folder) because I have Indigo running from there, so my Indigo files are backed up by Backblaze.

Anyway, with the tail -f still going, I repeated the steps as you asked, but I never saw anything new written to plugin.log, and the device was not added. I tried several times.

I decided to try and reload the plugin, and in the Indigo log I saw

Reloading plugin "INDIGOplotD 7.20.71"
Stopping plugin "INDIGOplotD 7.20.71" (pid 48028)
Stopped plugin "INDIGOplotD 7.20.71"
Starting plugin "INDIGOplotD 7.20.71" (pid 73932)
INDIGOplotD initializing ...
INDIGOplotD path To files: =================
INDIGOplotD indigo /Library/Application Support/Perceptive Automation/
INDIGOplotD installFolder /Library/Application Support/Perceptive Automation/Indigo 7.4/
INDIGOplotD plugin.py /Users/home/Documents/Perceptive Automation/Indigo 7.4/Plugins/INDIGOplotD.indigoPlugin/Contents/Server Plugin/
INDIGOplotD Plugin params /Library/Application Support/Perceptive Automation/Indigo 7.4/Preferences/Plugins/com.karlwachs.INDIGOplotD/
INDIGOplotD (testing logger; see >/Library/Application Support/Perceptive Automation/Indigo 7.4/Logs/com.karlwachs.INDIGOplotD/plugin.log< for detailed logging
INDIGOplotD Error !!!!INFO ONLY!!!! logger enabled for ERROR !!!!INFO ONLY!!!!
INDIGOplotD Critical Error !!!!INFO ONLY!!!! logger enabled for CRITICAL !!!!INFO ONLY!!!!
INDIGOplotD check /Library/Application Support/Perceptive Automation/Indigo 7.4/Logs/com.karlwachs.INDIGOplotD/plugin.log <<<< for detailed logging
INDIGOplotD Plugin short Name INDIGOplotD
INDIGOplotD my PID 73932
INDIGOplotD set params 4 indigo V 7.4
Started plugin "INDIGOplotD 7.20.71"
INDIGOplotD initializing ... ; debuglevel=['General', 'SQL']
INDIGOplotD number of days per bin category are: [8, 38, 390] for [days, hours, minutes] data
INDIGOplotD using '/usr/bin/python2.7' for utily programs
INDIGOplotD testing if variable enableTimeTracking_INDIGOplotD is == on/off/print-option to enable/end/print time tracking of all functions and methods (option:'',calls,cumtime,pcalls,time)
INDIGOplotD SQLMode: ; GNUPLOT/MATPLOT: mat; GNUplotVersion= ; PLOT-Directory= /Library/Application Support/Perceptive Automation/Indigo 7.4/Preferences/Plugins/com.karlwachs.INDIGOplotD/
INDIGOplotD Error self.timeDataNumbers wrong number of columns: 2/1trying to fix
INDIGOplotD Error self.timeDataNumbers wrong number of columns: 2/2trying to fix
INDIGOplotD checking py-restore files


Note the "wrong number of columns" Error messages near the end.

After that I was surprised that I was then able to add four devices using the steps I posted previously, and then...the fifth did not work, nor the next, etc. I was unable to add any more.

I still never saw any changes written to plugin.log — the last entry is the "main loop stopped" — which is from BEFORE I was able to successfully add the four devices.

I can try and wipe everything out and start over if you think best—in fact I tried to do that but it didn't seem to help. If you think that's worth trying again, please let me know which things I need to delete, steps to take, etc. to completely start over, and I will try.

Thank you for the help!

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Thu Aug 06, 2020 3:43 pm
by kw123
I would need to look at it directly, If I could.
would a screen sharing session work for you?

Karl

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Thu Aug 06, 2020 3:56 pm
by welchgregusa
I would be happy to do/host screen sharing session(s). Suggest we transition to private messages and then later update the public thread, unless you think otherwise. Thank you.

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Thu Aug 06, 2020 4:03 pm
by kw123
Send me email at.
Karlwachs me com


Sent from my iPhone using Tapatalk

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Fri Aug 07, 2020 9:22 am
by kw123
posted v 7.20.72

now supports device states w name "index"

Indigoplotd retrieves data from the indigo sql database as input for data in plots.

"index" has a special meaning in a DB, hence "select index from ..." does not work

sqllite wants [index] and postgres "index"

In addition it seems that indigo uses the states names as lower case for the column names.

That is ignored when just using select xxx from... but not with select "xxx" from ...

This version will take care of that using: (select [xxx.lower()] and select "xxx.lower()"
(not proper sql syntax, but explains what it does.)


Karl

Re: INDIGOplotD -- a plot tool for indigo -- program downlo

PostPosted: Wed May 18, 2022 9:11 am
by kw123
posted v 7.21.73

fixed settings for indigo 2022.1
programs still run under python2.7, using the indigo install.

Karl

ps

working on moving to py3,
there are some environment setting missing. everything compiles under py3 and is running within a terminal,
When running from with plugin under py3 it does not find some libs.