Device Usage Report IWS Plugin

Posted on
Fri Mar 05, 2010 8:43 pm
berkinet offline
User avatar
Posts: 2407
Joined: Nov 18, 2008
Location: Mougins, France & Berkeley, CA, USA

Re: Device Usage Report IWS Plugin

This can easily wait until Jay get back or someone gets a chance to hack at the code. Just in case there is a doubt there is a problem...

I have not been in my living room for 2 days. But the usage report for today shows...
Code: Select all
Living Room Cabinet Lamp       731
Living Room Chandeliers        731
Living Room Fireplace Mantel   731
Living Room Sconces (KPL)      731
Living Room Table Lamp         731
In fact that same number (731) appears for any device that has not been used today. Similarly, for yesterday, unused devices reported 1261.

I have taken a quick look at the code and my non-python eyes didn't see anything obvious. But, the problem could well be in the db.
Does anyone know a simple sql reporting tool to dump the data to a text file?

EDIT: I found /usr/bin/sqlite3

Posted on
Mon Mar 08, 2010 10:30 am
jay (support) offline
Site Admin
User avatar
Posts: 8881
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Device Usage Report IWS Plugin

OK, I'm back and I'll look at it today to see if I can spot the bug.

Jay (Indigo Support)
Twitter | Facebook

Posted on
Mon Mar 08, 2010 10:52 am
jay (support) offline
Site Admin
User avatar
Posts: 8881
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Device Usage Report IWS Plugin

OK, I did find something unusual in my SQLite db - I found a day where there were a ton of entries that were obviously not accurate - devices that I have that aren't even plugged in were showing up in the DB log. According to Matt, this sometimes happens when the server starts up or the DB switches - and it's going to be next to impossible for me to filter it out successfully. So, for days where you see devices represented in the list that you know weren't used I'm afraid you'll just need to skip that day - particularly if they have an unusual amount of time associated with it.

I'm going to look some more to make sure that there are no other logic errors in the plugin, but just be warned. Also, another user pointed out that they have a light that comes on very dim at night and only brightens when a motion sensor detects motion - because the device is on (dimmed) it's still counted because I don't attempt to use the brightness value for anything.

Just want to remind everyone that this tool is good for identifying patterns but certainly isn't meant to replace a dedicated energy monitoring system... ;)

Jay (Indigo Support)
Twitter | Facebook

Posted on
Tue Mar 09, 2010 1:18 pm
jay (support) offline
Site Admin
User avatar
Posts: 8881
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Device Usage Report IWS Plugin

Version 1.1 of the plugin is now available for download (just download it again from the file library and replace the old usageReporter folder). Another forum user, berkinet, is cutting his python teeth on the plugin and has made some significant enhancements and fixes:

  • Changed the calculation logic to fix a some errors
  • Added a filtering mechanism so you can filter out devices you don't want to see
  • Added the ability to export the data to a text file
  • Added a nice calendar popup date picker

A couple of notes:
  1. The problem with incorrect data on days when you restart the Indigo server may have been fixed - I'm not completely convinced it is, but in any case it should be much more accurate for those days. I'd still be a little dubious of those days - the ultimate solution is to do some more work on the SQL logger - which won't happen until 5.0 probably.
  2. We removed the assumption that lights are ON if the first command we see is an OFF - so if you turn on a light at 10pm and leave it on all night, it won't show as being on. We're kicking around a couple of ideas on how to address that so watch this thread for more information.

Enjoy!

Jay (Indigo Support)
Twitter | Facebook

Posted on
Tue Mar 09, 2010 10:33 pm
berkinet offline
User avatar
Posts: 2407
Joined: Nov 18, 2008
Location: Mougins, France & Berkeley, CA, USA

Re: Device Usage Report IWS Plugin

jay wrote:
    ...
  • We removed the assumption that lights are ON if the first command we see is an OFF - so if you turn on a light at 10pm and leave it on all night, it won't show as being on. We're kicking around a couple of ideas on how to address that so watch this thread for more information.

Kick, kick... Here is a quick, but useful, little AppleScript that should address the problem Jay noted. The script checks Indigo's internal status for every device and then updates the sqlite db with pseudo On records for each device that is on. The net effect is that the Usage Report will only be off by at most a minute for devices that were on at mid-night.

The script should be saved as an external AppleScript (I called it "what-is-on.scpt") and then run daily at 12:01AM* as the Action of an Indigo Time/Date Action.
Code: Select all
set devIsOnTable to {} as list

tell application "IndigoServer"
   repeat with currentDevice in devices
      tell currentDevice
         set dev_name to name of currentDevice
         set dev_dim to supports dimming of currentDevice
         set dev_state to on state of currentDevice
         
         if dev_state then
            if not dev_dim then
               set dev_value to "NULL"
            else
               set dev_value to brightness of currentDevice
            end if
            set dev_state to "True"
            set the end of devIsOnTable to {dev_name, dev_state, dev_value}
         end if
      end tell
   end repeat
end tell

repeat with dev_name in devIsOnTable
   set db to quoted form of "/Library/Application Support/Perceptive Automation/Indigo 4/IndigoSqlClient/sqlite_db"
   set s to "/usr/bin/sqlite3 " & db
   set s to s & "  \"INSERT INTO device_history_basic (dev_name, dev_state, dev_value) VALUES('"
   set s to s & item 1 of dev_name & "','" & item 2 of dev_name & "'," & item 3 of dev_name & ");\""
   do shell script s
   tell application "IndigoServer"
      log "Saved ON state of " & item 1 of dev_name using type "sql update"
   end tell
end repeat


*Minor Peeve: Why doesn't Indigo support, as an option, a 24 hour clock? IMHO, 00:01 is much clearer than 12:01AM

Posted on
Wed Mar 10, 2010 7:57 am
jay (support) offline
Site Admin
User avatar
Posts: 8881
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Device Usage Report IWS Plugin

berkinet wrote:
*Minor Peeve: Why doesn't Indigo support, as an option, a 24 hour clock? IMHO, 00:01 is much clearer than 12:01AM


Because nobody but programmers think 00:01 is clearer than 12:01AM... ;)

Jay (Indigo Support)
Twitter | Facebook

Posted on
Wed Mar 10, 2010 8:11 am
matt (support) offline
Site Admin
User avatar
Posts: 14024
Joined: Jan 27, 2003
Location: Texas

Re: Device Usage Report IWS Plugin

berkinet wrote:
*Minor Peeve: Why doesn't Indigo support, as an option, a 24 hour clock? IMHO, 00:01 is much clearer than 12:01AM

Actually if you change your Time format in the OS Language & Text System Preferences you can get 24 hour time. :-)

Image

Posted on
Wed Mar 10, 2010 10:20 am
eme jota ce offline
Posts: 331
Joined: Jul 09, 2009
Location: SW Florida

Re: Device Usage Report IWS Plugin

jay wrote:
Version 1.1 of the plugin is now available for download (just download it again from the file library and replace the old usageReporter folder). Another forum user, berkinet, is cutting his python teeth on the plugin and has made some significant enhancements and fixes:

  • ...
  • Added a filtering mechanism so you can filter out devices you don't want to see


Thanks Jay and Berkinet (and thanks for the 12:01am applescript,supra too!). This is a very cool plugin.

Regarding the filtering mechanism... where is that?

Is there a way to have the report generate the same list of devices every time regardless of which devices are used? That would make it easy to copy and paste i usage minutes into a spreadsheet. My reports list have 25 devices some days, but 60 on others, depending upon what was used in a day.

Posted on
Wed Mar 10, 2010 10:39 am
jay (support) offline
Site Admin
User avatar
Posts: 8881
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Device Usage Report IWS Plugin

eme jota ce wrote:
Regarding the filtering mechanism... where is that?


Check the ReadMe.txt that came with the latest version - it's described there.

eme jota ce wrote:
Is there a way to have the report generate the same list of devices every time regardless of which devices are used? That would make it easy to copy and paste i usage minutes into a spreadsheet. My reports list have 25 devices some days, but 60 on others, depending upon what was used in a day.


Not currently - only devices that were actually used show up.

Jay (Indigo Support)
Twitter | Facebook

Posted on
Wed Mar 10, 2010 11:15 am
berkinet offline
User avatar
Posts: 2407
Joined: Nov 18, 2008
Location: Mougins, France & Berkeley, CA, USA

Re: Device Usage Report IWS Plugin

eme jota ce wrote:
...Is there a way to have the report generate the same list of devices every time regardless of which devices are used?...

There would seem to be two approaches:
  1. Search the db for any device ever seen and then list the totals for each of those devices for the report date. The problem with this method is that the list of devices will tend to grow forever and disconnected or renamed devices will hang around.
  2. Read the list of devices from Indigo. This would limit the report to current devices only. That would be fine for "today." But, reports for previous days would not include devices that were no longer defined in Indigo.
It is also possible to take a daily snapshot of Indigo's device table and then save that snapshot into the db. But, that would require a fair amount of work for questionable additional value.

The main point is that regardless of how this problem is addressed, the reports for any two days could still be different - which means, if a user really wanted a fixed report format, it would be necessary for them to add some post-processing.

FWIW: I am inclined to go with the second idea.

Posted on
Wed Mar 10, 2010 4:46 pm
berkinet offline
User avatar
Posts: 2407
Joined: Nov 18, 2008
Location: Mougins, France & Berkeley, CA, USA

Re: Device Usage Report IWS Plugin

support wrote:
Actually if you change your Time format in the OS Language & Text System Preferences you can get 24 hour time. :-)
Yup, that was it. I had changed the format in the Date & Time Preferences, and that changed the menu bar clock. So, I didn't think to ALSO change Language & Text.

BTW,
Jay wrote:
Because nobody but programmers think 00:01 is clearer than 12:01AM...
... and just about anybody outside the US (and maybe Canada) who buys tickets, makes an appointment, asks when a movie starts... :P

Posted on
Wed Mar 10, 2010 5:19 pm
jay (support) offline
Site Admin
User avatar
Posts: 8881
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Device Usage Report IWS Plugin

berkinet wrote:
BTW,
Jay wrote:
Because nobody but programmers think 00:01 is clearer than 12:01AM...
... and just about anybody outside the US (and maybe Canada) who buys tickets, makes an appointment, asks when a movie starts... :P


WOW - people do all that between midnight and 1am?

Jay (Indigo Support)
Twitter | Facebook

Posted on
Fri Mar 12, 2010 1:20 pm
berkinet offline
User avatar
Posts: 2407
Joined: Nov 18, 2008
Location: Mougins, France & Berkeley, CA, USA

Re: Device Usage Report IWS Plugin

eme jota ce wrote:
...Is there a way to have the report generate the same list of devices every time regardless of which devices are used?...
and then I wrote:
There would seem to be two approaches:...

I have thought of another approach: Create a device file that would list the names of all devices to be included in the report. The user would be responsible for creating and maintaining this file, though it would probably be possible to have an AppleScript create the file and initially populate it with the names of every Indigo device. The Usage Reporter would then limit it's report to these devices and would report on these devices regardless of their usage.

A nice advantage/side-effect of having a device file is the ability to store meta-information, such as wattage, in the file. This would allow the report to show (and summarize) and possibly cost (in $, € or?)

Thoughts?

Posted on
Fri Mar 12, 2010 2:26 pm
jay (support) offline
Site Admin
User avatar
Posts: 8881
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Device Usage Report IWS Plugin

Seems like a good idea - but you should add a management page for devices, then each time the management page is hit, you can do another lookup of all devices in the DB, and if it sees a device that's not currently in the list, it can add it, with the appropriate hide/show button and other data. Eventually, this is where you could allow stuff like handling device renames, etc.

Jay (Indigo Support)
Twitter | Facebook

Posted on
Thu Mar 18, 2010 9:15 am
kpfriedberg offline

Re: Device Usage Report IWS Plugin

just installed today, i get an "undefined page requested' error after inputting my id/password and the following event log:

Connected to SmartHome PowerLinc USB E
PowerLinc address 0E.14.03, firmware version 2.13
PowerLinc standalone operation disabled
Indigo directly controlling automation logic
WebServer client authenticated
WebServer started on port 8176 -- digest authentication enabled
WebServer enabled browser access to plugin path "plugins/usageReporter/usageReporter/css"
WebServer enabled browser access to plugin path "plugins/usageReporter/usageReporter/images"
WebServer enabled browser access to plugin path "plugins/usageReporter/usageReporter/js"

Mar 18, 2010 11:08:39 AM
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Client authentication failed - bad XML received (63.71.3.45)
Client disconnected (63.71.3.45)
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Client authentication failed - bad XML received (63.71.3.45)
Client disconnected (63.71.3.45)
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Client authentication failed - bad XML received (63.71.3.45)
Client disconnected (63.71.3.45)
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.
Error XML Parse Error: syntax error
Error On character 0 of line number 1.

Who is online

Users browsing this forum: No registered users and 1 guest