Latest Version - 2.0.30 - Data Reporting & Exporting

Posted on
Sun Dec 06, 2015 9:12 am
Chameleon offline
Posts: 611
Joined: Oct 04, 2014

Latest Version - 2.0.30 - Data Reporting & Exporting

Hi all

One of the most frequently requested features for NEST Home has been the storage, analysis and export of Nest information. Of course all of the data is held in sqlLogger if you have that installed but for a lot of people the information is difficult to extract, export or report on unless you've got some background in SQL.

It's always been a limitation of NEST that Google only provides a maximum of 10 days of historical data back to users. To use the data you have to enter it manually into a spreadsheet and then perform your analysis there. If you forget to extract the information on time you'll end up with gaps in your information.

Finally I've added some data management features to NEST Home (all versions). These allow you to generate reports on devices directly to the indigo log, text file or export the information to a CSV file for use in most spreadsheets for follow up analysis. It allows you export all data or a date range for reporting or export using two options added to the NEST Home Menu Commands.


You can install these updates in the usual way.

What's new in version 2.0.30?

Two new fields have been added to the configuration screen 'Should NEST Home archive data changes?:' and Text File Directory.

Screen Shot 2015-12-06 at 14.35.35.png
Config dialog
Screen Shot 2015-12-06 at 14.35.35.png (105.27 KiB) Viewed 5673 times


You may have already filled in the Text File Directory if you've been using the other NEST Mapping Reports. If so you don't need to change this field. This is where all of your reports and CVS files will be saved for you. If you haven't completed this field yet it's VERY important that you enter a path to a folder you have access to. Check the spelling of the path name, the case and finally don't add a '/' on the end.

You should also ensure that the Should NEST Home archive data changes? box is checked. For some reason you'll find it more reliable if you uncheck and recheck the box before saving! I'll investigate why later.

Once you've updated the fields I recommend that you Save and then Reload the plugin to get data management working straight away. When you do this you'll get a message in the indigo log telling you that ** Data Management Online **

Now you'll be able to use the reporting and export options. Note that nothing will be reported at this stage because none of the NEST States have changed but soon you'll gain a rich resource of NEST information.

Export to CSV

If you select the Export NEST data to CSV for analysis option from the NEST Home Menu you'll be presented with the following dialog:

Screen Shot 2015-12-06 at 14.44.57.png
Export to CSV
Screen Shot 2015-12-06 at 14.44.57.png (110.25 KiB) Viewed 5673 times


The first option (Export all data) will get NEST Home to generate reports based on the whole dataset rather than a range. This could take a significant amount of time as your database gets larger and isn't recommended.

The next options allow you to define a date range for the export. The dates are inclusive and must be completed (dd/mm/yy - I'm a Brit so I've kept this format sorry :oops: ) If you want to have a single day, say 4th December, you would enter 04/12/15 for the start date and 04/12/15 as the end date.

Once you've selected your range you can then select the datasets you'd like to export to CSV. Experiment with them to see what's the most useful for you. CSV files can be read in a text editor or imported into most spreadsheets for your analysis.

The filenames will be similar to NEST Report (04-12-15 : 04-12-15) Ambient Temperature.csv

If you look at the indigo log it will tell you exactly where the files are stored and their names

Report to log or file

If you select the Report NEST data to CSV for analysis option from the NEST Home Menu you'll be presented with the following dialog:

Screen Shot 2015-12-06 at 14.55.11.png
Report dialog
Screen Shot 2015-12-06 at 14.55.11.png (122.2 KiB) Viewed 5673 times


The first option (Report all data to file) will get NEST Home to generate reports to text files rather than outputting to the indigo log. It's recommended that text file is the option you select. If left unchecked reports will be displayed in the indigo log

The next option (Report all data to log) will get NEST Home to generate reports on the whole dataset rather than a range. Clearly this could take sometime if there is a lot of data.

The next options allow you to define a date range for the export. The dates are inclusive and must be completed. If you want to have a single day, say 4th December, you would enter 04/12/15 for the start date and 04/12/15 as the end date.

Once you've selected your range you can then select the datasets you'd like to report on. Experiment with them to see what's the most useful for you. There's some basic analysis (hours heating, cooling or off for example) that are generated as part of the files.

The filenames will be similar to NEST Report (04-12-15 : 04-12-15) Ambient Temperature.txt

If you look at the indigo log it will tell you exactly where the files are stored and their names

This functionality is currently classed as Beta and I'm going to add more features in the coming days. However, it would be useful for me to get any reports of issues, bugs or changes so I can improve the software.

BTW - There's going to be an unexpected plugin release for an unusual device in the next day or so. I'll let you know when it's ready for you to play with :D

Best regards and thanks for your support as usual!

Mike

Posted on
Sun Dec 13, 2015 12:00 am
jgb offline
Posts: 20
Joined: Dec 29, 2013

Re: Latest Version - 2.0.30 - Data Reporting & Exporting

Mike, I've just installed 2 nests 3.0 (replaced venstar/insteon adapter thermostats which I had running on schedules, triggers, variable changes & an apple script). The nests are working in their simplest form with the plugin but I get the following error message on every action:

NEST Home 3 sent "Den Thermostat" increase heat setpoint to 71.0
NEST Home 3 Error Error in plugin execution ServerReplacedElem:

Traceback (most recent call last):
File "plugin.py", line 2083, in deviceUpdated
File "plugin.py", line 2215, in dataChange
File "plugin.py", line 2358, in nestFileWrite
<type 'exceptions.NameError'>: global name 'db' is not defined

Also, I'm not exactly sure how the python script requirement enters in? I'm familiar with the apple script that was used in my venstar setup.

Thanks,
Greg

Posted on
Sun Dec 13, 2015 3:17 pm
Chameleon offline
Posts: 611
Joined: Oct 04, 2014

Re: Latest Version - 2.0.30 - Data Reporting & Exporting

jgb wrote:
Mike, I've just installed 2 nests 3.0 (replaced venstar/insteon adapter thermostats which I had running on schedules, triggers, variable changes & an apple script). The nests are working in their simplest form with the plugin but I get the following error message on every action:

NEST Home 3 sent "Den Thermostat" increase heat setpoint to 71.0
NEST Home 3 Error Error in plugin execution ServerReplacedElem:

Traceback (most recent call last):
File "plugin.py", line 2083, in deviceUpdated
File "plugin.py", line 2215, in dataChange
File "plugin.py", line 2358, in nestFileWrite
<type 'exceptions.NameError'>: global name 'db' is not defined

Also, I'm not exactly sure how the python script requirement enters in? I'm familiar with the apple script that was used in my venstar setup.

Thanks,
Greg


Don't worry - leave it with me :D I'll be back in the morning!

Regards

Mike

Posted on
Mon Dec 14, 2015 7:21 am
Chameleon offline
Posts: 611
Joined: Oct 04, 2014

Re: Latest Version - 2.0.30 - Data Reporting & Exporting

Chameleon wrote:
jgb wrote:
Mike, I've just installed 2 nests 3.0 (replaced venstar/insteon adapter thermostats which I had running on schedules, triggers, variable changes & an apple script). The nests are working in their simplest form with the plugin but I get the following error message on every action:

NEST Home 3 sent "Den Thermostat" increase heat setpoint to 71.0
NEST Home 3 Error Error in plugin execution ServerReplacedElem:

Traceback (most recent call last):
File "plugin.py", line 2083, in deviceUpdated
File "plugin.py", line 2215, in dataChange
File "plugin.py", line 2358, in nestFileWrite
<type 'exceptions.NameError'>: global name 'db' is not defined

Also, I'm not exactly sure how the python script requirement enters in? I'm familiar with the apple script that was used in my venstar setup.

Thanks,
Greg


Don't worry - leave it with me :D I'll be back in the morning!

Regards

Mike


Greg

Sorry you're not the only one to get this error but I've released a new version of NEST Home that should resolve it.

When you use the NEST Home devices from the NEST Home Plugin your Mac is running a python script in the background to manage changes to that device and provide other features that are available. This is important because Indigo 6 doesn't have the NEST Thermostat or Protect as a native device so I create a custom one in the python script that's running and manage it for you. This should all be transparent as far as you're concerned - unless of course the developer makes an error in the plugin logic that affects you. Sorry about that :oops:

The plugin provides historical data tracking and reporting if required by the user and this is set using an option in the configuration dialog (Plugins/NEST Home 3/Configure) that can either be on or off. If you want to use tracking and/or reporting you also have to provide a text folder where it can save the results. For example in my system I create a folder called /Users/Michael/NESTMap that I use to store this information. I suspect that you had switched on the archive option (perhaps by accident - it's easy to do) but not entered a folder that the plugin could locate in the Text Directory field. As a result, when it tried to write information to a tracking log it couldn't and hence you got this error. Of course, my plugin should recover more gracefully than it did in your case!

The latest update looks to see if this problem exists. If it does it will then inform you that it can't track information and disable the option in the configuration dialog. If you want to use the reporting or tracking features of the plugin you'll need to enter a valid folder for text files and recheck the archive option mentioned above. If not, you can safely ignore the message and you shouldn't see it again.

You can find the latest version (2.0.32) under the latest version post at the top of the forum on this link:http://forums.indigodomo.com/viewtopic.php?f=170&t=15262

Regards


Mike

Posted on
Mon Dec 14, 2015 2:05 pm
jgb offline
Posts: 20
Joined: Dec 29, 2013

Re: Latest Version - 2.0.30 - Data Reporting & Exporting

MIke,

Thanks a million. I think what I had done originally was create the text file folder after I configured the plugin. So I reconfigured (unchecked the box) all that and created the folder first. Now, no problems with the error messages and this before I loaded the new plugin.

Just to be clear; there is nothing I need to do with python? No installing it on my mac? No configuring? No having it run in the background? No script writing?

Appreciate you and your plugin. You're exactly right, the nest are great in and of themselves, but having them interface with indigo is terrific.

Your quick response was great!
Greg

Posted on
Tue Dec 15, 2015 6:15 am
Chameleon offline
Posts: 611
Joined: Oct 04, 2014

Re: Latest Version - 2.0.30 - Data Reporting & Exporting

jgb wrote:
MIke,

Thanks a million. I think what I had done originally was create the text file folder after I configured the plugin. So I reconfigured (unchecked the box) all that and created the folder first. Now, no problems with the error messages and this before I loaded the new plugin.

Just to be clear; there is nothing I need to do with python? No installing it on my mac? No configuring? No having it run in the background? No script writing?

Appreciate you and your plugin. You're exactly right, the nest are great in and of themselves, but having them interface with indigo is terrific.

Your quick response was great!
Greg


Hi Greg

Python 2.6 is part of the standard Mac install so there's nothing that you need to do. The plugin clearly works (now anyway) so you're good to go. So you've nothing to set up in python (unlike another plugin iFindStuff that I've released for managing geofences). The advantage of the plugin is that you don't need any scripts, background tasks or anything else to run NEST Home... and

You get access to NEST event initiated triggers such as device setpoints, ambient temperature, home & away, etc... as well as NEST specific actions such as set it to Home or Away or all of the general thermostat actions to manage setpoints and mode... I'm particularity proud of the way that once you've set up your account it creates NEST devices automatically in Indigo for both Thermostats and Protects which makes it easy to maintain.

You should also look at the reporting options you'll find under Plugins/NEST Home 3. These provide you with reports and tracking of states for a much longer time than the current Google 10 day limit. I'm currently adding graphing capabilities using indigoPlot which should be released in the next few days so keep an eye on the forum.

If you fancy a more whimsical plug in then have a look at the Indigo Santa Tracker I released last week. You'll find details here: http://forums.indigodomo.com/viewtopic.php?f=181&t=15201. It uses a direct feed from NORAD. It requires no configuration or installations of other python stuff or plugins and runs independently. It's a bit of fun to show what indigo can do and I'll be using it later in the year as a beginners guide to python and writing plugins!

BTW - I've upgraded the plugin to version 2.032 to avoid problems like yours in the future but there's no need to upgrade your install at the moment - wait for new features.

Glad you like it and I appreciate the kind words.

Regards

Mike

Posted on
Tue Dec 15, 2015 8:48 am
roussell offline
User avatar
Posts: 1108
Joined: Aug 18, 2008
Location: Alabama

Re: Latest Version - 2.0.30 - Data Reporting & Exporting

Maybe it's just me but I get a 404 from Dropbox when trying to download from any of the links in your first post. Interested in your plugin and Nest to replace my current Insteon stats so would like to give it a try.

Thanks!
Terry

Posted on
Tue Dec 15, 2015 1:06 pm
Chameleon offline
Posts: 611
Joined: Oct 04, 2014

Re: Latest Version - 2.0.30 - Data Reporting & Exporting

roussell wrote:
Maybe it's just me but I get a 404 from Dropbox when trying to download from any of the links in your first post. Interested in your plugin and Nest to replace my current Insteon stats so would like to give it a try.

Thanks!
Terry


Hi Terry

You were right - I don't know what dropbox was doing they were working yesterday!

I've updated the links and tested them.

Let me know if you need any help. Don't forget to download the manual as a starting place.

Regards and good luck

Mike

Page 1 of 1

Who is online

Users browsing this forum: No registered users and 2 guests