questions about Nest plugin, states in Grafana
Posted: Sun Jan 27, 2019 1:49 pm
I'm stumbling over some issues with Nest Thermostats and trying to display data from them using Grafana.
I realize my situation may be different from others - I personally have a few Nest thermostats and also have some other types of thermostats as well, and am often trying to use Grafana to graph their data in similar ways when possible.
First, I believe it is the case that Nest thermostats don't update the basic states from indigo.thermostat class like heatIsOn, coolIsOn, and fanIsOn. This makes it hard for me to do things like write a single query across multiple thermostats to plot state, or show the amount of runtime for heat/cool per hour or whatever. Seems like this should be fairly easy to fix (on the Nest thermostat plugin) and I will post an issue there. Anyone know any convenient workarounds or query magic I can use to paper over the discrepancy?
In trying to inspect and understand what is going on with this plugin and GHD's treatment of it, I'm poking around in Indigo.
I am looking at Nest thermostat devices in Indigo (looking at the device details window). This (if I understand correctly) lists the device's custom states in scrolling window at the bottom (which you don't see by default unless you scroll or stretch the bottom part of the Indigo window. It seems there is a daunting list of stuff there which comes from a) Nest is a fairly complex device with regular setpoints, eco set points, locked set point limits, and many other details. b) supports Fahrenheit and centigrade and has separate states for each. and c) seems to support decimal value and convenience rounded integer values for centigrade temperatures (presumably for user convenience when presenting information on a control page or similar) . d) there seem to be a mix of naming conventions, not always consistent - sometimes camelCase, sometimes_underscores, sometimesuniformcase, sometimes.dots, etc. No question that there are quite a lot of states to sort thru.
Currently, my Nest thermostats are in my list of included devices, so it seems that all of their states should be sent to InfluxDB. So I believe in the bottom part of the Indigo window I'm seeing the custom states for the device, meaning those states are specifically defined by the Nest Home plugin. Other "regular" (non-custom) device states (aka "properties"?) are shown generally either in the top of the window or in the case of some properties like lastUpdated, as a separate column in the device list view, or in a few cases (folderID) may not be viewable directly.
From an earlier inquiry on this subject, my understanding is that GHD creates field names from custom states by prepending "state." to the name to avoid possible collisions with properties of the same name. So if a I have a device which is an instance of a plugin that defines custom state "foo" (and I include this state via GHD's config), then I expect to see a corresponding field in InfluxDB called "state.foo". Properties of a device, on the other hand, are represented in InfluxDB with field names that match the property names (that is, without "state." prepended).
Also, it seems that when there is a state whose values are strings, but whose values can also be converted to a number (including strings that parse as numbers or can be interpreted as boolean values like "yes", "no" "true", "false", etc.), GHD creates an additional field with the same name with ".num" appended to the end. So for example, if I have a custom state called "isBar" and it takes on values "Yes" and "No", then GHD create a field with name "state.isBar" and an additional field with name "state.isBar.num" and in this field, it will store values converted to numerical values (presumably Yes->1.0, No->0.0).
Using Plugins>Grafana Home Dashboard>Explore Device, I'm poring over a bunch of information dumped to the logs, but still not quite able to understand what I'm seeing there. Some specific questions and issues:
As an aside... It could greatly simplify user comprehensibility of the "explore device..." dumps produced by GHD if a) a state were listed on a line with a simple annotation when the ".num" state is also included (this would cut the number of lines almost in half and greatly reduce unnecessary entropy in the output). b) the states were sorted by name, either simply by the actual field name or (bonus!) by the "root" name, so that for example fields state.humidity, humidifierIsOn, and state.humidityInput1 would come out adjacent in the sort. c) maybe either eliminate "special" fields/states from the output or flag them as such, since things like "name", "folder", "folderId" and friends are not things the user can really include/exclude via settings.
I realize my situation may be different from others - I personally have a few Nest thermostats and also have some other types of thermostats as well, and am often trying to use Grafana to graph their data in similar ways when possible.
First, I believe it is the case that Nest thermostats don't update the basic states from indigo.thermostat class like heatIsOn, coolIsOn, and fanIsOn. This makes it hard for me to do things like write a single query across multiple thermostats to plot state, or show the amount of runtime for heat/cool per hour or whatever. Seems like this should be fairly easy to fix (on the Nest thermostat plugin) and I will post an issue there. Anyone know any convenient workarounds or query magic I can use to paper over the discrepancy?
In trying to inspect and understand what is going on with this plugin and GHD's treatment of it, I'm poking around in Indigo.
I am looking at Nest thermostat devices in Indigo (looking at the device details window). This (if I understand correctly) lists the device's custom states in scrolling window at the bottom (which you don't see by default unless you scroll or stretch the bottom part of the Indigo window. It seems there is a daunting list of stuff there which comes from a) Nest is a fairly complex device with regular setpoints, eco set points, locked set point limits, and many other details. b) supports Fahrenheit and centigrade and has separate states for each. and c) seems to support decimal value and convenience rounded integer values for centigrade temperatures (presumably for user convenience when presenting information on a control page or similar) . d) there seem to be a mix of naming conventions, not always consistent - sometimes camelCase, sometimes_underscores, sometimesuniformcase, sometimes.dots, etc. No question that there are quite a lot of states to sort thru.
Currently, my Nest thermostats are in my list of included devices, so it seems that all of their states should be sent to InfluxDB. So I believe in the bottom part of the Indigo window I'm seeing the custom states for the device, meaning those states are specifically defined by the Nest Home plugin. Other "regular" (non-custom) device states (aka "properties"?) are shown generally either in the top of the window or in the case of some properties like lastUpdated, as a separate column in the device list view, or in a few cases (folderID) may not be viewable directly.
From an earlier inquiry on this subject, my understanding is that GHD creates field names from custom states by prepending "state." to the name to avoid possible collisions with properties of the same name. So if a I have a device which is an instance of a plugin that defines custom state "foo" (and I include this state via GHD's config), then I expect to see a corresponding field in InfluxDB called "state.foo". Properties of a device, on the other hand, are represented in InfluxDB with field names that match the property names (that is, without "state." prepended).
Also, it seems that when there is a state whose values are strings, but whose values can also be converted to a number (including strings that parse as numbers or can be interpreted as boolean values like "yes", "no" "true", "false", etc.), GHD creates an additional field with the same name with ".num" appended to the end. So for example, if I have a custom state called "isBar" and it takes on values "Yes" and "No", then GHD create a field with name "state.isBar" and an additional field with name "state.isBar.num" and in this field, it will store values converted to numerical values (presumably Yes->1.0, No->0.0).
Using Plugins>Grafana Home Dashboard>Explore Device, I'm poring over a bunch of information dumped to the logs, but still not quite able to understand what I'm seeing there. Some specific questions and issues:
- In Indigo, I can see custom state for my Nest thermostat devices: isheating =Yes. I don't see the corresponding fields I would expect to see (state.isheating and maybe state.isheating.num) listed in the json dump generated by GHD, nor does it exist in influxDB. Why is this?
- In the json dump, "name" and "folderId" appear. These I believe are tag keys (not fields) in influxDB. Also, I notice that "folder" does not appear - why, since it presumably has the same status?
- In the json dump, I notice both "folderId" and "folderId.num are present - is it expected/desired that these both exist? Is one a tag and one a field? Is there lost efficiency by having the tag ("folderId") be a string instead of a number - does influxDB even care about this?
- Basically same as previous question for "Id" and "Id.num".
As an aside... It could greatly simplify user comprehensibility of the "explore device..." dumps produced by GHD if a) a state were listed on a line with a simple annotation when the ".num" state is also included (this would cut the number of lines almost in half and greatly reduce unnecessary entropy in the output). b) the states were sorted by name, either simply by the actual field name or (bonus!) by the "root" name, so that for example fields state.humidity, humidifierIsOn, and state.humidityInput1 would come out adjacent in the sort. c) maybe either eliminate "special" fields/states from the output or flag them as such, since things like "name", "folder", "folderId" and friends are not things the user can really include/exclude via settings.