Extending "valuesDict" in callback functions

Posted on
Fri Jul 10, 2020 6:00 pm
hannt offline
User avatar
Posts: 81
Joined: Jul 08, 2011
Location: TN USA

Extending "valuesDict" in callback functions

I'm feeling like a real newbie right now so I'm hoping somebody can provide some clarification and/or direction.

As I understand it, valuesDict is generally used in various plugin callback methods to validate and possibly change UI values. It's very valuable in that changes can be made in various callbacks (menu change, button press, validation method, etc.) but the values only get written permanently as properties if the user does not cancel the configuration dialog. This is probably an oversimplification but hopefully works for this discussion.

Anyway, I'm working on a custom device plugin that gets values from the UI, and they are converted to device properties if the user does not cancel the configuration dialog. Pretty straight forward so far. However, in a button callback method, I'd like to create some 'properties' that are not part of the UI. These properties would also get permanently saved as device properties if the user does not cancel the configuration dialog. They would get discarded if the user cancels.

I though I could use valuesDict for this purpose, however, it doesn't seem to always work. For example, in the button callback method, I added a simple string to valuesDict that did not correspond to any UI element. As I hoped, it would get converted to a property when the configuration dialog was closed, but it was forgotten if the dialog was canceled. Yay.

The problem I ran into was trying to crate a more complicated variable type and save the value in valuesDict. Specifically, I want to create a list where each item in the list is a dictionary. When I create the list and add it to valuesDict, then return valuesDict, the list does not appear to actually get added. At first I thought this might a mutable vs. immutable issue, but I tried various workarounds with no avail.

First question: Can valuesDict be used in this way? Stated otherwise, can I randomly insert values of various types in valuesDict that would eventually get saved as device properties? I'm assuming no, but wanted to confirm.

Now here's where the newbie part comes in. I'd like to be able to maintain this list in a variable until the user either confirms or cancels the configuration dialog. At that point I can either write the variable as a device properly or just discard it if the user cancels, mimicking valuesDict functionality. I can't figure out where to create the variable so that it can be referenced by the various callback methods.

I thought about defining this in the plugin startup method, like I have with other global variables, but this variable is specific to the device instance, not the plugin.

Thoughts? Advice? Thanks!

Posted on
Sat Jul 11, 2020 6:06 am
FlyingDiver offline
User avatar
Posts: 4303
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Extending "valuesDict" in callback functions

Post the code that you tried (but didn't work).

I suspect the problem you're running into is that Indigo can only store certain device types in a device property, and a list of dicts is too complicated for that method. One solution would be to convert the complex object to a JSON string, and store that. The convert it back when you read that property.

Now here's where the newbie part comes in. I'd like to be able to maintain this list in a variable until the user either confirms or cancels the configuration dialog. At that point I can either write the variable as a device properly or just discard it if the user cancels, mimicking valuesDict functionality. I can't figure out where to create the variable so that it can be referenced by the various callback methods.


All callback methods are part of the plugin object. So you make it a property (Python term) of the Plugin object.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Sat Jul 11, 2020 8:17 am
hannt offline
User avatar
Posts: 81
Joined: Jul 08, 2011
Location: TN USA

Re: Extending "valuesDict" in callback functions

FlyingDiver wrote:
Post the code that you tried (but didn't work).

This is one variation of a button pressed callback method:

Code: Select all
def savePlotButtonPressed(self, valuesDict, typeId, devId):
      if 'Plots' in valuesDict.keys():
         tempPlots = valuesDict['Plots'])
      else:
         tempPlots = []
      tempPlots.append({
               'PlotDeviceID':valuesDict['PlotDeviceIdUI'],
               'PlotDeviceStateKey':valuesDict['PlotDeviceStateKeyUI'],
               'LegendLabel':valuesDict['LegendLabelUI'],
               'LineWidth':valuesDict['LineWidthUI'],
               'LineColor':valuesDict['LineColorUI'],
               'LineAlpha':valuesDict['LineAlphaUI'],
               'LineStyle':valuesDict['LineStyleUI'],
               'LineMarker':valuesDict['LineMarkerUI']
            })
      valuesDict['Plots'] = tempPlots
      #Also tried this:
      #valuesDict['Plots'] = list(tempPlots)
      return valuesDict
In troubleshooting, I could see that valuesDict['Plots'] contained the correct values right before returning from the method, but it didn't seem to get saved after the return.

FlyingDiver wrote:
I suspect the problem you're running into is that Indigo can only store certain device types in a device property, and a list of dicts is too complicated for that method.

Exactly. That's what I was thinking. Looking for a confirmation here.

FlyingDiver wrote:
One solution would be to convert the complex object to a JSON string, and store that. The convert it back when you read that property.

That's an excellent idea!! Thanks!! Think I'll give that a try.

FlyingDiver wrote:
All callback methods are part of the plugin object. So you make it a property (Python term) of the Plugin object.

I get that. I think what I'm having trouble understanding is the difference between a plugin and a device created with the plugin (in terms of code and memory space). For example:

Code: Select all
################################################################################
class Plugin(indigo.PluginBase):
   ########################################
   def __init__(self, pluginId, pluginDisplayName, pluginVersion, pluginPrefs):
      super(Plugin, self).__init__(pluginId, pluginDisplayName, pluginVersion, pluginPrefs)

   self.plots = []
I understand that I can reference self.plots from any callback method or anywhere within the class for that matter. However, is self.plots part of the device object or the plugin object? Maybe a better question is if there are multiple devices created with this plugin, would they each have a unique instance of self.plots (part of the device object) or would all devices share the same instance (plugin object)? Even though this is a plugin class, does a unique instance get created for each device created by the plugin class?

I don't think I quite get the big picture yet on plugins.

Posted on
Sat Jul 11, 2020 8:25 am
FlyingDiver offline
User avatar
Posts: 4303
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Extending "valuesDict" in callback functions

hannt wrote:
I understand that I can reference self.plots from any callback method or anywhere within the class for that matter. However, is self.plots part of the device object or the plugin object? Maybe a better question is if there are multiple devices created with this plugin, would they each have a unique instance of self.plots (part of the device object) or would all devices share the same instance (plugin object)? Even though this is a plugin class, does a unique instance get created for each device created by the plugin class?


There is only ONE Plugin object, for the entire plugin. So any properties of that (Plugin) object are available to all code, for any device (owned by that plugin).

From the point of view of the plugin, Indigo devices are just a data structure that you can access. They are not objects (in the Python sense) unless you also create your own classes that correspond to your different device types.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Sat Jul 11, 2020 8:32 am
FlyingDiver offline
User avatar
Posts: 4303
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Extending "valuesDict" in callback functions

hannt wrote:
This is one variation of a button pressed callback method:

Code: Select all
def savePlotButtonPressed(self, valuesDict, typeId, devId):
      if 'Plots' in valuesDict.keys():
         tempPlots = valuesDict['Plots'])
      else:
         tempPlots = []
      tempPlots.append({
               'PlotDeviceID':valuesDict['PlotDeviceIdUI'],
               'PlotDeviceStateKey':valuesDict['PlotDeviceStateKeyUI'],
               'LegendLabel':valuesDict['LegendLabelUI'],
               'LineWidth':valuesDict['LineWidthUI'],
               'LineColor':valuesDict['LineColorUI'],
               'LineAlpha':valuesDict['LineAlphaUI'],
               'LineStyle':valuesDict['LineStyleUI'],
               'LineMarker':valuesDict['LineMarkerUI']
            })
      valuesDict['Plots'] = tempPlots
      #Also tried this:
      #valuesDict['Plots'] = list(tempPlots)
      return valuesDict


In this case, try changing the definition of the tempPlots variable to:
Code: Select all
         tempPlots = indigo.List()

Which creates a special variation of List object that Indigo CAN serialize. You might also need to change your code that creates the dict it's storing in the list to create it first as an indigo.Dict() object, then populate it and store it in the List object.

See https://wiki.indigodomo.com/doku.php?id ... data_types

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Sat Jul 11, 2020 5:06 pm
hannt offline
User avatar
Posts: 81
Joined: Jul 08, 2011
Location: TN USA

Re: Extending "valuesDict" in callback functions

Thank you, FlyingDiver... for both posts!

FlyingDiver wrote:
There is only ONE Plugin object, for the entire plugin. So any properties of that (Plugin) object are available to all code, for any device (owned by that plugin).

From the point of view of the plugin, Indigo devices are just a data structure that you can access. They are not objects (in the Python sense) unless you also create your own classes that correspond to your different device types.

Wow, I'm surprised some of my other plugins actually work. That information helps a lot.

FlyingDiver wrote:
In this case, try changing the definition of the tempPlots variable to:
Code: Select all
         tempPlots = indigo.List()

Which creates a special variation of List object that Indigo CAN serialize. You might also need to change your code that creates the dict it's storing in the list to create it first as an indigo.Dict() object, then populate it and store it in the List object.

See https://wiki.indigodomo.com/doku.php?id ... data_types

I've read that documentation multiple times, but this time it actually makes sense when to use the indigo versions of dict and list. Again, thanks for the information. I've been away from home all day, but will try your suggestions when I get back.

Posted on
Sun Jul 12, 2020 8:16 am
hannt offline
User avatar
Posts: 81
Joined: Jul 08, 2011
Location: TN USA

Re: Extending "valuesDict" in callback functions

Had some time this morning for testing - here are the results.

As suggested, I created an Indigo list of python dictionaries and tried to save that using valuesDict. That did not work, nor did an Indigo list of Indigo dictionaries.

After thinking about it more, I figure Indigo is probably expecting a "list" (in the context of a device configuration UI) to be a list of tuples, where each tuple consists of 2 strings. This "list" would be used for popupMenu UIs and list UIs. So I created a list of tuples where each tuple consisted of a string and a dictionary. This did not work either. But what did work is to create a random list of tuples where each tuple was 2 strings and try to save that to valuesDict. Even though the random list was not associated with an actual popupMenu or list, it still saved properly.

Interesting side note though - creating an Indigo list did not work for a popupMenu UI or list UI.

This worked in a popupList UI list generator callback method:
Code: Select all
plotListUI = list()  # An empty list

And this did not work:
Code: Select all
plotListUI = indigo.List()  # An empty list
The thing that DID work was to use json to convert the list into a string and save that into valuesDict. However, this feels like a hack of sorts and I wonder if it might break at some later date if Indigo's handling of valuesDict changes in some way. So I think I'm going to change strategies.

After the device is configured, I'd like for the it to contain a property that is the list of dictionaries, or more appropriately, an Indigo list of dictionaries. To accomplish this, when the device config UI is open, I'll maintain a device property (not using valuesDict to store it) called 'tempPlots' (or similar). When a callback method is executed, I'll get the value from the device properties and save it back at the end of the method. When the UI is validated and closed, I'll save the property as 'Plots' and delete 'tempPlots'. While this is a little more work, I'm hoping it won't break sometime in the future.

Posted on
Sun Jul 12, 2020 10:34 am
kw123 offline
User avatar
Posts: 7127
Joined: May 12, 2013
Location: Dallas, TX

Re: Extending "valuesDict" in callback functions

I believe you should put the json.dumps into valuesDict and not into props=indigo.devcies.pluginProps while you are in the method validateDeviceConfigUi
After return True .. I believe the props are being replaced by the valuesDict (they are the temporary contents of the dev props) when you do return False .. they are not overwritten and no property in the device gets changed.

I have been using v
Code: Select all
aluesDict["my_list"] = json.dumps(myList/dict/...)
for some time and it works very well.
then you can access my_list with
Code: Select all
props = indigo.devices.props[devID]
my_list = json.loads(props["my_list"]
anywhere in the plugin.

I don't see any reason that that would break as you store a simple test string the device property


Karl

or you store the dict/ list in /Library/Application Support/Perceptive Automation/Indigo 7.4/Preferences/Plugins/YourpluginID/yourfile.json
and write and read the dict w json.read / json.dum yourself

Posted on
Sun Jul 12, 2020 1:13 pm
hannt offline
User avatar
Posts: 81
Joined: Jul 08, 2011
Location: TN USA

Re: Extending "valuesDict" in callback functions

kw123 wrote:
I believe you should put the json.dumps into valuesDict and not into props=indigo.devcies.pluginProps while you are in the method validateDeviceConfigUi
After return True .. I believe the props are being replaced by the valuesDict (they are the temporary contents of the dev props) when you do return False .. they are not overwritten and no property in the device gets changed.
...
I don't see any reason that that would break as you store a simple test string the device property
Karl

Thanks for the suggestion/validation, Karl. I originally wrote it that way, but had a feeling that it was a hack. My thinking was that at some point, Indigo might start validating that the values in valuesDict actually corresponded with a UI element. Although now that I think about it, even if that did happen, I could just create a hidden Text Field that corresponded with the value. Duh.

From the bigger picture perspective, all of this is to manage a list of plot data within a device that creates a plot using matplotlib, but it could be to manage a list of any kind of "thing" really. I'm creating a UI that's similar to the way HomeKit Bridge allows the user to add a device, delete a selected device or edit a selected device. It seems to me this is a very common UI not just in an Indigo configuration UI, but in just about any computer/device application. Has there been some sort of enhancement request to make this process easier? Just off the top of my head, here are a few thoughts:

  • Allow buttons to be arranged next to each other. This would allow the standard 3-button UI (A list and below it: Add, Edit and Delete buttons)
  • Enable or disable buttons based on if an item in the list is selected
  • Allow a second UI window to open when the user clicks the Add or Edit button
  • Allow a popupMenu or list to store additional data for items in the list.

Here's the typical flow using a contact manager as an example. Start with an empty list of contacts so you click the Add button. A new window opens up where you type in the Name, Address, Email and other information about that person. Upon closing the dialog, that person's name is now in the list and the additional information for that person is stored somewhere in valuesDict. Next the user might add another person to the list so now the list displays 2 people and maintains data for each. Next the user might want to delete or edit a particular person, etc. When the configuration UI is closed, list of contacts (along with data for each) gets written to the device as a property.

I've implemented this with visible bindings, etc. but it's still very clunky with the current limitations.

-Tommy

Posted on
Sun Jul 12, 2020 3:24 pm
kw123 offline
User avatar
Posts: 7127
Joined: May 12, 2013
Location: Dallas, TX

Re: Extending "valuesDict" in callback functions

one more comment:

You can add properties to indigo.devices.[12345].pluginProps that are not in device.xml. That is allowed and encouraged and widely used.
There are also examples in the indigo wiki on how to use it.

Karl

Posted on
Mon Jul 13, 2020 3:36 am
hannt offline
User avatar
Posts: 81
Joined: Jul 08, 2011
Location: TN USA

Re: Extending "valuesDict" in callback functions

Oh I understand that and use properties often. What is new to me is using valuesDict as an easy way to temporarily store properties during configuration and then they automatically get saved (as properties) to the device, or discarded, depending on if the user cancels the config UI or not. I had never saved anything to valuesDict until now, other than what was in Devices.xml.

Thanks to you and FlyingDiver for your help. I learned a lot though this thread.

Posted on
Tue Jul 14, 2020 7:01 am
hannt offline
User avatar
Posts: 81
Joined: Jul 08, 2011
Location: TN USA

Re: Extending "valuesDict" in callback functions

Well, I thought I had it working, but as it turns out, it wasn't completely working. To summarize, it doesn't appear that you can write a property to a device from validateDeviceConfigUi or from closedDeviceConfigUi.

As suggested, I used a jason dumps string during device configuration to store 'tempPlots' which is a list of dictionaries:
Code: Select all
if 'TempPlots' in valuesDict:
         tempPlots = json.loads(valuesDict['TempPlots'])
      else:
         tempPlots = []
   #Append dictionaries to the list tempPlots
   valuesDict['TempPlots'] = json.dumps(tempPlots)
   return valuesDict
Then in validateDeviceConfigUi or closedDeviceConfigUi, I tried to write the actual data structure as a device property:
Code: Select all
      dev = indigo.devices[devId]
      localPropsCopy = dev.pluginProps
      localPropsCopy['Plots'] = json.loads(localPropsCopy['TempPlots'])
      dev.replacePluginPropsOnServer(localPropsCopy)
This did not work! The data was not written as a property. Is this expected?

In the end, I used deviceUpdated to write the property:
Code: Select all
   def deviceUpdated(self, origDev, newDev):
      dev = indigo.devices[newDev]
      localPropsCopy = dev.pluginProps
      if 'TempPlots' in localPropsCopy and localPropsCopy['TempPlots']:
         localPropsCopy['Plots'] = json.loads(localPropsCopy['TempPlots'])
         localPropsCopy['TempPlots'] = ""
         dev.replacePluginPropsOnServer(localPropsCopy)
This appears to be working as expected, but I was just curious if the behavior I saw was expected, and why.

Posted on
Tue Jul 14, 2020 7:09 am
FlyingDiver offline
User avatar
Posts: 4303
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Extending "valuesDict" in callback functions

You can't do that in validateDeviceConfigUi or closedDeviceConfigUi because at the point those are called, the device object may not exist yet, especially for devices that are currently being created.

You should be doing this in deviceStartComm().

Doing it in deviceUpdated() is going to break badly because that gets called for EVERY change to EVERY Indigo device. Not just your plugin's devices. Your code doesn't even check to see if the device belongs to your plugin.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Tue Jul 14, 2020 9:17 am
jay (support) offline
Site Admin
User avatar
Posts: 16302
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Extending "valuesDict" in callback functions

FlyingDiver wrote:
Doing it in deviceUpdated() is going to break badly because that gets called for EVERY change to EVERY Indigo device. Not just your plugin's devices. Your code doesn't even check to see if the device belongs to your plugin.


Slight correction: it will get called for all devices if you subscribe to device changes. Otherwise you just get changes to devices owned by your plugin.

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Posted on
Tue Jul 14, 2020 9:23 am
jay (support) offline
Site Admin
User avatar
Posts: 16302
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Extending "valuesDict" in callback functions

I have to admit that I've read through the thread a couple of times and get lost along the way, so let me ask this: would you mind restating succinctly what you're trying to accomplish, but with as little implementation details as possible? It's often easier to start with the base requirement/use case to work your way to a solution rather than to jump in with implementation details and their failings. At least that's been my experience... ;)

I read your first post multiple times but I'm still not getting the picture of what you're trying to do overall (vs how you'd like to implement it).

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Who is online

Users browsing this forum: No registered users and 0 guests

cron