Subscription options

Wouldn't it be the same approach either way? I want a device state (virtual or otherwise) to update based on an incoming message.

I have not used Virtual Devices, but to my understanding they are just abstract device shells, leaving the implementation to the author (like virtual device for Nest thermostat uses web API to Nest cloud instead of local PLM device communication, etc).

If that is the case, a device is a virtual device is a device. For the weather station, I might want a Virtual Device to update its state (not sure what my options are for a weather station device?). For all actual/hardwired Indigo devices, I want the same thing - to mirror the state from a topic into any given device (and trigger actions, and read/update a variable). Something more abstract than Indigo's API would be preferred (to Jay's point about finding standard HA MQTT schemas) but, even a 1 to 1 with Indigo (schema-wise), would work for now.

Matt

[jay (support)]

Is calling the local REST web API an option from within the python script?

To specifically answer your question, yes. We include the requests python module that easily handles digest authentication.

jay (support) wrote:
We intentionally prevent external write access to device states/properties/configs which would present both a security risk and the possibility that the device would become inoperative based on another plugin/script manipulating it's states...

Seems to me there are straight forward ways to deal with these concerns.

Security - we are inside an inline python script within an Indigo process. Add the concept of a sandbox to these inline scripts (or offer an additional kind of Secure Script that has a higher privilege).
Concurrency - use thread locking for device state writes within said Secure Scripts.

Done!

I can certainly understand the concerns you lay out above, but not having access to modify device states from within a plugin seems like a pretty big gap. Can someone explain the "shim" device approach a bit more to me?

Thanks,
Matt

jay (support) wrote:
We intentionally prevent external write access to device states/properties/configs which would present both a security risk and the possibility that the device would become inoperative based on another plugin/script manipulating it's states...

Seems to me there are straight forward ways to deal with these concerns.

Security - we are inside an inline python script within an Indigo process. Add the concept of a sandbox to these inline scripts (or offer an additional kind of Secure Script that has a higher privilege).
Concurrency - use thread locking for device state writes within said Secure Scripts.

Done!

Well, yeah, if you add the security levels and sandboxing. And plugins have some way to block any scripts or identify authorized scripts or plugins. But that's a non-trivial amount of work to be done.

I can certainly understand the concerns you lay out above, but not having access to modify device states from within a plugin seems like a pretty big gap. Can someone explain the "shim" device approach a bit more to me?

The design intent is that only the plugin itself has write-access to it's own internal state. Which is a good thing, IMO. And there's really no good reason for anything external to a plugin to need to do this for plugins that are actually managing their own devices. It's when you're trying to maintain a "device" that represents something that doesn't have a native plugin, or doesn't actually exist. The Virtual Device plugin does this, but it's limited in the types of devices it can create. And the communication to it using action groups and variables is pretty cumbersome.

What we need is either a rework of Virtual Devices or an alternative that uses some sort of inter-process API to manage the devices. Something that plugins and scripts can use, without having to do the variables and action groups. This "lighter weight" implementation is what I think of as a "shim". But I don't know if Jay and Matt agree.

[jay (support)]

The design intent is that only the plugin itself has write-access to it's own internal state. Which is a good thing, IMO.

Exactly. Chaos would undoubtedly ensue if you didn't restrict write access to critical state information. And adding some kind of security layer is quite large and, in all honesty, not something we're particularly interested in doing given the demand. We do, however, understand that there are situations where some separate communication layer (a la MQTT) is useful to add, and that inherently means the need for some way to connect them to the standard Indigo device types. In the past it's been up to the plugin developer to add that into the plugin that's built to use that comm layer. We're thinking about alternatives though (see next comment).

What we need is either a rework of Virtual Devices or an alternative that uses some sort of inter-process API to manage the devices. Something that plugins and scripts can use, without having to do the variables and action groups. This "lighter weight" implementation is what I think of as a "shim". But I don't know if Jay and Matt agree.

The Virtual On/Off Device in the Virtual Devices Interface (the only one that applies really to this conversation) probably does need some work. The intention all along (notice the top "execution model" popup) was to have a variety of ways of using it. Action Groups and variables were the initial implementation - which for many cases is exactly what non-technical users needed (no scripts, etc) since it builds on the UI tools that they are already familiar with. I think that there's a "script" model, so that the various actions will just execute python scripts. I also now believe that automatic state management should be an option.

But for this particular use case, there may be yet another model: None (which doesn't really do anything at all except create a shell device). It might be that a script model that doesn't have any scripts associated with it might suffice as well. But in either case, there might (or might not) need to be actions added to directly modify properties/states. I've yet to hear why using the built-in indigo.device.turnOn(123) command wouldn't suffice to modify the devices onOffState, but it's possible that there is a reason.

This needs more careful consideration to make sure that we're keeping the device simple, but also capable enough to solve this type of problem. We don't want to half-a$$ anything as most of you know - we're really quite deliberate when adding functionality to make sure that adhering to our KISS principle (as much as possible with an advanced product like Indigo).

We're having some side conversations about this, and feedback is welcome.

[jay (support)]

Concurrency - use thread locking for device state writes within said Secure Scripts.

That issue isn't about concurrency (that's already handled). It's about some outside script munging state data such that the device can no longer function correctly. For instance, setting the "onOffState" to "neither" or brightness to "10000000". Device states mean something to somebody (likely the plugin or interface that created them), so allowing other scripts or plugins to arbitrarily change state data is just dangerous.

That issue isn't about concurrency (that's already handled). It's about some outside script munging state data such that the device can no longer function correctly. For instance, setting the "onOffState" to "neither" or brightness to "10000000". Device states mean something to somebody (likely the plugin or interface that created them), so allowing other scripts or plugins to arbitrarily change state data is just dangerous.

That makes a lot more sense. Thanks for clarifying.

So you're saying that there is the ability to write state though?

From your comment

I've yet to hear why using the built-in indigo.device.turnOn(123) command wouldn't suffice to modify the devices onOffState, but it's possible that there is a reason.

So then:

Code: Select all

input = '{"deviceValue": "ON", "deviceType": "OUTLET", "deviceId": 12345}'.   
data = json.loads(input)
device = indigo.devices[data["deviceId"]]
type = data["deviceType"]
value = data["deviceValue"]
(if value == 1 || value == 'ON')
   indigo.device.turnOn(12345)


?

Matt

Code: Select all

input = '{"deviceValue": "ON", "deviceType": "OUTLET", "deviceId": 12345}'.   
data = json.loads(input)
device = indigo.devices[data["deviceId"]]
type = data["deviceType"]
value = data["deviceValue"]
(if value == 1 || value == 'ON')
   indigo.device.turnOn(12345)


Close.

Code: Select all

input = '{"deviceValue": "ON", "deviceType": "OUTLET", "deviceId": 12345}'.   
data = json.loads(input)
value = data["deviceValue"]
devId = data["deviceId"]
if (value != 0) or (value == 'ON'):
    indigo.device.turnOn(devId)


Assuming I want to create an inline script, what is the variable substitution syntax so I can get the actual incoming message vs declaring 'input' in this example? I will give this a whirl. If there is an equivalent function for Actions (like action.execute()) then I would have about 90% of my use cases for HA covered. I will still want a way to represent weather station data, sensors and potentially geo located items, but we'll get to those in due time.

Thanks again!
Matt

What is the variable substitution syntax so I can get the actual incoming message vs declaring 'input' in this example? I will give this a whirl. If there is an equivalent function for Actions (like action.execute()) then I would have about 90% of my use cases for HA covered. I will still want a way to represent weather station data, sensors and potentially geo located items, but we'll get to those in due time.

Where is the data actually coming from? My MQTT plugin? If so, you want the "last_payload" state.

Code: Select all

mqtt_broker_devID = XXXXX
broker_dev = indigo.devices[mqtt_broker_devID]
input = broker_dev.states["last_payload"]

# input = '{"deviceValue": "ON", "deviceType": "OUTLET", "deviceId": 12345}'.   

data = json.loads(input)
device = indigo.devices[data["deviceId"]]
type = data["deviceType"]
value = data["deviceValue"]
(if value == 1 || value == 'ON')
   indigo.device.turnOn(12345)


FWIW, I just finished the first version of a fancy UI for the Event Triggers.

I just had an interesting thought. If you want some easy "shim" devices, grab the Indigo plugin SDK (http://www.indigodomo.com/Indigo7SDK) and use that to create switches or dimmers. I think that's the only one that you can actually control to mirror your IoT device state.

That's a temporary solution, honestly. There will be a better one soon enough.

Yup, this is in the context of running a script in response to a given device update topic, with an incoming payload (from your plugin).

Code: Select all

import json
mqtt_broker_devID = 694068968
broker_dev = indigo.devices[mqtt_broker_devID]
input = broker_dev.states["last_payload"]
data = json.loads(input)
#device = indigo.devices[data["deviceId"]]
#type = data["deviceType"]
value = data["deviceValue"]
if (value == 1) or (value == 'ON'):
   indigo.device.turnOn(data["deviceId"])
elif (value == 0) or (value == 'OFF'):
   indigo.device.turnOff(data["deviceId"])


This works. So yeah, just following the wiki content from the linked SDK shows what would be needed to get variables and action groups working also.
https://wiki.indigodomo.com/doku.php?id=indigo_7_documentation:plugin_scripting_tutorial
That should get me what I was after with my initial request!

Matt

[jay (support)]

So you're saying that there is the ability to write state though?

From your comment

I've yet to hear why using the built-in indigo.device.turnOn(123) command wouldn't suffice to modify the devices onOffState, but it's possible that there is a reason.

So then:

Code: Select all

input = '{"deviceValue": "ON", "deviceType": "OUTLET", "deviceId": 12345}'.   
data = json.loads(input)
device = indigo.devices[data["deviceId"]]
type = data["deviceType"]
value = data["deviceValue"]
(if value == 1 || value == 'ON')
   indigo.device.turnOn(12345)


?

Matt

If a device is one of the built-in device types, and there is an indigo command to manipulate the state in some way, then yes. If the device is a custom device, then the plugin author must provide some way to manipulate the states for that device. Usually done through actions (including scripts if they provide the information), though sometimes menu items.

Yeah, then it is simple enough to get incoming MQTT messages to control standard devices/actions/variables, which was a major part of my initial request.

1) You could get two or more Indigo instances to be in sync with one another - I'm not sure why you'd want to do this, but it sounds pretty cool
2) I could edit my Alexa skill to send the much lighter/simpler MQTT messages in order to control Indigo, instead of using the heavier HTTP w/ Digest auth calls through the reflector. I will probably try this out next, before I bite off more in the native Smart Home Skill space. I am hoping this is approach is much faster and less prone to issues with caching. What I see with my skill today (with http and digest auth) is that the first call takes a bit and will sometimes time out. Subsequent calls with in the next few minutes are much faster and more likely to succeed. I'm not sure why this is this case, but it reeks of caching... somewhere...

So next up will be my remote Weather Station project. I will only have 1 at first, so I can just stuff its data into Indigo variables if I want to see them on a control page, etc. Eventually I will have one at my house and one at a remote property, so having something more robust would be desired. The author of the GreenSky plugin indicated his plugin included a Virtual Device for a Weather Station... @Jay what would be your recommendation for establishing a WeatherStation device type that would be updatable via the MQTT events from FlyingDiver's plugin? Clone the Weather Station virtual device from GreenSky, minus that plugins inherent integration with its MQTT functionality?

Matt