Can you stop and re-start runConcurrentThread()?

I'm working on a plugin that responds to an action. When the action is called, it does some stuff and it's done. However, 99.9% of the time it doesn't do anything.

When the action is called, the plugin will need to run a concurrent thread. This all works fine, but I'd like to be able to start and stop that thread so that during the 99.9% of the time when the plugin isn't doing anything, the thread isn't running either.

I've tried stopConcurrentThread() and that does in fact shutdown the thread (because self.StopThread is set to True). However, Indigo tries to restart it every 10 seconds. I suspect this is because stopConcurrentThread() is not something the plugin should call, but instead, a method to implement if extra code is needed when Indio is shutting down the concurrent process. Correct?

I've also tried prepareToSleep(), but that applies to devices and triggers, which this plugin doesn't use.

I also tried using a Python thread, and that mostly works. The issue I ran into there was that exception logging from the thread did not show up in the Indigo log. It seems like this is fixable, but I'm not sure how to do that. Any thoughts on using a Python thread?

So the functionality I'm most interested in: Can the plugin tell Indigo to stop trying to run the concurrent thread defined in runConcurrentThread(), then tell it to start running again?

Thanks for any help!

If you are using runConcurrentThread() in a plugin then it has to run all the time.

The default implementation is something like:

Code: Select all

   def runConcurrentThread(self):
      try:
         while True:
            if taskNeedsDoing:
               self.goAndDoIt()
            self.sleep(60) # in seconds
      except self.StopThread:
         # do any cleanup here
         pass

Thus it runs all the time, wakes up at a period you define to see if anything needs doing, then (or otherwise) goes back to sleep.

Peter

If you are using runConcurrentThread() in a plugin then it has to run all the time.

That's what I thought. I'll look more at Python threads.

Thanks, Peter!

[jay (support)]

I'm working on a plugin that responds to an action. When the action is called, it does some stuff and it's done. However, 99.9% of the time it doesn't do anything.

Is what the action needs to do a long-running operation? Any reason why you'd need it to spawn to run async?

If so, I'd recommend taking @howartp's recommendation: create a Python Queue when the plugin starts up. When the action is called, add something to the queue. In runConcurrentThread, periodically check the queue to see if there's something to do, and if so then pull it off the queue and process it. Much easier than starting/stopping threads, and having runConcurrentThread run in a relatively tight loop isn't a resource contention.

Is what the action needs to do a long-running operation? Any reason why you'd need it to spawn to run async?

A lot of what I'm playing with right now is to communicate with Z-Wave devices using raw Z-Wave commands. For example, I'd like to query a device for what configuration options it supports (as opposed to what the documentation might say). This requires sending raw Z-Wave commands, getting responses, then sending more commands based on the responses. From what I've been able to determine, some sort of parallel threading is required in order to use zwaveCommandReceived() in this way.

In other words, I don't know how to send a command and wait on a response, then send another command and wait on a response. Etc. The only way I know how to get responses is to use indigo.zwave.subscribeToIncoming() and catch the incoming with zwaveCommandReceived().

So the way I have been trying to architect this is with a separate thread to generate the report. It first sends some raw commands and loops until some "task completed" variables are updated, sleeping between loops. Then it would send some more commands and wait again. Etc. It would also time out if the variable never got updated and handle the error. Once all commands are sent and all information is received, it would then log the report and quit.

The main thread would implement zwaveCommandReceived() and when something comes in that it's interested in, it would be decoded and a report dictionary would be updated. As it finishes each task, the "completed" variable would be updated so the report thread would know.

This could certainly be implemented using runConcurrentThread(). I even had this working, but I don't like the idea of a thread running 24/7 when it is only needed once in a while (ex. the report in the example above might get logged once every week or month or year). Even though it's not used often, I would want it to be responsive so then it would sleep only about 1 second between loops. Maybe I'm just being over concerned with this thread running.

I've not done anything with Python Queues, but I'll take a look to see if that makes sense.

Thanks, Jay.

[jay (support)]

Maybe I'm just being over concerned with this thread running.

In a word, yes. There are a LOT of plugins that do that exactly: sleep for a second, check to see if it needs to do something, sleep another second, and spend a lot of time just sleeping. Performance is not an issue because it's running it it's own thread (and the plugin is running in it's own separate macOS process) and not consuming any real resources unless it actually does some work. It becomes problematic when you don't sleep at all, but a second of yield time is an eternity (be sure to use self.sleep(), not Python's time.sleep(), as it does a couple of special things under the hood).

In a word, yes. There are a LOT of plugins that do that exactly

Thanks, Jay. Honestly, that is exactly what was worrying me. I noticed a lot of the plugins I'm running are using runConcurrentThread() and I didn't want to add to it all if there was a better way. I'll finish the implementation I had (mostly) working with runConcurrentThread(), then I'll take a look at Python Queues..

[jay (support)]

Yeah, each plugin runs in it's own process, and the RCT is a separate thread in that process. So extremely low-impact as long as it specifically yields some time using the self.sleep() method. It's definitely the recommended way unless a more complex multithreaded approach is needed for other reasons.

I'd recommend taking @howartp's recommendation: create a Python Queue when the plugin starts up.

Note: The Queue module has been renamed to queue in Python 3.

Are they purposely trying to make 2.7 to 3.x migration difficult?!?

[jay (support)]

Are they purposely trying to make 2.7 to 3.x migration difficult?!?

In migrating the backend systems from Python 2 to Python 3, I've found lots of seemingly silly things like that. Another one is objects that used to be iterables (directly iterable) now require you to explicitly call an iterator method or cast to a different iterable type. Django queryset objects are an example (there are other straight-up Python examples but I forget what they are ATM). Formerly:

Code: Select all

# Get all rows from some database table
query_set = DatabaseModelObject.objects()
for object in query_set:
    # do something with object


Now fails, because query_set is apparently no longer iterable. This is what you have to do now:

Code: Select all

# Get all rows from some database table
query_set = DatabaseModelObject.objects()
for object in query_set.all():
    # do something with object


In general I think they did a reasonable job of backwards compatibility given the large amount of change that happened under the hood. However, I think there were some things that may not have won the "it's better this way" vs "it'll break backwards compatibility" argument...

Thanks Jay and and Peter for the tip on using a queue for processing jobs on the concurrent thread. That works great.

There's one scenario though that I still can't figure out how to implement. The plugin defines an action with a UI. Within the UI is a button so I've defined a callback function for when the button is pressed. When the button is pressed, I'd like to query a Z-Wave device, get the response and return some data back to the UI by populating valuesDict and returning it.

The problem though is that zwaveCommandReceived() and the action button callback function are both on the main (same) thread, so the callback function blocks zwaveCommandReceived(). I don't know of a way to split these out to different threads.

Further, assuming there is a way to get past this challenge there appears to be another one. Say the user is creating an Action Group with one of the actions being the one defined by the plugin, and the user presses the button in the UI. If the button callback function sleeps while waiting for a response from the Z-Wave device, the UI reports that the Action Group UI has lost connection with the plugin. It comes back after the sleep times out though.

Any thoughts about how to code for this scenario?

Thanks!

Hi

I can't see that the first question is going to work how you describe it - callbacks will (I'm pretty sure) have a timeout on them that is always going to be shorter than a Zwave query and response - and it doesn't sound like user-friendly UI design/flow.

Can you describe what you're trying to achieve (what is it you're querying, and what data info returning to an Action UI) then we can maybe propose an alternative way of doing it?

Peter

One option might be to take advantage of the refreshCallbackMethod which will be called every second to refresh the UI. It is briefly described in the announcements for Indigo 7.0 in this post.

I plan on using it in a somewhat similar situation of pairing with an Android client device... user opens the device edit screen and it can refresh the UI upon receiving a pairing from the Domotics Pad client. That can't be done in a button press or anything due to the fact that they must send the value from another client. Not exactly the same, but kind of similar flow if you consider the Android client the Z-Wave responding device.

Adam

Can you describe what you're trying to achieve (what is it you're querying, and what data info returning to an Action UI) then we can maybe propose an alternative way of doing it?

See the Attachment for the specific UI I referenced in the previous post. It's defined in an Action ConfigUI. This is similar to Interfaces -> Z-Wave -> Modify Configuration Parameter..., but at least for now, I just want to read the value and it's size.

So I'd like a way to query a device for Configuration Command Class type information, then display that information in a UI. I've been able to obtain the information using a Queue as discussed previously, but the only thing I've been able to do with that information is send it to the log. The dialog below would later be updated to include the parameter name, info and properties.

One option might be to take advantage of the refreshCallbackMethod which will be called every second to refresh the UI.

Thanks, Adam! I think that might solve the problem. I also found another nugget of information in the Announcements for Indigo 7.0 that will solve another issue for this plugin.