Plugins with the exclamation point badge

Posted on
Sun Oct 02, 2022 11:56 am
Mark offline
User avatar
Posts: 262
Joined: Apr 21, 2005
Location: California

Plugins with the exclamation point badge

I just noticed on the plug-in page: "The plugins below with the exclamation point badge may not be compatible with versions after Indigo 2022.1.2. Contact the developer to inquire about an update."

It seems a huge number of plugins carry "the badge."

Is this based on their age, and when they were last updated? Or are you planning a change to the plugin architecture in a future release that will likely break all those plugins? I have two in particular that I'd be crippled without.

Posted on
Sun Oct 02, 2022 12:51 pm
matt (support) offline
Site Admin
User avatar
Posts: 21411
Joined: Jan 27, 2003
Location: Texas

Re: Plugins with the exclamation point badge

Plugins will need to be upgraded to python 3 compatibility to work in the next major release of Indigo. Our blog post here goes into the details of why.

Image

Posted on
Sun Oct 02, 2022 3:55 pm
Mark offline
User avatar
Posts: 262
Joined: Apr 21, 2005
Location: California

Re: Plugins with the exclamation point badge

I have a c-load of Python, all upgraded from AppleScript. Yikes. Well, hopefully there won't be too much recoding.

Is there any sort of resource that can analyze a Python 2 script for anything that won't work in Python 3? That would certainly take the sting out of the transition...

Posted on
Mon Oct 03, 2022 9:23 am
jay (support) offline
Site Admin
User avatar
Posts: 18200
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Plugins with the exclamation point badge

This document on github will help. I found that installing pylint and running it on your plugin source files:

Code: Select all
pylint --py3k your_source_file.py


identifies a lot of the issues that you'll need to address. This wiki article also contains a lot of useful information for plugin devs.

The one thing that has bit several of us is the str vs bytes change. In Python 3, all strings are unicode. When you are using any kind of network/socket communication, you will almost always have to convert to/from bytes objects since those don't generally understand unicode. So if any of your scripts or plugins do network communication, that's something you'll want to pay special attention to.

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Posted on
Mon Oct 03, 2022 10:03 am
Mark offline
User avatar
Posts: 262
Joined: Apr 21, 2005
Location: California

Re: Plugins with the exclamation point badge

Thanks Jay!

Posted on
Mon Oct 03, 2022 10:09 am
autolog offline
Posts: 3988
Joined: Sep 10, 2013
Location: West Sussex, UK [GMT aka UTC]

Re: Plugins with the exclamation point badge

Mark wrote:
... I have two in particular that I'd be crippled without.

What are the two plugins you are concerned about?

Posted on
Mon Oct 03, 2022 10:14 am
Mark offline
User avatar
Posts: 262
Joined: Apr 21, 2005
Location: California

Re: Plugins with the exclamation point badge

I use:
Email+
iRobot - Roomba
NOAA Weather Plus
PowerView

It's "PowerView" I'm most concerned about. I suppose I could modify the Python myself, or just extract from the Plug-in what I need (is either allowed?). But I'd prefer the developer do that dirty work!

NOAA Weather doesn't have a badge, but NOAA Weather Plus does ("Plus" is the one I'm currently using, not sure the difference).

iRobot - Roomba doesn't have a badge, so hopefully the developer is on top of that one.

I don't see "Email+" listed. Where did I get that? Was that a built-in/included plug-in?

Posted on
Mon Oct 03, 2022 10:50 am
autolog offline
Posts: 3988
Joined: Sep 10, 2013
Location: West Sussex, UK [GMT aka UTC]

Re: Plugins with the exclamation point badge

EMail+ functionality has been incorporated into the latest version of Indigo and therefore uses Python 3.

iRobot - Roomba is "unbadged" and has therefore already been converted to Python 3.

NOAA Weather has been upgraded and is now using Python 3 - It is included as standard in the latest version of Indigo.
The NOAA Weather Plus Plugin page in the Indigo Store lists the functionality that is improved over the NOAA Weather plugin included with Indigo.
However, how up-to-date that list is given that the NOAA Weather plugin included with Indigio has been recently updated, I am not sure?

PowerView would need to be converted to Python 3. As you say, ideally this would be done by the developer.

Posted on
Mon Oct 03, 2022 11:59 am
DaveL17 offline
User avatar
Posts: 6744
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Plugins with the exclamation point badge

Most (but not all) of the things that NOAA Weather Plus added on top of Indigo's NOAA Weather plugin are now included with the latest version of the plugin that ships with Indigo 2022. These revisions were done as a part of moving to NOAA's newest API.

One word of caution is that, while the updates to the NOAA Weather plugin tried to remain backwards compatible with existing NOAA Weather plugin devices, they may not be backward compatible with the NOAA Weather Plus plugin. For example, the NOAA Weather plugin presents these data in different ways:

  • Weather Station device: should be backward compatible with legacy NOAA Weather plugin devices.
  • Current Conditions device: New. Based on geo position rather than station ID. Largely the same as the legacy Weather Station device. Legacy devices should convert to this device type after geo data are provided.
  • Weather Forecast device: New. Based on geo position. Provides 14-day forecast information if available.
  • Weather Alerts device: New. Based on geo position. Provide up to 5 severe weather alerts.

Geo data must be added on each device (since many users track weather for multiple locations).

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Mon Oct 03, 2022 1:16 pm
Mark offline
User avatar
Posts: 262
Joined: Apr 21, 2005
Location: California

Re: Plugins with the exclamation point badge

Thank you both. Most helpful. I have a ton of dependencies on that NOAA Weather Plus. Hopefully that gets updated...

I have a huge amount of scripts. It was something converting from AppleScript (which I don't miss, I have to admit). Not looking forward to testing/updating them all for Python 3. Hopefully that will last us a good long while. I know time marches on, and things "improve." So we adapt. The problem is, I use a dual-edged sword when it comes to HA. When my system is working, it does amazing things, thanks to Indigo scripting support and plug-ins. But when upgrades happen (OS, Indigo, etc), I'm sometimes at the mercy of those same scripts!

I've always known that my HA saves me less time than it takes to build/configure/maintain. I write most of that off as "hobby time." And the rest I justify that my system provides me great convenience when I need convenience. But it's times like this that my "rationalization" is, uh, challenged! :wink:

Posted on
Fri Nov 04, 2022 5:00 am
Mark offline
User avatar
Posts: 262
Joined: Apr 21, 2005
Location: California

Re: Plugins with the exclamation point badge

Happy to announce...

The 2022.1/Python 3 upgrade went well. The PowerView and NOAA Weather Plus plugins seem to be working fine. As well as they ever did, anyway (NOAA+ errors a lot, but I think that's NOAA, not the plugin, I digress). Roomba is working fine, too. As is email. All good.

The Python 2 to 3 upgrade went well. I had one script fail because of some errant indenting (spaces vs tabs or something like that). And as predicted the unicode issue was all over the place. I use the unicode function liberally, in almost all of my scripts. I fixed my external Includes one by one, that wasn't too bad. But I have dozens and dozens of Python scripts in Indigo itself. So I got a hair-brained idea to shortcut the process:

- I shut down Indigo (so it would close/save the database file).
- I opened the file with BBEdit.
- I did a global replace of "unicode" with "str".
- And then I fired Indigo back up.

Voila, so far so good. I check the log for errors at the end of each day, and the "unicode search and destroy" has so far caused no problems.

I didn't run scripts through any of the suggested "testers." Maybe I will, maybe I won't. Or I'll just wait for failures, if any. I write "Python" in all my Action's and Trigger's Notes fields, so I can keep track of where they are. If I get inspired I can just go through them all and test them, when I have the time.

If I'm going to get stung, it will most likely be when one of my scripts calls on a seldom-used bit of code, like an error handler, or rare-condition handler, something like that.

It'd be great if Indigo had a command that could extract all the Python scripts and output them to text files! Hint, hint!

Posted on
Fri Nov 04, 2022 6:13 am
DaveL17 offline
User avatar
Posts: 6744
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Plugins with the exclamation point badge

Mark wrote:
I opened the file with BBEdit.

I'm glad that this effort worked out. A word of warning--it's best to create a copy of the database file [CMD-D] when working with it directly (and especially when editing it). Just in case you need to revert.

Another tool you can use to find scripts in the Indigo DB file is Grep (note the spaces are escaped and the single quotes around the search string which contains double quotes).

Code: Select all
grep -n '<ScriptSource type="string">' /Library/Application\ Support/Perceptive\ Automation/Indigo\ 2022.1/Databases/development\ environment.indiDb
I used grep extensively in converting embedded scripts to Python 3.

Mark wrote:
I write "Python" in all my Action's and Trigger's Notes fields, so I can keep track of where they are.

That's a good idea.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Fri Nov 04, 2022 6:55 pm
Mark offline
User avatar
Posts: 262
Joined: Apr 21, 2005
Location: California

Re: Plugins with the exclamation point badge

Thanks for the tips. I'm aware of Grep, but I barely understand it. I've no doubt I could figure out how to extract the Python out of the database, one way or another. But doesn't all the Python (as did AppleScript) get re-formatted into one long string somehow? Perhaps line breaks and indents are replaced, and later "reconstituted" by Indigo? I just assumed that, because while looking at the database I can see the scripts, but not formatted as they are in the Indigo editor, only always on one line in the database.. Anyway, not knowing how that actually works, or how to get it back into that one-line format after "fixing" errant Python 2, I didn't explore that. Perhaps I have that all wrong?

I examined one external file last night with pylint, and found errors, mostly indent errors that I didn't understand. Several errors regarding the formatting of constants not in all-caps, extra spaces and returns at the end of lines, things of that nature. Nothing that appears to keep the script from working (because it is working). pylint gave me a score. I got 0.0/10.0 or something like that. Is that the worst, or the best possible? :lol: (I actually don't know!) And pylint doesn't find everything, right?

I think I understand that Indigo is currently executing all my internal and include scripts in Python 3, not 2. Only plug-ins are temporarily allowed to call to Python 2, in this "transition" period. But my scripts don't get to call Python 2, right? So if they're working they're working? I suppose the next iteration of Python could break many of my scripts, if it is more strict about some of the code-foibles that Python 3 seems to be ignoring. Is that possible?

To be honest, based on what it took to examine that one include last night, I wouldn't want to have to do that across all my dozens and dozens of scripts. I just counted: Triggers and Actions and Includes that I tagged as utilizing Python number about 270!! Granted, many are near duplicates, with only a few words altered. Many or very short. But there are also many that are quite involved, pages long. Just opening and copy-pasting 270 code snippets into 270 files that could be examined by pylint would be a daunting task!

There's got to be a better way. I wasn't kidding about building some sort of code extractor into Indigo...

I've obviously spent a lot of time scripting. TWICE, because most of these scripts were once AppleScript. I realize I've brought some of this pain on myself, complicating my HA in this way, but it is what it is. There's no going back now. But I have to ask myself, how many more times will I have the energy (or brain-capacity) to completely overhaul hundreds scripts?
Last edited by Mark on Sat Nov 05, 2022 11:50 am, edited 1 time in total.

Posted on
Sat Nov 05, 2022 6:16 am
DaveL17 offline
User avatar
Posts: 6744
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Plugins with the exclamation point badge

I don't mean to edit the database file directly -- I can't emphasize this strongly enough -- don't edit the database unless you really know what you're doing. I meant to use grep to spot where the embedded scripts are "hidden" in your setup. Then you can look at the database code to know which object contains the script and edit it via the Indigo user interface.

Regarding Pylint, it reports a lot more than errors. A script can run just fine and Pylint will still report ways that it can be improved. You've seen some of its messages are about formatting (adding/removing extra spaces, unnecessary parentheses, etc.), but it will also report things like unused variables and function calls, and other "nuts and bolts" stuff like that. Lastly, it also reports things like style "issues" -- for example, it will flag a variable called `myVariableValue` and suggest that it should be called `my_variable_value` -- that's just a matter of convention. (Google PEP 8 for more information.) I wouldn't recommend using Pylint to spot the changes from Python 2 to Python 3 as you'll have to sort through loads of stuff that's not mission critical.

I understand your frustration. In both instances, the changes were the result of Apple dropping support for AppleScript and Python. Matt & Jay supported Python 2 long after it was deprecated to try to reduce the pain. Most simple scripts will require minimal changes if any to go from Python 2 to Python 3 and there's a helpful guide which you may have already seen to point you to many of the things that need to change:

https://github.com/IndigoDomotics/IndigoSDK/blob/main/Updating%20to%20API%20version%203.0%20(Python%203).md

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Sat Nov 05, 2022 12:20 pm
Mark offline
User avatar
Posts: 262
Joined: Apr 21, 2005
Location: California

Re: Plugins with the exclamation point badge

Thanks again, Dave, for your insights.

OK, grep just to find them, understood. As I mentioned, I tag Notes fields with the word "Python," and while I undoubtably missed a few, I pretty much know where my scripts are. That's how I counted them to get 270! Maybe I could use Grep somehow to see if there are any Actions and Triggers that use Python that don't have a Note that contains "Python." That would be great to clean that up.

I did read that GitHub page. Thanks. I was pleased to discover that, somehow, most of the things to watch out for I had already addressed. Like open() instead of file(), for example. A good portion of my Python has been "learned" by finding examples online. Mostly via stackoverflow.com and sometimes w3schools.com, both amazing resources. Likely the examples I found were already "future-proofed" by their great contributors.

- Is there a better utility than Pylint that will more reliably (or simply) find code that will fail in Python 3? I could run my includes through it without too much work.

- Can you decipher Pylint's result score? I get "0.0."

And my complaint about the necessity for these changes was no reflection on Matt and Jay, not at all. Of all the software I use, and have ever used, I'd say Indigo is hands-down the most reliable and least buggy. I know that is because of their meticulous coding practices and their long-term strategies. I hated losing AppleScript, but less so than actually using it! Now I'm all-Python-all-day and appreciate the guys' wisdom making that move.

Apple is to blame, of course, and frankly it's why MacOS hasn't, and never will, become the dominant OS on this planet. Apple sucks at maintaining mission critical. (I just upgraded my OS and had to remove 1600 duplicate Contacts!! After decades of trying, they still don't have cloud syncing figured out!!!) They'll abandon decades-old tech just because they don't want to support it. 32-bit is a classic example. Airport devices. AppleScript. HyperCard. The list is endless. Using Apple and MacOS means you'll have to accept their changes, and give up what they remove, at their whim, sometimes without notice. That's why they'll never be the mainstream corporate OS. Most companies just can't justify the risk of investing in Mac. I accept it, but that doesn't mean I can't complain about it!!

Who is online

Users browsing this forum: No registered users and 6 guests