Optimizing Python Script Execution

Posted on
Tue Jul 23, 2019 8:50 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Optimizing Python Script Execution

Use "Execute Script", not "Open File".

https://wiki.indigodomo.com/doku.php?id ... er_actions

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Tue Jul 23, 2019 8:58 am
whmoorejr offline
User avatar
Posts: 762
Joined: Jan 15, 2013
Location: Houston, TX

Re: Optimizing Python Script Execution

FlyingDiver wrote:
Use "Execute Script", not "Open File".


I feel like a dork. Thanks (again). I have seriously lost count of how many times you have helped me with a plugin or a script. There should be a "send a beer" button on the forum.

Bill
My Plugin: My People

Posted on
Tue Jul 23, 2019 1:21 pm
matt (support) offline
Site Admin
User avatar
Posts: 21411
Joined: Jan 27, 2003
Location: Texas

Re: Optimizing Python Script Execution

Nevermind this post: didn't see the last couple of posts in the thread since they were on the 2nd page!

You don't want the action type Open File. Instead choose Server Actions->Script and File Actions->Execute Script, then select the File radio button (not Embedded) and you can choose your .py script file.

Image

Posted on
Tue Jul 23, 2019 2:55 pm
whmoorejr offline
User avatar
Posts: 762
Joined: Jan 15, 2013
Location: Houston, TX

Re: Optimizing Python Script Execution

I now have my sample script running as an external script (yay!). To test the scripts, I'm running them as embedded.

To make sure I don't have scripts stuck in a loop or running indefinitely or not working without explanation, I'm putting in some error handling..... but I'm having a hard time getting my head wrapped around that. My script now does two thing... 1) get a camera image from a URL. 2) Save that on the indigo server with a set filename.

I've got the first part working... so if I put in the wrong URL, wrong password, camera offline, etc... I get a message in the log file. For the second part, I figured I'd just copy/paste what worked above. No dice. "Invalid syntax". I've tried rearranging the same stuff.

Code: Select all
import requests
from requests.auth import HTTPDigestAuth
try:
   r = requests.get('http://192.168.1.141:8141/cgi-bin/snapshot.cgi?', auth=HTTPDigestAuth('username', 'password'), timeout = 2.0)
   r.raise_for_status()

except:
   indigo.server.log("Can't get camera image", isError = True)

try:
   with open('/Users/williammoore/Desktop/IPCamRotate/Testing.jpg', 'wb') as image_file:
      image_file.write(r.content)

except:
   indigo.server.log(“Can’t write the rotation image”, isError = True)
Am I close?

Bill
My Plugin: My People

Posted on
Tue Jul 23, 2019 3:06 pm
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Optimizing Python Script Execution

This line has fancy quotes, which Python doesn't like:

Code: Select all
   indigo.server.log(“Can’t write the rotation image”, isError = True)


What text editor are you using for your Python code?

Highly recommend BBEdit. There's a free mode which is all you really need for basic stuff.

https://www.barebones.com/products/bbedit/download.html

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Tue Jul 23, 2019 3:41 pm
jay (support) offline
Site Admin
User avatar
Posts: 18199
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Optimizing Python Script Execution

FlyingDiver wrote:
Highly recommend BBEdit. There's a free mode which is all you really need for basic stuff.


+1.

I've owned every version of BBEdit since the early 90's. It's solid and reliable, does what it does very well. Those guys deserve the meager amount they charge.

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Posted on
Tue Jul 23, 2019 3:46 pm
whmoorejr offline
User avatar
Posts: 762
Joined: Jan 15, 2013
Location: Houston, TX

Re: Optimizing Python Script Execution

FlyingDiver wrote:
This line has fancy quotes, which Python doesn't like:


And now I owe you another beer. I hope you are keeping tract of my tab. :D

When I was fiddling with AppleScript, I used TextEdit to store snippets and ScriptEditor or Indigo to test and piece things together.

I have BBedit on my iMac and TextWrangler on my indigo server (Mac mini running Yosemite). But I typically save all my working snippets to text files with TextEdit and copy/past them into indigo to test them or build new scripts.

FYI: after all of your input and some re-arranging, all is right in the world. I had to put part 2 in the middle of part 1. The way I had it above, if the URL crapped out, it would still try to run part 2 which would create a weird blank .jpg

Code: Select all
ThisScript = 'Python Script With Error Handling'
import requests
from requests.auth import HTTPDigestAuth
try:
   r = requests.get('http://192.168.1.141:8141/cgi-bin/snapshot.cgi?', auth=HTTPDigestAuth('username', 'password'), timeout = 2.0)
   r.raise_for_status()

   try:
      with open('/Users/williammoore/Desktop/IPCamRotate/Testing.jpg', 'wb') as image_file:
         image_file.write(r.content)
   except:
      indigo.server.log(ThisScript + ' :Rotation image error', isError = True)
   
except:
   indigo.server.log(ThisScript + ' :No camera image available', isError = True)

Bill
My Plugin: My People

Posted on
Tue Jul 23, 2019 3:51 pm
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Optimizing Python Script Execution

This construct is a little easier to follow. It keeps the except adjacent to the try. The else construct after a try only runs if it doesn't trigger the except.

Code: Select all
ThisScript = 'Python Script With Error Handling'
import requests
from requests.auth import HTTPDigestAuth
try:
   r = requests.get('http://192.168.1.141:8141/cgi-bin/snapshot.cgi?', auth=HTTPDigestAuth('username', 'password'), timeout = 2.0)
   r.raise_for_status()

except:
   indigo.server.log(ThisScript + ' :No camera image available', isError = True)

else:
   try:
      with open('/Users/williammoore/Desktop/IPCamRotate/Testing.jpg', 'wb') as image_file:
         image_file.write(r.content)
   except:
      indigo.server.log(ThisScript + ' :Rotation image error', isError = True)
   


joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Tue Jul 23, 2019 5:00 pm
whmoorejr offline
User avatar
Posts: 762
Joined: Jan 15, 2013
Location: Houston, TX

Re: Optimizing Python Script Execution

FlyingDiver wrote:
This construct is a little easier to follow....


Thank goodness. I hated the way mine flowed even though it was working. Greatly appreciate all your help!

Bill
My Plugin: My People

Posted on
Tue Jul 23, 2019 5:08 pm
jay (support) offline
Site Admin
User avatar
Posts: 18199
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Optimizing Python Script Execution

While we're on our voyage of learning, Here's my modification:

Code: Select all
import requests

THIS_SCRIPT = 'Python Script With Error Handling: {}'
IMAGE_PATH = '/Users/williammoore/Desktop/IPCamRotate/Testing.jpg'

try:
    # Get the image
    reply = requests.get(
                'http://192.168.1.141:8141/cgi-bin/snapshot.cgi?',
                auth=requests.auth.HTTPDigestAuth('username', 'password'),
                timeout = 2.0
            )
    # Raise an exception if the HTTP GET failed
    reply.raise_for_status()
    # Open the image file and write the image data
    with open(IMAGE_PATH, 'wb') as image_file:
        image_file.write(reply.content)
except requests.HTTPError:
    # The request to get the image failed
    indigo.server.log(
        THIS_SCRIPT.format('No camera image available'),
        isError=True
    )
except:
    # Something else went wrong, probably writing to the file
    indigo.server.log(
        THIS_SCRIPT.format('Rotation image error'),
        isError=True
    )


There's a standard for formatting Python scripts called PEP8, and while I myself don't always follow the guidelines, it really does lead to more readable code in most cases. I've reformatted the script to follow (I think) most of the guidelines including the line length of 79 characters. Some editors (PyCharm for instance) will highlight PEP8 violations so it makes it easy to write consistent code.

Here's what I did to the script, starting at the top:

Imports should go first.

Constants don't really exist in Python, but by convention they are UPPERCASE so it's easier later in the script to identify a variable vs a constant that should never change.

Also note, because you always want to put some extra stuff at the end of the line when you log, I'm adding the standard format() syntax to the end here in the string. This will make adding the additional stuff when you actually do the log cleaner. Also, you should use the format() method rather than + for string manipulation.

I've moved your image path to the top and made it a constant to 1) help with line length and 2) to make it clearer that it shouldn't change.

I've consolidated everyting into a single try block for readability. There is a specific except block that will catch the requests.HTTPError exception that's always raised by the raise_for_status() method. So, if that's raised you'll get the right log line and the script will end. The second except block will catch any other exception that might be raised in the try block - so that's where your other logging is happening.

You'll notice that I've wrapped the function parameters for requests.get() and indigo.server.log() so that they will fit in the PEP8 79 character column formatting. That's definitely the PEP format that I most often violate, but I'm really trying hard not to do it. It really does make for cleaner code.

I don't know if it's in the PEP8 guide or not, but I like to either import all individual elements from a module or just import the module itself and use the full dotted notation. I usually do the latter except in cases where the hierarchy is just ridiculously deep. Again, it makes for easier understanding of what's going on.

I've removed most of the extra blank lines. PEP8 says to blank lines sparingly and only to help identify related blocks of code. Others may not agree, but having lots of blank lines makes it harder for me to read. Also, there's a section about where to put spaces in other places (and where not to).

Finally, I added brief comments throughout the script to help anyone else reading the script (or to remind me) what's going on. This is definitely a personal habit, not one defined in PEP8 or anywhere else. But as I was taught back in the 80's - comments are free and usually very helpful.

Ok, that's enough for today's class!

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Posted on
Tue Jul 23, 2019 5:13 pm
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Optimizing Python Script Execution

I know PEP8 has that line length thing. Foo on them. I've got a huge monitor and I'm going to use it. ;)

Also, personal preference, but I wouldn't lump two wildly different operations (get and image write) into the same try block. But that's just me. Is that really in PEP8? I guess I should go look.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Tue Jul 23, 2019 5:21 pm
jay (support) offline
Site Admin
User avatar
Posts: 18199
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Optimizing Python Script Execution

FlyingDiver wrote:
I know PEP8 has that line length thing. Foo on them. I've got a huge monitor and I'm going to use it. ;)


I really only mentioned it because if you use short lines when you post scripts onto the forum it generally makes it easier to read and because saying close to that limit will eliminate most wrapping in the embedded script editor window in the Mac Client at it's default width... :lol:

FlyingDiver wrote:
Is that really in PEP8?


Don't know (notice that's one place I didn't mention PEP8 :wink: ). I put it there because there are really only 2 lines that are running and it seems wasteful to have two completely separate try blocks for them. Also, it's a good opportunity to illustrate another way that you can handle multiple exceptions.

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Posted on
Tue Jul 23, 2019 8:00 pm
DaveL17 offline
User avatar
Posts: 6742
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Optimizing Python Script Execution

PEP8 is a beautiful thing. Taken in moderation.

One spot where I disagree with PEP8 (that hasn't already been mentioned) is that I think this:
Code: Select all
foo['bar'] = self.pluginPrefs['key']
var = indigo.variables[12345678].value
sandwich = "ham"
ice_cream_flavor = "Blackberry Swirl"
foo = "bar"

is much less readable than this:
Code: Select all
foo['bar']       = self.pluginPrefs['key']
var              = indigo.variables[12345678].value
sandwich         = "ham"
ice_cream_flavor = "Blackberry Swirl!"
foo              = "bar"

I just ran an inspection against the Matplotlib plugin, and have 11 PEP8 style violations (in 5,600 lines of code):
- 7 module level imports not at top of file (I group my imports by built-ins, third-party, and then mine).
- 2 lines missing whitespace after ',' (now fixed).
- 2 lines too long (in the spirit of full disclosure, my default line length is set to 186 characters).

I think the line length limit of 79 originated because people used to print their code out on letter width paper--but I defer to those with degrees in computer science. :D

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Tue Jul 23, 2019 9:40 pm
jay (support) offline
Site Admin
User avatar
Posts: 18199
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Optimizing Python Script Execution

DaveL17 wrote:
PEP8 is a beautiful thing. Taken in moderation.


Agreed. It's the definition of "moderation" that varies widely!

DaveL17 wrote:
One spot where I disagree with PEP8 (that hasn't already been mentioned) is that I think this:
Code: Select all
foo['bar'] = self.pluginPrefs['key']
var = indigo.variables[12345678].value
sandwich = "ham"
ice_cream_flavor = "Blackberry Swirl"
foo = "bar"

is much less readable than this:
Code: Select all
foo['bar']       = self.pluginPrefs['key']
var              = indigo.variables[12345678].value
sandwich         = "ham"
ice_cream_flavor = "Blackberry Swirl!"
foo              = "bar"


I personally don't like it - all the extra white space sometimes requires me have to select the whole line to make sure I'm seeing the right assignments. To each his own... :wink:

DaveL17 wrote:
I think the line length limit of 79 originated because people used to print their code out on letter width paper--but I defer to those with degrees in computer science. :D


PEP8 wrote:
Limiting the required editor window width makes it possible to have several files open side-by-side, and works well when using code review tools that present the two versions in adjacent columns.


Among a bunch of other rationale, including docstrings and comments (which they want limited to 72 characters).

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Posted on
Wed Jul 24, 2019 8:17 am
RogueProeliator offline
User avatar
Posts: 2501
Joined: Nov 13, 2012
Location: Baton Rouge, LA

Re: Optimizing Python Script Execution

I know PEP8 has that line length thing. Foo on them. I've got a huge monitor and I'm going to use it

I'm with you here - even if I put windows side-by-side on my monitor, I could easily get several hundred characters line length in each window without wrapping. Of course I don't go that far with it, but definitely am not going to worry going a bit over 79!

I personally don't like it - all the extra white space sometimes requires me have to select the whole line to make sure I'm seeing the right assignments. To each his own...

With Jay on this one... I get the OCD of lining everything up, but sometimes I feel it hard to "match up" if you have similar variable values and names. Like, say, items in a list or a few similar URLs. Unless the variable names are ridiculously long, though, this is just a minor personal preference.

Also, personal preference, but I wouldn't lump two wildly different operations (get and image write) into the same try block. But that's just me. Is that really in PEP8? I guess I should go look.

I would normally write that like Jay... my basic rule would be to have multiple handlers in two cases:
1. Where you might can recover and continue such that subsequent code would still be error handled
2. Where specific, detailed information is required for the error -- e.g. I am working with an Excel library and if something goes wrong with a cell read, it throws an error that does not include the cell location. Thus, those reads have their own try/catch block so that the error can detail the cell information. Different language in that case, but same concept.

Adam

Who is online

Users browsing this forum: No registered users and 3 guests