Converting AppleScript that reads data from remote URL

Posted on
Mon Jun 29, 2020 10:08 am
gmusser offline
Posts: 290
Joined: Feb 12, 2005
Location: New Jersey

Converting AppleScript that reads data from remote URL

I'm on to my next AppleScript conversion, which retrieves data that one of my environmental sensors uploads to a remote website. The data is returned in an array that looks like this:
{"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

The AppleScript I've been using is:
Code: Select all
tell application "IndigoServer"
   
   --retrieve from GMC Map
   set TheCommand to "curl -m 5 --request GET http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812"
   
   try
      set myResult to do shell script TheCommand
      
      --extract Geiger reading
      set AppleScript's text item delimiters to ","
      set theArray to every text item of myResult
      set radLevelText to item 2 of theArray
      
      -- extract value from text string
      set radLevel to word 2 of radLevelText
      set value of variable "Radiation_Basement" to radLevel
      
   on error
      log "Error executing curl to retrieve from gmcmap.com"
   end try
   
end tell

Based on examples in the Indigo forums, I tried this Python code:
Code: Select all
import requests
import simplejson

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
response = simplejson.loads(result)
indigo.variable.updateValue(385534937, response["CPM"])

But I get this error:
Script Error GMC Map.py: expected string or buffer
Script Error Exception Traceback (most recent call shown last):

GMC Map.py, line 8, at top level
File "/Library/Application Support/Perceptive Automation/Indigo 7.3/IndigoPluginHost.app/Contents/Resources/PlugIns/simplejson/__init__.py", line 307, in loads
return _default_decoder.decode(s)
File "/Library/Application Support/Perceptive Automation/Indigo 7.3/IndigoPluginHost.app/Contents/Resources/PlugIns/simplejson/decoder.py", line 335, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer

I'm not sure what to do now. Thanks for your help. It is greatly appreciated!

Posted on
Mon Jun 29, 2020 10:23 am
jay (support) offline
Site Admin
User avatar
Posts: 18199
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Converting AppleScript that reads data from remote URL

Try this for the second to last line:

Code: Select all
response = result.json()


The requests library knows how to automatically decode json for you. Your version might have worked if you passed result.text to the simplejson.loads function.

FYI, when using the json library you should:

Code: Select all
import json


We included simplejson because when we first released Python support, Python 2.5 was the standard version on Macs and it didn't have the built-in library. If you are still running on a very old Mac OS version (before Yosemite I think) then you should continue to use simplejson.

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Posted on
Mon Jun 29, 2020 10:24 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Converting AppleScript that reads data from remote URL

The result variable is a complex object containing all the info about the request call. You need:

Code: Select all
import requests
import simplejson

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
response = simplejson.loads(result. text)
indigo.variable.updateValue(385534937, response["CPM"])


But you don't even need to do that much, as requests can do the JSON for you:

Code: Select all
import requests

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
response = result.json
indigo.variable.updateValue(385534937, response["CPM"])

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Mon Jun 29, 2020 10:55 am
gmusser offline
Posts: 290
Joined: Feb 12, 2005
Location: New Jersey

Re: Converting AppleScript that reads data from remote URL

Ah, that clarifies matter. But the revised code still give an error that, for a noobie, is incomprehensible:
Script Error GMC Map.py: 'instancemethod' object is not subscriptable
Script Error Exception Traceback (most recent call shown last):

GMC Map.py, line 8, at top level
TypeError: 'instancemethod' object is not subscriptable

Posted on
Mon Jun 29, 2020 10:59 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Converting AppleScript that reads data from remote URL

gmusser wrote:
Ah, that clarifies matter. But the revised code still give an error that, for a noobie, is incomprehensible:
Script Error GMC Map.py: 'instancemethod' object is not subscriptable
Script Error Exception Traceback (most recent call shown last):

GMC Map.py, line 8, at top level
TypeError: 'instancemethod' object is not subscriptable


Post your complete script. You probably have a typo.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Mon Jun 29, 2020 11:02 am
gmusser offline
Posts: 290
Joined: Feb 12, 2005
Location: New Jersey

Re: Converting AppleScript that reads data from remote URL

It is a direct copy of the above.

Posted on
Mon Jun 29, 2020 11:03 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Converting AppleScript that reads data from remote URL

gmusser wrote:
It is a direct copy of the above.


Which one above? There are multiple scripts. And at least one (of mine) has an errant space in it (that I see now).

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Mon Jun 29, 2020 11:10 am
gmusser offline
Posts: 290
Joined: Feb 12, 2005
Location: New Jersey

Re: Converting AppleScript that reads data from remote URL

Code: Select all
import requests

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
response = result.json
indigo.variable.updateValue(385534937, response["CPM"])

Posted on
Mon Jun 29, 2020 11:12 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Converting AppleScript that reads data from remote URL

Oops. Jay had it right. Should be:

Code: Select all
import requests

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
response = result.json()
indigo.variable.updateValue(385534937, response["CPM"])

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Mon Jun 29, 2020 11:37 am
gmusser offline
Posts: 290
Joined: Feb 12, 2005
Location: New Jersey

Re: Converting AppleScript that reads data from remote URL

Great, I added the parentheses and an "import json" line. But now I get the following error. I checked the URL and it is properly returning output in the format I mentioned.
Code: Select all
 Script Error                    GMC Map.py: No JSON object could be decoded
   Script Error                    Exception Traceback (most recent call shown last):

     GMC Map.py, line 7, at top level
     File "/Library/Application Support/Perceptive Automation/Indigo 7.3/IndigoPluginHost.app/Contents/Resources/PlugIns/requests/models.py", line 816, in json
       return complexjson.loads(self.text, **kwargs)
     File "/Library/Application Support/Perceptive Automation/Indigo 7.3/IndigoPluginHost.app/Contents/Resources/PlugIns/simplejson/__init__.py", line 307, in loads
       return _default_decoder.decode(s)
     File "/Library/Application Support/Perceptive Automation/Indigo 7.3/IndigoPluginHost.app/Contents/Resources/PlugIns/simplejson/decoder.py", line 335, in decode
       obj, end = self.raw_decode(s, idx=_w(s, 0).end())
     File "/Library/Application Support/Perceptive Automation/Indigo 7.3/IndigoPluginHost.app/Contents/Resources/PlugIns/simplejson/decoder.py", line 353, in raw_decode
       raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Posted on
Mon Jun 29, 2020 11:41 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Converting AppleScript that reads data from remote URL

Actually, that URL is returning HTML with a JSON string embedded in it:

Code: Select all
jkeenan@Kraken ~ % python
>>> import requests
>>> result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
>>> result.text
u' \r\n\r\n\r\n<!DOCTYPE HTML>\r\n\r\n<html>\r\n  <head>\r\n   <meta charset="utf-8">\r\n\r\n    \r\n\r\n  </head>\r\n  <body>\r\n\r\n        \r\n\r\n\r\n\r\n \r\n\r\n\r\n\r\n\r\n\r\n    <table class="table table-hover">\r\n\r\n     \r\n\r\n    {"time": "2020-06-29 17:39:16",\r\n    "CPM": "24", \r\n    "ACPM": "244.54",\r\n    "uSv": "0.16"}\r\n  \r\n\r\n\r\n\r\n \r\n</table>\r\n\r\n\r\n\r\n\r\n  \r\n\r\n\r\n\r\n\r\n\r\n\r\n  </body>\r\n</html>\r\n\r\n'
>>>


That's going to be a lot harder to parse out.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Mon Jun 29, 2020 11:49 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Converting AppleScript that reads data from remote URL

Yeah, you skipped converting this section of your original script;

Code: Select all
      --extract Geiger reading
      set AppleScript's text item delimiters to ","
      set theArray to every text item of myResult
      set radLevelText to item 2 of theArray
     
      -- extract value from text string
      set radLevel to word 2 of radLevelText
      set value of variable "Radiation_Basement" to radLevel
     


Code: Select all
import requests

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
lines = result.text.split(',')
words = lines[1].split(":")
indigo.variable.updateValue(385534937, words[1])

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Mon Jun 29, 2020 12:26 pm
gmusser offline
Posts: 290
Joined: Feb 12, 2005
Location: New Jersey

Re: Converting AppleScript that reads data from remote URL

Ahhhh I naïvely thought the parsing would be automatic. One issue remains: the Indigo variable value includes quotation marks. How do I strip them out? This wasn’t an issue with AppleScript. I tried applying strip() but that leaves the leading quote mark. When I run strip() a second time to try to eliminate that quote mark, it remains. The function remove() is no better. Presumably there is some more elegant solution, anyway.
import requests

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
lines = result.text.split(',')
words = lines[1].split(":")
cpm_value = words[1].strip('"')
cpm_value = cpm_value.strip('"')
indigo.variable.updateValue(385534937, cpm_value)

Posted on
Mon Jun 29, 2020 12:40 pm
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Converting AppleScript that reads data from remote URL


joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Mon Jun 29, 2020 1:28 pm
gmusser offline
Posts: 290
Joined: Feb 12, 2005
Location: New Jersey

Re: Converting AppleScript that reads data from remote URL

Yes, those are the directions I followed, but it’s not working.

Who is online

Users browsing this forum: No registered users and 1 guest