Page 1 of 2

Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 10:08 am
by gmusser
I'm on to my next AppleScript conversion, which retrieves data that one of my environmental sensors uploads to a remote website. The data is returned in an array that looks like this:
{"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

The AppleScript I've been using is:
Code: Select all
tell application "IndigoServer"
   
   --retrieve from GMC Map
   set TheCommand to "curl -m 5 --request GET http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812"
   
   try
      set myResult to do shell script TheCommand
      
      --extract Geiger reading
      set AppleScript's text item delimiters to ","
      set theArray to every text item of myResult
      set radLevelText to item 2 of theArray
      
      -- extract value from text string
      set radLevel to word 2 of radLevelText
      set value of variable "Radiation_Basement" to radLevel
      
   on error
      log "Error executing curl to retrieve from gmcmap.com"
   end try
   
end tell

Based on examples in the Indigo forums, I tried this Python code:
Code: Select all
import requests
import simplejson

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
response = simplejson.loads(result)
indigo.variable.updateValue(385534937, response["CPM"])

But I get this error:
Script Error GMC Map.py: expected string or buffer
Script Error Exception Traceback (most recent call shown last):

GMC Map.py, line 8, at top level
File "/Library/Application Support/Perceptive Automation/Indigo 7.3/IndigoPluginHost.app/Contents/Resources/PlugIns/simplejson/__init__.py", line 307, in loads
return _default_decoder.decode(s)
File "/Library/Application Support/Perceptive Automation/Indigo 7.3/IndigoPluginHost.app/Contents/Resources/PlugIns/simplejson/decoder.py", line 335, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer

I'm not sure what to do now. Thanks for your help. It is greatly appreciated!

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 10:23 am
by jay (support)
Try this for the second to last line:

Code: Select all
response = result.json()


The requests library knows how to automatically decode json for you. Your version might have worked if you passed result.text to the simplejson.loads function.

FYI, when using the json library you should:

Code: Select all
import json


We included simplejson because when we first released Python support, Python 2.5 was the standard version on Macs and it didn't have the built-in library. If you are still running on a very old Mac OS version (before Yosemite I think) then you should continue to use simplejson.

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 10:24 am
by FlyingDiver
The result variable is a complex object containing all the info about the request call. You need:

Code: Select all
import requests
import simplejson

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
response = simplejson.loads(result. text)
indigo.variable.updateValue(385534937, response["CPM"])


But you don't even need to do that much, as requests can do the JSON for you:

Code: Select all
import requests

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
response = result.json
indigo.variable.updateValue(385534937, response["CPM"])

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 10:55 am
by gmusser
Ah, that clarifies matter. But the revised code still give an error that, for a noobie, is incomprehensible:
Script Error GMC Map.py: 'instancemethod' object is not subscriptable
Script Error Exception Traceback (most recent call shown last):

GMC Map.py, line 8, at top level
TypeError: 'instancemethod' object is not subscriptable

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 10:59 am
by FlyingDiver
gmusser wrote:
Ah, that clarifies matter. But the revised code still give an error that, for a noobie, is incomprehensible:
Script Error GMC Map.py: 'instancemethod' object is not subscriptable
Script Error Exception Traceback (most recent call shown last):

GMC Map.py, line 8, at top level
TypeError: 'instancemethod' object is not subscriptable


Post your complete script. You probably have a typo.

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 11:02 am
by gmusser
It is a direct copy of the above.

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 11:03 am
by FlyingDiver
gmusser wrote:
It is a direct copy of the above.


Which one above? There are multiple scripts. And at least one (of mine) has an errant space in it (that I see now).

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 11:10 am
by gmusser
Code: Select all
import requests

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
response = result.json
indigo.variable.updateValue(385534937, response["CPM"])

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 11:12 am
by FlyingDiver
Oops. Jay had it right. Should be:

Code: Select all
import requests

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
response = result.json()
indigo.variable.updateValue(385534937, response["CPM"])

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 11:37 am
by gmusser
Great, I added the parentheses and an "import json" line. But now I get the following error. I checked the URL and it is properly returning output in the format I mentioned.
Code: Select all
 Script Error                    GMC Map.py: No JSON object could be decoded
   Script Error                    Exception Traceback (most recent call shown last):

     GMC Map.py, line 7, at top level
     File "/Library/Application Support/Perceptive Automation/Indigo 7.3/IndigoPluginHost.app/Contents/Resources/PlugIns/requests/models.py", line 816, in json
       return complexjson.loads(self.text, **kwargs)
     File "/Library/Application Support/Perceptive Automation/Indigo 7.3/IndigoPluginHost.app/Contents/Resources/PlugIns/simplejson/__init__.py", line 307, in loads
       return _default_decoder.decode(s)
     File "/Library/Application Support/Perceptive Automation/Indigo 7.3/IndigoPluginHost.app/Contents/Resources/PlugIns/simplejson/decoder.py", line 335, in decode
       obj, end = self.raw_decode(s, idx=_w(s, 0).end())
     File "/Library/Application Support/Perceptive Automation/Indigo 7.3/IndigoPluginHost.app/Contents/Resources/PlugIns/simplejson/decoder.py", line 353, in raw_decode
       raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 11:41 am
by FlyingDiver
Actually, that URL is returning HTML with a JSON string embedded in it:

Code: Select all
jkeenan@Kraken ~ % python
>>> import requests
>>> result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
>>> result.text
u' \r\n\r\n\r\n<!DOCTYPE HTML>\r\n\r\n<html>\r\n  <head>\r\n   <meta charset="utf-8">\r\n\r\n    \r\n\r\n  </head>\r\n  <body>\r\n\r\n        \r\n\r\n\r\n\r\n \r\n\r\n\r\n\r\n\r\n\r\n    <table class="table table-hover">\r\n\r\n     \r\n\r\n    {"time": "2020-06-29 17:39:16",\r\n    "CPM": "24", \r\n    "ACPM": "244.54",\r\n    "uSv": "0.16"}\r\n  \r\n\r\n\r\n\r\n \r\n</table>\r\n\r\n\r\n\r\n\r\n  \r\n\r\n\r\n\r\n\r\n\r\n\r\n  </body>\r\n</html>\r\n\r\n'
>>>


That's going to be a lot harder to parse out.

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 11:49 am
by FlyingDiver
Yeah, you skipped converting this section of your original script;

Code: Select all
      --extract Geiger reading
      set AppleScript's text item delimiters to ","
      set theArray to every text item of myResult
      set radLevelText to item 2 of theArray
     
      -- extract value from text string
      set radLevel to word 2 of radLevelText
      set value of variable "Radiation_Basement" to radLevel
     


Code: Select all
import requests

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
lines = result.text.split(',')
words = lines[1].split(":")
indigo.variable.updateValue(385534937, words[1])

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 12:26 pm
by gmusser
Ahhhh I naïvely thought the parsing would be automatic. One issue remains: the Indigo variable value includes quotation marks. How do I strip them out? This wasn’t an issue with AppleScript. I tried applying strip() but that leaves the leading quote mark. When I run strip() a second time to try to eliminate that quote mark, it remains. The function remove() is no better. Presumably there is some more elegant solution, anyway.
import requests

# output from gmcmap.com looks like this:
# {"time": "2019-08-04 10:44:53", "CPM": "19", "ACPM": "18.22", "uSv": "0.12"}

result = requests.get("http://www.gmcmap.com/historyData-plain.asp?Param_ID=68108722812")
lines = result.text.split(',')
words = lines[1].split(":")
cpm_value = words[1].strip('"')
cpm_value = cpm_value.strip('"')
indigo.variable.updateValue(385534937, cpm_value)

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 12:40 pm
by FlyingDiver

Re: Converting AppleScript that reads data from remote URL

PostPosted: Mon Jun 29, 2020 1:28 pm
by gmusser
Yes, those are the directions I followed, but it’s not working.