Extending Indigo: Strange problem with json.loads()

I'm developing a plugin for the Heatmiser Neo range of thermostats and have a problem with json decoding. I developed a Vera plugin for the same range of devices a few months ago and didn't encounter this issue in Lua, so I'm wondering what's different about Python. Comms is via TCP; I receive a json reply then use:

Code: Select all

data = json.loads(reply)
return data

The problem is that in something like one time out of four or five I get an error along the lines of:

Traceback (most recent call last):
File "plugin.py", line 60, in runConcurrentThread
File "plugin.py", line 99, in updateReadings
File "plugin.py", line 143, in getNeoData
File "/Library/Application Support/Perceptive Automation/Indigo 6/IndigoPluginHost.app/Contents/PlugIns/simplejson/__init__.py", line 307, in loads
File "/Library/Application Support/Perceptive Automation/Indigo 6/IndigoPluginHost.app/Contents/PlugIns/simplejson/decoder.py", line 338, in decode
ValueError: Extra data: line 2 column 1 - line 2 column 2 (char 2832 - 2833)

The strange part is that, since the thermostat data doesn't change frequently, one reply will be decoded correctly while the next reply (identical to the last in every respect) produces an error. I've attached text file of a typical reply received from the hub. This reply passed, failed and passed again on successive polls despite being unchanged.

Any thoughts would be appreciated!

Its hard to say without seeing the actual file that produced the error because the file you attached is only 2831 characters. If that IS the file then the JSON may be encoded wrong because you are missing or have some rotten character at the end of the string.

The file I posted is 2830 characters long and ends with:

"NeoPlug"}]}

It parses fine in online checkers - and json.loads() is happy with it most of the time.

The strange part is that, since the thermostat data doesn't change frequently, one reply will be decoded correctly while the next reply (identical to the last in every respect) produces an error. I've attached text file of a typical reply received from the hub. This reply passed, failed and passed again on successive polls despite being unchanged.

The same exact reply just shouldn't behave that way, so the logical answer is that something else is in the file... obviously you have looked over the strings and they appear identical -- but there could possibly be out-of-ascii control characters append to the end. I am not sure how Python or the JSON parser would deal with, say, an EOF or null character at the end of the string. I would suggest writing a regex to strip all non-"normal" characters out of the response and see if that helps.

Obviously this is a shot in the dark that I can't test, but it makes sense given that an online or other validator may trim the beginning and ending characters out first or run it through a regular expression to eliminate anything it considers unsafe. Copying to a file could also potentially eliminate a character that we can't see.

Adam

Bingo!

Code: Select all

datak = re.sub('[^\s!-~]', '', dataj)

fixed the issue.

Thanks guys!

I'd be interested in the plugin when you're done.

I've got 2 Heatmiser non-Neo's at the moment, but possibly upgrading to 6 Neo's.


Sent from my iPhone using Tapatalk