UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

[Jann]

Hi all.

Having issues adding or editing ANY of my lights in Hue Lights plugin..

Getting the following:

Code: Select all

Hue Lights Error                Error in plugin execution UiValidate:

Traceback (most recent call last):
  File "plugin.py", line 877, in validateDeviceConfigUi
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 181: ordinal not in range(128)

ONLY when the lamp names in the Hue lamp have apostrophes... (ie: "Jann's Office Lamp") ... When no apostrophe in the same Hue Lamp and I reload Hue Lights (to get new device names) everything works..

Ideas please?

[jay (support)]

I suspect it's a smart apostrophe that's causing the problem.

This is a normal one: '
This is a "smart" one: ’

iOS and macOS like to "fix" them for you...

[nsheldon]

I’m sure Jay is correct. I don’t exactly know how to fix this in the plugin. Python and Unicode haven’t always been the best of friends. Jay: have any pointers on how to get the plugin to not puke when it encounters Unicode in REST data?

[FlyingDiver]

I’m sure Jay is correct. I don’t exactly know how to fix this in the plugin. Python and Unicode haven’t always been the best of friends. Jay: have any pointers on how to get the plugin to not puke when it encounters Unicode in REST data?

Do all your string handling in unicode, not ascii.

However, I can't figure out how the OP is getting that error on that specific line of code. Unless they're not running the current version linked from the plugin store.

[Jann]

I am sure it is the apostrophe being a “smart apostrophe” or some-such...but all the software (indigo & hue lights) are updated.

Seems to me that Hue Lights are assuming EVERYthing that the JSON hue sends is ascii...not Unicode...and therein lies the issue.

[nsheldon]

As far as I can tell, I am handling everything as Unicode in Hue Lights. Unless you can point out examples in the code where I’m not (along with the recommended Unicode alternate).

[FlyingDiver]

I am sure it is the apostrophe being a “smart apostrophe” or some-such...but all the software (indigo & hue lights) are updated.

Seems to me that Hue Lights are assuming EVERYthing that the JSON hue sends is ascii...not Unicode...and therein lies the issue.

Can you turn on debug logging and get that error again? That will show the actual data returned from the hub.

[DaveL17]

I've found that using this construction has basically eliminated Unicode errors for me:

Code: Select all

u"Something interesting: {0}".format(some_var)

But that doesn't always work:

Code: Select all

#! /usr/bin/env python
# -*- coding: utf-8 -*-

some_var = 'º'
print(u"Something interesting: {0}".format(some_var))

Yields:

Code: Select all

Traceback (most recent call last):
  File "untitled text 136", line 5, in <module>
    print(u"Something interesting: {0}".format(some_var))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal not in range(128)

So you can do this:

Code: Select all

some_var = 'º'
print(u"Something interesting: {0}".format(some_var.decode('utf-8')))

Which yields:

Code: Select all

================================================================================
Apr 19, 2018, 6:47:03 AM
untitled text 136
--------------------------------------------------------------------------------
Something interesting: º

So I *think* you'll need to trap on the error and decode the string when needed.

Unicode is a pain.

[FlyingDiver]

This works without the trap:

Code: Select all

#! /usr/bin/env python
# -*- coding: utf-8 -*-

some_var = u'º'
print(u"Something interesting: {0}".format(some_var))

[Jann]

Well, as far as unicode in this plug-in goes, since Hue says the api is unicode, there's no issue with de-unicoding when you shouldn't, right?

So, assuming "ascii" in the plug-in wrt the item names (the names WE gave the hue bulbs) would be invalid, right? Just use the unicode line above when de-unicoding the item name, then somehow keep it unicode wrt Indigo. I wouldn't know how that is done... does Indigo use unicode when displaying names in the drop-down "item chooser" when assigning a Indigo "device" to a third-party "device?"

Jann

[FlyingDiver]

The problem is that the line number from the error message you posted doesn't have any unicode conversion going on. Which is why I wondered if I'm looking at the same version of the plugin you are. I'm looking at 1.6.10.

Also, capturing the debug output will show the JSON data that the plugin is working with, which will also help figure out what's going wrong.

[Jann]

@flyingdiver: app version is 1.6.11

Give me a minute to turn on logging, change the name BACK To apostrophe, then add it and let it give error

Will respond in 10 or so..

[FlyingDiver]

Ah. The plugin store does not have the latest version. I found 1.6.11, and here's the line that's giving the error:

Code: Select all

			self.debugLog(u"Data from hub: " + r.content)

So it's actually the debug statement itself. Changing that to the following will probably fix the error:

Code: Select all

			self.debugLog(u"Data from hub: {}".format(r.content))

[Jann]

Okay I found a LOT of places where "Data from hub: " was unicoded but the r.content was not. Should I change them all to use format(r.content)?

Code: Select all

grep -n '"Data from hub:' "./Hue Lights.indigoPlugin/Contents/Server Plugin/plugin.py":

450:			self.debugLog(u"Data from hub: " + r.content)
554:			self.debugLog(u"Data from hub: " + r.content)
658:			self.debugLog(u"Data from hub: " + r.content)
773:			self.debugLog(u"Data from hub: " + r.content)
877:			self.debugLog(u"Data from hub: " + r.content)
981:			self.debugLog(u"Data from hub: " + r.content)
1145:			self.debugLog(u"Data from hub: " + r.content)
1231:			self.debugLog(u"Data from hub: " + r.content)
1341:			self.debugLog(u"Data from hub: " + r.content)
4784:		self.debugLog(u"Data from hub: " + r.content)
4844:		self.debugLog(u"Data from hub: " + r.content)
4907:		self.debugLog(u"Data from hub: " + r.content)

[FlyingDiver]

Can't hurt. All those other ones are for the different device types, I think. If you only have the one Hue device type you'll never hit that code.