UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

Posted on
Wed Apr 18, 2018 3:38 pm
Jann offline
Posts: 120
Joined: Mar 12, 2006
Location: Auburndale, FL

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

Hi all.

Having issues adding or editing ANY of my lights in Hue Lights plugin..

Getting the following:
Code: Select all
Hue Lights Error                Error in plugin execution UiValidate:

Traceback (most recent call last):
  File "plugin.py", line 877, in validateDeviceConfigUi
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 181: ordinal not in range(128)


ONLY when the lamp names in the Hue lamp have apostrophes... (ie: "Jann's Office Lamp") ... When no apostrophe in the same Hue Lamp and I reload Hue Lights (to get new device names) everything works..

Ideas please?

Posted on
Wed Apr 18, 2018 4:31 pm
jay (support) offline
Site Admin
User avatar
Posts: 18199
Joined: Mar 19, 2008
Location: Austin, Texas

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

I suspect it's a smart apostrophe that's causing the problem.

This is a normal one: '
This is a "smart" one: ’

iOS and macOS like to "fix" them for you... :roll:

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Posted on
Wed Apr 18, 2018 8:11 pm
nsheldon offline
Posts: 2469
Joined: Aug 09, 2010
Location: CA

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

I’m sure Jay is correct. I don’t exactly know how to fix this in the plugin. Python and Unicode haven’t always been the best of friends. :wink: Jay: have any pointers on how to get the plugin to not puke when it encounters Unicode in REST data?

Posted on
Thu Apr 19, 2018 12:12 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

nsheldon wrote:
I’m sure Jay is correct. I don’t exactly know how to fix this in the plugin. Python and Unicode haven’t always been the best of friends. :wink: Jay: have any pointers on how to get the plugin to not puke when it encounters Unicode in REST data?


Do all your string handling in unicode, not ascii.

However, I can't figure out how the OP is getting that error on that specific line of code. Unless they're not running the current version linked from the plugin store.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Thu Apr 19, 2018 12:51 am
Jann offline
Posts: 120
Joined: Mar 12, 2006
Location: Auburndale, FL

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

I am sure it is the apostrophe being a “smart apostrophe” or some-such...but all the software (indigo & hue lights) are updated.

Seems to me that Hue Lights are assuming EVERYthing that the JSON hue sends is ascii...not Unicode...and therein lies the issue.

Posted on
Thu Apr 19, 2018 1:43 am
nsheldon offline
Posts: 2469
Joined: Aug 09, 2010
Location: CA

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

As far as I can tell, I am handling everything as Unicode in Hue Lights. Unless you can point out examples in the code where I’m not (along with the recommended Unicode alternate).

Posted on
Thu Apr 19, 2018 5:45 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

Jann wrote:
I am sure it is the apostrophe being a “smart apostrophe” or some-such...but all the software (indigo & hue lights) are updated.

Seems to me that Hue Lights are assuming EVERYthing that the JSON hue sends is ascii...not Unicode...and therein lies the issue.


Can you turn on debug logging and get that error again? That will show the actual data returned from the hub.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Thu Apr 19, 2018 5:49 am
DaveL17 offline
User avatar
Posts: 6742
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

I've found that using this construction has basically eliminated Unicode errors for me:
Code: Select all
u"Something interesting: {0}".format(some_var)

But that doesn't always work:
Code: Select all
#! /usr/bin/env python
# -*- coding: utf-8 -*-

some_var = 'º'
print(u"Something interesting: {0}".format(some_var))

Yields:
Code: Select all
Traceback (most recent call last):
  File "untitled text 136", line 5, in <module>
    print(u"Something interesting: {0}".format(some_var))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal not in range(128)

So you can do this:
Code: Select all
some_var = 'º'
print(u"Something interesting: {0}".format(some_var.decode('utf-8')))

Which yields:
Code: Select all
================================================================================
Apr 19, 2018, 6:47:03 AM
untitled text 136
--------------------------------------------------------------------------------
Something interesting: º

So I *think* you'll need to trap on the error and decode the string when needed.

Unicode is a pain.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Thu Apr 19, 2018 5:51 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

This works without the trap:

Code: Select all
#! /usr/bin/env python
# -*- coding: utf-8 -*-

some_var = u'º'
print(u"Something interesting: {0}".format(some_var))

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Thu Apr 19, 2018 10:54 am
Jann offline
Posts: 120
Joined: Mar 12, 2006
Location: Auburndale, FL

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

Well, as far as unicode in this plug-in goes, since Hue says the api is unicode, there's no issue with de-unicoding when you shouldn't, right?

So, assuming "ascii" in the plug-in wrt the item names (the names WE gave the hue bulbs) would be invalid, right? Just use the unicode line above when de-unicoding the item name, then somehow keep it unicode wrt Indigo. I wouldn't know how that is done... does Indigo use unicode when displaying names in the drop-down "item chooser" when assigning a Indigo "device" to a third-party "device?"

Jann

Posted on
Thu Apr 19, 2018 10:59 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

The problem is that the line number from the error message you posted doesn't have any unicode conversion going on. Which is why I wondered if I'm looking at the same version of the plugin you are. I'm looking at 1.6.10.

Also, capturing the debug output will show the JSON data that the plugin is working with, which will also help figure out what's going wrong.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Thu Apr 19, 2018 11:03 am
Jann offline
Posts: 120
Joined: Mar 12, 2006
Location: Auburndale, FL

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

@flyingdiver: app version is 1.6.11

Give me a minute to turn on logging, change the name BACK To apostrophe, then add it and let it give error

Will respond in 10 or so..

Posted on
Thu Apr 19, 2018 11:15 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

Ah. The plugin store does not have the latest version. I found 1.6.11, and here's the line that's giving the error:

Code: Select all
         self.debugLog(u"Data from hub: " + r.content)


So it's actually the debug statement itself. Changing that to the following will probably fix the error:

Code: Select all
         self.debugLog(u"Data from hub: {}".format(r.content))

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Thu Apr 19, 2018 11:34 am
Jann offline
Posts: 120
Joined: Mar 12, 2006
Location: Auburndale, FL

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

Okay I found a LOT of places where "Data from hub: " was unicoded but the r.content was not. Should I change them all to use format(r.content)?

Code: Select all
grep -n '"Data from hub:' "./Hue Lights.indigoPlugin/Contents/Server Plugin/plugin.py":

450:         self.debugLog(u"Data from hub: " + r.content)
554:         self.debugLog(u"Data from hub: " + r.content)
658:         self.debugLog(u"Data from hub: " + r.content)
773:         self.debugLog(u"Data from hub: " + r.content)
877:         self.debugLog(u"Data from hub: " + r.content)
981:         self.debugLog(u"Data from hub: " + r.content)
1145:         self.debugLog(u"Data from hub: " + r.content)
1231:         self.debugLog(u"Data from hub: " + r.content)
1341:         self.debugLog(u"Data from hub: " + r.content)
4784:      self.debugLog(u"Data from hub: " + r.content)
4844:      self.debugLog(u"Data from hub: " + r.content)
4907:      self.debugLog(u"Data from hub: " + r.content)

Posted on
Thu Apr 19, 2018 11:35 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2

Can't hurt. All those other ones are for the different device types, I think. If you only have the one Hue device type you'll never hit that code.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Who is online

Users browsing this forum: No registered users and 0 guests