Handling Non Ascii Characters

Posted on
Thu Mar 15, 2018 3:03 pm
Colorado4Wheeler offline
User avatar
Posts: 2794
Joined: Jul 20, 2009
Location: Colorado

Handling Non Ascii Characters

I seem to get hit with this all the time and I'm sure that it's something simple but what are you guys doing to prevent non-ascii characters from throwing errors when you reference a device name? Between the logging and everything else it seems like a constant chase and I'm wondering if other plugin devs have a particular way they reference, say, dev.name without having to always do unicode(dev.name) to make sure it does't puke.

My Modest Contributions to Indigo:

HomeKit Bridge | Device Extensions | Security Manager | LCD Creator | Room-O-Matic | Smart Dimmer | Scene Toggle | Powermiser | Homebridge Buddy

Check Them Out Here

Posted on
Thu Mar 15, 2018 3:07 pm
FlyingDiver offline
User avatar
Posts: 7190
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Handling Non Ascii Characters

I guess it depends on how you "reference it". If you want to print or log it, then I would unicode() it. The point is you need to make sure the libraries know it's possibly unicode and not ascii, which seems to be the default. I think there's a way to set the default once in the python app, but I don't know the syntax.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Thu Mar 15, 2018 3:08 pm
Colorado4Wheeler offline
User avatar
Posts: 2794
Joined: Jul 20, 2009
Location: Colorado

Re: Handling Non Ascii Characters

FlyingDiver wrote:
I guess it depends on how you "reference it". If you want to print or log it, then I would unicode() it.

Bearer of good news eh? :P. I was hoping there was some other method than that. I'll just have to be super careful about when I reference the name in my plugins and always unicode it.

My Modest Contributions to Indigo:

HomeKit Bridge | Device Extensions | Security Manager | LCD Creator | Room-O-Matic | Smart Dimmer | Scene Toggle | Powermiser | Homebridge Buddy

Check Them Out Here

Posted on
Thu Mar 15, 2018 3:12 pm
FlyingDiver offline
User avatar
Posts: 7190
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Handling Non Ascii Characters

Just for a test, try putting this at the start of the .py file:

Code: Select all
#!/usr/bin/env python
# -*- coding: utf-8 -*-


And see if it all works without the unicode() calls.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Thu Mar 15, 2018 3:16 pm
Colorado4Wheeler offline
User avatar
Posts: 2794
Joined: Jul 20, 2009
Location: Colorado

Re: Handling Non Ascii Characters

FlyingDiver wrote:
Just for a test, try putting this at the start of the .py file:

Code: Select all
#!/usr/bin/env python
# -*- coding: utf-8 -*-


And see if it all works without the unicode() calls.


That was the first thing I tried. I mean, it should be there anyway if I'm being proper about Py coding in pre 3.0 Python (guilty as charged) but it doesn't fix it. I've tried that in several plugins that users complain about the non-ascii issue on. I did just try it again just for giggles and forced a foreign character and no love.

My Modest Contributions to Indigo:

HomeKit Bridge | Device Extensions | Security Manager | LCD Creator | Room-O-Matic | Smart Dimmer | Scene Toggle | Powermiser | Homebridge Buddy

Check Them Out Here

Posted on
Thu Mar 15, 2018 3:23 pm
DaveL17 offline
User avatar
Posts: 6744
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Handling Non Ascii Characters

I unicode() everything. In fact I go one step further and
Code: Select all
u”{0}”.format(x)
wherever possible.

Tempting the fates, those errors have been silent for me since I started doing that.


Sent from my iPhone using Tapatalk

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Thu Mar 15, 2018 3:32 pm
FlyingDiver offline
User avatar
Posts: 7190
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Handling Non Ascii Characters

Colorado4Wheeler wrote:
That was the first thing I tried. I mean, it should be there anyway if I'm being proper about Py coding in pre 3.0 Python (guilty as charged) but it doesn't fix it. I've tried that in several plugins that users complain about the non-ascii issue on. I did just try it again just for giggles and forced a foreign character and no love.


Yeah, I was afraid of that. It's really meant to make sure that unicode in the code (like unicode characters in variable names) works. I suspect that the real solution to this would be for all the C++ interfaces to the server to force strings to Unicode. I think.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Thu Mar 15, 2018 3:33 pm
Colorado4Wheeler offline
User avatar
Posts: 2794
Joined: Jul 20, 2009
Location: Colorado

Re: Handling Non Ascii Characters

DaveL17 wrote:
I unicode() everything. In fact I go one step further and
Code: Select all
u”{0}”.format(x)
wherever possible.

Tempting the fates, those errors have been silent for me since I started doing that.


Sent from my iPhone using Tapatalk

That's what I just did across the entire plugin. I've always regarded the 'u' as not really needed but I'm changing my thinking on that now, I'm going to use it on any string that hits Indigo I think.

My Modest Contributions to Indigo:

HomeKit Bridge | Device Extensions | Security Manager | LCD Creator | Room-O-Matic | Smart Dimmer | Scene Toggle | Powermiser | Homebridge Buddy

Check Them Out Here

Posted on
Thu Mar 15, 2018 3:34 pm
FlyingDiver offline
User avatar
Posts: 7190
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Handling Non Ascii Characters

Colorado4Wheeler wrote:
That's what I just did across the entire plugin. I've always regarded the 'u' as not really needed but I'm changing my thinking on that now, I'm going to use it on any string that hits Indigo I think.


I need to do better about that myself.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Thu Mar 15, 2018 3:37 pm
Colorado4Wheeler offline
User avatar
Posts: 2794
Joined: Jul 20, 2009
Location: Colorado

Re: Handling Non Ascii Characters

FlyingDiver wrote:
I need to do better about that myself.

Yea, I'm super bad with it. It's not just the u"" stuff but I realize there are so many places where this pops up, like model and submodel as well (if and when you reference those as I do in HKB). It's becoming quite the chase to find them all. It's kind of like fixing a little problem with a huge sledgehammer but if it works and teaches me to be less code-lazy then it's probably a good lesson.

My Modest Contributions to Indigo:

HomeKit Bridge | Device Extensions | Security Manager | LCD Creator | Room-O-Matic | Smart Dimmer | Scene Toggle | Powermiser | Homebridge Buddy

Check Them Out Here

Posted on
Thu Mar 15, 2018 3:55 pm
DaveL17 offline
User avatar
Posts: 6744
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Handling Non Ascii Characters

I actually like format(x) very much. It has powerful format specifiers that are more intuitive to me than those for str(x). I sometimes get complaints from the PyCharm syntax checker--like "expected string but got type unicode instead"--but those still seem to be handled by the interpreter in most cases.

To me, this is very easy to see what's going on:
Code: Select all
u"{0:*^80}".format(" Hello World ")

>>>  ********************************** Hello World *********************************

and many of the things it can do aren't available with %s formatting.

This is a nice summary:
https://pyformat.info

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Thu Mar 15, 2018 4:11 pm
Colorado4Wheeler offline
User avatar
Posts: 2794
Joined: Jul 20, 2009
Location: Colorado

Re: Handling Non Ascii Characters

I love format(), it really reads well when you have to review your code. Doing string concatenating is horrible to read. But, yea, the extras that format offers is great. I feel the same way about .NET format too and for the same reasons.

My Modest Contributions to Indigo:

HomeKit Bridge | Device Extensions | Security Manager | LCD Creator | Room-O-Matic | Smart Dimmer | Scene Toggle | Powermiser | Homebridge Buddy

Check Them Out Here

Posted on
Thu Mar 15, 2018 7:08 pm
jay (support) offline
Site Admin
User avatar
Posts: 18200
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Handling Non Ascii Characters

DaveL17 wrote:
I unicode() everything. In fact I go one step further and
Code: Select all
u”{0}”.format(x)
wherever possible.

Tempting the fates, those errors have been silent for me since I started doing that.


Yep, exactly. Every string that comes out of Indigo is a unicode string. When there are just latin characters, Python helpfully converts it for you when you use formatting into a string not specifically marked as a unicode string. But, when there are other unicode characters in them, that implicit conversion doesn't work. So the safest bet is to always format into a unicode string.

I'm also not particularly consistent at doing this, but it is a worthwhile habit to form.

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Page 1 of 1

Who is online

Users browsing this forum: No registered users and 4 guests