Voice control of Indigo - anyone interested?

Posted on
Mon Feb 10, 2014 11:36 am
dstrickler offline
User avatar
Posts: 340
Joined: Oct 08, 2010
Location: Boston, MA

Voice control of Indigo - anyone interested?

Although this would seem a rhetorical question, I'm asking because I am developing one, and I am asking to sense the level of interest.

What I have currently working is a text interface that allow you to type in things like "Raise the dinner lights" or "Lower the house by 2 degrees". All the natural language processing works, and I am now going to work on the voice integration so you can say these things instead of type them (although you will always be able to type them).

This interface will never replace Indigo's screens, or even the Indigo iOS app. It's simply an easy way of saying commands like "Set the house in Away mode", which is easier than firing up the interface on your iPhone and tapping around to find your Group that sets the house in Away mode. My goal is to have most of your daily interaction with Indigo done by talking to an app on your iPhone. I like to think of it as a streamlined way of getting things done with Indigo.

Also note that app will only run on iOS and it will require your Indigo server be accessible from the Internet. It is designed in full compliance with Indigo's and Apple's terms, so I doubt there is anything Apple could/would do to break it. It will be free to the those who help me test it, but there will be a small fee once it's up and running.

Any interest in using, and possibly paying for, this product once I am done (hopefully in the summer of 2014, but that's just a guess)?

Dave

Posted on
Mon Feb 10, 2014 1:19 pm
durosity offline
User avatar
Posts: 4320
Joined: May 10, 2012
Location: Newcastle Upon Tyne, Ye Ol' England.

Re: Voice control of Indigo - anyone interested?

Strangely it’s something that my other half was just asking if was possible. I myself find voice recognition technology too unreliable for day to day use (If you ever want to know why watch this: http://www.youtube.com/watch?v=5FFRoYhTJQQ (caution.. adult language)) but my girlfriend has a very crisp clean english voice and would very much be interested in such a facility! I assume it’d be an iOS app of it’s own?

Computer says no.

Posted on
Mon Feb 10, 2014 1:22 pm
mat offline
Posts: 769
Joined: Nov 25, 2010
Location: Cambridgeshire - UK

Re: Voice control of Indigo - anyone interested?

I had a look at this a little while back.

Not sure if there is anything useful here….viewtopic.php?f=11&t=8201

regards


Mat

Late 2018 mini 10.14

Posted on
Mon Feb 10, 2014 1:31 pm
dstrickler offline
User avatar
Posts: 340
Joined: Oct 08, 2010
Location: Boston, MA

Re: Voice control of Indigo - anyone interested?

Yes, I know what you mean about voice commands; tricky in even the best of circumstances. Like auto-correct on phones, they are definitely first generation. The good news is they are getting better all the time, and in my case, I'm using Machine Learning, so they more people that use it, the better it gets. In theory at least ;-)

So yes, it would be an iOS app of it's own, or that's what I am thinking. It would be fun for the Indigo guys used my AP in their existing Indigo Touch version, but I think they are busy enough at this point. I'll suggest that to them when it's all up and running.

Dave

Posted on
Mon Feb 10, 2014 1:39 pm
dstrickler offline
User avatar
Posts: 340
Joined: Oct 08, 2010
Location: Boston, MA

Re: Voice control of Indigo - anyone interested?

mat wrote:
Not sure if there is anything useful here….viewtopic.php?f=11&t=8201


Thanks for the tip on this - good to know others are hacking away at it. I'm using some NLP (Natural Language Processing) calls, so the end result is more flexible. For instance, I taught it "turn on lights" and it now understands that, as well as "turn the lights on, I'm in the dark in here!" without me re-teaching it. Less coding on my part, and a better user experience.

It's not perfect by any means, but as I use it, the machine learning gets more experienced, and the overall experience is better.

Dave

Posted on
Thu Feb 13, 2014 10:03 pm
mundmc offline
User avatar
Posts: 1060
Joined: Sep 14, 2012

Re: Voice control of Indigo - anyone interested?

Hey Dstrickler,
I've been toiling over this for some time (and I'm a poor programer at best). I was using Siri Proxy on a Raspberry Pi with great success... until the protocol was changed in ios7, effectively breaking Siri Proxy (for now).

The main problem I have encountered is easy, quality voice recognition. My latest workaround is utilizing "Notes" on both my iphone and Mac. I prefer Notes to using a "Messages" client on the Mac as I don't have to push the Siri button a second time to confirm sending the message. Basically, I just want to easily pass a spoken string into an indigo variable, and do language parsing in python from there.

I am VERY interested in any natural language processing wisdom you pass on.

In the meantime, here is a hacky workaround for iphones: Press the Siri button and say "Note KEYWORD YOUR COMMAND HERE".
This automatically creates a note with great language recognition, and it syncs via icloud to the host mac.
A client applescript app runs on the host, searching for notes that begin with KEYWORD ("Jarvis" in my case as I'm an Ironman fan). The word "Jarvis" then get's stripped off the title, and the rest of the note gets passed to Indigo via a RESTful call.

Here is the code if anybody finds it helpful:
Code: Select all
on run
   -- just here to get things started...
end run

on idle
   
   tell application "Notes"
      repeat with i from 1 to the count of notes
         set n to note i
         if name of n contains "Jarvis" then
            set note_name to name of n as string
            set note_name to ((characters 8 thru -1 of note_name) as string)
            do shell script "/usr/bin/curl --digest -u USERNAME:PASSWORD -X PUT -d value=" & "'" & note_name & "'" & " http://10.0.1.2:8176/variables/siriproxy_in"
            delete note i
            set noteTitle to "Message received:"
            make new note at folder "Notes" with properties {name:noteTitle, body:note_name}
            
         end if
      end repeat
   end tell
   
   return 1
end idle


Hope somebody finds this useful, i'll stay tuned.

Posted on
Thu Feb 13, 2014 10:35 pm
dstrickler offline
User avatar
Posts: 340
Joined: Oct 08, 2010
Location: Boston, MA

Re: Voice control of Indigo - anyone interested?

Very clever!

Yes, the speech-to-text is not very reliable. If there's something that Siri gets right 95% of the time, it's that. It's impressive.

My code is working well, except for the speech-to-text, but I'm working on that each day. I'll let you know if I can get it close to a Siri quality via the iPhone.

The good news is that the process I am using learns with each command given, so it's getting better all the time. However, I have only tested it with my voice so far.

I'll keep you posted via this thread.

Dave

Posted on
Fri Feb 14, 2014 8:10 am
mundmc offline
User avatar
Posts: 1060
Joined: Sep 14, 2012

Re: Voice control of Indigo - anyone interested?

Outstanding, very excited, happy to help with testing, and happy to donate when up and running!

Posted on
Fri Feb 21, 2014 8:38 am
dstrickler offline
User avatar
Posts: 340
Joined: Oct 08, 2010
Location: Boston, MA

[Update] Voice control of Indigo - anyone interested?

I am progressing with the software, but have hit a snag.

It seems that I'll need to create an iOS app to records voice commands and upload them to my servers for processing. While my testing from my Mac works pretty well, there's nothing like having a real iOS app to capture voice just like the final product will need. Unfortunately, I'm not an iOS developer.

So, while the project is certainly not dead, it may take a little longer to get going as I work around this snag.

Just wanted to keep you all posted,

Dave

Dave

Posted on
Fri Feb 21, 2014 8:51 am
dstrickler offline
User avatar
Posts: 340
Joined: Oct 08, 2010
Location: Boston, MA

Re: Voice control of Indigo - anyone interested?

mundmc wrote:
In the meantime, here is a hacky workaround for iphones: Press the Siri button and say "Note KEYWORD YOUR COMMAND HERE".
This automatically creates a note with great language recognition, and it syncs via icloud to the host mac.
A client applescript app runs on the host, searching for notes that begin with KEYWORD ("Jarvis" in my case as I'm an Ironman fan). The word "Jarvis" then get's stripped off the title, and the rest of the note gets passed to Indigo via a RESTful call.


mundmc,

I thought I would play around with this for a little weekend project, but I had some questions:

1. I assume the Mac this runs on is locally the Indigo server, but it looks like it could go anywhere as long as the Mac has the same iCloud account - right?

2. Where is the script placed? It sounds like it needs to run all the time, so is it put in Indigo with "run every 10 seconds" loop? That's a lot of work for Indigo. I would think some sort of crontab on the Mac would be a better choice, but I've never put an apple script in crontab (just PHP, Perl, etc).

Thanks for the help,

Dave

Dave

Posted on
Sun Feb 23, 2014 5:08 pm
mundmc offline
User avatar
Posts: 1060
Joined: Sep 14, 2012

Re: Voice control of Indigo - anyone interested?

1. Yes and yes. It can run on any mac with a synced icloud account. I tested it on my laptop (not the server) using icloud and changing variables via the RESTful interface, though i have a bug somewhere in my server that causes it to timeout on some RESTful calls, so i prefer to use tell-blocks to command Indigo

2. I realized I have not yet given proper credit to the guy who came up with the idea of using a Note-reader app to control one's mac: http://www.tuaw.com/2013/03/27/talking- ... with-siri/
One just needs to export the applescript as an application that runs on idle, as described above. I keep it on the desktop and just let it run outside of Indigo. This makes it less reliant on Indigo, which is good or bad depending on how you approach it.

I am using the same method to connect IFTTT with Indigo, utilizing the Reminders app:
viewtopic.php?f=13&t=11230

Hope this helped and let me know any thoughts! Again, my programming is very, very rusty and not up to date.

Posted on
Sun Feb 23, 2014 8:20 pm
dstrickler offline
User avatar
Posts: 340
Joined: Oct 08, 2010
Location: Boston, MA

Re: Voice control of Indigo - anyone interested?

mundmc wrote:
One just needs to export the applescript as an application that runs on idle, as described above.


I know I'm pretty clueless about Applescript, but I don't know what "runs on idle" is. It sounds like you use the Applescript editor to compile the script into an executable, but then how do I tell it to "run on idle"?

I'm afraid I am a linux programmer, and things like that on a Mac are a bit of a mystery.

Thanks,

Dave

Posted on
Mon Feb 24, 2014 7:36 pm
mundmc offline
User avatar
Posts: 1060
Joined: Sep 14, 2012

Re: Voice control of Indigo - anyone interested?

Yes, you use the applescript editor:
1) instead of saving the file, click "export"
2) choose to export as "application"
3) as an application, you can click the "run on idle button"
4) fire up the applet you just made, and it will run in the system tray

I have NO idea how the OS decides timing and priority, but it hasn't been an issue

Posted on
Mon Feb 24, 2014 7:40 pm
dstrickler offline
User avatar
Posts: 340
Joined: Oct 08, 2010
Location: Boston, MA

Re: Voice control of Indigo - anyone interested?

Wow - not being an Applescript guy, I have never tried this. Good for Apple for making it so easy!

I don't have time to try this tonight, but it looks too easy.

Thanks!

Dave

Posted on
Wed Feb 26, 2014 2:09 am
mundmc offline
User avatar
Posts: 1060
Joined: Sep 14, 2012

Re: Voice control of Indigo - anyone interested?

Full credit goes here: http://www.tuaw.com/2013/03/27/talking- ... with-siri/

It's also a good tutorial.

I realize i linked out to this post on another thread but not here!

Who is online

Users browsing this forum: No registered users and 4 guests