New gitHub Branch:
https://github.com/Ghawken/iMessagePlugin/tree/wit.ai
Beta Functions: From Version 0.2.2
As a mechanism of controlling all indigo devices and requesting information with minimal setup I have turned to a free service wit.ai
The aim of this internet service is to process any text messages you might send and enable the Plugin to reply, recognising what you want to do but also which device to apply this to.
e.g
Aiming to bring to life.. the following with minimal setup.
- Turn off pool Pump.
- Dim the living room light to 20%
- Set Living Room light to 100% brightness
- What is the temperature in Maxs Room?
- Tell me a joke?
- What is the temperature of the pool?
- Can you help with some advice?
- How hot is the Living Room?
- What is Glenn's Location?
Any Natural language can be trained to have the desired result within Indigo. As the Beta progresses can add other example sentences/structures and samples depending on own usage.
Basically Wit.ai - recognises message:
intent, currently device_action, insult, yes_no_decision, joke, dim_set, temperature, advice
& entities such as device_name (which is Indigo Device), plus others such as number/sentiment/on_off etc.
This parsed result is sent back to the plugin to action. (very quickly)
So if message is sent 'Please Indigo could you be a dear and turn on the living room light?'
The (Plugin after suitable wit.ai training/samples) should recognise intent: device_action, device_name:'Living Room light', and on_off = on.
The plugin parses the reply and then Turns on living room light for example.
Within Wit.ai you can train any sentence structure to the correct intent and result. The aim is for the plugin to probably do this at Wit.ai app creation which is what currently happens, obviously others may speak in a different manner. and I can add any samples/sentence structures wished.
It also enables speech->text via iMessage audioMessage, and a few fun extras perhaps I got carried away with.
Consider this a beta - BUT if disabled wit.ai in PluginConfig - rest of plugin unchanged and none of this is run.
NB:
The wit.ai processing of the sent message only occurs after any trigger check.
So if you have a trigger action set for the message 'Gate', this will work unchanged.
After receiving that message the Plugin will active your Trigger and end any thinking about the message.
If you sent 'Indigo please open gate' then that message will be sent to wit.ai and hopefully the same result will occur.
(the benefit here is natural language, and no trigger setup is needed)
Requirements:
Need your own Wit.Ai API Key (free)
We each need our own as our devices are named something different
So while the NLP logic may be the same the devices referred to will be different.
Setup:
New Plugin Config Settings
1. Create Wit.AI Api
- Log in to, with either github or facebook.
https://wit.ai
2/ Next Go to the already existing App called 'MyFirstApp'
- Go to Settings, top Right
- Find the 'Server Access Token' and copy this in the PluginConfig
Enable Use Wit.Ai Processing to use this service for processing of Messages received.
Checkbox: Use all Indigo Devices
- This will send every indigo device name to Wit.Ai to be recognised as a device down the track.
- This is fine if you have limited devices and the names of your devices make sense for control
- But if like me you have hundreds will names like (U) Computer Room Light, or PiBeacon_Pool Temperature
- These names make sending a text message 'what is the PBeacon_Pool Temperature' will work but isn't much fun.
- I have added the ability for WitAI to recognise synonyms for the same device
- eg. device is PiBeacon_Pool Temperature, other names can be 'Pool Temp', 'Pool Temperature' or even 'Pool'
This is performed via the individual Device Notes of each Indigo Device. (once there is re-read)
On the first line of the notes (the other lines are ignored)
a :
- Code: Select all
witai|Pool|Pool Temp|Pool Temperature
- Must start with witai and then Vertical bar Characters separating names
- Sorry the character is the | Vertical Bar Character
eg.
- This will give different names which refer to the same device for all future commands
Coming back to the Checkbox - if this is unchecked only those devices so marked with witai| in device notes will be included.
Okay:
So ideally - give you funny named devices some better names, mark those you want to send, or send all
Then:
1. Press the Button called 'Generate Wit.Ai App' [once we are done only need to do this once]
This will send a whole lot of text based data to Wit.ai naming devices, and setting up you wit.ai device which the plugin uses
You can access this online at wit.ai if needed but really only for finetuning.
This will take a while as can only send a few devices/samples at once; so will take at least 5 if not 10 minutes to be done.
Once finished also takes a while at Wit.Ai end as well - before everything is recognised probably - probably at least 1/2 hour
If any issues or want to resend different devices following some playing - press the delete wit.ai App button and then recreate after a short pause.
Following this
- You should be able to 'turn on INDIGODEVICE'
- 'turn off INDIGODEVICE',
- 'set INDIGODEVICE brightness to 50%',
- 'What is the Temperature of INDIGODEVICE',
- 'Tell me a joke'
- 'What is IFRIENDEVICE location'
- Please give me some advice
- amongst others and does have constant learning on its side... please you can adjust from within wit.ai if there are sentences you use
(the intent in wit.ai is the most important - and should be self-explanatory)
I have also added iMsg AudioMessage speech to text via wit.ai - which works okay within limits of speech recognition.
From within iMSG press and hole down microphone - say command and viola!
Fine Tuning:
- Probably more to come I suspect:
- Basically can add whatever phrase seems to make sense to its samples and can modify and build the training data which is sent when the app is created.
- From within Wit.ai - can go to Inbox and validate audio received for training purposes .
- Can also can to device_name and add any extra synomyms for this device you wish - pays to do it in Indigo Notes as well/so survives delete and recreate of the wit.ai app
Other changes:
- Add send HTTP Image - takes http/web location downloads fine and sends it to buddy.
- Few other fixes/changes
New gitHub Branch:
https://github.com/Ghawken/iMessagePlugin/tree/wit.ai
(Clone or Download)