ChatGPT powered Siri

I'm not at all certain I'm smart enough to put this all together, but just out of curiosity what kind of costs are you looking at for API access to make this work?

Apologies if I've misunderstood the question.

Apologies if I've misunderstood the question.

I am not sure I have understood the question!

MY goals were to have a smarter more intuitive way to talk to my home and get devices, lights , thermostats, AV equipment controlled. Some smart scheduling would be nice.

We need it to be able to action changes, answer questions on state, and maybe do both, ie "turn the thermostat in the kitchen up 2 degrees".
We can put logic into ChatGPT, Shortcuts or Indigo.
We can control devices via Indigo API or devices published into Homekit, and likewise respond to queries about device state from both.

Having it all in Shortcuts using only Homekit published devices, and action groups etc as devices makes it simpler potentially, assuming we can parameterise the names, otherwise doing if statements in shortcuts for each room name is painful.
Indigo gives us more flexibility with python but needs device id mappings.
Shortcuts gives us some scheduling options, maybe... I am not sure how we can dynamically ask Indigo to turn on a device in 30 minutes?

As to what is the best solution? I think that is what we are investigating. So yes, we can try and use the Indigo API into Shortcuts to pull states there to have displayed or spoken, or we can keep it simple fo rthose use cases and use Homekit devices.

I'm not at all certain I'm smart enough to put this all together, but just out of curiosity what kind of costs are you looking at for API access to make this work?

Not a huge amount assuming its only used a handful of times a day, unless one feels the need to pay for the $20/mo premium access for faster response and dedicated access.

If the kids get hold of it and start using... that is anyones guess.

[jay (support)]

Apologies if I've misunderstood the question.

I am not sure I have understood the question!

MY goals were to have a smarter more intuitive way to talk to my home and get devices, lights , thermostats, AV equipment controlled. Some smart scheduling would be nice.

My current thinking is this: if you want a few/limited number of things you want to tell Siri to do, then use the HTTP API in your shortcuts. If you have a shortcut that is identical but based on a different device (and again, just a few) then you can just duplicate the shortcut with a different name and change the ID. For instance, if you have 2 devices (Office Lamp and Dining Room) that you want to toggle, implement the shortcut for one (name it "toggle office lamp") and embed the ID, then duplicate it, change the name to "toggle dining room" and change the embedded ID. That gives you simple control of two devices, which may be all you need, so it's probably a lower entry for siri control. Of course, there are variations on the theme, and you can do the same with shortcuts that are a little more complex, etc. I think the key here is how much you want the shortcut to do and how many you need.

If, however, you have more advanced needs, then you probably want to use the HomeKitLink Siri plugin so your devices are published to HomeKit and you can use those in your shortcuts. This has the advantage of working directly based on the spoken name of the device, so you don't need a shortcut name that embeds the device (or action group or variable) name in the name of the shortcut and embed the ID in the shortcut logic (providing the map).

If you need lots of logic, I suggest that it's probably better to try to do that in Indigo itself, to encapsulate logic into one environment vs in multiple places. This is just more of a personal preference for me perhaps, but I find it better to not have to look in Indigo and in the Shortcuts app to find something that might be doing some kind of logic that's causing unintended consequences.

And, honestly, I don't see any advantage to ChatGPT, but perhaps I'm biased by its ability to authoritatively tell you something that is patently wrong.

You said it better than I could.

I think use of Homekit Siri plugin is the simplest to just get basic use with a more intelligent AI but to be seen if we can parameterise the names. Ie ChatGPT returns "turn of office lamp" or turn off all kitchen lights" and we need that LinkedIn shortcuts to the right device without 53 If statements.

Yes if it needs more Al in indigo so not debugging 2 places.

Reason for it is that it can infer the correct action from a request where Siri (and Alexa etc) would just get it wrong or say they don't understand.

[jay (support)]

Reason for it is that it can infer the correct action from a request where Siri (and Alexa etc) would just get it wrong or say they don't understand.

I'm just not convinced. But I guess time will tell.

Nice.. I wonder if we can set up a github for sharing updated code, maybe easier than continually posting on here? Would like to see where you have got to..

https://github.com/anyone2/ChatIndigo

Look at intents.py for how I explained it to ChatGPT. It can be done better than this I am sure.

My plan is to implement a connection to the newIndigoAPI.py to allow ChatGPT to perform the action it wants and then with some input, refine the intent.

Can someone post a public link to their iOS shortcut, so I can catchup with where everyone else is with that?

Thanks.

Cheers. I will post my shortcut tomorrow.

Could we make a request to Indigo before calling ChatGPT to get the list of devices to deviceid and insert into the URL call?

I have such a low bar with this. All I want is it not have to say an exact phrase to do something.

If there's something else that does that, I'll play with that also.

Just FYI, Jay has posted a public shortcut to the Indigo File Library.

Hopefully, many more will appear there over time.

[MODERATOR NOTE]: fixed the link to go directly to the library entry.

Hi all,

Reading the thread with interest..

I currently do something similar with wit.ai (natural language processing, predating chatGPT by 5 years or so). Some years ago with iMessage plugin I added support for it - basically enabling natural language commands to be sent for device control. Suspect hasn’t got much use, but was interesting to tinker with and some of the other aspects have had some usage in my household.

If I follow the flow correctly:

Sending to chatGPT a setup/household details etc. and then asking natural language questions. Getting output back in json and parsing this through Shortcuts??
(pretty much all done in Shortcuts?)

Would it be quicker to send the questions to a plugin via eg. plugin /message/ command - have the a plugin send to chatGPT (?other free options even) and then action the result. This probably would mean same shortcut for all (API bearer code aside)

There may be an issue then feeding that back to Shortcuts and user though? Wonder whether can send http via shortcuts and wait for reply? Presumably that is what happens with chatGPT now - so should be doable.

[jay (support)]

Wonder whether can send http via shortcuts and wait for reply?

Yes. Check out the example shortcut I posted in the Library. The new HTTP API is well suited for use from the Get Contents of URL shortcut action.

Hi all,

Wonder how is everyone’s experience that is trying this?

I spent a bit of time supporting chatGPT within iMessage plugin - relatively simple drop in replacement for current Natural language processing wit.ai model that predates and is supported. I will do my best to fine tune this and release, this like the wit.ai stuff sits to one side unless enabled.

Working well: all the chatGPT chatting bits
iMsg to ‘Indigo’: “Whats the capital of Turkey?”
iMsg to ‘Indigo’: “ Tell me a joke”
iMsg to ‘Indigo’: “Write a year 10 essay on modernism in 400 words.”
Working about 90% of the time:
iMsg to “Indigo”: “Turn on Kitchen main lights”

Feedback

- ChatGPT API has a new model gpt-turbo-3.5 with different setup mechanism (via user/system roles) x10 cheaper than davinci-text. Seems similar although not quite the same in my testing.
- chatGPT doesn’t remember any prior info - so you need to send the whole setup info each time, including deviceID and aliases. This leads to big requests and quite a lot of token usage (cost only a few cents still though from a day of heavy testing)
- ChatGPT often, and I mean often (probably 1 in 5 - 1 in returns misformed JSON which can’t be parsed without a crash. Typically it seems to want to add something before the starting {. Catching this and editing out fixes this about 1/2 the times - but a awful lot of time the json is unreadable. Or it forgets and sends plain text.
eg. ask it to “turn on light”. without specifying location - should ask for clarification - but as is often the case seems to just make up a location and/or more likely to mis form JSON in these cases.
- the new gpt-turbo-3.5 model seems slightly worse at creating json in my testing. It does seem more targeted to language (not surprising for a language model). If this trend continues then JSON output won’t really be usable anytime soon I would guess
- It doesn’t use correct JSON quotes if single or double quotes in text given will typically be invalid JSON.

- for device control it seems in my limited tested that wit.ai is probably more accurate for standard commands. wit.ai now needs a facebook login to create Api-Key annoyingly - thankfully my older ones still work. The setup for Wit.ai is far more complicated (but that is all done by plugin), needs devices, entities, intents created within an online app which plugin creates (just needs API key), at start the plugin sends hundreds of examples - based on devices marked as per plugin’s documentation.

I suspect the AI future probably still is HomeKit and Siri/Apple will expand leaps and bounds based on this and Google competitor, and apples own in-house AI projects…

But interested in how accurate others are finding it?

I’ll see if can tidy up plugin, make sure supporting both davinci and gpt-turbo-3.5, and see if can add support for a /message/ url link to send messages in and out for use in shortcuts…

Glenn


Sent from my iPad using Tapatalk

I have to say I've probably only issued <50 commands in test so not really tested reliability and accuracy yet.
However I just ask it "turn on the light" and it replied ok, which light do you want and also has pop up of all my rooms to choose from.
I've also not seen any misformed JSON yet either.