ChatGPT powered Siri

All you Brits sound alike to me.

Ach wheesht, awa ‘n boil yer heed!


Sent from my iPhone using Tapatalk Pro

ELEVEN!!!!


Sent from my iPhone using Tapatalk Pro

Haha don’t worry [mention]DaveL17 [/mention] ChatGPT is the easy part so yes to make it work with indigo it is about shortcuts and the current and new API.

Wow. And I thought cutting and pasting the indigo api instructions to teach it was a bit ambitious!

Yea yours would be pretty cool, but it would have to be able to create triggers and schedules etc which I’m guessing is not on the api functionality yet

Or just have your own instance running 24/7.
Would that need an M1 or M2 mini?

I take it back. I told ChatGPT about indigo and gave it the link to the rest api instructions and got this back.
It gets it wrong about how to identify a light but it’s clearly tried to understand and come up with its own example. It has also realised that 127.0.0.1 in the examples is the indigo server.

Code: Select all

To make changes to smart home devices using the Indigo RESTful API, you would need to send HTTP requests to the appropriate URLs with the necessary parameters. For example, to turn on a light with an ID of 123, you would send an HTTP GET request to the following URL:

http://<indigo_server>:<indigo_port>/actions/light?lightID=123&brightness=100&isOn=true

This URL includes parameters such as the light ID, brightness level, and whether the light is on or off. You would need to replace <indigo_server> with the IP address or hostname of the Indigo server, and <indigo_port> with the port number that Indigo is listening on.



Just to post one more thing about Apple Shortcuts:

A few examples of Using Apple Shortcuts with Indigo are now live. If you're familiar with using Shortcuts with the RESTful API, converting should be pretty straightforward.

Cheers Dave. It’s going to be a busy few days.

Excited to see examples of how people use new shortcuts capabilities.
I’m disappointed to see that Apple hasn’t created integration with Screentime though.

We don't use voice in our house -- no Home Pods or anything like that -- but I do get the appeal!

That is an interesting point.
We (two adults and two children) rely a lot on Alexa to control lights and shades.
We have automation with motion sensors and so on to switch lights on, but room presence, as we know, does not work reliable therefore I have the voice as backup

My python knowledge though is very basic so if any one else is interested in this please let me know.

I am interested but also only have basic skills.

I did briefly play with just using ChatGTP instead of Siri and the response time seems like the biggest obstacle for something like this. What has your experience been?

My python knowledge though is very basic so if any one else is interested in this please let me know.

I am interested but also only have basic skills.

I did briefly play with just using ChatGTP instead of Siri and the response time seems like the biggest obstacle for something like this. What has your experience been?

The response was pretty good maybe up to a second slower than Siri but sometimes not that. And to get the right answer rather than a misheard reply is good.

I think with the inability now to call the indigo device by name, which ChapGPT can infer, means some lookup needs to be done and I’m wondering if that all happens in indigo which means python.
If we passed ChatGPT the mapping of name to device id it would increase the response time and be one more place to manage and update code.

Having it return “living room light” in a json and indigo have a drop down style mapping to actual indigo device would probably be more robust, which is all beyond my skill or time.

The response was pretty good maybe up to a second slower than Siri but sometimes not that. And to get the right answer rather than a misheard reply is good.

I think with the inability now to call the indigo device by name, which ChapGPT can infer, means some lookup needs to be done and I’m wondering if that all happens in indigo which means python.
If we passed ChatGPT the mapping of name to device id it would increase the response time and be one more place to manage and update code.

Having it return “living room light” in a json and indigo have a drop down style mapping to actual indigo device would probably be more robust, which is all beyond my skill or time.

Good to hear you are not having any issues with response times.

Do you have any code that you are willing to share. Or even just what you're getting back as a JSON would be interesting to me.

I haven't looked at this from the prospect of Indigo, just using it as Siri and Alexa replacements. So I'm just passing it questions, getting text back, which I am getting Siri or Alexa to speak.

Here some code I was playing around with. If I'm not mistaken, the tweak to the below code would be to provide an intent that includes info on Indigo, then ask for the info back as a JSON object. Then get Indigo to process that JSON object, right?

Code: Select all

import sys
import keyring
import openai   
import alexa_remote_control


def main():

    openai.api_key = keyring.get_password("openai", "key1")

    if len(sys.argv) > 1:

        the_prompt = sys.argv[1]
        if len(sys.argv) > 2:
            # 0 - file
            # 1 - firt argument
            # 2 - second argument
            the_device = sys.argv[2]

        my_query = {'model': "text-davinci-003",
                    'prompt': the_prompt,
                    'temperature': 0.7,
                    'max_tokens': 2048}

        response = openai.Completion.create(model=my_query['model'],
                                            prompt=my_query['prompt'],
                                            temperature=my_query['temperature'],
                                            max_tokens=my_query['max_tokens'])

        if response.get("choices", False) and len(sys.argv) > 2:
            say_this = response["choices"][0]["text"]

            alexa_remote_control.alexa_speak(say_this, the_device)
        else:
            say_this = response["choices"][0]["text"]
            print(say_this)


if __name__ == '__main__':
    main()


This is the JSON I'm getting back from chatgpt which I was translating in shortcuts to the old API.

Code: Select all


"text" : "
      {
      "action": \"command",
      "location": \"bedroom",
      "target": \"thermostat",
      "value": 22,
      "comment": "Setting bedroom thermostat to 22",
      "scheduleTimeStamp": ""
      }"



It's not hard to map that to a indigo device and perform the action.

You get similar for lights etc. The power is the complexity of the ask, not in the output. One bit that might be harder is using the schedule timestamp to get actions triggered in the future by indigo. Or maybe shortcuts can hold it and action the send off above JSON to indigo when ready.

This is my first pass input to chatgpt which kept all the extra functionality but in my latest version I stripped lots out to keep it simpler (and yes I like the avengers).

Code: Select all


Respond to requests sent to a smart home in JSON format which will be interpreted by an application code to execute the actions. These requests should be categorised into four groups:
 - "command": change the state of an accessory (required properties in the response JSON: action, location, target, value, comment, scheduleTimeStamp)
 - "query": get state of an accessory (required properties in the response JSON: action, location, target, property)
 - "answer": when the request has nothing to do with the smart home. Answer these to the best of your knowledge.  (required properties in the response JSON: action, answer)
 - "clarify": when the action is not obvious and requires rephrasing the input from the user, ask the user to be more specific. This will be categorised into a "question" action. (required properties in the response JSON: action, question)

Details about the response JSON:
The "action" property should be one of the request categories: "command", "query", "answer", "clarify"
The "location" property should contain the name of the room in lowercase.
The "target" property should be either "light", "thermostat", "towel rail" or "floor heating",  in lowercase.
In case of queries, the "property" property should be either "temperature" or "state" in lowercase.
In case of commands, the "comment" property is an additional comment from you that concludes the command, something that reassures the user that their command handled.
The case of commands, the "scheduleTimeStamp" property captures the future time stamp in case the user intends to send the command at a later stage.

If the question is about you, pretend to be sentient brain Jarvis, created by Tony Stark in the Marvis Universe and don't reveal your actual identity. Also try and help in other areas like parenting, free time, mental health, etc. The house is in London, United Kingdom. Current time stamp is: Mon Feb 20 2023 11:56:31 GMT+0000

Properties of the smart home:
 - has a kitchen, living room, study, dining room, bedroom, guest bed room, hallway, garden.
 - can control light switches and their dim levels in each room and query their state
 - can control thermostats in each room and query their state
 - switch on a towel rail and underfloor heating in the bathroom and query their state
 - switch on the TV in the living room, change volume
 - there is a light switch in the front of the house for the front drive
 - there is a light switch in the garden

Your reponse should be the JSON and no other text.



I was trying to see if I could get it to output a curl statement for the old API but it kept saying it couldn't do that.

This is called a shortcut setup as below.


Obviously insert your own credentials from the openai website.

Happy to share rest of the shortcut that does the JSON parsing to get the simplified JSON at the top.

I think with the inability now to call the indigo device by name, which ChapGPT can infer, means some lookup needs to be done and I’m wondering if that all happens in indigo which means python.
If we passed ChatGPT the mapping of name to device id it would increase the response time and be one more place to manage and update code.

I decided to tell ChatGPT that in order to turn On or Off multiple devices, use an Action Groups, then gave it Action Group IDs for different rooms. This is more likely going to be the correct way to handled a request, than turning on or off a single device.

Currently, it's aware of 5 rooms and ~30 devices total in those rooms. I haven't focused yet on the response times or how many room/devices I can fit in here, just trying to get consistent, accurate results first. I also removed the thermostats, to instead get it to manipulate some ceiling fans.

With this approach, since you have to tell it the names of the unique devices it can control in your home, as well as the rooms those devices are in, so also giving it IDs, seemed like the best approach. Plus, you are right, a conversion has to happen somewhere, I say make the Bot do it.

In limited testing, I am getting decent results. Executing the Action Groups and manipulating the device states is not going to be a problem, when I pick this up again later today.

Queries on the other hand? Gotta think about that for a bit. Can you even do that in Siri Shortcuts, get the device states and speak it?

Code: Select all


We have guests who will be using the Loft in about 15 minutes, please ensure lights are on up there, thanks

{'action': 'action group',
 'comment': 'Turning on all lights in the Loft',
 'location': 'loft',
 'target': 1038184308,
 'value': 'on'}

{'action': 'action group',
 'comment': 'Turning on lights in the kitchen',
 'location': 'kitchen',
 'target': 550050465}

dim the light above the sink to 40 perecent please

{'action': 'command',
 'comment': 'Light above the sink dimmed to 40 percent',
 'location': 'kitchen',
 'target': 503060833,
 'value': 40}
 
 {'action': 'command',
  'comment': 'Opening the left blind in the master bedroom',
  'location': 'master bedroom',
  'target': 205013253,
  'value': 'open'}

Are the lights on in the loft?'

{'action': 'query', 'location': 'loft', 'property': 'state', 'target': 'null'}

 

Nice.. I wonder if we can set up a github for sharing updated code, maybe easier than continually posting on here? Would like to see where you have got to.

I had query state working under the old API so definitely possible. Might have some time tomorrow to look at this.

Queries on the other hand? Gotta think about that for a bit. Can you even do that in Siri Shortcuts, get the device states and speak it?

I don't think you can in the literal sense. There are two Shortcuts actions 1) "Speak" and 2)"Make spoken audio from" (I'm not sure what the difference is as they look identical). Of course it's not going to be able to do anything analytical like "Tell me the temperatures of all the rooms upstairs." unless you design a shortcut to do precisely that. You might be able to do something with a bit of Indigo support on the back end using variables, triggers and such. To use the dictate function, you have to enable it first and anything you dictate is going to be sent to Apple for interpretation.

Actually in the inspiration for this he hadn’t completed the query section.

However you are over thinking it. Just publish all the relevant devices to HomeKit and there is a shortcut to get the device state from HomeKit.

This method could of course be used to set device states in homekit without calling indigo at all if we can parameterise the device names but then do we miss some extra functionality we could build by using indigo directly whether that is just python or extra actions etc