zigbee2mqtt & zigbee USB for Hue bulbs

Posted on
Tue Jun 09, 2020 9:42 am
siclark offline
Posts: 1960
Joined: Jun 13, 2017
Location: UK

zigbee2mqtt & zigbee USB for Hue bulbs

What: I want to control my Hue bulbs from Indigo without the Hue hub, using a zigbee USB stick
Why: To expand mains zigbee devices in my network to enable more zigbee battery sensors to be used, plus the zigbee USB sticks are cheaper than the Hue hub and removes our dependence on proprietary hubs.


I have been using the Philips Hue for longer than I have had Inidgo, and recently upgraded to v2. However I have never had perfect signal strength, despite bulbs being in neighbouring rooms. I have also wanted to move the Hub but have been limited due to the need to wired into my network (despite having cat6 in most rooms)

Since finding the Aqara sensors and wireless switches, I wanted to expand my zigbee network to use more of these battery sensors. However as these dont act as routers, I would be forced to use lots of USB zigbee sticks as routers. So, I got thinking about my Hue bulbs. Could I run just one zigbee network.

Taking full advantage of Joe's amazing MQTT plugin, and node red, I have created a dummy Meta Device Dimmer and I can now use that to control my a Hue bulb.

I now need to create a dummy RGB device and see how to get the colour states published over MQTT from Indigo, and then move all my Hue bulbs over so that I can test my zigbee network.

Posted on
Tue Jun 09, 2020 9:51 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: zigbee2mqtt & zigbee USB for Hue bulbs

When you get this all working, can you post a diagram of the system, showing what kinds of signals (and protocols) are between each of the components? Maybe with links to the non-Indigo parts?

I don't use Hue, but having a good way to use Zigbee components might be useful.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Tue Jun 09, 2020 10:11 am
siclark offline
Posts: 1960
Joined: Jun 13, 2017
Location: UK

Re: zigbee2mqtt & zigbee USB for Hue bulbs

Sure.

Actually, I have been wondering if we can bypass node-red entirely. For the On/Off and brightness for example, I am converting the incoming MQTT message from Indigo

indigo/devices/626280811/update : msg.payload : string[88]
Code: Select all
"{'name': 'dummy hue bulb', 'states': { 'brightnessLevel': '61', 'onOffState': 'True' } }"


to a JSON for the zigbee2mqtt USB stick as

indigo/devices/626280811/update : msg.payload : Object
Code: Select all
{"state":"ON","brightness":"61"}.
which is sent to zigbee2mqtt/TestHueBulb/set/

I do this with some renames of the state names, and a convert. I could have removed the name in the publishing template but left it in for now .

I was wondering if a simple plugin to create the dummy device that has the right state names, ie state and brightness, not onOffState and brightnessLevel (unless they are indigo hardcorded). couldnt be set to publish direct to the zigbee stick?

Posted on
Tue Jun 09, 2020 10:27 am
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: zigbee2mqtt & zigbee USB for Hue bulbs

Those names are hardcoded in Indigo, so you can't change them as the actual state names. But...

You might be able to do this with some fancy coding in the Pystache (based on Mustache ) template that's being used to publish the device info.

The Mustache spec is here: https://mustache.github.io/mustache.5.html

I'm trying to think if there's any good way to call a Python script to do the data dictionary conversion, before publishing the device. That might be too much overhead.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Tue Jun 09, 2020 11:31 am
siclark offline
Posts: 1960
Joined: Jun 13, 2017
Location: UK

Re: zigbee2mqtt & zigbee USB for Hue bulbs

Ok. Once I’ve done the conversion in node red it can be copied and even shared on here so not much of a hassle.

I’m also wondering if I can make it generic using the same indigo device id as zigbee name and using one flow for all similar zigbee devices,

Posted on
Tue Jun 09, 2020 4:14 pm
siclark offline
Posts: 1960
Joined: Jun 13, 2017
Location: UK

Re: zigbee2mqtt & zigbee USB for Hue bulbs

Ok... its working for an RGB bulb. I can set on/off, brightness or RGB via indigo colour picker or Homekit and it changes the Hue bulb via my zigbee USB stick rather than the Hue hub.

The flow is neatish for one (other than those painful 17 changes) but might be cumbersome for many devices. I believe I can set the topic of the right MQTT message to have the topic come from the payload, which in tern can be the device name or id, so I might only need to have this one flow with many inputs, and it would see run the same flow for multiple indigo devices, and send the MQTT message to zigbee2usb with the topic being the indigo device name, assuming you had indigo and zigbee names in line.

A bit more testing to be done, but this is looking promising. Tomorrow I will start moving some more lights over and see how the network looks.
Attachments
Screenshot 2020-06-09 at 23.10.51.png
Screenshot 2020-06-09 at 23.10.51.png (83.05 KiB) Viewed 4044 times

Posted on
Tue Jun 09, 2020 4:24 pm
siclark offline
Posts: 1960
Joined: Jun 13, 2017
Location: UK

Re: zigbee2mqtt & zigbee USB for Hue bulbs

Forgot to say, I am now using the SDK example dimmer plugin to create the dummy colour device.

I still need to decide how to do Alexa control. If I link Alexa direct from node red to zigbee, I need to work out how to update the device state in indigo.

Posted on
Tue Jun 09, 2020 4:28 pm
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: zigbee2mqtt & zigbee USB for Hue bulbs

If you could eliminate the conversion flow with a different topic/payload format, post the format you'd like (as complete as possible) and I'll see how hard it would be to make that available in the plugin.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Tue Jun 09, 2020 4:32 pm
siclark offline
Posts: 1960
Joined: Jun 13, 2017
Location: UK

Re: zigbee2mqtt & zigbee USB for Hue bulbs

Thanks

Format would
Topic =zigbee2mqtt/devicename/set

With payload of

{"state":"ON",
"brightness":"180",
"color":
{"r":"255",
"g":"226",
"b":"202"}}

For an rgb bulb. A white temp one would be different, that I’d need to check.

Posted on
Tue Jun 09, 2020 4:34 pm
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: zigbee2mqtt & zigbee USB for Hue bulbs

What's the possible values (ranges) for each field? I'm assuming I'd need to convert.

Or can you export the nodes you're using to do the conversion and I could read them directly?

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Tue Jun 09, 2020 4:44 pm
siclark offline
Posts: 1960
Joined: Jun 13, 2017
Location: UK

zigbee2mqtt & zigbee USB for Hue bulbs

The full description of the message structure is here if you are interested. It’s just the set part at the bottom that we need.

https://www.zigbee2mqtt.io/information/mqtt_topics_and_message_structure.html

There are choices on colour format and despite me sending rgb commands it reports back in xy format.

Brightness is 0-255 so I scale from indigo’s % value. RGB same 0-255.

Posted on
Tue Jun 09, 2020 4:51 pm
siclark offline
Posts: 1960
Joined: Jun 13, 2017
Location: UK

Re: zigbee2mqtt & zigbee USB for Hue bulbs

This is my node red flow

Code: Select all

[{"id":"912583bd.193ff","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"44708c4d.a519d4","type":"mqtt in","z":"912583bd.193ff","name":"","topic":"indigo/devices/1965234729/update","qos":"2","datatype":"utf8","broker":"628714c4.ba20ac","x":280,"y":700,"wires":[["88ef062f.8fde48"]]},{"id":"88ef062f.8fde48","type":"yaml","z":"912583bd.193ff","property":"payload","name":"","x":510,"y":700,"wires":[["ac41f170.8474d"]]},{"id":"a3cf2cde.fdb27","type":"mqtt out","z":"912583bd.193ff","name":"","topic":"zigbee2mqtt/SimonHueBulb/set/","qos":"","retain":"","broker":"628714c4.ba20ac","x":1420,"y":700,"wires":[]},{"id":"5598aefe.1b8ab","type":"change","z":"912583bd.193ff","name":"","rules":[{"t":"change","p":"payload.states.onOffState","pt":"msg","from":"True","fromt":"str","to":"ON","tot":"str"},{"t":"change","p":"payload.states.onOffState","pt":"msg","from":"False","fromt":"str","to":"OFF","tot":"str"},{"t":"set","p":"payload.states.state","pt":"msg","to":"payload.states.onOffState","tot":"msg"},{"t":"set","p":"payload.states.brightness","pt":"msg","to":"payload.states.brightnessLevel","tot":"msg"},{"t":"delete","p":"payload.states.brightnessLevel","pt":"msg"},{"t":"delete","p":"payload.states.onOffState","pt":"msg"},{"t":"set","p":"payload.brightness","pt":"msg","to":"$number(payload.brightness)","tot":"jsonata"},{"t":"delete","p":"payload.name","pt":"msg"},{"t":"set","p":"payload.states.r","pt":"msg","to":"payload.states[\"redLevel.ui\"]","tot":"msg"},{"t":"set","p":"payload.states.b","pt":"msg","to":"payload.states[\"blueLevel.ui\"]","tot":"msg"},{"t":"set","p":"payload.states.g","pt":"msg","to":"payload.states[\"greenLevel.ui\"]","tot":"msg"},{"t":"delete","p":"payload.states.redLevel","pt":"msg"},{"t":"delete","p":"payload.states.blueLevel","pt":"msg"},{"t":"delete","p":"payload.states.greenLevel","pt":"msg"},{"t":"delete","p":"payload.states[\"redLevel.ui\"]","pt":"msg"},{"t":"delete","p":"payload.states[\"blueLevel.ui\"]","pt":"msg"},{"t":"delete","p":"payload.states[\"greenLevel.ui\"]","pt":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":800,"y":700,"wires":[["66cce37f.746a1c"]]},{"id":"66cce37f.746a1c","type":"change","z":"912583bd.193ff","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.states","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":1000,"y":700,"wires":[["890f5da0.1d74"]]},{"id":"ac41f170.8474d","type":"range","z":"912583bd.193ff","minin":"0","maxin":"100","minout":"5","maxout":"255","action":"clamp","round":true,"property":"payload.states.brightnessLevel","name":"","x":640,"y":700,"wires":[["5598aefe.1b8ab"]]},{"id":"890f5da0.1d74","type":"template","z":"912583bd.193ff","name":"","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"{\"state\":\"{{payload.state}}\",\n\"brightness\":\"{{payload.brightness}}\",\n\"color\":\n{\"r\":\"{{payload.r}}\",\n\"g\":\"{{payload.g}}\",\n\"b\":\"{{payload.b}}\"}}","output":"str","x":1180,"y":700,"wires":[["a3cf2cde.fdb27"]]},{"id":"628714c4.ba20ac","type":"mqtt-broker","z":"","name":"qnap mqqt","broker":"192.168.0.104","port":"1883","clientid":"","usetls":false,"compatmode":false,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""}]


Posted on
Wed Jun 10, 2020 2:54 am
siclark offline
Posts: 1960
Joined: Jun 13, 2017
Location: UK

Re: zigbee2mqtt & zigbee USB for Hue bulbs

And this is the MQTT reply from the Hue bulb

Code: Select all
{"state":"ON","last_seen":"2020-06-10T08:53:24.754Z","linkquality":36,"brightness":255,"color":{"x":0.1664,"y":0.1385},"color_temp":31}

Posted on
Wed Jun 10, 2020 3:24 pm
siclark offline
Posts: 1960
Joined: Jun 13, 2017
Location: UK

Re: zigbee2mqtt & zigbee USB for Hue bulbs

Along with the RGB bulb, I have added 2 Hue colour temp bulbs as well. They needed slightly different processing in node-red as they take colour temperature not RGB, but it also helped me realise I could simplify things a bit.

I am wondering if I can build a more complicated flow, but that recognises the incoming payload format and formats the outgoing template automatically. If that could be packaged into a single node then this could be a simple setup.

Posted on
Wed Jun 10, 2020 3:42 pm
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: zigbee2mqtt & zigbee USB for Hue bulbs

There's certainly some things you could generate in the flow to make this more generic. For example, you could build the output MQTT topic based on the incoming MQTT payload, which has the device name in it. Just use the exact same name on both systems.

Here's the complete inbound MQTT msg object:

Code: Select all
{"topic":"indigo/devices/587622500/update","payload":{"name":"Example Dimmer","address":"123456","deviceId":"587622500","deviceTypeId":"myDimmerType","model":"Example Dimmer Module","subModel":"","protocol":"Plugin","states":{"backlightBrightness":"0","brightnessLevel":"51","onOffState":"True"},"capabilities":["supportsStatusRequest"]},"qos":0,"retain":true,"_msgid":"138afa9d.779fd5"}


I expect you could branch in your flow based on the deviceTypeId/model/subModel fields, and then do the appropriate mods for each flow.

Are you using node-red for anything else in the project, or just for converting the MQTT messages from what the Indigo plugin is generating to what zigbee2mqtt is expecting?

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Who is online

Users browsing this forum: No registered users and 0 guests