Misinterpreted Z-Wave command shut off my chest freezer

Posted on
Sun Sep 27, 2020 1:35 pm
beaker offline
Posts: 19
Joined: Apr 30, 2016

Misinterpreted Z-Wave command shut off my chest freezer

I noticed today that my basement chest freezer was shut off. I have it connected to an Aeotec DSC06106 switch, but it's not part of any scenes, triggers, schedules, or anything else - I'm only using it to monitor the amount of power being used. The "Show Dependencies" option returns nothing.

Since it's not part of any actions, and the only way to turn it off is at the switch or manually within Indigo, I didn't have any triggers to tell me when power was shut off. That was a big, expensive, wateful mistake because it had been off for two weeks, spoiling around $1,000 worth of food.

I checked the logs, and it wasn't sent a command to turn it off, but it was turned off when I triggered my "going to bed" action that turns down the heat and turns off a bunch of other things. I've been using this action for around two years and haven't changed it since it was created. It does trigger other Aeotec DSC06106 switches, but not the one connected to the freezer.

The logs show that when I triggered that action, the freezer switch turned off, even though no commands were sent to it.

I have a trigger now to send me a message when the device state changes, but this is really concerning to me. I assumed that it wasn't possible for a Z-Wave device to respond to a command not sent to it, but that's clearly not the case.

I doubt there are any answers for this other than "sometimes weird things happen", but I thought I'd post a message to let people know about it, just in case my experience could help others.

Posted on
Sun Sep 27, 2020 5:09 pm
agame offline
Posts: 514
Joined: Jul 13, 2017
Location: Melbourne, Australia

Re: Misinterpreted Z-Wave command shut off my chest freezer

I've had a z-wave controlled window open totally without explanation. Quite weird.

For this reason I also monitor anything that might have bad consequences, or in the case of irrigation valves (that might empty my fire tanks) I chain two physical relays.

Posted on
Mon Sep 28, 2020 9:41 am
peszko offline
Posts: 311
Joined: Mar 07, 2012

Re: Misinterpreted Z-Wave command shut off my chest freezer

I had these kind of problems with Insteon quite frequently. After analysing the protocol (and related statistical literature) I figured that with a large installation the possibility of signal mis identification was statistically likely once a month (given my setup size). Which is generally what I was seeing. To add insult to injury, Insteon has an All on feature (super bad in large installations) which got triggered several times, bot in the middle of the night and when I was away on vacations. Wife was not happy, I must tell you. All lights on, garage opening and alarm triggered at around 3am. Not fun. I'm still amazed my wife let me continue with automation after that.

With Insteon rogue signals, Indigo would not even be aware that the state changed, so it was even harder to mitigate them. Specifically because of this, I'm moving most of my devices to z-wave. I haven't had a single rogue event with z-wave devices. I had signal failure, but not mis identification.

But what I ended up doing, and will do for more critical devices on z-wave as well after hearing your story, was to implement a second layer of status. I would have a variable with the status of a device and operate on that variable. A separate process (trigger) would then react and turn the device on or off to follow. Another script would run every 30 min to query the state of all the devices, and if discrepancy was found it would re-send the appropriate command based on the associated variable. With Insteon, I would see this correction regularly.

I still use this system for the sauna and the pool heating ( even on z-wave now), but I haven't seen it trigger a correction yet.

I'll have to analyze the z-wave protocol to see if it is worth applying this method to more devices.

Posted on
Mon Sep 28, 2020 12:02 pm
DaveL17 offline
User avatar
Posts: 6756
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Misinterpreted Z-Wave command shut off my chest freezer

I still refuse to put mission-critical things on Z-Wave switches for this exact reason. I monitor our sump pump with an Aeotec energy meter instead of an energy-reporting switch. This way nothing (including a very unlikely hacking event) can keep the pump from running when it needs to. I haven't done anything to monitor our freezer--but at some point may add a DS18B20 temperature sensor to that. I did suspend my rule for thermostats--but I have extra safety nets in Indigo to ensure that set points are not out of range and units can't run counter to the outside temperature (i.e., no AC in winter). I second peszko's suggestion that something like a trigger is a great idea. If [DEVICE X] turns off, turn [DEVICE X] on.

I still think a plug-in Z-Wave device that can report energy but not affect the load is a huge hole in the market.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Mon Sep 28, 2020 6:07 pm
beaker offline
Posts: 19
Joined: Apr 30, 2016

Re: Misinterpreted Z-Wave command shut off my chest freezer

I've had cases where a command wasn't received by a device, but those could happen due to range or temporary interference (although a simple ACK/NAK and re-transmit should be part of the process). This is the first time I've had a device say "Oh, I see you're turning off some devices. You know what? That looks like fun - I think I'll join in".

I had assumed that Z-Wave devices, since they had a built-in device ID, had a layer of reliability that would make something like this impossible. Add to that an assumed checksum, and I was confident in using Z-Wave for things like monitoring the power in my freezer without turning it off for no reason.

I guess I was wrong.

I don't use Z-Wave for any security-related things like locks, and after this, there's no chance I will.

Posted on
Mon Sep 28, 2020 7:05 pm
agame offline
Posts: 514
Joined: Jul 13, 2017
Location: Melbourne, Australia

Re: Misinterpreted Z-Wave command shut off my chest freezer

Z-wave messages do get routinely get delayed or lost on occasion. My speculation has been that perhaps a message could get held up somewhere in the mesh and subsequently 'pop-out'. Presumably there are safeguards against this but I wonder if they are infallible?

Mind you I've had loads of zwave devices many years and only seen one 'inexplicable' case - I do think user-error (schedules or triggers buried and forgotten) is far more likely to occur!

Posted on
Tue Sep 29, 2020 11:04 am
beaker offline
Posts: 19
Joined: Apr 30, 2016

Re: Misinterpreted Z-Wave command shut off my chest freezer

Delaying a message, or losing it in the mesh is one thing. Having a device turn off without receiving a command to do it should be impossible, but that's exactly what happened.

This device wasn't a part of any schedules or triggers, and wasn't activated manually in Indigo or physically at the switch. It just decided to turn itself off. It reported the event to Indigo, and I was able to turn it back on, so it was communicating with the server just fine.

Posted on
Tue Sep 29, 2020 12:13 pm
peszko offline
Posts: 311
Joined: Mar 07, 2012

Re: Misinterpreted Z-Wave command shut off my chest freezer

What I would like to know, and will investigate is if this is a protocol weakness or just a malfunctioning device. Everything fails at some point. You could loose your freezer contents when your compressor breaks, or coolant leaks. The more important question for me is is it common problem or a freak event. (at least the device reported it as opposed to my Insteon issues)

Same goes for security or other critical devices. Standard locks on most doors are super simple to pick, most alarm systems communicate with monitoring using single accessible from outside phone line, most garage door openers remotes can be spoofed, doors forced with minimal effort, etc... Similarly with other critical systems, furnaces break, pipes break, gas leaks etc..

The question for me is not if it's possible rather is it likely enough to cause a problem. I even drive a car and that kills around 1.4 million people a year world wide.

Posted on
Tue Sep 29, 2020 12:25 pm
peszko offline
Posts: 311
Joined: Mar 07, 2012

Re: Misinterpreted Z-Wave command shut off my chest freezer

DaveL17 wrote:
This way nothing (including a very unlikely hacking event) can keep the pump from running when it needs to.


The breaker can flip, pump malfunctions, intake blocked, pressure switch stuck, etc. There are other things that can prevent the pump from running. But yes, the more critical the system is the more reliability should be designed in to it.

As an example, for the sauna, I have two temperature sensors, several triggers related to them, schedule that double check the on time and temperature, a visible light that comes on when heater is on, notifications to me when it comes on and when it goes off, notification of temperature extremes. And on top of that, a thermal electricity cut off if all of this fails. Things could still go wrong but I feel comfortable with the safety margin even when automated.

Posted on
Tue Sep 29, 2020 1:59 pm
DaveL17 offline
User avatar
Posts: 6756
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Misinterpreted Z-Wave command shut off my chest freezer

peszko wrote:
DaveL17 wrote:
This way nothing (including a very unlikely hacking event) can keep the pump from running when it needs to.


The breaker can flip, pump malfunctions, intake blocked, pressure switch stuck, etc. There are other things that can prevent the pump from running. But yes, the more critical the system is the more reliability should be designed in to it.

Of course. My statement was referring to its interaction with Z-Wave and Indigo.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Tue Sep 29, 2020 5:30 pm
beaker offline
Posts: 19
Joined: Apr 30, 2016

Re: Misinterpreted Z-Wave command shut off my chest freezer

peszko wrote:
Standard locks on most doors are super simple to pick, most alarm systems communicate with monitoring using single accessible from outside phone line, most garage door openers remotes can be spoofed, doors forced with minimal effort, etc... Similarly with other critical systems, furnaces break, pipes break, gas leaks etc..

The question for me is not if it's possible rather is it likely enough to cause a problem. I even drive a car and that kills around 1.4 million people a year world wide.


That's a bit of a non-sequitur, it's it? If I unlock one lock, another lock next to it doesn't unlock by itself for no reason. My alarm system doesn't disarm itself because I shutoff my oven. My furnace doesn't shut off when I turn off a light.

Yes, devices fail, things break, security systems can be hacked. I'm not saying that any of those things happened in my case. What happened was that a command sent to one device was instead executed by a different, unrelated device. A device that isn't a member of any triggers, schedules, or scenes. The device didn't die or otherwise fail - it reported the event to Indigo, and then shut itself off. Turning it back on in Indigo worked fine.

I'm not trying to talk people into throwing away their Z-Wave devices, I just saw something weird that shouldn't be possible, and thought people might want to know that it could happen.

If your car decided to take a left turn because the guy ahead of you made a left, I have a feeling you'd be a bit surprised by it.

Posted on
Tue Sep 29, 2020 5:40 pm
peszko offline
Posts: 311
Joined: Mar 07, 2012

Re: Misinterpreted Z-Wave command shut off my chest freezer

beaker wrote:
If your car decided to take a left turn because the guy ahead of you made a left, I have a feeling you'd be a bit surprised by it.


I'm surprised by cars taking left ahead of me all the time :-)

BTW, non of what I said was meant as a criticisms, just my approach to automation (and life in general). If so taken, I apologize.

Posted on
Tue Sep 29, 2020 5:46 pm
peszko offline
Posts: 311
Joined: Mar 07, 2012

Re: Misinterpreted Z-Wave command shut off my chest freezer

BTW, Tesla car has already killed some people by taking left in to barriers for some strange reasons. I suspect, we will see our AI overlords killing us at faster pace in the future.

Just in case... I'm trying to make a joke here.

Posted on
Thu Oct 01, 2020 11:35 am
howartp offline
Posts: 4559
Joined: Jan 09, 2014
Location: West Yorkshire, UK

Re: Misinterpreted Z-Wave command shut off my chest freezer

I have TKB plugin switches all over at home.

I put the energy versions on my PCs and server at home and left them running; no on/off commands, just metering.

After some long period, quite possibly 512, 1024, 999, 9999 hours (etc), they all turned off. (At different times, relative to when I would have bought and added them)

I’m presuming the on/off switches aren’t expected to be running constantly and do turn themselves off at some ‘round number’ in IT figures.

Possibly the same happened here, rather than rogue commands?

Peter



Sent from my iPhone using Tapatalk Pro

Posted on
Thu Oct 01, 2020 3:17 pm
peszko offline
Posts: 311
Joined: Mar 07, 2012

Re: Misinterpreted Z-Wave command shut off my chest freezer

howartp wrote:
After some long period, quite possibly 512, 1024, 999, 9999 hours (etc), they all turned off. (At different times, relative to when I would have bought and added them)

I’m presuming the on/off switches aren’t expected to be running constantly and do turn themselves off at some ‘round number’ in IT figures.


Thats kind of what I was getting at. There is a difference with protocol not working and specific device from a brand or even a specific device. Its easier to change the malfunctioning device, or replace a badly implemented brand devices. But if the underlying protocol is deficient, all devices from all brands will share the same issue.

The Insteon thermostat I had seems to have a 7 bit register for the temperature, and would wrap around below zero and above 127. Bad design in my opinion. Since I had it installed in a outside place (Canada) in winter when it went below zero, it jumped to 127 (round IT number :-)), setting off several overheat alarms for me.

Who is online

Users browsing this forum: No registered users and 17 guests