Remoting a Z-Wave Stick

For those among you interested in locating your Z-Wave interface at a location remote to your Indigo server, take a look at this post. I used an Aeotec Z-Stick Gen5. But, it should work well with the Z-Wave.me UZB1.

Note: This approach might work with the older Aeotec Z-Stick 2. Though it will require some effort and may require a Raspberry-Pi. The issue is installation of the SILABS USB VCP drivers.

Interesting!!!! Good work

Matt, Jay... Not a big deal, but it seems the Z-Wave plugin does not trap an error resulting from a selected Socket interface being unavailable when it is called:

Code: Select all

 Z-Wave                          Optimizing - starting "014 - Umbrella Lights"
   Z-Wave Error                    Error in plugin execution:

Traceback (most recent call last):
  File "zwavehealer.py", line 144, in healThreadFunc
  File "zwavehealer.py", line 103, in _processHeal
  File "zwaveinterface.py", line 715, in sendTimedPing
  File "/Library/Application Support/Perceptive Automation/Indigo 7.2/IndigoPluginHost.app/Contents/Resources/PlugIns/serialinterface.py", line 453, in putQueueOutAction
    return completeEvent.getResultOrThrowException()
  File "/Library/Application Support/Perceptive Automation/Indigo 7.2/IndigoPluginHost.app/Contents/Resources/PlugIns/serialinterface.py", line 73, in getResultOrThrowException
    raise self.exception
SerialException: connection failed ([Errno 54] Connection reset by peer)

   Z-Wave Error                    serial port communication error: connection failed ([Errno 54] Connection reset by peer)
   Z-Wave Error                    Z-Wave 4.54 static controller might be disconnected
   Z-Wave Error                    silently retrying serial connection every 5.0 seconds
   Interface Failure               lost connection with interface


It appears Indigo has a problem with an RFC2217 connection to a ser2net server.

Code: Select all

   Z-Wave Debug                    starting serial connection loop
   Z-Wave Debug                    "Z-Wave 4.54 static controller" serial port open error: write_timeout is currently not supported
   Z-Wave Debug                    make sure remote serial server IP address and port number are correct
   Z-Wave Debug                    Z-Wave 4.54 static controller might be disconnected
   Z-Wave Debug                    silently retrying serial connection every 5.0 seconds
   Z-Wave Debug                    starting serial connection loop


I assume you are using pyserial, and the comments (from github.com/pyserial/pyserial/pyserial/setrial/rfc2217.py) say: - write timeout not implemented at all.
Indeed, at line 507...

Code: Select all

if self._write_timeout is not None:
raise NotImplementedError('write_timeout is currently not supported')

FWIW, my server config is exactly as described in pyserial at line43...

Code: Select all

# - run a server: run ser2net daemon, in /etc/ser2net.conf:
#     2000:telnet:0:/dev/ttyS0:9600 remctl banner

As a test, I commented out the following two lines in ...IndigoPluginHost.app/Contents/Resources/PlugIns/serial/rfc2217.py

Code: Select all

if self._write_timeout is not None:
        raise NotImplementedError('write_timeout is currently not supported')

and that allowed the RFC2217 connection to complete.

I am not sure why the pyserial authors chose to raise an exception rather than issue a warning. But, oh well. Not a big deal anyway since Network Socket is available, and, in any case, I may be the only Indigo user not using a local Controller.

soon to be two
Have to try this.
Ive been hoping someone would break this barrier.
Want to have the z-stick a more central place in my house. this is the perfect solution. Homeseer guys have been doing this for a long time.

Thanks!

A little more information about using VirtualHere to share a USB Z-Stick controller. I have now received a Z-Wave.Me USB Smart Home Stick. This USB interface on this product is reported to be the same as the Aeotec Z-Wave Gen5. Unfortunately, I can confirm the Z-Wave.Me controller has the same problem with VirtualHere sharing as the Aeotec Z-Stick Gen5. Also, as others have reported elsewhere on this forum, the same backup and restore tool provided by Aeotec works with the Z-Wave.Me Stick. In fact, as reported by umtauscher. I was able to back-up the Aeotec and restore to the Z_Wave.me.

So, the good news is, the Z-Wave.me stick should work just as well as the Aeotec stick when creating a Z-Wave controller server.

[matt (support)]

I will look at adding exception handling for the socket interface not being available.

That is surprising pyserial throws an exception if there is a write timeout versus just logging an error. I'm not really familiar with RFC2217 / ser2net server at all. So does the server config get passed from the server to the client? That is, there is no way to override the config to remove the write timout to avoid the exception?

I will look at adding exception handling for the socket interface not being available.

Thanks.

That is surprising pyserial throws an exception if there is a write timeout versus just logging an error. I'm not really familiar with RFC2217 / ser2net server at all. So does the server config get passed from the server to the client? That is, there is no way to override the config to remove the write timout to avoid the exception?

It looks like rfc2217.py simply iterates over what it receives in the session setup negotiation, which appears to be performed in serialutil.py. As far as I can tell, which isn't very far, ser2net sends wait_timeout as part of the connection initialization. I have created an issue on GitHub (https://github.com/pyserial/pyserial)... maybe someone will respond.

Lots of progress here. For those who have not been following the Serving a "remote " building discussion...

• I installed Z-Way (from Z-Wave.me) on a Raspberry-Pi (R-Pi). Z-Way is a general home automation product designed for the Z-Wave.me UZB1 and RaZberry Z-Wave controllers. But it also provides controller management features like backup/restore and firmware updates.
  Z-Way on the R-Pi is controlled using a web interface. So, you can manage it from a browser on any platform. For example, your Indigo server.
  
  • Z-Way may also support the Aeotec Z-Stick Gen5. But, a license would be required.
• I also installed ser2net on the R-Pi to allow sharing of the UZB1 to Indigo over a Socket connection.
  
• Because ser2net and Z-Way cannot both access the controller the same time, you must switch between functions.
  
  • To make this switching easier, I created an Indigo action to control whether the R-Pi is sharing the Z-Wave controller, or using it directly for maintenance purposes.
• I can now place my Z-Wave controller wherever I want and can also do backups directly from my desktop Mac. The best part, the backup is saved directly to my Mac.
• This is essentially a HomeSeer Z-NET (FrankenSeer?) controller, at about 1/2 the price.

So, there are now options for remoting a Z-Wave controller:

• The R-Pi, for around $75 with backup, restore and firmware upgrade capabilities. Requires the Z-Wave.me UZB1
• The GL.iNET GL-MT300N-V2 Mango Mini router for around $55. Works with the Aeotec or Zwavw.me controller

Both prices include the Z-Wave.me UZB1. So, if you already have an Aeotec Z-STick Gen5,, at $20 the Mango is the way to go.

BTW, The Mango based controller has now been up for three days without a hiccup.

That is surprising pyserial throws an exception if there is a write timeout versus just logging an error. I'm not really familiar with RFC2217 / ser2net server at all. So does the server config get passed from the server to the client? That is, there is no way to override the config to remove the write timout to avoid the exception?

It looks like rfc2217.py simply iterates over what it receives in the session setup negotiation, which appears to be performed in serialutil.py. As far as I can tell, which isn't very far, ser2net sends wait_timeout as part of the connection initialization. I have created an issue on GitHub (https://github.com/pyserial/pyserial)... maybe someone will respond.

More data. In place of ser2net I used rfc2217_server.py to share the USB device. Same result in Indigo.

Code: Select all

   Z-Wave Debug                    starting serial connection loop
   Z-Wave Debug                    "Z-Wave Controller" serial port open error: write_timeout is currently not supported
   Z-Wave Debug                    make sure remote serial server IP address and port number are correct
   Z-Wave Debug                    Z-Wave Controller might be disconnected
   Z-Wave Debug                    silently retrying serial connection every 5.0 seconds
   Z-Wave Debug                    starting serial connection loop

But, this is rather odd since the server example is from the same package as the client, rfc2217.py and both import serial.rfc2217 and serialutil.

I am now thinking something in the session setup (telnet sub negotiation?) is actually timing out and the code can't handle that (rather than not handling a session attribute). And, finally, FWIW, the value of self._write_timeout when the exception is raised is 2.0

EDIT: I found this article which discusses the same issue and ultimately punts to use socket://

Because ser2net and Z-Way cannot both access the controller the same time, you must switch between functions.
To make this switching easier, I created an Indigo action to control whether the R-Pi is sharing the Z-Wave controller, or using it directly for maintenance purposes.

could you share how you do that?

I have it up and running with ser2net, want to try out the z-wave.me software.

Do you use a license for the UZB stick?
Does the Z-Way software backup/restore function work without a license for UZB stick?

I bought UZB stick along time ago but it didnt have a license... wasn't aware that there was such a thing either
just tried to restore aeotec gen5 backup on the UZB stick.. great success!

...could you share how you do that? ...
...I have it up and running with ser2net, want to try out the z-wave.me software...

Sure. Note, these instructions are for an R-Pi. This won't work on a OpenWRT router like the Mango.

The basic idea is to have a shell script on the R-Pi that starts and stops the appropriate daemons. Here is the R-Pi script (I placed it in /home/pi/bin/zcontrol):

Code: Select all

#!/bin/sh

MODE=$1

case $MODE in
   local)
      /etc/init.d/ser2net stop >/dev/null
      /etc/init.d/zbw_connect start >/dev/null
      /etc/init.d/z-way-server start >/dev/null
      echo Z-Wave controller is reserved for local use
      ;;
   remote)
      /etc/init.d/ser2net start >/dev/null
      /etc/init.d/zbw_connect stop >/dev/null
      /etc/init.d/z-way-server stop >/dev/null
      echo Z-Wave controller is reserved for local use
      ;;
   *)      
      echo invalid argument. arg1 must be local or remote
      exit 1
      ;;
esac

/usr/bin/logger -i Z-Wave controller set to $MODE mode

exit 0


Then, on the Indigo server I created two scripts (just to make things easier). These scripts use expect so they can pass the R-Pi's root password to the R-Pi to execute the script there with the right permissions. First, to set the R-Pi to local mode. (I.e. Z-WAY mode) I named this zlocal and it can be placed anywhere you like

Code: Select all

#!/usr/bin/expect

log_user 0
set password "password"
eval spawn /usr/bin/ssh -q -t pi@192.168.5.139 "sudo /home/pi/bin/zcontrol local"
expect "assword:"
log_user 1
send "$password\r";
expect "closed"
exit


and the companion script to set the R-Pi to remote (I.e. ser2net) mode is

Code: Select all

#!/usr/bin/expect

log_user 0
set password "password"
eval spawn /usr/bin/ssh -q -t pi@192.168.5.139 "sudo /home/pi/bin/zcontrol remote"
expect "assword:"
log_user 1
send "$password\r";
expect "closed"
exit


Finally, create two Indigo Actions of type: Server Actions -> Script and File Actions -> Run Shell Script
I called mine z switch local and z switch remote. Click Edit Action Settings...Enter the full path to the appropriate script.

That's it.

As to getting Z-WAY running . Once you have Z-WAY installed (which is a pain, the documentations is not well translated) the first thing you need to do is tell Z-WAY which device you have. The default is for the RaZberry at /dev/ttyAMA0. However, if you are using the UZB1, you will need to set it to /dev/tty ACM0. You can do this in the Z-WAY web interface. Click ☰ (The menu symbol) in the upper-right corner and select Apps, then select Z-Wave Network Access and under Serial port to Z-Wave dongle enter the device.

BTW, if you don't want to run ser2net, there is an alternative, the built-in python rfc2217_server.py. You can execute this as:

Code: Select all

sudo python /usr/share/doc/python-serial/examples/rfc2217_server.py -v -v -v -v -p 7000 /dev/ttyACM0

where -p is the port number. However, if you go this route, you will have to modify the zcontrol script accordingly. Starting the server is easy, just enter the line above. But, killing the server is a bit more complex. The following line seems to work

Code: Select all

kill  $(ps aux | grep 'rfc2217_server.py' |grep -v "sudo\|grep" | awk '{print $2}')

...That is surprising pyserial throws an exception if there is a write timeout versus just logging an error. I'm not really familiar with RFC2217 / ser2net server at all. So does the server config get passed from the server to the client? That is, there is no way to override the config to remove the write timout to avoid the exception?

I found the source of the problem. In IndigoPluginHost.app/Contents/Resources/PlugIns/serialinterface.py, at line 501 writeTimeout is set to 2.0. Setting it to None resolves the issue with rfc2217 connections. However, I am not suggesting this change is the fix, it may well cause other issues, I just wanted to report that I had found the root of the problem

Nice! Thank you

ill give it a try on my rpi

I noticed its possibel do use ser2net with the RFX433 dongel (and zigbee dongel) aswell, but sadly the network socket or rfc2217 is not implemented in the rfxcom plugin. only usb.
So no luck getting that to work.