Z-Wave Node Matrix

Posted on
Fri May 13, 2016 9:09 am
DaveL17 offline
User avatar
Posts: 5897
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Z-Wave Node Matrix

The following script creates a node/neighbor matrix for Z-Wave devices. It requires the matplotlib and numpy modules.

I recommend saving the image to the Indigo static images folder. It's probably not necessary to run the script regularly (assuming that the network doesn't change that frequently) and it's probably most useful to run the script right before examining the output. When a device is included or excluded, the chart will adjust accordingly (the next time the script is run.) Color is meaningful and I've added a "color blind" feature in case the color differences are too subtle for some (see below.) Note that the script can take a fair bit of time to complete--especially on slower servers and/or larger networks. The script will write a message to the log when it's done. I will continue to try to add lightness where I can. Each node on the X axis displays its corresponding neighbors vertically. The resulting figure can be used to answer questions such as:

  • which devices are not associated with node 1?
  • which devices show neighbors that are no longer in the network?

EDIT 1: corrects mixture of tabs and spaces.
EDIT 2: Adds features to plot battery devices in a different color (white dot, red border in example), highlight devices not associated with node 1 (red dot in example) and own node (gray dot in example.) These new features are optional.
EDIT 3: Adds feature to hide unused node numbers, moves indication of devices not associated with node 1 to the x-axis label to avoid obscuring other indicators (colored text on X axis), adds indication of neighbors with unused nodes (colored text on Y axis), adds optional legend.
EDIT 4: Adds optional color blind feature to make certain indications more prominent.

Code: Select all
#! /usr/bin/env python2.6
# -*- coding: utf-8 -*-

try:
    import sys
    import matplotlib.pyplot as plt
    import numpy as np
except ImportError, e:
    sys.exit(u"The matplotlib and numpy modules are required to use this script.")


# =================== User Settings ===================
output_file = '/Library/Application Support/Perceptive Automation/Indigo 6/IndigoWebServer/images/controls/static/neighbors.png'

title_font_size = 9
chart_title = 'node matrix'
x_axis_title = 'node'
y_axis_title = 'neighbor'

background_color = '#000000'
font_color = '#888888'
font_name = 'Lato Light'
tick_font_size = 6
foreground_color = '#888888'
node_border = '#66FF00'
node_color = '#FFFFFF'
node_marker = '.'

# If True, each node that is battery powered will be highlighted.
plot_battery = True
node_border_battery = '#FF0000'

# If True, devices with node 1 missing will be highlighted.
plot_no_node_1 = True
node_1_missing = '#FF0000'

# If True, neighbors without a corresponding node will be highlighted.
plot_no_node = True
node_missing = '#000099'

# If True, each node will be plotted as its own neighbor.
plot_self = True
plot_self_color = '#333333'

# If True, unused node addresses will be plotted.
plot_unused_nodes = False

# If True, display a chart legend.
show_legend = True

# If True, additional annotations will be used beyond color.
color_blind = False
color_blind_marker = 'x'

# =================== kwarg Settings ===================
# To minimize the blank space surrounding the chart, add bbox_inches='tight' to kwarg_savefig.  This causes the chart's dimensions to be dynamic.
kwarg_savefig = {'bbox_extra_artists': None,
                 'dpi': 100,
                 'edgecolor': background_color,
                 'facecolor': background_color,
                 'format': None,
                 'frameon': None,
                 'orientation': None,
                 'pad_inches': 0.1,
                 'papertype': None,
                 'transparent': True}
kwarg_title = {'color': font_color,
               'fontname': font_name,
               'fontsize': title_font_size}

# =====================================================
# Matplotlib will decide the optimal figure size on its own. If you want to give the chart constrained dimensions, uncomment the next line.
# Figsize values are in inches. The saved image will be inches x DPI. In other words, a chart that is 7" x 7" will yield a chart 700 x 700 at
# 100 DPI (set above in the kwarg_savefig parameters.).
# plt.figure(figsize=(7, 7))

# =============================================================
# CODE BELOW THIS LINE SHOULD GENERALLY NOT NEED TO BE CHANGED.
# =============================================================

# Build the master dictionary of the Z-Wave Mesh Network.
address_list = []
final_devices = []
no_node_1 = []
max_addr = 0

# Iterate through all the Z-Wave devices and build a dictionary.
for dev in indigo.devices.itervalues('indigo.zwave'):
    try:
        neighbor_list = list(dev.globalProps['com.perceptiveautomation.indigoplugin.zwave']['zwNodeNeighbors'])
        supports_battery = dev.globalProps['com.perceptiveautomation.indigoplugin.zwave']['SupportsBatteryLevel']

        address_list.append(int(dev.address))
        final_devices.append([int(dev.address), dev.name, neighbor_list, supports_battery])

        if max_addr < int(dev.address):
            max_addr = int(dev.address)

    except Exception, e:
        pass

# Take the master list, sort it, and add a counter element for later charting.
counter = 1
for item in sorted(final_devices):
    item.append(counter)
    counter += 1

# Assign consecutive keys to neighbor nodes for plotting when unused nodes are not plotted.
neighbor_list = []
for device in sorted(final_devices):
    for neighbor in device[2]:
        if neighbor not in neighbor_list:
            neighbor_list.append(neighbor)
for device in sorted(final_devices):
    if device[0] not in neighbor_list:
            neighbor_list.append(device[0])
counter = 1
dummy_y = {}
for neighbor in sorted(neighbor_list):
    if neighbor not in dummy_y.keys():
        dummy_y[neighbor] = counter
        counter += 1

# ================== Lay Out The Plots ==================
for device in sorted(final_devices):
    try:

        # This plot will show all Z-Wave neighbors on the plot.
        if plot_unused_nodes:
            for neighbor in device[2]:
                plt.plot(device[0], neighbor, color=node_border, marker=node_marker, markerfacecolor=node_color, zorder=10)
        else:
            for neighbor in device[2]:
                plt.plot(device[4], dummy_y[neighbor], color=node_border, marker=node_marker, markerfacecolor=node_color, zorder=10)

        # This plot provides an overlay for battery devices.
        if device[3] and plot_battery and plot_unused_nodes:
            for neighbor in device[2]:
                if color_blind:
                    plt.plot(device[0], neighbor, color=node_border_battery, marker=color_blind_marker, markerfacecolor=node_color, zorder=10)
                else:
                    plt.plot(device[0], neighbor, color=node_border_battery, marker=node_marker, markerfacecolor=node_color, zorder=10)
        elif device[3] and plot_battery:
            for neighbor in device[2]:
                if color_blind:
                    plt.plot(device[4], dummy_y[neighbor], color=node_border_battery, marker=color_blind_marker, markerfacecolor=node_color, zorder=10)
                else:
                    plt.plot(device[4], dummy_y[neighbor], color=node_border_battery, marker=node_marker, markerfacecolor=node_color, zorder=10)

        # This plot provides an additional overlay over devices that don't report node 1 as a neighbor.
        if plot_no_node_1 and not device[2]:
            indigo.server.log('  Device %s is valid, but has no neighbors. Skipping.' % device[0])
        elif plot_no_node_1 and device[2][0] != 1 and plot_unused_nodes:
            for neighbor in device[2]:
                no_node_1.append(int(device[4]) - 1)
        elif plot_no_node_1 and device[2][0] != 1 and not plot_unused_nodes:
            for neighbor in device[2]:
                no_node_1.append(int(device[4]) - 1)

        # This plot will assign a point to a node for its own address (plot itself.)
        if plot_self and plot_unused_nodes:
            for me in np.arange(2, max_addr, 1):
                plt.plot(me, me, color=plot_self_color, marker=node_marker, markerfacecolor=plot_self_color, zorder=10)
        elif plot_self:
            plt.plot(device[4], dummy_y[device[0]], color=plot_self_color, marker=node_marker, markerfacecolor=plot_self_color, zorder=10)

    except IndexError, e:
        indigo.server.log("%s - %s" % (device[0], e), isError=True)

    except KeyError, e:
        indigo.server.log("%s - %s. Skipping." % (device[0], e), isError=True)
        pass

    except Exception, e:
        indigo.server.log("%s - Problem building node matrix: %s" % (device[0], e), isError=True)
        pass

# =================== Chart Settings ===================
plt.title(chart_title, **kwarg_title)
for spine in ['top', 'bottom', 'left', 'right']:
    plt.gca().spines[spine].set_color(foreground_color)
plt.tick_params(axis='both', which='both', labelsize=tick_font_size, color=foreground_color)

# =================== X Axis Settings ===================
plt.xlabel(x_axis_title, fontsize=title_font_size, color=foreground_color)
plt.tick_params(axis='x', bottom=True, top=False)

if plot_unused_nodes:
    plt.xticks(np.arange(1, max_addr + 1, 1), fontsize=tick_font_size, color=foreground_color)
    plt.xlim(1, max_addr + 1)
else:
    plt.xticks(np.arange(1, len(final_devices) + 1, 1), sorted(address_list), fontsize=tick_font_size, color=foreground_color)
    plt.xlim(0, len(final_devices) + 1)

# =================== Y Axis Settings ===================
plt.ylabel(y_axis_title, fontsize=title_font_size, color=foreground_color)
plt.tick_params(axis='y', left=True, right=False)

if plot_unused_nodes:
    plt.yticks(np.arange(1, max_addr + 1, 1), fontsize=tick_font_size, color=foreground_color)
    plt.ylim(0, max_addr + 1)
else:
    plt.yticks(np.arange(1, max_addr, 1), sorted(dummy_y.keys()), fontsize=tick_font_size, color=foreground_color)
    plt.ylim(0, len(dummy_y) + 1)

# =================== Legend Settings ===================
if show_legend:
    foo1, = plt.plot([], color=node_border, linestyle='', marker=node_marker, markerfacecolor=node_color)
    if color_blind:
        foo2, = plt.plot([], color=node_border_battery, linestyle='', marker=color_blind_marker, markerfacecolor=node_color)
    else:
        foo2, = plt.plot([], color=node_border_battery, linestyle='', marker=node_marker, markerfacecolor=node_color)
    foo3, = plt.plot([], color=plot_self_color, linestyle='', marker=node_marker, markerfacecolor=plot_self_color)
    foo4, = plt.plot([], color='blue', linestyle='', marker=node_marker, markerfacecolor='blue')
    foo5, = plt.plot([], color=node_1_missing, linestyle='', marker=node_marker, markerfacecolor=node_1_missing)
    legend = plt.legend([foo1, foo2, foo3, foo4, foo5], ['neighbor', 'battery', 'self', 'no node', 'no node 1'], bbox_to_anchor=(1, 0.5), fancybox=True, loc='best', ncol=1, numpoints=1, prop={'size': 6.5})
    legend.get_frame().set_alpha(0)
    for text in legend.get_texts():
        text.set_color(font_color)

# ======== Color labels for nodes with no node 1 ========
if plot_no_node_1 and no_node_1:
    foo_x = [i for i in plt.gca().get_xticklabels()]
    if not plot_unused_nodes:
        for node in no_node_1:
            foo_x[node].set_color(node_1_missing)
            if color_blind:
                foo_x[node].set_weight('bold')
                foo_x[node].set_backgroundcolor('white')
    else:
        for node in no_node_1:
            foo = sorted(final_devices)
            a = foo[node][0]
            foo_x[a-1].set_color(node_1_missing)
            if color_blind:
                foo_x[node].set_weight('bold')
                foo_x[node].set_backgroundcolor('white')

# ========= Color labels for neighbors with no node ========
if plot_no_node:
    foo_y = [i for i in plt.gca().get_yticklabels()]
    for a in dummy_y.keys():
        if a not in address_list:
            if a != 1:
                if not plot_unused_nodes:
                    a = dummy_y[a] - 1
                    foo_y[a].set_color(node_missing)
                    if color_blind:
                        foo_y[a].set_weight('bold')
                        foo_y[a].set_backgroundcolor('white')
                else:
                    a -= 1
                    foo_y[a].set_color(node_missing)
                    if color_blind:
                        foo_y[a].set_weight('bold')
                        foo_y[a].set_backgroundcolor('white')

# ========= Output the Z-Wave Node Matrix Image =========
try:
    plt.savefig(output_file, **kwarg_savefig)
except Exception, e:
    indigo.server.log("Chart output error: %s:" % e)

# Wind things up.
plt.close()
indigo.server.log("Z-Wave Node Matrix generated.")
Attachments
neighbors.png
Color Only
neighbors.png (31.77 KiB) Viewed 3986 times
neighbors.png
Color Blind
neighbors.png (34.4 KiB) Viewed 3986 times

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Fri May 13, 2016 9:30 am
matt (support) offline
Site Admin
User avatar
Posts: 20538
Joined: Jan 27, 2003
Location: Texas

Re: Z-Wave Node Matrix

That is really cool. I'm not sure of the answer to your questions, but it is possible that resyncing/healing the entire network might be easier in a future version of Indigo.

Thanks for sharing the script!

Image

Posted on
Fri May 13, 2016 9:40 am
Different Computers offline
User avatar
Posts: 2115
Joined: Jan 02, 2016
Location: East Coast

Re: Z-Wave Node Matrix

All I have to say is

WOW.

Awesome work! I'm going to try this over the weekend to see if it helps me figure out some device flakiness.

SmartThings refugee, so happy to be on Indigo. 10.13.6 on an i5 MBP w/Harmony Hub, Hue, DomoPad, Dynamic URL, Device Extensions, HomeKit Bridge, MatplotLib, Plex, uniFAP, Wunderground, Nanoleaf, LED Simple Effects, Airfoil Pro, Grafana.

Posted on
Fri May 13, 2016 9:44 am
DaveL17 offline
User avatar
Posts: 5897
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Z-Wave Node Matrix

matt (support) wrote:
That is really cool. I'm not sure of the answer to your questions, but it is possible that resyncing/healing the entire network might be easier in a future version of Indigo.

Thanks for sharing the script!

Thanks Matt! My pleasure.

ETA: +1 on more native network interaction.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Fri May 13, 2016 9:45 am
DaveL17 offline
User avatar
Posts: 5897
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Z-Wave Node Matrix

Different Computers wrote:
All I have to say is

WOW.

Awesome work! I'm going to try this over the weekend to see if it helps me figure out some device flakiness.

Thanks DC. Make sure to come back for the most recent version when you do decide to try it out. I expect that a couple tweaks will be forthcoming.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Fri May 13, 2016 10:01 am
durosity offline
User avatar
Posts: 4046
Joined: May 10, 2012
Location: Newcastle Upon Tyne, Ye Ol' England.

Re: Z-Wave Node Matrix

Very cool!


Sent from my iPad using Tapatalk

Computer says no.

Posted on
Fri May 13, 2016 10:03 am
autolog offline
Posts: 3497
Joined: Sep 10, 2013
Location: West Sussex, UK [GMT aka UTC]

Re: Z-Wave Node Matrix

Hi Dave,
This is a really useful matrix - I must congratulate you on your efforts :)

If you are looking for enhancement suggestions (based on having run it on my network), then here goes :lol: :
  • I have gaps in my z-Wave addressing (probably due to resyncing devices) and the matrix could have less white space if thoses addresses weren't included i.e. if no device is in the address and no neighbour points to one of those addresses, then remove from X and Y axis.
  • put a different colour box where the X and Y is for the same device i.e. a diagonal from bottom left to top right
  • highlight battery devices in a different colour
  • highlight devices which don't have a neighbour of 1 in a different colour

Posted on
Fri May 13, 2016 10:47 am
DaveL17 offline
User avatar
Posts: 5897
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Z-Wave Node Matrix

durosity wrote:
Very cool!


Sent from my iPad using Tapatalk

Thanks brother!

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Fri May 13, 2016 10:57 am
DaveL17 offline
User avatar
Posts: 5897
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Z-Wave Node Matrix

autolog wrote:
Hi Dave,
This is a really useful matrix - I must congratulate you on your efforts :)

If you are looking for enhancement suggestions (based on having run it on my network), then here goes :lol: :
  • I have gaps in my z-Wave addressing (probably due to resyncing devices) and the matrix could have less white space if thoses addresses weren't included i.e. if no device is in the address and no neighbour points to one of those addresses, then remove from X and Y axis.
  • put a different colour box where the X and Y is for the same device i.e. a diagonal from bottom left to top right
  • highlight battery devices in a different colour
  • highlight devices which don't have a neighbour of 1 in a different colour

Thanks Jon!

1. Yeah, me too. I believe that it's inherent in the protocol that controllers don't reuse node numbers when they've been freed (I think this is due in part to what I show above--devices still in a neighbors list where the device no longer exists.) I will think about adding this as an option (display/hide unused node numbers).

2. This one I'm not entirely clear on. I understand the idea of running something from the lower left to the upper right, but I'm not sure what you mean by a different color box. I'm also not entirely clear on the utility. Are you suggesting we point out that a device is not a neighbor to itself?

3. That's a great idea. I'll add this as an option.

4. That's also a great idea. I'll add that too.

Dave

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Fri May 13, 2016 11:13 am
autolog offline
Posts: 3497
Joined: Sep 10, 2013
Location: West Sussex, UK [GMT aka UTC]

Re: Z-Wave Node Matrix

DaveL17 wrote:
... 2. This one I'm not entirely clear on. I understand the idea of running something from the lower left to the upper right, but I'm not sure what you mean by a different color box. I'm also not entirely clear on the utility. Are you suggesting we point out that a device is not a neighbor to itself?

Basically, Yes - Take a look at page 13/14 of this manual: Razberry Pi - In this case it runs from top left towards bottom right. :)

Posted on
Fri May 13, 2016 11:42 am
DaveL17 offline
User avatar
Posts: 5897
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Z-Wave Node Matrix

autolog wrote:
Basically, Yes - Take a look at page 13/14 of this manual: Razberry Pi - In this case it runs from top left towards bottom right. :)

Got it!

I see now where the inspiration for several of your suggestions comes from. :D

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Fri May 13, 2016 12:21 pm
durosity offline
User avatar
Posts: 4046
Joined: May 10, 2012
Location: Newcastle Upon Tyne, Ye Ol' England.

Re: Z-Wave Node Matrix

DaveL17 wrote:
Thanks brother!


Have you been watching Lost? :p

Computer says no.

Posted on
Fri May 13, 2016 12:27 pm
DaveL17 offline
User avatar
Posts: 5897
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Z-Wave Node Matrix

durosity wrote:
Have you been watching Lost? :p

I *am* lost most of the time. Does that count?

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Fri May 13, 2016 12:28 pm
durosity offline
User avatar
Posts: 4046
Joined: May 10, 2012
Location: Newcastle Upon Tyne, Ye Ol' England.

Re: Z-Wave Node Matrix

Mostly. Now get lost!

Computer says no.

Posted on
Fri May 13, 2016 1:29 pm
mat offline
Posts: 768
Joined: Nov 25, 2010
Location: Cambridgeshire - UK

Re: Z-Wave Node Matrix

Another wow .... really useful!

Love the matrix green! Can we use letters instead of coloured dots? Will it scroll for a few seconds while refreshing?

:shock: :mrgreen:

Late 2018 mini 10.14

Who is online

Users browsing this forum: No registered users and 0 guests