Skip the first three paras if you just want the technical.
My home automation has been running indigo since around 2011 (ish) and with the exception of one house move, has worked flawlessly. Like many I have gone quiet on the boards as it's just working. Until about 18 months ago. My gen5 stick failed. With limited time and other things taking priority it got pushed to the bottom of the pile. Everything else was working except 106 z-wave modules. I just couldn't face added them all back. We had arguments about lights being left on. My daughter had a melt down as she didn't know where any light switches were <- weeks of crashing into things in the dark which I personally found hilarious, and at the same time being called a "dick" with every trip. Those with teenage daughters will understand it was my fault, but that's a generation that have never needed to use a switch to turn a light on! We all know what adding 106 modules back really takes, especially those hidden in the back of cupboards and in the loft.
So I'm currently working on a new suit of software for DJ's. After 100 days coding I decided to take a break. With winter drawing in and the lights on more and more I decided it was time to tackle the stick! I had a fresh one ready.
Before starting, I thought I'd get claude to have a look at what was wrong with the stick. As expected the chip had lost the mapping table. Now im not a tech tech like lots of you, but I asked claude if it could fix the stick. 4 hours work, and we found the method, and a Mac save and restore script to boot. I'll leave claude to finish the story in its own words. I hope this helps someone.
Recovering a Z-Wave 500-series controller's lost node list WITHOUT re-pairing (Aeotec Z-Stick Gen5)
Mats Z-Stick Gen5 lost its entire node table: Indigo still listed all of the devices, but the controller reported no nodes, so nothing could be commanded ("can't find node"), and he had had no controller backup from before the loss (dick). The standard answer to that is "re-pair every device" — around a hundred here, many behind faceplates. Instead, working through it with Claude (Anthropic's AI), it rebuilt the controller's node table from Indigo's own device database and brought the whole network back with zero re-pairing. It's all confirmed working — on/off, dimming, blinds, colour, meters. Full method and generic scripts below in case it saves someone else the pain.
The situation
A Z-Wave 500-series controller that is the primary can end up with an empty node table (Mats did — cause unknown, possibly a glitch/failing NVM). The Home ID and every physical device are still fine and still bound to the network; it's only the controller that forgot its list. Indigo can't command devices it can't find on the controller, so the house goes dark. Without a pre-loss backup, everyone (zwave-js, HomeSeer, Silicon Labs, the "rebuild your mesh" guides) says the same thing: re-pair everything. A node table can normally only be built by the inclusion protocol.
Why the obvious fixes don't work
- The Aeotec backup tool can clone a controller's identity (Home ID) onto a fresh same-model stick — but it carries no node list.
- zwave-js (@zwave-js/nvmedit) can build and inject a full node table — the nodes read back — but on this firmware its jsonToNVM500 output isn't byte-native (it's ~57 bytes short and drops the routing tables), so the firmware sees a not-quite-valid NVM and zeroes the protected Home ID on boot. You get the nodes back but lose the Home ID.
The key insight
Don't rebuild the NVM (lossy). Seed the Home ID with the Aeotec clone, then edit that byte-native image IN PLACE — write only the node-table and routing entries via zwave-js's low-level NVM500 class, leaving every validity marker, the Home ID and the S0 key byte-identical — and write it back raw (restoreNVMRaw, never restoreNVM, which re-runs the lossy conversion). The node data comes from Indigo, which still stores each device's node ID, its [basic, generic, specific] device class, and its neighbour list.
Confirmed working
Network fully recovered, no devices re-included. The final rebuild writes every device's correct class into the node table — dimmers, blinds, colour, meters, not a generic placeholder — so the controller reports each device's true type and full function returns across the board once you reconnect the stick. Important gotcha we hit: a device's available commands follow the class the stick reports, and Indigo's per-device "Sync" (Edit device → Define and Sync… → Sync) just re-reads that class. So (a) rebuild the table with the correct classes first; and (b) if you'd already Synced a device earlier, while the stick still had a wrong/placeholder class, that Sync "corrected" its Indigo record down to on/off — re-Sync it against the now-correct stick to repair it. That was the only manual fixup we needed.
Requirements — two same-model 500-series controllers, target on a zwave-js-supported SDK (≥ 6.61; Gen5 v1.02 / "6.07" works, v1.00 doesn't). Node.js 20 + npm i zwave-js. Aeotec Backup Tool (Windows) for the Home-ID seed.
Scripts (generic — fill the CONFIG blocks for your own network):
1) backup.js
Code: Select all
const { Driver } = require("zwave-js"); const fs = require("fs");
const PORT = process.argv[2], OUT = process.argv[3], CACHE = process.argv[4] || "./cache";
const d = new Driver(PORT, { logConfig:{enabled:false}, storage:{cacheDir:CACHE} });
const wd = setTimeout(()=>{console.error("timeout");process.exit(3)},180000);
d.on("error",e=>console.error("ERR:",e.message));
d.once("driver ready", async()=>{ try{
const nvm = await d.controller.backupNVMRaw((c,t)=>process.stdout.write("\r "+c+"/"+t+" "));
fs.writeFileSync(OUT, nvm);
console.log("\nSAVED",nvm.length,"bytes ->",OUT,"| homeId",d.controller.homeId?.toString(16));
} catch(e){ console.error("\nERR:",e.stack||e.message);} finally{ clearTimeout(wd); await d.destroy().catch(()=>{}); process.exit(0);} });
d.start().catch(e=>{console.error("START ERR (stick on this machine?):",e.message);process.exit(1)});2) extract_nodes.py — pull per-node data from Indigo's database (the .indiDb is XML):
Code: Select all
import xml.etree.ElementTree as ET, shutil, json, sys
SRC = sys.argv[1] # path to your Indigo .indiDb
ZW = "com.perceptiveautomation.indigoplugin.zwave"
shutil.copy(SRC, "/tmp/_hadb"); root = ET.parse("/tmp/_hadb").getroot()
parent = {c:r for r in root.iter() for c in r}
def txt(e,t): x=e.find(t); return (x.text or "").strip() if (x is not None and x.text) else ""
def ints(e,t):
x=e.find(t); return [int(i.text) for i in x.findall("Item") if i.text and i.text.strip().lstrip('-').isdigit()] if x is not None else []
nodes={}
for el in root.iter():
if el.tag!=ZW: continue
mp=parent.get(el); dev=parent.get(mp) if mp is not None else None
if dev is None or dev.tag!="Device": continue
a=txt(el,"address")
if not a or not a.isdigit(): continue
n=nodes.setdefault(a,{"battery":False,"neighbors":[],"classIds":[]})
if txt(el,"SupportsBatteryLevel")=="true": n["battery"]=True
nb=ints(el,"zwNodeNeighbors"); n["neighbors"]=nb if len(nb)>len(n["neighbors"]) else n["neighbors"]
ci=ints(el,"zwClassIds"); n["classIds"]=ci if len(ci)>len(n["classIds"]) else n["classIds"]
print(json.dumps({"nodes":nodes}))3) inject.js — rebuild the node table into a native image (base = a backup that already has the Home ID):
Code: Select all
const nv = require("@zwave-js/nvmedit"), fs = require("fs");
const { NVMMemoryIO } = require(require.resolve("@zwave-js/nvmedit").replace(/index.js$/,"lib/io/NVMMemoryIO.js"));
const BASE=process.argv[2], NODESJSON=process.argv[3], OUT=process.argv[4];
// ---------- CONFIG for your network ----------
const SECURE_NODES = new Set([]); // node IDs included with S0 security (from Indigo), else []
const CLASS_OVERRIDES = {}; // {nodeId:[generic,specific]} for any node whose stored class is wrong
// ---------------------------------------------
(async()=>{
const io=new NVMMemoryIO(Uint8Array.from(fs.readFileSync(BASE)));
const nvm=new nv.NVM500(io); await nvm.init();
const nt=await nvm.get("EX_NVM_NODE_TABLE_START_far"); // 232-slot NodeInfo array (empty = undefined)
const rt=await nvm.get("EX_NVM_ROUTING_TABLE_START_far"); // 232 neighbour-lists
const base=nt[0];
const data=JSON.parse(fs.readFileSync(NODESJSON,"utf8"));
const ids=Object.keys(data.nodes).map(Number).filter(n=>n>=2).sort((a,b)=>a-b);
for(const id of ids){
const d=data.nodes[String(id)]; const cls=Array.isArray(d.classIds)?d.classIds:[];
let gen = cls.length>=2 ? cls[1] : (d.battery?0x20:0x10);
let spec= cls.length>=3 ? cls[2] : 1;
if(CLASS_OVERRIDES[id]){ gen=CLASS_OVERRIDES[id][0]; spec=CLASS_OVERRIDES[id][1]; }
nt[id-1]={ isListening:!d.battery, isFrequentListening:false, isRouting:!d.battery,
supportedDataRates:base.supportedDataRates, protocolVersion:base.protocolVersion,
optionalFunctionality:false, nodeType:1, supportsSecurity:SECURE_NODES.has(id),
supportsBeaming:true, genericDeviceClass:gen, specificDeviceClass:spec };
rt[id-1]=(d.neighbors||[]).filter(x=>x>=1&&x<=232);
}
await nvm.set("EX_NVM_NODE_TABLE_START_far", nt);
await nvm.set("EX_NVM_ROUTING_TABLE_START_far", rt);
await nvm.set("EX_NVM_MAX_NODE_ID_far",[Math.max(...ids)]);
await nvm.set("EX_NVM_LAST_USED_NODE_ID_START_far",[Math.max(...ids)]);
fs.writeFileSync(OUT, Buffer.from(io._buffer));
const chk=await nv.nvm500ToJSON(Buffer.from(io._buffer));
console.log("built:",OUT,"| homeId:",chk.controller.ownHomeId,"| nodes:",Object.keys(chk.nodes).length);
})().catch(e=>console.log("ERR",e.stack||e.message));4) restore_raw.js — write RAW (never restoreNVM); prints a clear next step, not a scary error:
Code: Select all
const { Driver } = require("zwave-js"); const fs = require("fs");
const PORT=process.argv[2], NVMFILE=process.argv[3], CACHE=process.argv[4]||"./cache";
const nvm=fs.readFileSync(NVMFILE);
const BENIGN=/requires a driver restart|did not respond after soft-reset|being destroyed/i; // expected, not failures
let wrote=false;
const d=new Driver(PORT,{logConfig:{enabled:false},storage:{cacheDir:CACHE}});
const wd=setTimeout(()=>{console.error("\nWATCHDOG: no completion in 180s.");process.exit(3)},180000);
d.on("error",e=>{ if(!BENIGN.test(e.message||"")) console.error("REAL ERROR:",e.message); });
function finish(){ clearTimeout(wd);
console.log("\n=================================================================");
if(wrote){ console.log(" WRITE COMPLETE -> image written to the stick.");
console.log(" NEXT: unplug the stick, wait ~3s, plug it back in (power-cycle to activate).");
console.log(" 'driver restart' / 'soft-reset' messages above are EXPECTED, not errors.");
console.log(" THEN VERIFY: python3 verify.py"); }
else console.log(" WRITE DID NOT CONFIRM — see REAL ERROR above; do NOT replug/verify yet.");
console.log("=================================================================");
d.destroy().catch(()=>{}).finally(()=>process.exit(0));
}
d.once("driver ready",async()=>{ try{
let last=0; await d.controller.restoreNVMRaw(nvm,(c,t)=>{ if(c-last>=20000||c===t){last=c;process.stdout.write("\r writing "+c+"/"+t+" ");}});
wrote=true;
} catch(e){ if(BENIGN.test(e.message||"")) wrote=true; else console.error("\nREAL ERROR:",e.stack||e.message);} finally{ finish(); } });
d.start().catch(e=>{console.error("START ERR — stick on THIS machine? (ls /dev/cu.usbmodem*)\n ",e.message);process.exit(1)});5) verify.py — read Home ID + node list over the raw Serial API:
Code: Select all
import os, termios, time, glob, sys
p=sorted(glob.glob("/dev/cu.usbmodem*"))
if not p: sys.exit("no usbmodem port")
PORT=p[0]
def frame(f,d=b""):
b=bytes([0,f])+d; L=len(b)+1; c=0xFF
for x in bytes([L])+b: c^=x
return bytes([1,L])+b+bytes([c])
fd=os.open(PORT,os.O_RDWR|os.O_NOCTTY|os.O_NONBLOCK); a=termios.tcgetattr(fd)
a[2]|=(termios.CLOCAL|termios.CREAD); a[2]&=~termios.CSIZE; a[2]|=termios.CS8
a[2]&=~(termios.PARENB|termios.CSTOPB|termios.CRTSCTS)
a[0]&=~(termios.IXON|termios.IXOFF|termios.IXANY|termios.INPCK|termios.ISTRIP)
a[1]&=~termios.OPOST; a[3]&=~(termios.ICANON|termios.ECHO|termios.ECHOE|termios.ISIG); a[4]=a[5]=termios.B115200
termios.tcsetattr(fd,termios.TCSANOW,a); os.write(fd,b"\x15"); time.sleep(0.3)
def rd(t=1.5):
buf=b""; end=time.time()+t
while time.time()<end:
try:
c=os.read(fd,256)
if c: buf+=c; end=time.time()+0.3; os.write(fd,b"\x06")
except BlockingIOError: time.sleep(0.02)
return buf
os.write(fd,frame(0x20)); r=rd(); j=0; home=None
while j<len(r)-3:
if r[j]==1 and r[j+2]==1 and r[j+3]==0x20:
pl=r[j:j+r[j+1]+2]
if len(pl)>=9: home=pl[4:8].hex().upper()
break
j+=1
os.write(fd,frame(0x02)); r=rd(2.0); j=0; nodes=[]
while j<len(r)-3:
if r[j]==1 and r[j+2]==1 and r[j+3]==0x02:
fr=r[j:j+r[j+1]+2]; bm=fr[7:7+fr[6]]
nodes=[bi*8+bit+1 for bi,by in enumerate(bm) for bit in range(8) if by&(1<<bit)]; break
j+=1
os.close(fd)
print("HOME ID:", ("0x"+home) if home else "?", "| NODES(%d):"%len(nodes), nodes)Step-by-step
1. node backup.js <oldstickport> old.bin (or Aeotec Read EEPROM) — keep it as a safety copy.
2. Aeotec tool: Read EEPROM the old (Home-ID-bearing) stick → Write EEPROM to a same-model, same-firmware target stick. This seeds the correct Home ID onto the target.
3. node backup.js <targetport> seeded.bin — native image: Home ID present, empty node list.
4. python3 extract_nodes.py "<your .indiDb>" > nodes.json — each device's node ID, [basic,generic,specific] class, and neighbours.
5. node inject.js seeded.bin nodes.json injected.bin — rebuilds the whole node table with every device's real class (from nodes.json). Only touch the CONFIG for SECURE_NODES, and CLASS_OVERRIDES for any single device whose stored class you know is wrong.
6. node restore_raw.js <targetport> injected.bin → power-cycle the stick → python3 verify.py (expect the correct Home ID + full node count).
7. Reconnect the stick to your controller machine and point the software at it. Because the table now carries each device's correct class, devices regain full function on their own — there's no per-device step to run.
Only repair case: if you Synced a device earlier, while the stick still had a wrong/placeholder class, that Sync corrupted its record down to on/off — re-Sync only that device against the now-correct stick. Never Sync against a stick that has wrong classes.
Expected messages (look like errors, aren't) — "Activating the NVM backup requires a driver restart", "did not respond after soft-reset", "controller instance is being destroyed": the write succeeded; just power-cycle the stick. First post-replug read showing "?"/empty: re-read.
Real problems — "Failed to open the serial port … No such file or directory" = stick not on this machine/wrong port. "Did not find a matching NVM 500 parser … SDK 6.61 or higher" = firmware too old for zwave-js. Home ID reads 0x00000000/random on a settled re-read = the write was rejected (non-native image) → rebuild in place + restoreNVMRaw, or restore your backup.
Do this now that it works (so it never recurs)
Back up after any change: node backup.js <port> backup_YYYYMMDD.bin, stored off-device. Restore to the same stick: node restore_raw.js <port> backup.bin → power-cycle. Restore to a new same-model, same-firmware stick: Aeotec Write EEPROM the backup (or restore_raw.js it) — carries Home ID + node table, no re-pairing. The one rule: always restore with restoreNVMRaw (raw/native) or the Aeotec tool — never restoreNVM, whose lossy conversion zeroes the Home ID.
Credit — the reverse-engineering and these scripts were worked out with Claude (Anthropic). Sharing generically in case it helps anyone else who's staring down a re-pair of their whole house.
Honest status: UNTESTED via this exact from-blank path. I recovered mine by cloning a still-readable original, so I never had to write a Home ID from scratch. It should work by the logic above — and if the firmware rejects it, you're no worse off than before (re-pair). If you try it, please reply with the result — that would confirm the fully-dead-controller case for everyone.
) put together, so the plugin now backs up, verifies and restores the newer sticks as well as the 500 series it already handled: Zooz ZST10 700 and ZST39, Aeotec Z-Stick 7 and Z-Stick 10 Pro, Silicon Labs UZB-7 and similar.
to build the plugin that recreates the data for the stick from indigo if you don’t have a backup, and replace the correct house ID. 