Help creating Shim devices for Pool Controller

[FlyingDiver]

So I wasn't creating a new trigger, based on my understanding that one trigger could do all the work needed. It never occurred to me to create a 2nd trigger because I don't see how to differentiate between triggers when it comes to the Shims Device. But maybe I missed something... ?

The triggers will have different match strings, so they'll trigger for different topic strings. You define the message type in the trigger definition, which links it to specific shim devices.

Right now you're using ##easytouch## for your message type. That won't work for different kinds of devices - you'll need to define different ones for different device types.

The Wiki page I linked to describes the use of the message type field. You should probably re-read that page. If you have questions about what something means, stop and ask.

[jltnol]

So I think I've made a lot of progress. I've managed to come to an understanding about creating triggers, and have successfully created a few that fire off when there is a data change from the Mosquito server on my Pi.

That leaves the info needed for the Shims devices. When I started all this, I decided to work on the "time" data, since it seemed to me clear that it wasn't a JSON field, and I could test out a bunch of stuff without having to worry about how to configure JSON fields. And the "time" field changed a LOT, so it wouldn't take long to see if I had things setup correctly or not. I've created a trigger for the "time" data, and have created a Shims device for time as well. I set this up as a Sensor Device since none of the other options seemed to apply. I think I've got the Shims device configured correctly,

Unique ID Location "Topic Component", Topic Field "2", Unique ID Value "time", State Value Location "Payload Field", Payload Type "Raw", Sensor Type "Generic"

but every time my time trigger fires, I get this error message:

Code: Select all

Trigger                         MQTT Trigger 4 TIME
   MQTT Shims Error                pool time: update() is unable to convert '2022-05-30T13:15:25.000-0500' to float

So from this error message, I'm guessing my data is actually getting to my Shims device, but I either don't have it set up exactly correctly. (very likely) or the plugin isn't designed to handle this kind of data natively (much less likely).

I really have no interest in harvesting the data from the "time" field, it was more of a test for me to understand MQTT Triggers, something I think I have a better understanding of.

In trying to get the main pump (easytouch2-4/state/temps/bodies/1/pool) into Indigo, I realized the only real way for me to test all this was to repeatedly turn the pump on and off again as I made changes, something that didn't seem like the best idea, so instead, I settled on this field:

Code: Select all

easytouch2-4/state/circuits/4/aux3

as this is just a relay with nothing connected, so toggling it on and off again while testing just seems like a better idea.

Again, I've got the trigger setup correctly as when I do turn it off and on, that particular trigger fires. But when it comes to the Shims setup, I'm not getting anywhere. I have this setup as a MQTT Switch Device, and agin think I've got the upper most part of the plugin configured correctly

Message Type: "easytouch4", Unique ID Location "Topic Component", Topic Field "4", Unique ID Value "aux3"

but the middle and end are mysteries. The JSON field of my first successful device was {"temp":84}, and I was told the correct Payload Key was simply "temp" (without the quotes). But the field for aux3 is more complicated--

{"id":4,"isOn":"on","endTime":"2022-05-31T01:52:12:00-0500"}

so I'm at loss as to what the Payload Key is.

What I have is this:

State Value Location "Payload Fields", Payload Type "JSON", Payload key "id", and I'm guessing the On value is "isOn" which is the same whether it is on or off, , and Sensor type is "Power On/Off",

but I get is this error message when I turn in on:

Trigger MQTT Trigger
MQTT Shims Error AUX 3: No JSON payload data for state_value

so am guessing my error lies in the Payload Key.

I'm also surmising that the "On Value" is what Indigo looks for to indicate the device is On (or not on), the Action Payloads are what Indigo sends out to turn the device on or off, but have no idea what should go in those fields either.

I've read the Wiki several times, but my problem is I'm trying to compare what I guess is some kind of terminal output in the examples, and what I'm looking at in MQTT Explorer, and just can't make any meaningful comparisons to try to figure out what needs to go where in my Shims device.

So I guess I'm only mildly interested in what the error is for the "time" field, but am honestly interested in the "aux3" field, since its JSON field looks very similar to the maim pump field so am hoping translating settings from one to the other will be easy.

As always, any suggestions are most welcome.

[FlyingDiver]

I've read the Wiki several times, but my problem is I'm trying to compare what I guess is some kind of terminal output in the examples, and what I'm looking at in MQTT Explorer, and just can't make any meaningful comparisons to try to figure out what needs to go where in my Shims device.

You're going to have to disabuse yourself of this notion. There is no "Terminal Output" in the wiki. The wiki makes no assumptions about how you're looking at the MQTT data. It could be a GUI app like MQTT Explorer, or is could be a command line tool in a Terminal, or you could be looking at the debug output of the MQTT Connector plugin in the Indigo log. There are two parts to an MQTT message - the Topic and the Payload. Whenever I discuss this in the wiki or here on the forums, I always do with the topic as a plain text string. Payload is usually a string as well, either a simple string (raw), or formatted as JSON.

As you stated, the Topic of interest is:

Code: Select all

easytouch2-4/state/circuits/4/aux3

The Payload for this topic is:

Code: Select all

{"id":4,"isOn":"on","endTime":"2022-05-31T01:52:12:00-0500"}

Sometimes I'll use a shorthand like:

Code: Select all

easytouch2-4/state/circuits/4/aux3 --> {"id":4,"isOn":"on","endTime":"2022-05-31T01:52:12:00-0500"}

I prefer to reformat that like you see in the lower right corner of the MQTT Explorer window:

Code: Select all

{    
    "id":4,
    "isOn":"on",
    "endTime":"2022-05-31T01:52:12:00-0500"
}

This is a dict (dictionary) in JSON. It consists of a set of key/value pairs. So for this topic, the key that's of interest is the "isOn" key, which currently has a value of "on".

So just from that description, you should be able to see what you did wrong in the Shim. Your state key is "isOn" and your On value is "on".

I will say that I think whoever designed this topic scheme did a very poor job. Like why is the "bodies" topic a sub-topic of "temps"? And why does the circuits topic have numbered subtopics, each of which as a different name topic (which doesn't really match the number topic). Then there's an "id" field in the payload which DOES match the number topic. Screwy.

Oh, for your "time" device, there really is no appropriate device type because that payload can't be converted to anything useful for Indigo. So I didn't make a device to match. Perhaps that was an oversight on my part.

Last thing for now - when I'm testing things like this, I don't bother with an actual physical device. I use MQTT Connector plugin actions to publish a message that matches what I would get from the device, both topic and payload, then I just Execute the action group to test my triggers or shims. Like:

[jltnol]

Hey Joe

It is ONLY with your incredible patience and support that I’ve been able to get all the devices into Indigo from my pool controller. Thanks so much AGAIN for your help.

One (and a half) piece of my puzzle is left, and that is to get Indigo to actually control the pool equipment. My switch devices need an Action Topic as well as the On/ Off/ Toggle payloads. I’ve tried a few different things for the Action Topic and finally noticed that as I tried different things, all I was doing was adding topics in the MQTT Explorer. So with that in mind, back in the Explorer, I copied the Topic path for my device… in my case “easytouch2-4/state/circuits/4/aux3”, and put that in the Action Topic. So I’m halfway guessing that is close to what it should be.

For the On Action Payload, I essentially copied the JSON field while the device was on, and ditto for the Off Payload, but all I was doing was replacing the right line in the Explorer with what I had cobbled together. So while the Explorer indicated a change in state, this had no real effect on the actual equipment.

I’ve been back to the Wiki but don’t see any info on Action Topics and the Payloads, and have also skimmed the MQTT forum here, but can’t find anything that looks like it applies. If there is a resource for me to read, point me in the right direction and I’ll dive in.

My half-piece of the puzzle is, while all my Indigo devices correctly reflect the state of the devices, I’m still getting an error message, but ONLY when it is turned On.

Code: Select all

Trigger                         MQTT Trigger Circuit 3
   MQTT Shims Debug                message_handler: MQTT message ##easytouch5## from MQTT Connector to Pi
   Trigger                         MQTT Trigger Circuit 3
   MQTT Shims Debug                message_handler: MQTT message ##easytouch5## from MQTT Connector to Pi
   MQTT Shims Debug                pool AUX 3: processMessages: '##easytouch5##' easytouch2-4/state/circuits/4/aux3 -> {"id":4,"isOn":"on","endTime":"2022-05-31T21:58:51.000-0500"}
   MQTT Shims Debug                pool AUX 3: Raw payload = {"id":4,"isOn":"on","endTime":"2022-05-31T21:58:51.000-0500"}
   MQTT Shims Debug                pool AUX 3: states_key= 
   MQTT Shims Debug                pool AUX 3: Updating state to True
   MQTT Shims Debug                pool AUX 3: processMessages: '##easytouch5##' easytouch2-4/state/circuits/4/aux3/endTime -> {"id":4,"endTime":2022-05-31T21:58:51.000-0500}
   MQTT Shims Error                pool AUX 3: No JSON payload data for state_value

Not sure if this is related to my Indigo On/Off issue or not, but the Indigo devices definitely reflect the right state of the equipment. I also find it curious that the trigger fires twice every time the data is updated...

[FlyingDiver]

You can't tell what the correct topic is for controlling the equipment by looking at the update topics it sends out. They are often very different. You're going to need to look at the documentation for whatever's acting as the MQTT interface for the pool equipment for that info.

The reason you're getting that error is because this topic:

Code: Select all

   MQTT Shims Debug                pool AUX 3: processMessages: '##easytouch5##' easytouch2-4/state/circuits/4/aux3/endTime -> {"id":4,"endTime":2022-05-31T21:58:51.000-0500}

Matches the same trigger as your normal status update:

Code: Select all

   MQTT Shims Debug                pool AUX 3: processMessages: '##easytouch5##' easytouch2-4/state/circuits/4/aux3 -> {"id":4,"isOn":"on","endTime":"2022-05-31T21:58:51.000-0500"}

Poorly designed MQTT messages, IMO. The fix is to change your trigger to something like this, which would prevent it from triggering on the topic with the extra "endTime" on the end.

[cuhouse]

Jltnol,

Try something like this for the Action Topic>>easytouch2-4/state/circuits/4/aux3/toggleState

Use on and off for the On/Off Action Payload.

If that doesn't work then use your Pentair system controller and command the aux3 to change state. Watch the topics on MQTT Explorer for the command string to pop up.

Jody

[FlyingDiver]

If that doesn't work then use your Pentair system controller and command the aux3 to change state. Watch the topics on MQTT Explorer for the command string to pop up.

I don't think that will work unless the system controller is using MQTT to send the command.

[jltnol]

Hey guys so here's what I've tried:
Adding "End" to the Match list stopped the trigger from firing, and give me this error:

Code: Select all

MQTT Connector Error            Caught exception in on_message: list index out of range

And just as a note, only when the device is turned ON is there a timestamp in the JSON field, which is the time it will automatically be turned off. When I turn the device off, the timestamp is replaced with "null".

I also tried adding "toggleState" to the end of my Action Topic, and used on and off for the payloads, but that didn't work.

I don't think that will work unless the system controller is using MQTT to send the command.

So of course, I have no idea how the system controller operates, but part of my Pi project was to install a web interface which mimics everything on the main control panel. And with that said, I'm also halfway guessing that the web interface DOESN'T operate via MQTT, but am far from sure that's the case. The Pi is also running the Mosquito Service, which I guess works like a conduit between the System Panel (server...), and Explorer and Indigo(clients). So while I haven't actually used the panel to change the state of anything, when I change the state of something via the web interface, those changes are reflected on the Control Panel, change the equipment operation, and updates the state of those devices in Indigo. So the only messages I can check in Explorer are the same whether they come from the web interface or directly from the System Control Panel..

Looking for more documentation from Pentair makes sense as its possible that what I'm trying to do simply isn't implemented in what at least seems to be a poorly designed MQTT interface to start with. At the very least, I need to find what commands are actually needed, as it appears they don't actually show up in Explorer. In my feeble mind, both the web interface and the control panel communicate as they should, but Explorer isn't really showing those message exchanges, just the result of those messages. Does that make sense?

Guess its time for more Pentair research.

[FlyingDiver]

Hey guys so here's what I've tried:
Adding "End" to the Match list stopped the trigger from firing, and give me this error:

Code: Select all

MQTT Connector Error            Caught exception in on_message: list index out of range

And just as a note, only when the device is turned ON is there a timestamp in the JSON field, which is the time it will automatically be turned off. When I turn the device off, the timestamp is replaced with "null".

I'll have to look at that error. Not sure if something is wonky in your trigger or my code.

The other contents of the JSON payload are irrelevant. It's only looking at the item matching the state key

[FlyingDiver]

I also tried adding "toggleState" to the end of my Action Topic, and used on and off for the payloads, but that didn't work.

I don't think that will work unless the system controller is using MQTT to send the command.

So of course, I have no idea how the system controller operates, but part of my Pi project was to install a web interface which mimics everything on the main control panel. And with that said, I'm also halfway guessing that the web interface DOESN'T operate via MQTT, but am far from sure that's the case. The Pi is also running the Mosquito Service, which I guess works like a conduit between the System Panel (server...), and Explorer and Indigo(clients). So while I haven't actually used the panel to change the state of anything, when I change the state of something via the web interface, those changes are reflected on the Control Panel, change the equipment operation, and updates the state of those devices in Indigo. So the only messages I can check in Explorer are the same whether they come from the web interface or directly from the System Control Panel..

Looking for more documentation from Pentair makes sense as its possible that what I'm trying to do simply isn't implemented in what at least seems to be a poorly designed MQTT interface to start with. At the very least, I need to find what commands are actually needed, as it appears they don't actually show up in Explorer. In my feeble mind, both the web interface and the control panel communicate as they should, but Explorer isn't really showing those message exchanges, just the result of those messages. Does that make sense?

Guess its time for more Pentair research.

If the web interface is running on the rPi, and is controlling the Pentair via MQTT, then you should see those commands in MQTT explorer. Make sure you're subscribed to the '#' topic just to make sure you see ALL the traffic.

Mosquitto is the MQTT broker. It's the conduit between all the MQTT clients, which includes MQTT Explorer, the Indigo plugin, and whatever is on the rPi communicating with the Pentair. If the rPi is using MQTT to control the Pentair directly, then they should show up in Explorer. What connections do you have from the rPi to the Pentair? Serial cable? Ethernet?

[jltnol]

If the web interface is running on the rPi, and is controlling the Pentair via MQTT

And THAT is the $64,000,00 question. All of the web interface stuff was set and working long before I dove into the MQTT pool... so to speak.

So just doing some research, I ran across this link https://www.npmjs.com/package/nodejs-poolcontroller. This is the rPI service I'm using. You can scroll down to this headine

Home Automation Bindings (previously Integrations)

I'm wondering if I need to run the the "MQTT binding" operation, or because I can already "listen" to MQTT, if that has most likely been done.

[jltnol]

Hey Joe

Hope all is well.

You've been a huge help with this in the past, and I think we both agreed that it wasn't going to be possible for Indigo to control the pool stuff, but at least I could get data via MQTT into indigo.... mainly interested in the water temp. And while I DID have this working, I've just noticed that it has stopped.

Not sure what's changed but no doubt a lot. The IP address for the PI hasn't changed so I can rule that issue out.

I'm hoping to start on the Pi end of things to see if it's still sending data, but vaguely recall there is an app concerning MQTT on the Pi. I seem to recall an option for "Apps", under the Raspberry icon, which I don't see anymore. Should there be an APPS menu? Or do I need to configure the MQTT stuff on the PI via Terminal? If there is an app, I'll work on getting the menu item back so I can access the apps.

We're in the middle of a snow storm right now, very unusual for this far south, so I'm in no hurry on this.

And as always... thanks for your help

[FlyingDiver]

Honestly, I know nothing about MQTT on an rPi, OTHER than you can install an MQTT broker there.

The first thing you need to do is make sure that the MQTT broker is still running, the MQTT Connector device (Indigo plugin) is still working and connected to it, and that the MQTT Shim device is using that Connector. You can enable Debug logging in both of those plugins and see if the MQTT messages are being received. And use an MQTT client app (MQTT Explorer or MQTTx or others) connected to the same broker and is seeing the messages.

[jltnol]

Got it.

I really need to re-read this whole post, as I'm guessing most of my questions are already answered. But finding the MQTT Broker and making sure it's still running seems like the 1st order of business.

I'll also set up debugging as well.

Thanks!

[jltnol]

First, I’m assuming that basically, there is an MQTT broker on the PI, and that the Indigo MQTT Connector Device connects to that broker which is the point of contact between the two.

But the MQTT Connector plugin/Device isn’t connecting with the broker on the Pi. There isn’t anything to configure in the Indigo plugin, and all of the info I have for the device is correct…. Including the IP address, port, username and password. So am just going to have to work on the MQTT Broker on the PI to make sure I’ve got it set up and configured correctly.

But I’m also pasting the debug log so if you see something that needs addressing from the Indigo side, please let me know. From what I can tell, it connects, but then disconnects for whatever reason. As always, thanks for your time and help.

Code: Select all

Started plugin "MQTT Connector 2023.2.1"
   MQTT Connector                  MQTT Connector to Pi: Starting Device
   MQTT Connector Debug            MQTT Connector to Pi: Device is current version: 1
   MQTT Connector Debug            MQTT Connector to Pi: Broker __init__ address = 192.168.7.128, port = 1883, protocol = 4, transport = tcp
   MQTT Connector Debug            MQTT Connector to Pi: Enabling library level debugging
   MQTT Connector Debug            MQTT Connector to Pi: do_connect
   MQTT Connector Debug            MQTT-Pool *** B O T H ***: Adding Trigger
   MQTT Connector Debug            MQTT-Pool AIR Temps: Adding Trigger
   MQTT Connector Debug            Sending CONNECT (u1, p1, wr0, wq0, wf0, c1, k60) client_id=b'indigo-mqtt-219216085'
   MQTT Connector Debug            Received CONNACK (0, 5)
   MQTT Connector Debug            MQTT Connector to Pi: Connected with result code 5
   MQTT Connector                  MQTT Connector to Pi: Subscribing to: easytouch2-4/# (0)
   MQTT Connector Debug            Sending SUBSCRIBE (d0, m1) [(b'easytouch2-4/#', 0)]
   MQTT Connector Error            MQTT Connector to Pi: Disconnected with result code 5
   MQTT Connector Debug            Sending CONNECT (u1, p1, wr0, wq0, wf0, c1, k60) client_id=b'indigo-mqtt-219216085'
   MQTT Connector Debug            Received CONNACK (0, 5)
   MQTT Connector Debug            MQTT Connector to Pi: Connected with result code 5