Cellular GPS: Indigo, MQTT, Owntracks

This is definatley a work in progress, but I think I'm making enough progress that I thought I should share. (Hopefully someone smarter than I can polish this process up).

There are a lot of cell phone GPS options that may or may not work with indigo. For various reasons, all had a shortcoming somewhere given what I wanted to be able to do.

I want to be able to have a list of devices which represent all the member of my house. Each device should be able to contually update with their location. I want geofence areas that let me know who is in that geofence. I want some mapping ability. I want the ability to get notifications based on movement. (My favorite so far, I call a geo-lock. It is an action that creates a geofence on the fly given the user's current location with triggers pre-set to notifiy me... more on that later.

Owntracks is very similar to Life360. Biggest difference, Owntracks is free, it runs on your server (no cloud) so you (and only you) have the data. There are some docker methods, I found it easiest to use their quicksetup guide and dedicate a raspberry pi to the job.

Once owntracks is setup on a raspberry pi, the next step is getting it linked up with Indigo. The best fit for this task is Flying Diver's MQTT stuff: MQTT Connector and MQTT Shims.
MQTT Connector: https://www.indigodomo.com/pluginstore/211/
MQTT Shims: https://www.indigodomo.com/pluginstore/210/

STEP 1:
To connect to the MQTT Broker (set up on the Raspberry Pi... packaged part of the Owntracks Install), You will need to use the login / password for the user "_lr"
Connection Info:

IP Address (or hostname) of your Raspberry Pi
Port: 8883
MQTT Protocol: MQTTv311
Transport Protocol: TCP
Username: _lr
Password: find it by typing this in terminal or on the raspberry pi.

Code: Select all

sudo nano /usr/local/owntracks/userdata/.lr.pw


I saved this device as "Owntracks Broker". Call it whatever, but that's what I'll refer to from here on.

Save and that's it with the MQTT Connector setup.

STEP 2:
To add a device, I used MQTT Shims.... Start with a trigger, for this I'll use my phone as the example:
Type: MQTT Connector Event
Event: Topic Component Match
Event Settings: Select the Owntracks Broker (the step above) and you will want the topic strings to look like this:

Match: owntracks
Match: bill
Any:

Queeu Message (check)
Message Type: ##billPhone##
Done with that step. On to the last step.

STEP 3:
Create a shim device (This will be the device that actually represents my phone)

Type: MQTT Shims
Model: MQTT Generic JSON Device
Name: MQTT-Bill-Phone (obviously you will name it whatever... unless your name is also Bill)

Settings:

MQTT Broker: Owntracks Broker
Message Type: ##billPhone## <--- Super important this matches the Message Type from the trigger.
Unique ID Location: Payload Field
Payloadk Key: _type
Unique ID Value: location <--- since my trigger will fire on any owntracks/bill... that will include any message my phone sends (contact card, waypoints, etc.). Adding location here makes sure that only the "location" messages populate the states of this device.
Multi-States Key: . <---- hard to see, but don't forget that little period there.
Custom Decoder: None
Device Supports status requests (check) <--- I havn't figured out if this does anything.... but that's what I have.
Status Requet Topic: owntracks/bill/myphone/cmd
Status Request Payload: {"_type":"cmd", "action":"reportLocation"}
Device reports battery status (check)
Battery Payload Key: batt
Energy (blank)
Power Load (blank)

That's it for the shim device.

Go to your phone, in the owntracks app, once you send your location, you should see the information in the shim device populate with your phone data.

Geofences...

In owntracks, they call their geofences several things in the documentation. On the phone app, they are called "Regions" . Once your device is in one of them, you will have two states appear: inregions and inrids. inrids is the key for the region. If you push a new geofence (called waypoints on the MQTT side), if it has the same inrids variable, it will overwrite the other one. This is handy.

To keep the "Regions" the same across devices, I found it easier to type up the JSON code first, plop it into an MQTT Publish action and adjust the topic to align with each phone. Now all the phones in my house have the same "Regions" with the same "inrids" and "inregions" names. If I add geofences to the list, I just re-publish them out to the devices so I can centrally manage it all.

It looks like this:
MQTT Connector Actions (Publish Message)
Device: Owntracks Broker

Action Settings:
Topic: owntracks/bill/myphone/cmd (FYI, for everyone's device, it is "myphone". So the name "bill" is the unique value. I don't have to remember who has what type of phone, or change everything when someone upgrades, etc.

Payload:

Code: Select all

{
   "_type": "cmd",
   "action": "setWaypoints",
   "waypoints": {
      "_type": "waypoints",
      "waypoints": [
      {
         "_type": "waypoint",
         "tst": 1708703480,
         "rid": "my-region-id-001",
         "desc": "Home",
         "rad": 250,
         "lat": 29.61456,
         "lon": -94.84806
      },
      {
         "_type": "waypoint",
         "tst": 1708703495,
         "rid": "my-region-id-010",
         "desc": "Cline",
         "rad": 250,
         "lat": 30.14447,
         "lon": -95.51607
      }
      ]
   }
}


Note: "tst" doesn't have to be accurate, but you will have some weird recorder files on the backend of owntracks if you do any data mining later... "tst" is Unix time. I used the time that I created the list (roughly). You can find "Unix Time" by https://www.unixtimestamp.com/

So that's the indigo way to upload geofences / waypoints / regions onto your phones...

Now make a indigo geofence.....

STEP 1:
Make a Variable Device https://www.indigodomo.com/pluginstore/269/ for each Geofence you create.
You will need 2 variables for each variable device:

GeoFence_Home_Members
GeoFence_Home_Occupied

STEP 2:
Create 2 triggers (These two trigger will work for an unlimited number of geofences)
1st Trigger: I called mine: MQTT Match All Events

Type: MQTT Connector Event
Event: Topic Component Match
Select Broker: Owntracks Broker
Topic Strings for matching

Match: owntracks
Any:
Match: myphone <-- put another "Any" here if you used different things for different folks (iPhone5, Galaxy7, etc.)
Match: event
Queue Message (check)
Message Type: ##allEvents##

Before you move on to the last trigger, you need another shim. This shim will now update whenever any phone publishes an "Event". (Different from a location publish)
I called mine "MQTT-All Events"

MQTT Shims
MQTT Generic JSON Device
MQTT Broker: Owntracks Broker
Message Type: ##allEvents##
Unique ID Location: Payload Field
Payload Key: _type
Unique ID Value: transition <--- So this should only update if the "Event" is a "Transition" event with an Enter or Exit.
Multi-States Key: . <-- Hard to see, another period is there.

Save.

2nd Trigger: I called mine: MQTT Event: Geofence Processing
Type: MQTT Shims Event
Event: Device Updated
Shim Device: MQTT-All Events
(not sure if I needed this part, but)
Condition: If device MQTT-All Events sk_type is equal to transition
Action: Execute Script (This should update all geofence whenever anyone enters or exits any geofence)... so it should stay up-to-date without running unnecessarily.

Code: Select all

### Get regions from each device.  Use "Try" because if they aren't in a region, the state won't exist.  So return a " " value instead.


try:
   Bill = indigo.devices[1183167460].states["inregions"] # State "inregions" of "MQTT-Bill"
except:
   Bill = " "
   
try:    
   Vikki = indigo.devices[721909935].states["inregions"] # State "inregions" of "MQTT-Vikki"
except:
   Vikki = " "   

try:
   Anna = indigo.devices[204130087].states["inregions"] # State "inregions" of "MQTT-Anna"
except:
   Anna = " "
   
## Check Geofence Home ##
gfHomeMembers = ""
gfHome = "Home"
gfHomeOccupied = False

if gfHome in Bill:
   gfHomeMembers = gfHomeMembers + "Bill, "
   gfHomeOccupied = True
   
if gfHome in Vikki:
   gfHomeMembers = gfHomeMembers + "Vikki, "   
   gfHomeOccupied = True   
   
if gfHome in Anna:
   gfHomeMembers = gfHomeMembers + "Anna, "
   gfHomeOccupied = True      

if gfHomeOccupied == True:
   indigo.server.log("Geofence: " + gfHome + " is Occupied By: " + gfHomeMembers)
   indigo.variable.updateValue(836883327, value=gfHomeMembers) # "GeoFence_ISL_Home_Members"
   indigo.variable.updateValue(933218063, value="true") # "GeoFence_ISL_Home_Occupied"

else:
   indigo.server.log("Geofence: " + gfHome + " is Empty")
   indigo.variable.updateValue(836883327, value="Empty") # "GeoFence_ISL_Home_Members"
   indigo.variable.updateValue(933218063, value="false") # "GeoFence_ISL_Home_Occupied"      


## Check Geofence FHS ##
gfFHSMembers = ""
gfFHS = "FHS"
gfFHSOccupied = False

if gfFHS in Bill:
   gfFHSMembers = gfFHSMembers + "Bill, "
   gfFHSOccupied = True
   
if gfFHS in Vikki:
   gfFHSMembers = gfFHSMembers + "Vikki, "   
   gfFHSOccupied = True      

if gfFHS in Anna:
   gfFHSMembers = gfFHSMembers + "Anna, "
   gfFHSOccupied = True   

if gfFHSOccupied == True:
   indigo.server.log("Geofence: " + gfFHS + " is Occupied By:" + gfFHSMembers)
   indigo.variable.updateValue(1311857680, value=gfClineMembers) # "GeoFence_FHS_Members"
   indigo.variable.updateValue(869336078, value="true") # "GeoFence_FHS_Occupied"

else:
   indigo.server.log("Geofence: " + gfFHS + " is Empty")
   indigo.variable.updateValue(1311857680, value="Empty") # "GeoFence_FHS_Members"
   indigo.variable.updateValue(869336078, value="false") # "GeoFence_FHS_Occupied"

   

That's it. You now have geofences as devices.

Note: With Owntracks, you can be in multiple regions. The above should work with that. Owntracks will work with GPS (phone) or with iBeacons (which I havn't played with yet). So, you could easily be within multiple "Regions" or geofences, etc.

Other note... If you want to Delete a region from a phone remotely, I'll cover this in another post... but in short, make sure the key "inrids" is the same and change the Lat/Lon to something impossible. Like Lat: 100, Lon: 500. That will remove the "Waypoint" from the device.

One of my favorite tweaks to this.... GeoLocks. This is the ability to create a geofence on the fly from an indigo button/virtual on/off switch. Case use: You drop your kid off at a new location (not a regular location where a pre-existing geofence would be .. school, work, etc.). You want to get notified if your kid wanders away from that location. Traditionally, you would have to drive back over there, grab your kid's phone, create a geofence, drive back home, add that fence to indigo, write up some triggers... by this time, it's time to pick up your kid anyways, so you don't bother.

Step 1: Create an action for every device (i.e., person) you will want to GeoLock:

MQTT Connector Action / Publish Message
Device: Owntracks Broker

Settings:
Topic: owntracks/bill/myphone/cmd <--- Copy/paste the action for another user, change "bill" to the next user you want a GeoLock on.
Payload:

Code: Select all

{
   "_type": "cmd",
   "action": "setWaypoints",
   "waypoints": {
      "_type": "waypoints",
      "waypoints": [
      {
         "_type": "waypoint",
         "tst": %%d:1183167460:tst%%,
         "rid": "my-lock-id-001",
         "desc": "GEOLOCK_Bill|1|2",
         "rad": 50,
         "lat": %%d:1183167460:lat%%,
         "lon": %%d:1183167460:lon%%
      }
      ]
   }
}

Notice the device substitutions? "1183167460" is the shim device for Bill's phone. This way it will take the current user location and use it to create a geofence.
Note: (Optional) Change the description for each user.
Note: See the |1|2 after the description? That is optional. That tag on the end will change how the user's phone reports locations when entering and leaving that location. It will change the reporting to 1 (only a singificant movement will be reported after enter) and 2 (all movement will be reported after exit). My thought: Mimimize reporting while inside a geofence to save battery life.

Step 2: Create an action for every device (i.e., person) to remove a GeoLock:

MQTT Connector Action / Publish Message
Device: Owntracks Broker

Settings:
Topic: owntracks/bill/myphone/cmd <--- Copy/paste the action for another user, change "bill" to the next user you want a GeoLock on.
Payload:

Code: Select all

{
   "_type": "cmd",
   "action": "setWaypoints",
   "waypoints": {
      "_type": "waypoints",
      "waypoints": [
      {
         "_type": "waypoint",
         "tst": 1708703484,
         "rid": "my-lock-id-001",
         "desc": "GEOLOCK_Bill|1|2",
         "rad": 50,
         "lat": 100.0,
         "lon": 500.0
      }
      ]
   }
}

For that delete/remove thing to work... three of the elements in the payload have to be there:

The "rid" has to match the geofence you are removing
Lat and Lon have to be bad (don't exist on our planet) i.e., 100 / 500

Step 3: Create an trigger for any GeoLock enter/leave:
Very similar to the Geofence Processing trigger in my previous post
Trigger:

I called mine: MQTT Event: GeoLock Trigger
Type: MQTT Shims Event
Event: Device Updated -> Edit Event Settings : Shim Device: MQTT-All Events
Conditions: If device MQTT-All Events desc contains GEOLOCK

Action: This will now run anytime there is an enter or leave event with a GEOLOCK (waypoint/geofence/region) I chose to put it all into a python script. The script crates a friendly Date/Time variable to attach to the message as a reference. Then it grabs two things from the device "MQTT-All Events", the "tid" which is the two digit code for the person, and the "event" which will be either "enter" or "leave". Then the script matches the two digit code with a friendly name, i.e., BM --> "Bill", etc. Then it gets put into a message variable, theMessage , and sent via Pushover

Code: Select all

### Get the current date and time and put in a varible to add to the message later ###

import datetime
now = indigo.server.getTime()
dtg = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")

### Get the event type and "tid".  Match "tid" with a friendly name.

Event = indigo.devices[609931909].states["event"] # State "event" of "MQTT-All Events"
Person = indigo.devices[609931909].states["tid"] # State "tid" of "MQTT-All Events"
if Person == "VM":
   Name = "Vikki"

   if Person == "BM":
      Name = "Bill"

      if Person == "HM":
         Name = "Harper"

         if Person == "WM":
            Name = "William"
   
            if Person == "AH":
               Name = "Anna"

               if Person == "MH":
                  Name = "Mikey"

                  if Person == "RH":
                     Name = "Rylee"

                     if Person == "BX":
                        Name = "Bexley"

else:
   Name = "Unknown"

### Build the message ###
theMessage = (Name + " " + Event + " GEOLOCK at " + dtg)
indigo.server.log(theMessage)

### Send the message via pushover using the security token ###
import requests
r = requests.post("https://api.pushover.net/1/messages.json", data = {
  "token": "my_app_specific_token_for_security_events",
  "user": "my_pushover_user_key",
  "title": "GEOLOC Event!",
  "message": theMessage
})
print(r.text)

Lastly, you will want a Virtual Device for every person that will have a GEOLOCK.
Create a variable for each person, i.e., "GeoLock_Bill"
Now edit the two actions you made earlier to create and remove the GEOLOCK.
For the create GEOLOCK action, add a new action:

Modify Variable
Variable: GeoLock_Bill
Set to true.

For the remove GEOLOCK action: Do the same thing, but "Set to false"

Final step: Create the Virtual Device

Type: Virtual Devices
Model: Virtual On/Off Device
Name: MQTT_GEOLOCK_BILL (call it whatever... that's just how I'm naming stuff)

Device Settings:

Execution Model: Action Groups
On Group: MQTT Publish GeoLock - Bill
Off Group: MQTT Publish GeoLock - Bill (remove)
Supports Status (check)
Status Variable: GeoLock_Bill

Now you have a virtual on/off switch that will put a fence around someone or something.

Phone View (Owntracks Website Example.... becasue I'm not posting a picture of where all my children are on the internet. )

Device view of the software.

ipad-public-map.png (155.44 KiB) Viewed 1386 times

From my home computer, logging onto the raspberry pi at https://Your_DNS.com/owntracks
Table View with address information cropped out of image:

Table view... address information cropped out.

Screenshot 2024-02-26 at 10.39.47 AM.png (56.26 KiB) Viewed 1386 times

FrontEnd of the software. Lots of viewing options.

FrontEnd of computer interface

Screenshot 2024-02-26 at 10.38.47 AM.png (335.04 KiB) Viewed 1386 times

I'm trying to figure out how to graphically connect Indigo with Owntracks. This is what I'm trying to work around:
You can send a webpage to Owntracks. It will pop up on a users (Featured) tab. If you don't have that tab, that's just because you havn't used it. Once you send something to it... like a note or a web page (thinking control page), then the tab appears and will open directly to that page. But, with the control page security changes, I havn't figured out how to get an indigo control page to open on the Owntracks app.

Going in the other direction.... none of the pretty maps from the Owntracks interface are static .jpg or similar... so I can't figure out a way to add the owntracks graphics to an indigo control page.

I may go a different route and use the owntrack location data to build a seperate map within Indigo for use on a control page or for sending a notification with a map image.... I have the scripts for that somewhat worked out.

Some might be thinking: Shouldn't this just be a plugin? I agree, but it's beyond my skill level with plugin creation... but if anyone wants to steal what I've got so far and write a plugin for this... I'll help where I can.

This sounds very cool! It's not something that we would use--it's just my wife and me--she knows where she is already and doesn't care where I am.

In all seriousness, one thing that might work for you is to use the Indigo Web Server to display content. Doing that would require some manner of programming, but Ive found the framework to be very robust. It's a great feature of Indigo. I've written a personal web app that contains HTML, JavaScript, Vue JS, Bootstrap, and other libraries and the Indigo Web Server has just eaten it up. Once the files are created/compiled, all you have to do is save them to the ./Web Assets/static/ folder and then you can access them both locally and remotely (with the proper Indigo preferences configured, of course).

use the Indigo Web Server to display content. ...save them to the ./Web Assets/static/ folder and then you can access them both locally and remotely (with the proper Indigo preferences configured, of course).

Interesting. I'm going to have to polk around with that at some point. At the moment, I'm still up to my eyeballs figuring out MQTT stuff.... still wrapping my head around topics / shims / connector / etc.

This sounds very cool! It's not something that we would use--it's just my wife and me--she knows where she is already and doesn't care where I am.

In all seriousness, one thing that might work for you is to use the Indigo Web Server to display content. Doing that would require some manner of programming, but Ive found the framework to be very robust. It's a great feature of Indigo. I've written a personal web app that contains HTML, JavaScript, Vue JS, Bootstrap, and other libraries and the Indigo Web Server has just eaten it up. Once the files are created/compiled, all you have to do is save them to the ./Web Assets/static/ folder and then you can access them both locally and remotely (with the proper Indigo preferences configured, of course).

Help me wrap my head around this: you are not talking about using control pages... I'm familiar with using the indigo public folder, but did your web app require a custom plug-in or did it automatically interface with indigo?

Also, you guys rule

The project has two components:

1. Plugin - the primary purpose of the plugin is to provide a websocket communication channel which allows bi-directional communication between Indigo and the web app. I started this project before Indigo's API v2 which has a built-in websocket framework or I would've used that. The plugin also has a series of functions that respond to commands from the web app. For example, when you turn on a light, the command goes through the websocket to the plugin and then the plugin asks Indigo to turn on the light. Indigo's built-in websocket API handles that directly.
2. Web Content - this is all the compiled HTML/JavaScript/CSS etc. that comprises the web app. These files are placed in the folder mentioned above and calls to http://10.0.1.XXX:8176/static/index.html#/ cause the content to be displayed.
   
   I'm in the midst of a redesign of the UI, so it's super ugly at the moment. Here's the last iteration that didn't look too bad.

This sounds very cool! It's not something that we would use--it's just my wife and me--she knows where she is already and doesn't care where I am.

I'm pretty much in the same boat. This is just my nerdy side taking over. For tracking kids, my wife is content with Life360 and/or FindMy. (We are a blended family, so we are over the Apple limit of family members). FindMy doesn't cover all my portable devices and Life360 discontinued any home automation support.

Here's the last iteration that didn't look too bad.

Didn't look too bad? That looks amazing! That initial dashboard (first 3 seconds) very nice. The remianing shows a level of indigo integration that looks incredibly detailed, and professional. (Not sure how to word it....) Like, indigo touch is very limited, and very 1990 in feel. Domopad adds a lot more control, but still a 1990 sorta feel. This looks like a much more modern, almost like an industrial control system meets Home Assistant wrapped around indigo?

Damn... Now I have to get back into indgo and start playing with it again. (I took a break to learn about Ham Radios)

So, now I know the benefit of putting stuff in that folder, how to access that folder.... now I just have to figure out the content to put into that folder. Is your web app going to be a packaged product as some point?

This is just my nerdy side taking over.

I've got chronic nerd.

Didn't look too bad? That looks amazing!

You're too kind. I thought it looked a bit dated, but I could live with that. The main reason I dug back into the design was to make it more responsive--that is, to display nicely regardless of the device viewing it. This design -- especially the weather landing page -- didn't play too well with mobile. Bootstrap can only do so much when the design doesn't take full advantage of its features. Though I must admit that I'm pretty proud of this first effort since I didn't know what I was doing.