Glowmarkt access to live smart meter data (UK)

Posted on
Sun May 17, 2020 3:18 pm
neilk offline
Posts: 321
Joined: Jul 13, 2015
Location: Reading, UK

Glowmarkt access to live smart meter data (UK)

Hi all,
While waiting for my account to move to Octopus to get the consumption data, I came across https://shop.glowmarkt.com/collections/all/products/display-and-cad-combined-for-smart-meter-customers which replaces the rubbish in home display you get free and gives access to real time consumption data by talking directly to a SMETS 2 smart meter. The data still goes via the cloud as the regulator does not allow direct access to the data.

It looks very viable to match the currrnt variable rate from Octopus and to combine it with actual usage. This works independently from your provide as the supplier is licensed to access the meter data so should work with any SMETS 2 meter. Tempted to buy one, has anyone tried it ?

Neil

Posted on
Sun May 17, 2020 3:32 pm
howartp offline
Posts: 4083
Joined: Jan 09, 2014
Location: West Yorkshire, UK

Re: Glowmarkt access to live smart meter data (UK)

Well I was.... just on my way to bed!!!!


Sent from my iPhone using Tapatalk Pro

Posted on
Sun May 17, 2020 3:44 pm
howartp offline
Posts: 4083
Joined: Jan 09, 2014
Location: West Yorkshire, UK

Re: Glowmarkt access to live smart meter data (UK)

I've just signed up to the Bright app and given them my MPAN / SMETS2 meter details etc etc - i've to wait a couple of days for them to approve me then we'll see what happens.

(This stage is free)

Peter

Posted on
Sun May 17, 2020 4:06 pm
neilk offline
Posts: 321
Joined: Jul 13, 2015
Location: Reading, UK

Re: Glowmarkt access to live smart meter data (UK)

Same for me, let’s see what we get.
I think I will order the hardware as well, they have an API and a MQTT service.

Neil

Posted on
Thu May 21, 2020 2:05 pm
neilk offline
Posts: 321
Joined: Jul 13, 2015
Location: Reading, UK

Re: Glowmarkt access to live smart meter data (UK)

Peter,
My IHD/CAD arrived today, was pre-configured and starting working within minutes of adding my wifi details. I have managed to get the payload from the MQTT plugin working and I am getting my head around the shims plugin. I looks very do-able but the JSON is a bit cryptic and straight from the 600 page Zigbee smart meter standards.
Thanks,
Neil

Posted on
Fri May 22, 2020 1:56 am
siclark offline
Posts: 1234
Joined: Jun 13, 2017
Location: UK

Re: Glowmarkt access to live smart meter data (UK)

Does this mean I finally have a reason to get my meters upgraded to smart ones?

Posted on
Fri May 22, 2020 2:05 am
howartp offline
Posts: 4083
Joined: Jan 09, 2014
Location: West Yorkshire, UK

Re: Glowmarkt access to live smart meter data (UK)

I’m really interested in this; I was wondering how on earth they were expecting me to ring SSE and ask very nicely for them to unlock the smart meter so I could pair the new 3rd party device to it.

The fact that it’s come pre-configured suggests and proves that any authorised energy supplier can “log into” any SMETS2 smart meter in the country and add a new IHD or other device to the trusted/paired devices list in the meters config, and opens up a whole new world of possibilities for third party energy companies to offer new features like Glow are doing.


Sent from my iPhone using Tapatalk Pro

Posted on
Fri May 22, 2020 5:28 am
neilk offline
Posts: 321
Joined: Jul 13, 2015
Location: Reading, UK

Re: Glowmarkt access to live smart meter data (UK)

Peter,

The fact that it’s come pre-configured suggests and proves that any authorised energy supplier can “log into” any SMETS2 smart meter in the country and add a new IHD or other device to the trusted/paired devices list in the meters config, and opens up a whole new world of possibilities for third party energy companies to offer new features like Glow are doing.


That is exactly their model, they have the authority to pair with the smart meter and export the data into their cloud service (which is a requirement you cannot provide direct local access). They took the details from my meter and my existing IHD and as a result they do not need permission from your supplier to pair to your meter.

Their MQTT service basically passes on the zigbee message they get from the meter(s) (the gas meter connects via the electricity meter and is battery powered) wrapped in JSON every 10s. Getting my head around decoding that as it is a lower level from than the Octopus API.

Their API however can access historical data by the minute/30 min/hour/day/week/month as they access the same data your energy supplier uses for billing and again access to this is independent of your energy supplier. I am planning a plugin that will pull the historical data, and map that to the Octopus Agile tariff so you get realtime cost, but that will work for any SMETS2 meter (the ones that remain smart when you swap suppliers and mandated for new installs) regardless of supplier if you sign up for a Glowmarkt/Bright account. This may not be necessary if the tariff info is pushed, but I suspect that Octopus map it on after the event combining usage with the then applicable rate.

Currently getting my head around the right approach to use with MQTT, SHIMS or my own plugin that takes the raw JSON payload and maps to device states for incoming messages and suspect I will be calling for help over the weekend on the Shims configuration. I have a feeling some post processing will be required to make sense of the raw Zigbee data.


The endpoint design for me would be :-

Octopus Plugin
1) As now published tariff data without needing to be an Octopus Customer to track pricing (current functionality)
2) Consumption data for those people with an account and API Key that will get "yesterdays data and earlier" for Gas and Electricity all in a single plugin

Glowmarkt Plugin (API based)
1) Regardless of energy supplier historical data if you have a GlowMarkt account
2) Mechanism to combine the octopus tariff data with the consumption data from Glowmarkt

Glowmarkt MQTT Configuration (Ideally using the existing MQTT Plugin and potentially Shims)
1) Publish and consolidate real time usage and tariff data to give you current actual cost (for fixed rate contracts, and will include the ability to deal with economy 7 or EV tariffs that vary rates for certain hours). Goal is an Indigo device that gives current actual consumption and cost (every 10s)


I would ideally like to leverage the existing tooling around MQTT, I already am able to collect the JSON payload from that and the configuration was straightforward as that is much cleaner solution.

With the approach I suggest I think we will end up with greater flexibility for people who have different suppliers, tariffs and configuration. The Hildebrand support people are incredibly responsive and supportive, as are Octopus. My talent may be a limiting factor but with the help and patience I think the progress is good.

Thanks,
Neil

Posted on
Fri May 22, 2020 5:32 am
neilk offline
Posts: 321
Joined: Jul 13, 2015
Location: Reading, UK

Re: Glowmarkt access to live smart meter data (UK)

siclark wrote:
Does this mean I finally have a reason to get my meters upgraded to smart ones?


It was getting the EV that triggered me to consider it (the fuel costs are so much lower that it isn't material, but given making the move it made sense to optimise).

By the time the energy companies are in a position to resume smart installs I am confident we will have some strong capability for you to play with (just bear with me as it is just a hobby and my Python is still fledgling). Peter has been a great help in his QA effort and patience.

Posted on
Fri May 22, 2020 6:28 am
howartp offline
Posts: 4083
Joined: Jan 09, 2014
Location: West Yorkshire, UK

Re: Glowmarkt access to live smart meter data (UK)

My EnergyLink plugin is well under way which may play into this game as well.

It doesn’t negate the need for your MQTT or own plugin to parse the SMETS2 data (which incidentally id prefer a separate plugin parsing JSON but that’s your shout) but may handle the combining of data/tariffs you mention.

Should have the plugin ready to alpha/beta this weekend.


Sent from my iPhone using Tapatalk Pro

Posted on
Fri May 22, 2020 9:56 am
neilk offline
Posts: 321
Joined: Jul 13, 2015
Location: Reading, UK

Re: Glowmarkt access to live smart meter data (UK)

Peter,

Spot on, I may need to pick your brains at times (as well as the rest of the hive) but I think it will be possible to use the standard capabilities to consume the MQTT data, and then have a separate plugin that transforms that into a more readily consumable data set if necessary.

(which incidentally id prefer a separate plugin parsing JSON but that’s your shout)


Thanks,
Neil

Posted on
Fri May 22, 2020 3:50 pm
aaronlionsheep offline
Posts: 61
Joined: Feb 24, 2019
Location: Virginia, USA

Re: Glowmarkt access to live smart meter data (UK)

neilk wrote:
Peter,

Spot on, I may need to pick your brains at times (as well as the rest of the hive) but I think it will be possible to use the standard capabilities to consume the MQTT data, and then have a separate plugin that transforms that into a more readily consumable data set if necessary.

(which incidentally id prefer a separate plugin parsing JSON but that’s your shout)


Thanks,
Neil


You could also take advantage of FlyingDivers MQTTConnector plugin which can act as a "library" for another plugin. You can pass data between plugins in these later versions of Indigo (7), so you can have your plugin focused on consuming data over an MQTT topic and let Joes plugin handle managing subscriptions and establishing connections.

Posted on
Fri May 22, 2020 4:25 pm
neilk offline
Posts: 321
Joined: Jul 13, 2015
Location: Reading, UK

Re: Glowmarkt access to live smart meter data (UK)

aaronlionsheep wrote:

You could also take advantage of FlyingDivers MQTTConnector plugin which can act as a "library" for another plugin. You can pass data between plugins in these later versions of Indigo (7), so you can have your plugin focused on consuming data over an MQTT topic and let Joes plugin handle managing subscriptions and establishing connections.


Thanks - just trying to get up to speed with that, but indeed my preferred approach. I have the payload arriving in the MQTT plugin but it breaks the Shims plugin due to the numeric tags. Can I consume the raw payload without the Shims plugin ? I saw the script referenced https://forums.indigodomo.com/viewtopic.php?f=316&t=22774 but when I try that I need to specify the Shim device in the action (I was going to try the script). Any pointers as I am happy to parse the JSON as the format is pretty static, but struggling if I should be picking it up from the MQTT direct

Posted on
Fri May 22, 2020 4:31 pm
aaronlionsheep offline
Posts: 61
Joined: Feb 24, 2019
Location: Virginia, USA

Re: Glowmarkt access to live smart meter data (UK)

neilk wrote:
aaronlionsheep wrote:

You could also take advantage of FlyingDivers MQTTConnector plugin which can act as a "library" for another plugin. You can pass data between plugins in these later versions of Indigo (7), so you can have your plugin focused on consuming data over an MQTT topic and let Joes plugin handle managing subscriptions and establishing connections.


Thanks - just trying to get up to speed with that, but indeed my preferred approach. I have the payload arriving in the MQTT plugin but it breaks the Shims plugin due to the numeric tags. Can I consume the raw payload without the Shims plugin ? I saw the script referenced https://forums.indigodomo.com/viewtopic.php?f=316&t=22774 but when I try that I need to specify the Shim device in the action (I was going to try the script). Any pointers as I am happy to parse the JSON as the format is pretty static, but struggling if I should be picking it up from the MQTT direct


That deviceId is actually the indigo device id for the broker device created by the Connector plugin.

Posted on
Fri May 22, 2020 4:41 pm
neilk offline
Posts: 321
Joined: Jul 13, 2015
Location: Reading, UK

Re: Glowmarkt access to live smart meter data (UK)

Doh - stupid naming on my part, got it working now and collecting the queued messages with the payload.

So if my interpretation is correct then I can simply consume these queued messages periodically and do what I need to the JSON. Looks like I need to get my head into the zigbee spec document now, Thanks for the help !

Who is online

Users browsing this forum: No registered users and 0 guests

cron