Store series of values

Hi all,

I am integrating a script which connects via ModBus to my Solar Inverter; I get every 5 mins or so a value which represents how much kW my solar is generating; the target is to display them in Grafana, but how can I store them in Indigo?
The same is for the total generated kWh every day.

I though maybe with JSON with time/value pairs; the same for the daily kWh.

Thanks
Marco

[jay (support)]

I don't think we understand the environment: can you get it in JSON format? If not, what to you get from the Inverter? Is it getting it from a URL or something? The more specifics the better answer you're likely to get.

So basically I have a python script running in a schedule (for the moment) which downloads some data from my solar panel inverter.
I would like to store them somehow and then display them in a chart with Grafana.

Thanks

[jay (support)]

You didn't answer my questions:

I don't think we understand the environment: can you get it in JSON format? If not, what to you get from the Inverter? Is it getting it from a URL or something? The more specifics the better answer you're likely to get.

Specifics are key to helping you.

Hi Jay,

it is a simple python script which gives me a back a number, no JSON.
This python script uses a Modbus library to connect to the inverter and read the values it publishes
Therefore I need to find a way how to store this number and display then with Grafana.

If it's just 1 number every 5 minutes you can store it to a variable and then use SQL Logger to query for your reports.

[jay (support)]

How are you going to get it into Grafana? As @ryanbuckner you can store the value in an Indigo variable, but I don't know how to get it into Grafana after that.

I already thought to store the value in the variable, but the variable shall contain then the whole series, not only the last one.

As far as I know, it is possible with Grafana to show variables values, need to check.

If you want to save the whole series (data over time) then storing it in a variable would not be the best choice. I'd recommend saving the data to a file on the server machine in CSV or JSON format. I don't know anything about how to get the information into Grafana unfortunately.

Your best option in my opinion is to use a Python script attached to an Indigo schedule that reads the data from your solar array at a set interval, grabs the output value, and saves it to a data file. A couple things to think about if it were me doing it: 1) I would only run the schedule while the Indigo variable isDayLight is True, and 2) I would put some type of boundary on how much data to retain -- capturing every 15 minutes for a year is just over 35,000 data points (much less if you run it during daylight hours). Capturing it every 5 min as you suggest above would become huge very quickly.

How are you going to get it into Grafana? As @ryanbuckner you can store the value in an Indigo variable, but I don't know how to get it into Grafana after that.

I had suggested using a query against the data in SQL Logger to extract the data into Grafana

How are you going to get it into Grafana? As @ryanbuckner you can store the value in an Indigo variable, but I don't know how to get it into Grafana after that.

I had suggested using a query against the data in SQL Logger to extract the data into Grafana. 35,000 data points shouldn't be a problem for postgres

Yes, SQL is also an option. I'm partial to CSV, but I know others prefer the database approach. There's nothing wrong with either. One thing I like about CSV is its universality, but it's also nice because--if you need to edit the data for some reason later on--it's easy to do with any plain text editor.

If you can get the data into an Indigo variable, the Matplotlib Plugin's CSV Engine Device makes it easy to create CSV files. #ShamelessPlug

[jay (support)]

I think the bottom line here is that we need to know how you are getting data into Grafana. Based on that, we can recommend more specific solutions.

Grafana needs to query some time-series database., so you need to have one of those somewhere.

I think your easiest path would be to install the Grafana Home Dashboard plugin: https://www.indigodomo.com/pluginstore/167/ and store the data in the InfluxDB instance that creates.

This is roughly what I do - I use that plugin and send to an InfluxDB instance, and use Grafana for visualization. I just host that InfluxDB instance (and the Grafana that points to it) somewhere else (a Google Cloud VM), but the idea is the same and the logging mechanism is the same.

Depending on what your script is, it might make more sense to run the script outside of Indigo; it can still post values into the InfluxDB database.

well this looks like the answer sheet. Prob should move this to that forum