Kafka producer plugin

Posted on
Tue Jul 18, 2017 3:14 pm
smurfless1 offline
User avatar
Posts: 35
Joined: Sep 23, 2015
Location: Round Rock, TX

Kafka producer plugin

Publish JSON of device changes to Kafka!

https://github.com/smurfless1/indigo-kafka

Kafka is a useful way to keep extended logs of changes, and it's open source. If you have or use one, this is a way to keep device changes in json in a kafka topic.
Last edited by smurfless1 on Tue Jul 18, 2017 7:13 pm, edited 1 time in total.

Posted on
Tue Jul 18, 2017 4:19 pm
jay (support) offline
Site Admin
User avatar
Posts: 15146
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Kafka producer plugin

Another one, cool!

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Posted on
Sun Jul 30, 2017 8:07 pm
smurfless1 offline
User avatar
Posts: 35
Joined: Sep 23, 2015
Location: Round Rock, TX

Re: Kafka producer plugin

For those not familiar, Kafka is mostly like a DVR for log entries. It has channels and a limit of how many to keep. Then scripts/software can subscribe to a channel (called a topic) for either realtime or catch-up processing. So, you can store "1 TB of my Indigo info" or "1 year of Indigo info". And it lets clients keep track of where they left off, so if they disconnect, they just resume where they should, or they can decide to rewind. [ editor - I decided I hated my description. Retrying. ]

This is a terrible description of some awesome software, which I will point out is open source and free, and does much, much more. But the idea is that this is for storage, not querying or logic, but it makes querying and logic in external scripts easier.

Version 0.5.2:
* Strings that can be casted to floats will get a duplicate key.num, like "temperatureF" would also get "temperatureF.num" if possible.
* Add a Unix epoch timestamp field. This makes it possible to replay out of Kafka into InfluxDB and get nearly the same information as the Influx plugin.

Page 1 of 1

Who is online

Users browsing this forum: No registered users and 0 guests