Kafka producer plugin
Posted:
Tue Jul 18, 2017 3:14 pmby smurfless1
Publish JSON of device changes to Kafka!
https://github.com/smurfless1/indigo-kafkaKafka is a useful way to keep extended logs of changes, and it's open source. If you have or use one, this is a way to keep device changes in json in a kafka topic.
Re: Kafka producer plugin
Posted:
Sun Jul 30, 2017 8:07 pmby smurfless1
For those not familiar, Kafka is mostly like a DVR for log entries. It has channels and a limit of how many to keep. Then scripts/software can subscribe to a channel (called a topic) for either realtime or catch-up processing. So, you can store "1 TB of my Indigo info" or "1 year of Indigo info". And it lets clients keep track of where they left off, so if they disconnect, they just resume where they should, or they can decide to rewind. [ editor - I decided I hated my description. Retrying. ]
This is a terrible description of some awesome software, which I will point out is open source and free, and does much, much more. But the idea is that this is for storage, not querying or logic, but it makes querying and logic in external scripts easier.
Version 0.5.2:
* Strings that can be casted to floats will get a duplicate key.num, like "temperatureF" would also get "temperatureF.num" if possible.
* Add a Unix epoch timestamp field. This makes it possible to replay out of Kafka into InfluxDB and get nearly the same information as the Influx plugin.