Page 1 of 2

How are you using Machine Learning in Home Automation?

PostPosted: Sun Sep 01, 2019 8:48 am
by ryanbuckner
How are you using Machine Learning in your Indigo execution? From my perspective, so far we've use AI only in the area of NLP, but most use cases involve the following of specific rules and schedules rather than models that learn.

To advance into the next stages with Indigo I think we need a new discussion forum specific to development of Machine learning. Using the data we capture through IOT to have Indigo training and learning to make decision rather than dictating when triggers fire.

So how are you using Machine Learning, AI, Deep Learning, Neural Networks?

How are you using Machine Learning in Home Automation?

PostPosted: Mon Sep 02, 2019 11:03 am
by mundmc
I’m taking the fast.ai course right now- I plan to make a predictive agent that estimates the next room that will be entered. First, I need to clean up and select device states from the sql database. I might try to pigyback on the awesome Grafana Influxdb plugin.

Edit: On further thought, one of the problems with machine learning and home automation boils down to my major concern: to whom am I giving my training/ validation data?

I’m off Google/FB out of respect for what ML/DL can do now. Who knows what it will be able to classify/forecast in three years?

If there was a way to easily train a neural network locally, I’d be all in. However, the idea of some company having precise details on the goings on of my home? No way.

What are you thinking of doing?

Re: How are you using Machine Learning in Home Automation?

PostPosted: Wed Sep 04, 2019 5:09 pm
by ryanbuckner
I was thinking a few use cases:

      Predict which device will fail next
      Predict which device will run out of battery next
      Recommend the type of clothing to wear
      Learn who is home based on behaviors

Presence detection seems to be the one area where my data is shit

Check this out: http://mews.sv.cmu.edu/papers/buildsys-17.pdf

Re: How are you using Machine Learning in Home Automation?

PostPosted: Wed Sep 04, 2019 7:29 pm
by kw123
I need: which colors match.
Blue pants w black shoes seem to be a no no


Sent from my iPhone using Tapatalk

Re: How are you using Machine Learning in Home Automation?

PostPosted: Sat Sep 07, 2019 10:55 am
by mundmc
ryanbuckner wrote:
I was thinking a few use cases:

      Predict which device will fail next
      Predict which device will run out of battery next
      Recommend the type of clothing to wear
      Learn who is home based on behaviors

Presence detection seems to be the one area where my data is shit

Check this out: http://mews.sv.cmu.edu/papers/buildsys-17.pdf


Great paper- i skimmed it but i need to read more in depth. It would be awesome to have more deep learning and prediction in HA. I am in the super early proof-of-concept planning phase.

Re: what to wear- my wife and her mom are unable to plan what to wear based on the outdoor climate. Then they complain. An easy non-ML project I have been planning is a program that sends them a notification, asking if they dressed comfortable upon their return home. They can give a y/n response. I then can link a still image from Blue Iris from their return home to the day’s conditions. Eventually, it will be able to send them notifications in the morning of pics of them in weather-appropriate outfits.

It certainly could benefit from ML, but theres so little data to look at, i think i can make my own algorithm.

Re: How are you using Machine Learning in Home Automation?

PostPosted: Tue Sep 10, 2019 6:13 pm
by DaveL17
FYI - there's a chat tomorrow via Adafruit on Machine Learning with Microcontrollers (11 Sept @ 3PM ET).

Re: How are you using Machine Learning in Home Automation?

PostPosted: Wed Sep 11, 2019 7:38 pm
by mundmc
DaveL17 wrote:
FYI - there's a chat tomorrow via Adafruit on Machine Learning with Microcontrollers (11 Sept @ 3PM ET).


Totally missed this!

Re: How are you using Machine Learning in Home Automation?

PostPosted: Wed Sep 11, 2019 8:34 pm
by DaveL17
I couldn't watch the chat because I was busy with my day job, but it looks like they've posted a transcript of the chat (and there may be a link to the livestream in there someplace accessible from the link above.

Re: How are you using Machine Learning in Home Automation?

PostPosted: Fri Sep 13, 2019 2:44 am
by GlennNZ
mundmc wrote:
Totally missed this!


Just for you!

Well have been trying to bring Machine Learning into BlueIris Automation... and mostly there for those {maybe super} keen

https://github.com/Ghawken/DeepQuestAI
(and needs new version of BlueIris Plugin >0.1.2 - and enable Broadcast within BI Plugin Config)

Mostly Documented on github I hope.

But uses DeepStackAI (background is a interesting read)
https://deepquestai.com/about/
Particularly the bit about sharing a broken laptop in a third world country

But anyway - is local, AI service., which you need to install somewhere on home network

OSX and Windows version - have only test Windows version; which has been running well for days now

Plugin - sends any image (via Action) but mainly/also integrates with BlueIris Plugin to send any motion trigger to your local DeepStack incidence.

Then checks for objects, primarily currently Vehicles and People
Can Triggers on both
Saves Images for both

Plugin also runs if wanted a HTTP server to serve images of above to Control Pages (so can skip through last People seen or last vehicle seen)

Multiple actions to change settings - do find night versus day some differences this is useful for.

I run mine on a old i7 Micro Windows Computer and takes about 800mS to run recognition.
So number of cameras/images etc. would be limit.

Currently running with two cameras/entry enabled, saving all people/vehicles seen
(probably will save above forever)

DeepStack - can do face recognition, and can train faces - so was planning on adding that at some stage (think that's a paid option though)
BUT - found that most camera images pretty average for face, so actually hard to clearly fine face/recognise etc.
May still be case for press button - triggered, close up picture, recognised as 'Whomever' do action/open door etc - but photo would do the same, so not that secure....

Welcome to test out - just download from github
https://github.com/Ghawken/IndigoPlugin ... s/releases
&
https://github.com/Ghawken/DeepQuestAI/releases


Glenn

Re: How are you using Machine Learning in Home Automation?

PostPosted: Fri Sep 13, 2019 6:34 am
by mundmc
^ Locally run ML/DL!? This is really great. Thanks so much for your work

Re: How are you using Machine Learning in Home Automation?

PostPosted: Wed Sep 25, 2019 7:55 pm
by mundmc
I still haven’t gotten to this yet- but i’m pleased to say I got my Cuda toolkit enabled after much confusion, and now am playing around with household-member, locally run, neural networks. OpenCV is awesome. Thanks again for this awesome plugin and i look forward to reporting back!

Aside- Is this a bad time for a feature request :) ?

How are you using Machine Learning in Home Automation?

PostPosted: Wed Sep 25, 2019 11:39 pm
by GlennNZ
mundmc wrote:
I still haven’t gotten to this yet- but i’m pleased to say I got my Cuda toolkit enabled after much confusion, and now am playing around with household-member, locally run, neural networks. OpenCV is awesome. Thanks again for this awesome plugin and i look forward to reporting back!

Aside- Is this a bad time for a feature request :) ?


Welcome to!

Latest BlueIris and latest DeepState Plugin all function on my system well.

Have added a number of aspects, best of which in my usage is the control page image server. I find this something the whole household checks daily - last person seen, last car, last object of anytype served as image to control page. This enables me to have a control page showing last object type found. Most of the function is documented in github, certainly with changes in changelog at end of readme.

Main other change now that I think about it is creating and using a RamDisk for all the temporary images used - currently creates a 256mb MAC system Ramdisk and uses this if the option selected. Think have covered the issues this might create, from restarting to recreating etc. -- but I was in a couple of days processing 1.6gigs of images - which I figured better in ram.

Away for a another week or so, so won’t be to quick looking at any suggestions but welcome to make them!





Sent from my iPad using Tapatalk

Re: How are you using Machine Learning in Home Automation?

PostPosted: Mon Sep 30, 2019 12:37 pm
by mundmc
I am very excited to tinker with this!

Re: How are you using Machine Learning in Home Automation?

PostPosted: Mon Dec 23, 2019 3:21 pm
by mundmc
Update- For perpetuity, @GlennNZ ‘s alpha/beta plugin for locally hosted computer vision with Blue Iris and Deepstack is awesome (albeit not for the faint of heart to set up).

In other news, Magic Mirror has a plugin for self-hosting facial recognition on a raspberry pi- it utilizes the OpenCV4 python package.

Re: How are you using Machine Learning in Home Automation?

PostPosted: Mon Dec 28, 2020 8:13 pm
by ryanbuckner
mundmc wrote:
ryanbuckner wrote:
I was thinking a few use cases:

      Predict which device will fail next
      Predict which device will run out of battery next
      Recommend the type of clothing to wear
      Learn who is home based on behaviors

Presence detection seems to be the one area where my data is shit

Check this out: http://mews.sv.cmu.edu/papers/buildsys-17.pdf


Great paper- i skimmed it but i need to read more in depth. It would be awesome to have more deep learning and prediction in HA. I am in the super early proof-of-concept planning phase.

Re: what to wear- my wife and her mom are unable to plan what to wear based on the outdoor climate. Then they complain. An easy non-ML project I have been planning is a program that sends them a notification, asking if they dressed comfortable upon their return home. They can give a y/n response. I then can link a still image from Blue Iris from their return home to the day’s conditions. Eventually, it will be able to send them notifications in the morning of pics of them in weather-appropriate outfits.

It certainly could benefit from ML, but theres so little data to look at, i think i can make my own algorithm.


With the iMessages plugin, Indigo could has them a yes/no question and record the answer. Also, with the new breakthrough with Announcements, they could be alerted what to wear when the wake up routine fires.