Google Vision Plugin

Posted on
Wed Sep 27, 2017 3:58 pm
vtmikel offline
Posts: 628
Joined: Aug 31, 2012
Location: Boston, MA

Google Vision Plugin

Hello,

I've built a plugin to utilize Google's Vision API that I'm testing. You can read about it yourself here: https://cloud.google.com/vision. The first 1,000 images analyzed each month are free.

If you are interested in how it works, I recommend taking a snapshot using your own security camera and sending it to the service using the above site.

The plugin allows you to create dynamic events of three different types:
  • Face Detection - Looks for faces in the image. You can configure the level of certainty that Google returns back to the plugin. Also supported is a flag to look for "no faces detected". Google's API's also support things like the likelihood that the face is showing joy, anger, sorrow, etc. I have not implemented those.
  • Label (Object) Detection - Looks for objects in photos. Easiest way to understand Google's dictionary is to use debugging and send your own image a few times to the service. You'll see the labels that Google uses for things like a car, automobile, etc.
  • OCR - I haven't tested this thoroughly, but the plugin can trigger an event if Google is able to OCR text in a image and you are looking for a particular substring.

Note: This plugin does not require to create any devices.

Once you create the events, you can create triggers for those events in Indigo.

Finally, you trigger the processing of the events by sending an image to Google Vision, an Action that the plugin supports. Images can be local to your Indigo Server or via HTTP protocol. The location/URL can be set statically or via Indigo variable.

For me, I'm using the plugin to occasionally analyze images from my home security cameras. This works well with SecuritySpy's feature to upload a web camera image to a remote server.

Plugin can be downloaded here: https://github.com/mlamoure/Indigo-Goog ... n/releases
Attachments
Screen Shot 2017-09-27 at 10.17.46 PM.png
Screen Shot 2017-09-27 at 10.17.46 PM.png (64.89 KiB) Viewed 6291 times

Posted on
Wed Sep 27, 2017 10:10 pm
blysik offline
Posts: 213
Joined: Jan 06, 2015

Re: Google Vision Plugin

Very cool! I wonder if there's a way to chain a picture from the Security Camera plugin into this?

Posted on
Thu Sep 28, 2017 6:31 am
vtmikel offline
Posts: 628
Joined: Aug 31, 2012
Location: Boston, MA

Re: Google Vision Plugin

blysik wrote:
Very cool! I wonder if there's a way to chain a picture from the Security Camera plugin into this?


This was also my idea. Everyone's setup is different. For me, I have SecuritySpy creating continuous images. I create an activity timer when an event happens such as motion or a door opening. When the activity timer is active, I'll sample a few of the camera images and send them to Google Vision to figure out more information about what's going on.

I haven't perfected it. Still working on this. I have some other use cases in mind once I have my outdoor cameras installed.

Posted on
Thu Sep 28, 2017 7:13 am
RogueProeliator offline
User avatar
Posts: 2501
Joined: Nov 13, 2012
Location: Baton Rouge, LA

Re: Google Vision Plugin

Very cool! I use a version of this (though local, not cloud-based) in Domotics Pad to provide the option to wake whenever it sees a face. It is reasonably accurate even on older devices; but with more power and new features (e.g. OCR, object recognition) this sounds like something fun to play with. My only hesitation is the 1000/mo limit -- in an automated system that is going to be hit very quickly! $1.50/1000 images for object labeling could get steep unless properly regulated. Might be fun to play with nonetheless.

Posted on
Thu Sep 28, 2017 7:14 am
RogueProeliator offline
User avatar
Posts: 2501
Joined: Nov 13, 2012
Location: Baton Rouge, LA

Re: Google Vision Plugin

I wonder if the logo detection could detect delivery vehicles... an announcement that "UPS is here" or a generic "Deliveries Arrived" if it saw UPS or FedEx or USPS would be pretty damn bad@ss."

Posted on
Thu Sep 28, 2017 7:49 am
roussell offline
User avatar
Posts: 1108
Joined: Aug 18, 2008
Location: Alabama

Re: Google Vision Plugin

I subscribed to USPS Informed Delivery, a service from the US Post Office that sends scans of the mail (images of the envelops) that is coming to my email. It's pretty cool because I can quickly scan my email to see if a letter/bill/package I'm expecting is arriving, or if the mailbox is just filled with junk mail.

It would be pretty cool to feed this plugin with those letter images to ocr and then alert me if something I'm waiting for is in the mailbox. I'll have to play with that when I get some time.

Terry

Posted on
Thu Sep 28, 2017 7:58 am
matt (support) offline
Site Admin
User avatar
Posts: 21411
Joined: Jan 27, 2003
Location: Texas

Re: Google Vision Plugin

That is really cool. Lots of possibilities for adding some really intelligent home automation here.

Image

Posted on
Thu Sep 28, 2017 8:36 am
virgahyatt offline
Posts: 132
Joined: Jan 11, 2014

Re: Google Vision Plugin

That's really cool I will have to try it out. FYI, sighthound offers similar services through API with 5000 calls per month for free with a developer account, then goes to $50 for 50,000 per month. https://www.sighthound.com/products/cloud

Posted on
Thu Sep 28, 2017 8:41 am
vtmikel offline
Posts: 628
Joined: Aug 31, 2012
Location: Boston, MA

Re: Google Vision Plugin

RogueProeliator wrote:
I wonder if the logo detection could detect delivery vehicles... an announcement that "UPS is here" or a generic "Deliveries Arrived" if it saw UPS or FedEx or USPS would be pretty damn bad@ss."


I'll add the logo detection, it should be easy for me to do. I'll be interested in the success rate for you. I'm guessing the truck would need to be decently close and in-view of your outdoor camera. You could probably use the label and logo detection in concert to improve your detection rate.

Posted on
Thu Sep 28, 2017 9:01 am
bkmar1192 offline
Posts: 274
Joined: Sep 12, 2015

Re: Google Vision Plugin

blysik wrote:
Very cool! I wonder if there's a way to chain a picture from the Security Camera plugin into this?


Should be easy enough to do. Security Camera plugin has a snapshot action that saves a 5 picture rotating image list so the most recent picture is always the same name.

The flow would be:
- trigger started
- take snap shot with Security Camera
- send snapshot to Google plugin




Sent from my iPhone using Tapatalk

Posted on
Thu Sep 28, 2017 10:35 am
RogueProeliator offline
User avatar
Posts: 2501
Joined: Nov 13, 2012
Location: Baton Rouge, LA

Re: Google Vision Plugin

I'll add the logo detection, it should be easy for me to do. I'll be interested in the success rate for you. I'm guessing the truck would need to be decently close and in-view of your outdoor camera. You could probably use the label and logo detection in concert to improve your detection rate.

Cool, thanks! I have a camera pointed at the driveway which should be able to get a good, clear view of the side of any truck (at an angle, of course, the issue may be how much of a skew does it recognize). Bigger issue might be how often it would get triggered and how to prevent false positives. As one other user put it, he got motion "every time a bug decided to hump his security camera." Anyone with outdoor cameras will know what I am talking about!

Posted on
Thu Sep 28, 2017 10:48 am
vtmikel offline
Posts: 628
Joined: Aug 31, 2012
Location: Boston, MA

Re: Google Vision Plugin

RogueProeliator wrote:
I'll add the logo detection, it should be easy for me to do. I'll be interested in the success rate for you. I'm guessing the truck would need to be decently close and in-view of your outdoor camera. You could probably use the label and logo detection in concert to improve your detection rate.

Cool, thanks! I have a camera pointed at the driveway which should be able to get a good, clear view of the side of any truck (at an angle, of course, the issue may be how much of a skew does it recognize). Bigger issue might be how often it would get triggered and how to prevent false positives. As one other user put it, he got motion "every time a bug decided to hump his security camera." Anyone with outdoor cameras will know what I am talking about!


I'm playing around with the idea of enabling a schedule that runs every 1 minute to get the latest image and send that to Google Vision. I keep this schedule enabled for 10 minutes, triggered when motion is detected. Rather than take the results of any single run of the Google Vision results, I use a variable to detect the number of positives and negatives, and make a decision on if a state change or event needs to happen based on the multiple results that comes back from the API. In my use case, I'm trying to detect if a car is in the garage. The Vision API is very consistent with this so far, so I don't need to go overboard, but this approach might be helpful to the outside.

Posted on
Thu Sep 28, 2017 12:43 pm
jay (support) offline
Site Admin
User avatar
Posts: 18199
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Google Vision Plugin

Cool! Added to the social media post queue and the plugin list. Thanks for another cool contribution.

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Posted on
Fri Sep 29, 2017 12:27 am
Gusten offline
Posts: 171
Joined: Dec 30, 2015
Location: Sweden, Gothenburg

Re: Google Vision Plugin

Hi

This is really cool :D

But i have problem getting it to work

When running it it seems to shut down, saying i need the API key, but i can't enter it, i only have the following choices in the plugin menu

+Disable
+Reload
+About

Here is the log

Code: Select all
Reloading plugin "Google Vision 1.0.1"
   Starting plugin "Google Vision 1.0.1" (pid 79353)
   Google Vision                   Checking for updates...
   Google Vision                   No updates are available
   Google Vision Error             Error in plugin execution InitializeMain:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "plugin.py", line 46, in __init__
KeyError: key txtAPIKey not found in dict

   Stopping plugin "Google Vision 1.0.1" (pid 79353)
   Stopped plugin "Google Vision 1.0.1"


What can be the problem

Thanx

Martin

Posted on
Fri Sep 29, 2017 6:33 am
vtmikel offline
Posts: 628
Joined: Aug 31, 2012
Location: Boston, MA

Re: Google Vision Plugin

Gusten wrote:
Hi

This is really cool :D

But i have problem getting it to work

When running it it seems to shut down, saying i need the API key, but i can't enter it, i only have the following choices in the plugin menu

+Disable
+Reload
+About

Here is the log

Code: Select all
Reloading plugin "Google Vision 1.0.1"
   Starting plugin "Google Vision 1.0.1" (pid 79353)
   Google Vision                   Checking for updates...
   Google Vision                   No updates are available
   Google Vision Error             Error in plugin execution InitializeMain:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "plugin.py", line 46, in __init__
KeyError: key txtAPIKey not found in dict

   Stopping plugin "Google Vision 1.0.1" (pid 79353)
   Stopped plugin "Google Vision 1.0.1"


What can be the problem

Thanx

Martin


Sorry about that, I'll get this fixed in 1.0.2 soon.

Who is online

Users browsing this forum: No registered users and 0 guests