Image Analysis

Posted on
Mon Oct 09, 2017 2:09 pm
vtmikel offline
Posts: 628
Joined: Aug 31, 2012
Location: Boston, MA

Image Analysis

Hello,

I've merged my previous work on a Google Vision plugin into a generalized plugin for Image Analysis. The new plugin supports both Google Vision and AWS Rekognition.

The reason I've introduced different image analysis cloud services is that they are each different. This article talks about some of the pros and cons (https://engineering.musefind.com/we-com ... 54cff5ae62). For me, I found better person detection using the AWS API, and better scene/object detection with the Google Vision API.

In the future, I may attempt to build in support for AWS Rekognition's face recognition features. Google Vision only supports face detection. As of 1.0.4, face detection is the only thing implemented.

If you are interested in how it works, I recommend taking a snapshot using your own security camera and sending it to the service using the above site.

The plugin allows you to create dynamic events of four different types:
  • Face Detection - Looks for faces in the image. You can configure the level of certainty that Google returns back to the plugin. Also supported is a flag to look for "no faces detected". Google and AWS both support things like the likelihood that the face is showing joy, anger, sorrow, etc. I have not implemented those and this information is stored but ignored for now.
  • Label (Object) Detection - Looks for objects in photos. Easiest way to understand Google and Amazon's dictionary is test with your own images and see what results typically come back. Typical use cases is to detect the presence of a person, automobile, pet.
  • OCR (Google Vision only) - I haven't tested this thoroughly, but the plugin can trigger an event if Google is able to OCR text in a image and you are looking for a particular substring.
  • Logo Detection (Google Vision only) - Detects logos!

Note: This plugin does not require to create any devices.

Once you create the events, you can create triggers for those events in Indigo.

Finally, you trigger the processing of the events by sending an image to Google Vision or AWS Rekgonition, Actions that the plugin supports. Images can be local to your Indigo Server or via HTTP protocol. The location/URL can be set statically or via Indigo variable.

For me, I'm using the plugin to occasionally analyze images from my home security cameras. This works well with SecuritySpy's feature to upload a web camera image to a remote server.

Plugin can be downloaded here: https://github.com/mlamoure/Indigo-Image-Analysis

Posted on
Sat Jun 30, 2018 2:22 am
agame offline
Posts: 514
Joined: Jul 13, 2017
Location: Melbourne, Australia

Re: Image Analysis

As you say, the results are remarkably different between the services, and I find (aligning with your comment) that Google struggles with detecting humans in my setup.

I'm exploring using the plugin to filter out false movement alerts from two intercoms - so far with pretty good results on AWS..

I do have one conceptual issue I'm trying to get my mind around: it seems there is no way to identify which of my cameras has generated a trigger - only the event type (in my case both cameras are looking for the same things - people and parts of people). I can of course approximate by setting a variable when I send the image for recognition, but in the (infrequent but quite likely) event that both cameras sense motion at roughly the same time, things will get confusing (given its a few seconds think-time on the API call).

Am I missing something?


Curious, also if you've tried sending your stitched images for recognition...to save on API calls....? At another other Indigo setup I'm building I will be trying to detect people or cars in any of 4 images once again to filter out false movement alerts....

Posted on
Sun Jul 15, 2018 1:42 pm
vtmikel offline
Posts: 628
Joined: Aug 31, 2012
Location: Boston, MA

Re: Image Analysis

agame wrote:
As you say, the results are remarkably different between the services, and I find (aligning with your comment) that Google struggles with detecting humans in my setup.

I'm exploring using the plugin to filter out false movement alerts from two intercoms - so far with pretty good results on AWS..

I do have one conceptual issue I'm trying to get my mind around: it seems there is no way to identify which of my cameras has generated a trigger - only the event type (in my case both cameras are looking for the same things - people and parts of people). I can of course approximate by setting a variable when I send the image for recognition, but in the (infrequent but quite likely) event that both cameras sense motion at roughly the same time, things will get confusing (given its a few seconds think-time on the API call).

Am I missing something?


Curious, also if you've tried sending your stitched images for recognition...to save on API calls....? At another other Indigo setup I'm building I will be trying to detect people or cars in any of 4 images once again to filter out false movement alerts....



My apologies, I overlooked your post to the forum.

The easiest way to differentiate the camera that saw the event is to create a event for each camera. Not ideal. Then, when you create you actions, only make the camera applicable for that event. When you create the triggers for the event, you now know which camera, or group of cameras, it was possible to trigger the event. I can't think of any way I could make this easier in the plugin, since tiggers do not accept parameters.

No, I have not played around with sending stitched images to AWS or Google. I use the stitch plugin to send my alerts when something is detected, which is why I never ran into your first question - I dont care which camera saw the person, I send a still image of all cameras to Pushover when something is detected.

I personally just finished my install of 5 external cameras. I haven't been using this plugin myself for quite some time, that will change soon.

Posted on
Mon Aug 06, 2018 8:49 pm
mundmc offline
User avatar
Posts: 1060
Joined: Sep 14, 2012

Re: Image Analysis

Any updates on this?


Sent from my iPhone using Tapatalk

Posted on
Sun Aug 12, 2018 7:19 pm
vtmikel offline
Posts: 628
Joined: Aug 31, 2012
Location: Boston, MA

Re: Image Analysis

Sorry not yet


Sent from my iPhone using Tapatalk Pro

Posted on
Fri Jan 04, 2019 2:25 am
hensed offline
Posts: 59
Joined: Feb 01, 2012
Location: Oakland, CA

error attempting 2 events w/ AWS

Hi! I' trying to use AWS to to process a face rec event and a label event. It will do one or the other just fine fine, but doing 2 events get this error:
Code: Select all
Image Analysis Error            Error in plugin execution ExecuteAction:

Traceback (most recent call last):
  File "plugin.py", line 243, in sendImageAction
  File "/Library/Application Support/Perceptive Automation/Indigo 7.2/Plugins/Image Analysis.indigoPlugin/Contents/Server Plugin/AWSRekognitionAdapter.py", line 33, in sendImage
TypeError: coercing to Unicode: need string or buffer, file found

Posted on
Fri Jan 04, 2019 2:53 pm
hensed offline
Posts: 59
Joined: Feb 01, 2012
Location: Oakland, CA

Event triggers below score

Hi! I have an "event 2" that is set to look for Human or Person. I have set the score to .99, but I am still getting hits using AWS even though the certainty score falls well below .99. Seems to work as expected when I switch to Google vision, but Vision is not as good at spotting people.

Page 1 of 1

Who is online

Users browsing this forum: No registered users and 0 guests