Searching log now getting errors after installing 7.4

Hi Guys,
After installing 7.4. I'm now getting errors when searching log files. It worked before upgrading.

This is what I had before

theBody += os.popen('/usr/bin/grep -i error ' + logFile).read()

And I'm getting this error

Script Error embedded script: 'ascii' codec can't decode byte 0xe2 in position 5513: ordinal not in range(128)
Script Error Exception Traceback (most recent call shown last):

embedded script, line 107, at top level
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 5513: ordinal not in range(128)

So I also tried

with open(logFile) as f:
for line in f:
if line.find('error'):
theBody += line

And get the same error

Script Error embedded script: 'ascii' codec can't decode byte 0xcc in position 47: ordinal not in range(128)
Script Error Exception Traceback (most recent call shown last):

embedded script, line 112, at top level
UnicodeDecodeError: 'ascii' codec can't decode byte 0xcc in position 47: ordinal not in range(128)


Thanks guys for the help!!

Post the entire script, and use the CODE tags. We can't see how you've got 'logFile' defined, so we need the entire script. Or everything up to where you read the file, at least.

It is attached to the original post "full Error script.txt". I'm not sure what "CODE tags" are.

Thanks
Chris

Paste the code into the compose window between the tags created by this button:

Screen Shot 2019-10-29 at 1.15.36 PM.png (36.4 KiB) Viewed 1922 times

The file you attached is no good. It's an RTF file created with TextEdit or something similar.

Oh, this file in your script is suspect:

Code: Select all

logFile = '"/Library/Application Support/Perceptive Automation/Indigo 7.4/Logs/indigo_log.txt"'


Why is the name in single AND double quotes?

Also, there's an Indigo server call to get the install directory, so you don't need to hard code it. See https://wiki.indigodomo.com/doku.php?id ... r_commands

Code: Select all

name=indigo.server.getInstallFolderPath()
logFile = "{}/Logs/indigo_log.txt".format(name)


I suspect that using TextEdit to write your code is messing you up. Get BBEdit. There's a free version.

OK So I have made the changes to the log files. Thank you so much for that I have been looking for it, but could not find it. It makes it so much more simple and clean, and I no logger have to update after each software version change. Except I now have a bunch of scripts to update. As I'm sure you can tell I'm very new to python and scripting. Just a script kiddy.

And Wow BBEdit is so much easier, thanks a ton for that.

Over the years I have learned a lot from you FlyingDiver, and use many of your plugins

So I made the changes and no longer are getting errors, but its not returning any info for the grep

Code: Select all

###Search log for errors
name=indigo.server.getInstallFolderPath()
logFile = "{}/Logs/indigo_log.txt".format(name)

log("Running Script to Find Errors")
theBody += "\n\n*****Section 2: Today's Errors*****\n\n"

theBody += os.popen('/usr/bin/grep -i rror ' + logFile).read()

Here is the email
*****Section 2: Today's Errors*****


End of Report

Thanks
Chris

Are you using the SQL Logger plugin? If so, it would be easier to use a trigger from that plugin to capture errors in the log.

Otherwise, I'd add the name of the file to the output so you can make sure it's actually looking at the right file. I didn't test that code snippet I posted.

[matt (support)]

For the last line the logFile path being passed in isn't in quotes (probably why you had it in quotes before) so the spaces aren't escaped. Try this:

Code: Select all

theBody += os.popen('/usr/bin/grep -i rror "{}"'.format(logFile)).read()

ok, so it looks like changing to to

Code: Select all

theBody += os.popen('/usr/bin/grep -i rror "{}"'.format(logFile)).read()

is working, but I'm back to these UnicodeDecodeError errors

Script Error embedded script: 'ascii' codec can't decode byte 0xe2 in position 5513: ordinal not in range(128)
Script Error Exception Traceback (most recent call shown last):

embedded script, line 108, at top level
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 5513: ordinal not in range(128)

Thanks
Chris

Now you really are going to have to post the entire script. Your error is on line 108.

lol, 108 is the grep line. here is the full script

Code: Select all

#!/usr/bin/env python
## setup
myAddress = "acesandeights18@gmail.com"
mySubject = "HA Server Status Email"

## Low battery check setup
try:
   import indigo
except ImportError:
   print "Indigo control scripts can only be used from within Indigo"
   raise ImportError
from datetime import datetime, timedelta

## Convenience log function
def log(msg):
   indigo.server.log(msg, type="Low Battery Notification Script")

## Constants and global variables
kTrueStringValues = ['true', '1', 't', 'y', 'yes']
notifications = 0

## Get default overrides from variables
# Battery Threshold
v = indigo.variables.get("lbnBatteryThreshold", None)
try:
   batteryThreshold = int(v.value)
except:
   # default to 5%
   batteryThreshold = 10

# Time Threshold
v = indigo.variables.get("lbnTimeThreshold", None)
try:
   timeThreshold = int(v.value)
except:
   # default to anything older than 24 hours
   timeThreshold = 1
   
## Start main part of script
theBody = ''
log("Starting battery notification script with default battery threshold of %i%% and time threshold of %i days" % (batteryThreshold, timeThreshold))

theBody += "*****Section 1: Low Battery Report*****\n\n"

## Loop through devices looking for devices that have battery levels
for device in indigo.devices:
   if device.enabled == True:
         if device.model == "Open/Close Sensor":
            timeThreshold = (datetime.now() - device.lastChanged).days
            if timeThreshold > 1:
               log(device.name)
               theBody += device.name
               theBody += " last update is over 2 days\n"
               continue

for device in indigo.devices:
   if device.enabled == True:
         if device.model == "Leak Sensor":
            timeThreshold = (datetime.now() - device.lastChanged).days
            if timeThreshold > 1:
               log(device.name)
               theBody += device.name
               theBody += " last update is over 2 days\n"
               continue

for device in indigo.devices:
   if device.enabled == True:
         if device.model == "Motion Sensor (2844)":
            timeThreshold = (datetime.now() - device.lastChanged).days
            if timeThreshold > 1:
               log(device.name)
               theBody += device.name
               theBody += " last update is over 2 days\n"
               continue

for device in indigo.devices:
   if device.enabled == True:
         if device.model == "Door Sensor":
            timeThreshold = (datetime.now() - device.lastChanged).days
            if timeThreshold > 1:
               log(device.name)
               theBody += device.name
               theBody += " last update is over 2 days\n"
               continue

for device in indigo.devices:
   if device.enabled == True:
      if device.batteryLevel is not None:
         if device.batteryLevel < 10:
            batt = device.batteryLevel
            log(device.name)
            theBody += device.name
            theBody += " Battery is under 10%\n"
            continue

###Check if any low batts were found
bodyCount = theBody.count("is")
if bodyCount < 1:
   theBody += "No Low Batteries Found"

###Search log for errors
name=indigo.server.getInstallFolderPath()
logFile = "{}/Logs/indigo_log.txt".format(name)

log("Running Script to Find Errors")
theBody += "\n\n*****Section 2: Today's Errors*****\n\n"

theBody += os.popen('/usr/bin/grep -i rror "{}"'.format(logFile)).read()

###Send Email
log("Sending Email Report")

theBody += "\nEnd of Report"

if theBody:
   indigo.server.sendEmailTo(myAddress, subject=mySubject, body=theBody)



thanks
Chris

[matt (support)]

Try changing:

theBody = ''

to:

theBody = u''

Matt beat me to it. I was going to suggest two things.

Thing 1: Put this as the second line of your script, after the shebang line.

Code: Select all

# -*- coding: utf-8 -*-

So it would be:

Code: Select all

#!/usr/bin/env python
# -*- coding: utf-8 -*-

## setup


Thing 2 is what Matt said. Make a point of always using the construct:

Code: Select all

foo = u"{0}, {1}".format(thing1, thing2)

Ever since adopting those two things, I've virtually eliminated Unicode errors from my life.

Well, guys I figured it out. Even after I tried the suggested Construct/Formatting stuff, I was still getting the same errors. I found that it was adding the device.name to "theBody" that was causing the "UnicodeDecodeError" errors. So I just changed the script to just input all the low battery stuff to the indigo log and just grep the log later. That fix all the issue.

Thanks a ton to everyone for the help
I love being part of this community

Thanks
Chris

Code: Select all

#!/usr/bin/env python
# -*- coding: utf-8 -*-

## Setup
try:
   import indigo
except ImportError:
   print "Indigo control scripts can only be used from within Indigo"
   raise ImportError
from datetime import datetime, timedelta
name=indigo.server.getInstallFolderPath()
logFile = "{}/Logs/indigo_log.txt".format(name)

## Convenience log function
def log(msg):
   indigo.server.log(msg, type="HA Server Heath Check Script")
log("Starting HA Server Heath Check Script")

## Constants and global variables
kTrueStringValues = ['true', '1', 't', 'y', 'yes']
notifications = 0

## Get default overrides from variables
# Battery Threshold
v = indigo.variables.get("lbnBatteryThreshold", None)
try:
   batteryThreshold = int(v.value)
except:
   # default to 5%
   batteryThreshold = 10

# Time Threshold
v = indigo.variables.get("lbnTimeThreshold", None)
try:
   timeThreshold = int(v.value)
except:
   # default to anything older than 24 hours
   timeThreshold = 1
   
## Start main part of script
log("Starting battery notification script with default battery threshold of %i%% and time threshold of %i days" % (batteryThreshold, timeThreshold))

## Convenience log function
def log(msg):
   indigo.server.log(msg, type="Low Battery found on")
   
## Loop through devices looking for devices that have battery levels
for device in indigo.devices:
   if device.enabled == True:
         if device.model == "Open/Close Sensor":
            timeThreshold = (datetime.now() - device.lastChanged).days
            if timeThreshold > 1:
               log(device.name)
               continue

for device in indigo.devices:
   if device.enabled == True:
         if device.model == "Leak Sensor":
            timeThreshold = (datetime.now() - device.lastChanged).days
            if timeThreshold > 1:
               log(device.name)
               continue

for device in indigo.devices:
   if device.enabled == True:
         if device.model == "Motion Sensor (2844)":
            timeThreshold = (datetime.now() - device.lastChanged).days
            if timeThreshold > 1:
               log(device.name)
               continue

for device in indigo.devices:
   if device.enabled == True:
         if device.model == "Door Sensor":
            timeThreshold = (datetime.now() - device.lastChanged).days
            if timeThreshold > 1:
               log(device.name)
               continue

for device in indigo.devices:
   if device.enabled == True:
      if device.batteryLevel is not None:
         if device.batteryLevel < 10:
            batt = device.batteryLevel
            log(device.name)
            continue


## Convenience log function
def log(msg):
   indigo.server.log(msg, type="HA Server Heath Check Script")


## Email setup
myAddress = "youremail@gmail.com"
mySubject = "HA Server Status Email"
theBody = ''

log("Compile Email Report")
theBody += "*****HA Server Heath Check Script*****\n"

## Run Error Check
log("Finding Errors")
theBody += "\n*****Section 1: Today's Errors*****\n\n"

##Error Check Setup
keywords = ("rror", "update")

##Run Error Check script
with open(logFile) as f:
    for line in f:
      if any(words in line for words in keywords):
      theBody += line
      continue

log("Finding Low Batterys")
theBody += "\n\n*****Section 2: Low Battery Report*****\n\n"

##Find Low Batteries
keywords2 = ("Low Battery found on", "xxxxxxxxx")

##Run Error Check script
with open(logFile) as f:
    for line in f:
      if any(words in line for words in keywords2):
      theBody += line
      continue

###Send Email
log("Sending Email Report")

theBody += "\nEnd of Report"

if theBody:
   indigo.server.sendEmailTo(myAddress, subject=mySubject, body=theBody)