Question on captured data and its display?

Posted on
Fri Mar 22, 2019 7:56 am
norcoscia offline
User avatar
Posts: 845
Joined: Sep 09, 2015

Question on captured data and its display?

Hi Dave,

Is there a way to capture and save data but not display all of it. In other words, could I keep a years worth of data in my CVS file and then vary the amount of the information I show in a chart. The reason I'm asking is I have been adjusting the setting but when I change the values in the picture below the information of the chart is also changed.

Would it be possible to save a year (or more) of data in the CVS file but choose to display only a week on one chart but maybe by quarter on a different chart. Or would that require two different CVS files?
Attachments
2019-03-22_06-49-31.jpeg
2019-03-22_06-49-31.jpeg (115.25 KiB) Viewed 617 times
2019-03-22_06-48-54.jpeg
2019-03-22_06-48-54.jpeg (68.22 KiB) Viewed 617 times

_______
Norm

Posted on
Fri Mar 22, 2019 5:06 pm
DaveL17 offline
User avatar
Posts: 5000
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Question on captured data and its display?

norcoscia wrote:
Would it be possible to save a year (or more) of data in the CVS file but choose to display only a week on one chart but maybe by quarter on a different chart. Or would that require two different CVS files?

Hi Norm - Not currently but that request is on the list as they say. I am still squashing bugs with the most recent build and adding a few small enhancements, and then life got in the way. I will hopefully be pushing out an official release soon.

When I do implement this request, it will likely be on a chart by chart basis. That way you could have a year's worth of data and use that for one chart that shows all of it, another that shows a month's worth, and another that show the last 72 hours (these are just examples.) It will probably be something like, "Plot the last X day'w worth of data." or something like that.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Sat Mar 23, 2019 7:56 am
norcoscia offline
User avatar
Posts: 845
Joined: Sep 09, 2015

Re: Question on captured data and its display?

That would be the biggest upgrade ever - can't wait to test it out - It would be way cool to have a big file of data and then create different views as desired - that would represent a huge new capability w/o adding a ton of bits and pieces to fuss with / store and remember.

I having a hard time wrapping my head around how to set up the numbers to get the chart to display that way I would like. With 1800 observations, a duration of 400 and refresh Interval of 900. I think I will update the CVS file every 15 min (with new data). That would be 95 updates a day - with an 1800 observations - that would limit me to 18.75 days of collected data. Now with my refresh interval set to 400 that would be 16.6666666667 days.

I'm confused a bit there, does that set up mean I will collect less than the 1800 observation because the refresh interval will only last for 16.666 days? I'm trying to understand the relationship of the three setting and how they affect each other and if there is a conflict which one wins?

_______
Norm

Posted on
Sat Mar 23, 2019 9:55 am
DaveL17 offline
User avatar
Posts: 5000
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Question on captured data and its display?

Hi Norm,

  • Observations is the number of lines in the CSV file. Setting the value in the CSV Engine to 300 will result in no more than 300 lines in the file (Obviously, it could be less like when it's first created.)
  • Duration is in hours. Setting the value in the CSV Engine to 72 will result in no more than 72 hours' worth of data being saved in the CSV file. The plugin uses the timestamp values within the file and compares them to the current time to do the testing. If you want less than 1 hour's worth of data, you would use a fraction (i.e., 0.5 or 0.25). I didn't think that anyone would typically want to save data for less than an hour, but they could likely want to save data for less than a day. That's why that one's in hours.
  • Refresh Interval is in seconds. Setting the value in the CSV Engine to 900 will result in an observation being saved every 15 minutes (60 seconds x 15 minutes = 900.) Saving once per day would be 86,400 (60 seconds x 60 minutes x 24 hours = 86,400.)
I would have to dig into the code to be absolutely sure, but I believe that the order of importance is this (using the example above): The plugin will consider the Duration value to be more important than the Observations value. So if you tell the CSV Engine to save 300 observations but some of those are outside the 72 hour window, the CSV Engine will drop the older observations and there will be less than 300 after the cycle is complete. If all 300 observations are within the window, they will all be saved.

If I've done my job properly, for all plugin controls (and this should be true for all of my plugins) if you hover over a control, it will give you tips about the value that's expected and the units to use.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Sat Mar 23, 2019 10:08 am
norcoscia offline
User avatar
Posts: 845
Joined: Sep 09, 2015

Re: Question on captured data and its display?

Thanks Dave, I guess what was confusing me is why you went with 3 setting. It seems like just duration and # of observations specified would allow you to automatically calculate the refresh interval and avoid the possibility of conflicting values. Also, below did you mean to say saving once a second vice once per day. Sorry if dumb questions - I'm just trying to become one with Matplotlib :-)
Attachments
2019-03-23_09-04-44.jpeg
2019-03-23_09-04-44.jpeg (24.3 KiB) Viewed 548 times

_______
Norm

Posted on
Sat Mar 23, 2019 11:25 am
DaveL17 offline
User avatar
Posts: 5000
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Question on captured data and its display?

It's gotta be three settings--one is telling the plugin how often to save data to the file (Refresh Interval) and the other two are telling the plugin how much data to keep in the file (Observations, Duration). If you set the Duration value to 0 (zero), then the plugin will ignore the age of the CSV data and only limit it by the Observations value. Maybe an example would help.

Save a Value Once Per Day - Say you wanted to keep track of a device's battery level over time. You probably wouldn't need to save an observation more often than once per day. To set the CSV Engine device to save one observation once per day, we need to set the Refresh Interval value to the number of seconds in a day. 24 hours (in a day) times 60 minutes (in an hour) times 60 seconds (in a minute) or 24 x 60 x 60 = 86,400. Google has a handy tool to do the math (just Google 'how many seconds in a day').

Screen Shot 2019-03-23 at 11.52.27 AM.png
Screen Shot 2019-03-23 at 11.52.27 AM.png (19.53 KiB) Viewed 536 times


This will result in 300 daily observations in the file:
Observations: 300 (never keep more than 300 observations in the file)
Duration: 0 (ignore the age of the data in the file)
Refresh Interval 86,400 (save an observation every day)

This will result in observations only for the last 72 hours (the data will never exceed 72 observations because it's limited by the Duration Value):
Observations: 300
Duration: 72 (hours)
Refresh Interval: 86,400 (seconds)

This will result in 300 observations being saved (the data will never reach 72 hours because there are 1,440 minutes in a day):
Observations: 300
Duration: 72 (hours)
Refresh Interval: 60 (seconds)

Refresh Interval: 86,400 = once per day (60 x 60 x 24)
Refresh Interval: 3,600 = once per hour (60 x 60)
Refresh Interval: 900 = every 15 minutes (60 seconds x 15)
Refresh Interval: 60 = once per minute (60 seconds)

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Sat Mar 23, 2019 11:43 am
norcoscia offline
User avatar
Posts: 845
Joined: Sep 09, 2015

Re: Question on captured data and its display?

Ok, I got it now - thanks for taking to the to explain it to me - trying to figure it out with all three values makes my head hurt - I will move forward with the duration value set to 0 (zero). That helps keep my brain from twisting into a knot :-)

_______
Norm

Posted on
Sun Mar 24, 2019 12:16 pm
norcoscia offline
User avatar
Posts: 845
Joined: Sep 09, 2015

Re: Question on captured data and its display?

Just updated to 7.4.5 and it seems to be having a problem with text on one of my charts - see below
Attachments
2019-03-24_11-15-20.jpeg
2019-03-24_11-15-20.jpeg (73.44 KiB) Viewed 482 times

_______
Norm

Posted on
Sun Mar 24, 2019 12:23 pm
DaveL17 offline
User avatar
Posts: 5000
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Question on captured data and its display?

norcoscia wrote:
Just updated to 7.4.5 and it seems to be having a problem with text on one of my charts - see below

Do you have any custom axis labels defined for that chart?

Thanks for the report Norm.
(v0.7.45)

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Sun Mar 24, 2019 2:25 pm
norcoscia offline
User avatar
Posts: 845
Joined: Sep 09, 2015

Re: Question on captured data and its display?

This is what I have set up - it was working fine before I updated.
Attachments
2019-03-24_13-22-10.jpeg
2019-03-24_13-22-10.jpeg (69.81 KiB) Viewed 457 times

_______
Norm

Posted on
Sun Mar 24, 2019 4:58 pm
DaveL17 offline
User avatar
Posts: 5000
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Question on captured data and its display?

Thanks Norm, that helps a lot.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Mon Mar 25, 2019 5:45 am
DaveL17 offline
User avatar
Posts: 5000
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Question on captured data and its display?

Hi Norm - Thanks for reporting this bug. It seems that it's caused by having defined custom tick locations for the Y axis but not having custom tick labels defined. I will add a fix for this in the next build, but in the interim, it seems like adding custom tick labels will address the issue. Just copy the custom ticks and paste them into the custom ticks label field like this:

Screen Shot 2019-03-25 at 6.41.40 AM.png
Screen Shot 2019-03-25 at 6.41.40 AM.png (36.94 KiB) Viewed 435 times


The current build will not allow you to have tick locations defined without tick labels defined, but I've decided that's too rigid. I've got a better idea in mind.

Thanks again for the report.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Mon Mar 25, 2019 6:55 am
norcoscia offline
User avatar
Posts: 845
Joined: Sep 09, 2015

Re: Question on captured data and its display?

That fixed it Dave, thanks!!!!!!!!

_______
Norm

Posted on
Mon Mar 25, 2019 9:27 am
DaveL17 offline
User avatar
Posts: 5000
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Question on captured data and its display?

Great. Thanks for letting me know.


Sent from my iPhone using Tapatalk

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Tue Apr 09, 2019 4:27 pm
norcoscia offline
User avatar
Posts: 845
Joined: Sep 09, 2015

Re: Question on captured data and its display?

I'm still being this and having problems with data missing from my charts - can you tell me what it means?

Error client connection ID (CFBundleIdentifier) not unique: com.fogbert.indigoplugin.matplotlib

_______
Norm

Who is online

Users browsing this forum: No registered users and 1 guest