Daily limit exceeded

Posted on
Tue Jul 10, 2018 7:58 pm
lanbrown offline
Posts: 667
Joined: Sep 26, 2017

Daily limit exceeded

Has anyone ran into the 500 daily limit with the Wunderground plugin? The reason I ask is because I noticed that the temp was not updating and it stopped at 4am. So I restarted the plugin and it got an update and then an hour later I saw it hadn't updated. So I enabled dubugging and saw this:
WUnderground 7 Daily call limit (500) reached. Taking the rest of the day off.

The weird part, I have it set to update every 5 minutes. So that is 12 times an hour or 288 times a day. So has Wunderground actually lowered the daily limit and that 500 is hardcoded into the Wunderground plugin or is that 500 being received from Wunderground? Does anyone else see the same? You need to look at when the last update was as there was nothing in the logs about it.

Looking through the log this is all I see:
Jul 10, 2018 at 3:04:01 AM
WUnderground 7 Error Unable to decode data. (Line 1405) No JSON object could be decoded
No other logs messages for the rest of day until I restarted the plugin.

Posted on
Tue Jul 10, 2018 8:46 pm
Colorado4Wheeler offline
User avatar
Posts: 2723
Joined: Jul 20, 2009
Location: Colorado

Re: Daily limit exceeded

I wonder if that has to do with them shutting down public api access. Not sure if today was “the day” but I know the end is soon.

My Modest Contributions to Indigo:

HomeKit Bridge | Device Extensions | Security Manager | LCD Creator | Room-O-Matic | Smart Dimmer | Scene Toggle | Powermiser | Homebridge Buddy

Check Them Out Here

Posted on
Tue Jul 10, 2018 9:20 pm
lanbrown offline
Posts: 667
Joined: Sep 26, 2017

Re: Daily limit exceeded

That is exactly why I'm asking. I know the free API has a limit of 500, just not sure if that limit is hardcoded into the plugin so when it says I've reached my limit if that is the plugin saying it is 500 or if that is what is being returned by Wunderground. I guess I'll need to wait until DaveL17 chimes in on that.

I'll leave the debugging enabled and see what happens. I wouldn't be surprised if they decided to reduce the 500 to a lower value. They don't appear to offer a way to check what your limit is.

Maybe the plugin had an issue since it didn't update the weather since about 4am and it got stuck requesting information then killed my 500 calls per day limit. Hard to tell with nothing in the logs.

What is weird though is this:
Code: Select all
   Started plugin "WUnderground 7 7.0.10"
   WUnderground 7 Debug            Daily call limit reached: False
   WUnderground 7 Debug            New call day: False

It received the weather for the PWS
Then this:
Code: Select all
  WUnderground 7                  Daily call limit (500) reached. Taking the rest of the day off


If I restart the plugin, it gets the latest weather then I get the daily limit issue again.

Anyway, appreciate your response.

Posted on
Tue Jul 10, 2018 9:53 pm
lanbrown offline
Posts: 667
Joined: Sep 26, 2017

Re: Daily limit exceeded

So since I had a snapshot, I put it back to 10.13.5. The Wunderground plugin can retrieve weather data. However, it seems to be calling a little too frequently.

Code: Select all
Jul 10, 2018 at 10:41:54 PM
   WUnderground 7 Debug            Debug logging enabled.
   WUnderground 7 Debug            API calls left: 496


Code: Select all
Jul 10, 2018 at 10:42:25 PM
   WUnderground 7 Debug            Processing API: Call counter
   WUnderground 7 Debug            Daily call limit reached: False
   WUnderground 7 Debug            New call day: False
   WUnderground 7 Debug            API calls left: 495


Code: Select all
Jul 10, 2018 at 10:42:44 PM
   WUnderground 7 Debug            closedPrefsConfigUi called.
   WUnderground 7 Debug            User prefs dialog cancelled.
   WUnderground 7 Debug            Processing API: Call counter
   WUnderground 7 Debug            Daily call limit reached: False
   WUnderground 7 Debug            New call day: False
   WUnderground 7 Debug            API calls left: 494


Code: Select all
Jul 10, 2018 at 10:43:26 PM
   WUnderground 7 Debug            Processing API: Call counter
   WUnderground 7 Debug            Daily call limit reached: False
   WUnderground 7 Debug            New call day: False
   WUnderground 7 Debug            API calls left: 493


So it is polling a bit fast. It is set for 5 minutes. Let me try disabling the plugin and then enabling it even though I did a reboot after restoring a snapshot.

Posted on
Tue Jul 10, 2018 9:55 pm
lanbrown offline
Posts: 667
Joined: Sep 26, 2017

Re: Daily limit exceeded

Yep, back to a five minute poll period after disabling the plugin and then enabling it.

Let me try going back to 10.13.6 and rebooting it and see if the problem clears or returns.

Posted on
Tue Jul 10, 2018 10:17 pm
lanbrown offline
Posts: 667
Joined: Sep 26, 2017

Re: Daily limit exceeded

Well, went back to 10.13.6 and it missed two poll cycle, disabling the plugin, enabling it with debugging on returns the 500 API call limit issue. So it appears that 10.13.6 has an issue.

Posted on
Tue Jul 10, 2018 11:27 pm
lanbrown offline
Posts: 667
Joined: Sep 26, 2017

Re: Daily limit exceeded

So looking at the log from July 10th, the plugin got into a state where it was polling far to often. It was polling every 30 seconds, so 120 times a hour. A 500 call limit won't last long and that would take me to a little after 4am, which coincides with the last update I saw in the morning.

I went back to the July 9th and after the 10.13.6 was installed, when the system came up the plugin started to have issues.

Code: Select all
2018-07-09 20:47:41.775   DEBUG   Plugin.debugLog:   Initializing DLFramework...
2018-07-09 20:47:41.805   DEBUG   Plugin.debugLog:   DLFramework pluginEnvironment method called.
2018-07-09 20:47:41.807   DEBUG   Plugin.debugLog:   DLFramework convertDebugLevel method called.
2018-07-09 20:47:41.898   DEBUG        Plugin.deviceStartComm              Starting Device: xxx
2018-07-09 20:47:41.930   DEBUG        Plugin.runConcurrentThread          Starting main thread.
2018-07-09 20:47:46.936   DEBUG        Plugin.callDay                      Processing API: Call counter
2018-07-09 20:47:46.937   DEBUG        Plugin.callDay                      Daily call limit reached: False
2018-07-09 20:47:46.941   DEBUG        Plugin.callDay                      Reset call limit, call counter and call day.
2018-07-09 20:47:46.941   DEBUG        Plugin.debugLog                     versionCheck: Started
2018-07-09 20:47:46.941   DEBUG        Plugin.debugLog                     versionCheck: Version Server Url: https://raw.githubusercontent.com/DaveL17/WUnderground7/master/wunderground7_version.html
2018-07-09 20:47:47.226   DEBUG        Plugin.callDay                      New call day: True
2018-07-09 20:47:47.231   DEBUG        Plugin.refreshWeatherData           Processing device: xxx
2018-07-09 20:47:47.232   DEBUG        Plugin.getWeatherData               URL for pws:xxx http://api.wunderground.com/api/xxx/geolookup/alerts_v11/almanac_v11/astronomy_v11/conditions_v11/forecast10day_v11/hourly_v11/lang:EN/yesterday_v11/tide_v11/q/pws:xxx.json?apiref=xxx
2018-07-09 20:47:48.863   DEBUG        Plugin.getWeatherData               [  pws:xxx download: 01.631384 seconds  ]
2018-07-09 20:47:48.970   DEBUG        Plugin.getWeatherData               Adding weather data for pws:xxx to Master Weather Dictionary.
2018-07-09 20:47:48.970   DEBUG        Plugin.callCount                    API calls left: 500
2018-07-09 20:47:48.971   DEBUG        Plugin.fixCorruptedData             Imputing Solar Radiation data. Got: -- Returning: (-99.0, --)
2018-07-09 20:47:48.972   DEBUG        Plugin.fixCorruptedData             Imputing heatIndexF (S) data. Got: NA Returning: (-99.0, --)
2018-07-09 20:47:48.972   DEBUG        Plugin.fixCorruptedData             Imputing windChillF (S) data. Got: NA Returning: (-99.0, --)
2018-07-09 20:47:48.996   DEBUG        Plugin.refreshWeatherData           1 locations polled: [u'pws:xxx]
2018-07-09 20:47:49.026   DEBUG        Plugin.triggerProcessing            Rebuild Master Trigger Dict: {}
2018-07-09 20:47:49.036   DEBUG        Plugin.runConcurrentThread          [  Plugin execution time: 02.100613 seconds  ]
2018-07-09 20:47:49.037   DEBUG        Plugin.runConcurrentThread          ======== Plugin Cycle Complete =========
2018-07-09 20:48:19.043   DEBUG        Plugin.callDay                      Processing API: Call counter
2018-07-09 20:48:19.043   DEBUG        Plugin.callDay                      Daily call limit reached: False
2018-07-09 20:48:19.043   DEBUG        Plugin.callDay                      New call day: False
2018-07-09 20:48:19.050   DEBUG        Plugin.refreshWeatherData           Processing device: xxx
2018-07-09 20:48:19.050   DEBUG        Plugin.getWeatherData               URL for pws:xxx: http://api.wunderground.com/api/xxx/geolookup/alerts_v11/almanac_v11/astronomy_v11/conditions_v11/forecast10day_v11/hourly_v11/lang:EN/yesterday_v11/tide_v11/q/pws:xxx.json?apiref=xxx
2018-07-09 20:48:19.704   DEBUG        Plugin.getWeatherData               [  pws:xxx download: 00.654059 seconds  ]
2018-07-09 20:48:19.810   DEBUG        Plugin.getWeatherData               Adding weather data for pws:xxx to Master Weather Dictionary.
2018-07-09 20:48:19.810   DEBUG        Plugin.callCount                    API calls left: 499
2018-07-09 20:48:19.811   DEBUG        Plugin.fixCorruptedData             Imputing Solar Radiation data. Got: -- Returning: (-99.0, --)
2018-07-09 20:48:19.812   DEBUG        Plugin.fixCorruptedData             Imputing heatIndexF (S) data. Got: NA Returning: (-99.0, --)
2018-07-09 20:48:19.812   DEBUG        Plugin.fixCorruptedData             Imputing windChillF (S) data. Got: NA Returning: (-99.0, --)
2018-07-09 20:48:19.834   DEBUG        Plugin.refreshWeatherData           1 locations polled: [u'pws:xxx']
2018-07-09 20:48:19.840   DEBUG        Plugin.triggerProcessing            Rebuild Master Trigger Dict: {}
2018-07-09 20:48:19.846   DEBUG        Plugin.runConcurrentThread          [  Plugin execution time: 00.803483 seconds  ]


After that is was polling every 30 seconds.

Posted on
Wed Jul 11, 2018 3:54 am
DaveL17 offline
User avatar
Posts: 4245
Joined: Aug 20, 2013

Re: Daily limit exceeded

It is extremely unlikely that the change in OS affected the plugin in this way, but rather more likely that the plugin got into a bad state when it was restarted. My production installation looks to be working as expected (although WU is taking a LONG time to respond to API calls). However, when I restarted my development environment, I saw the behavior you described. I disabled and re-enabled the plugin and normal operation resumed. I've looked at the changes between 7.0.10 and 7.0.11 and don't see anything that would have caused this behavior. I did find a typo in the setting of the initial plugin configuration DEFAULTS though, which will be fixed in the next version.

The plugin checks every 30 seconds to see if an update is called for. If an update is called for, it does its thing and updates the next scheduled poll value (the next scheduled poll value is displayed within the plugin configuration dialog). I'm guessing that the plugin got into a state where it was not updating this value properly such that it was always time for an update (the current time exceeded next poll time every 30 seconds). You can always see when the plugin thinks the next poll is due by examining this value in the plugin configuration dialog box.

The API call limit is "hard coded" into the plugin such that it is a user-defined number set within the plugin configuration dialog (the default value is 500). The WU API doesn't return the number of calls to it--a long sought feature that WU never implemented--so as a proxy, the plugin attempts to track this on the users' behalf to keep it from burning through your raindrops when things go sideways.

If and until I can find (and fix) whatever caused this bad state, I would suggest that the workaround is to disable and re-enable the plugin.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Wed Jul 11, 2018 4:07 am
lanbrown offline
Posts: 667
Joined: Sep 26, 2017

Re: Daily limit exceeded

DaveL17,

Thank you. It does appear it got into a bad state. Another mitigating step I took was to lower the daily limit to 300 in the config. I shouldn't use more than 288 in a day and if I do, then hopefully it would just be at the end of the day. I added the extra 12 so that it would be an additional hour (think of when daylight savings time ends). If I go through the 300 because the plugin gets into a bad state again, then I at least have a reserve of 200 API calls.

Posted on
Wed Jul 11, 2018 5:40 am
DaveL17 offline
User avatar
Posts: 4245
Joined: Aug 20, 2013

Re: Daily limit exceeded

lanbrown wrote:
DaveL17,

Thank you. It does appear it got into a bad state. Another mitigating step I took was to lower the daily limit to 300 in the config. I shouldn't use more than 288 in a day and if I do, then hopefully it would just be at the end of the day. I added the extra 12 so that it would be an additional hour (think of when daylight savings time ends). If I go through the 300 because the plugin gets into a bad state again, then I at least have a reserve of 200 API calls.

Setting the API call limit to a value roughly equivalent to the total number of expected calls per day is excellent advice. If you see the behavior resurface, please tell me as much as you can about the circumstances so that I can try to recreate the problem.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Page 1 of 1

Who is online

Users browsing this forum: No registered users and 1 guest