Feature Requests

Posted on
Mon Jan 13, 2020 10:59 am
Umtauscher offline
User avatar
Posts: 566
Joined: Oct 03, 2014
Location: Cologne, Germany

Re: Feature Requests

hmm, my test uses a raspberry which delivers the json response using the digest method.
I just reboot the raspberry. This takes about 30 seconds to complete.
The timeout in the correspondig ghostXML device is set to 5 sec. , maxretries set to 100.
The refresh frequency is set to 15 sec.

Posted on
Mon Jan 13, 2020 11:44 am
DaveL17 offline
User avatar
Posts: 6744
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Feature Requests

Yeah, unfortunately I don't know precisely what state the rPi is when it becomes unresponsive. I have test devices that return empty data, malformed data, LAN and WAN devices that "aren't there", and bad auth credentials (among others).

Moreover, the hangs that you're seeing lead me to believe that there is an exception buried somewhere in a separate process and the process is getting hung up (thereby not telling the plugin what's wrong) and that's why it's stuck on 'processing'. We've seen this behavior before with other exceptions. The trick is figuring out where the hang is and rectifying that.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Mon Jan 13, 2020 12:15 pm
Umtauscher offline
User avatar
Posts: 566
Joined: Oct 03, 2014
Location: Cologne, Germany

Re: Feature Requests

how can I help?

Posted on
Mon Jan 13, 2020 12:24 pm
Umtauscher offline
User avatar
Posts: 566
Joined: Oct 03, 2014
Location: Cologne, Germany

Re: Feature Requests

I have another debug log snippet for you from a log earlier:
Code: Select all
2020-01-13 15:18:14.988   GhostXML   OTGW
2020-01-13 15:18:15.021   GhostXML Debug   HTTPS CURL result: * Adding handle: conn: 0x7fd35b80aa00
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7fd35b80aa00) send_pipe: 1, recv_pipe: 0
* About to connect() to otgw.voyager95.de port 80 (#0)
*   Trying 192.168.1.52...
* Failed connect to otgw.voyager95.de:80; Connection refused
* Closing connection 0

2020-01-13 15:18:15.021   GhostXML Debug   ReturnCode: 7
2020-01-13 15:18:15.022   GhostXML Warning   [OTGW] curl error * Adding handle: conn: 0x7fd35b80aa00 * Adding handle: send: 0 * Adding handle: recv: 0 * Curl_addHandleToPipeline: length: 1 * - Conn 0 (0x7fd35b80aa00) send_pipe: 1, recv_pipe: 0 * About to connect() to otgw.voyager95.de port 80 (#0) *   Trying 192.168.1.52... * Failed connect to otgw.voyager95.de:80; Connection refused * Closing connection 0 . [Return code: 7
2020-01-13 15:18:15.044   GhostXML Debug   [OTGW] Parse Error: No JSON object could be decoded
2020-01-13 15:18:15.044   GhostXML Debug   [OTGW] jsonRawData {u'controlsp_description': u'Control setpoint', u'setpoint_description': u'Room setpoint', u'flame_value': u'0', u'temperature_description': u'Room temperature', u'fault_description': u'Fault indication', u'modulation_value': u'0.00', u'dhwsetpoint_description': u'DHW setpoint', u'chmode_description': u'Central heating mode', u'dhwmode_description': u'Domestic hot water mode', u'dhwenable_description': u'Domestic hot water enable', u'otgw_software_version': u'4.3', u'fault_value': u'0', u'diag_description': u'Diagnostic indication', u'boilertemp_description': u'Boiler water temperature', u'dhwenable_value': u'1', u'returntemp_description': u'Return water temperature', u'modulation_description': u'Relative modulation level', u'dhwtemp_value': u'47.28', u'otgw_firmware_version': u'4.2.5', u'setpoint_value': u'23.00', u'controlsp_value': u'40.45', u'otgw_software_name': u'Opentherm Monitor', u'outside_description': u'Outside temperature', u'boilertemp_value': u'42.03', u'outside_value': u'10.11', u'otgw_firmware_name': u'Opentherm Gateway', u'diag_value': u'0', u'dhwsetpoint_value': u'35.00', u'flame_description': u'Flame status', u'temperature_value': u'23.47', u'chmode_value': u'1', u'dhwtemp_description': u'DHW temperature', u'otgw_time': u'2020-01-13 15:19:37 +0100', u'returntemp_value': u'41.12', u'dhwmode_value': u'0'}
2020-01-13 15:18:15.044   GhostXML Warning   There was a parse error. Will continue to poll.

Posted on
Mon Jan 13, 2020 1:40 pm
DaveL17 offline
User avatar
Posts: 6744
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Feature Requests

I'm adding some additional error trapping into the plugin and am testing that now. Hopefully, that will give us greater insight into why your device gets stuck on 'processing'. The parse error message above is not a plugin error per se. It's saying that the information returned couldn't be parsed. This is anticipated behavior and an expected outcome in lots of circumstances.

As you can imagine, it's a bit tricky to have a single plugin cover all potential XML and JSON payloads (or lack thereof!). Hopefully, the plugin will continue to improve with wider adoption.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Tue Jan 14, 2020 11:20 am
DaveL17 offline
User avatar
Posts: 6744
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Feature Requests

Hi Wilhelm - I've uploaded a new build which has even more error trapping. Hopefully, this will shed some light on the devices that get stuck on 'processing'.

Alas, I have still be unable to emulate your situation, so I don't know where to look.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Tue Jan 14, 2020 11:59 am
Umtauscher offline
User avatar
Posts: 566
Joined: Oct 03, 2014
Location: Cologne, Germany

Re: Feature Requests

It doesn't trigger any timeout errors any more amd stays online.

Here is the complete debug log.
The device that I used for the test is called OTGW.
I hope this helps....
Cheers
Wilhelm
Attachments
2020-01-14 Events Kopie.txt
(126.16 KiB) Downloaded 121 times

Posted on
Tue Jan 14, 2020 1:09 pm
DaveL17 offline
User avatar
Posts: 6744
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Feature Requests

Thanks Wilhelm. I'll take a look.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Sat Jan 18, 2020 7:03 am
DaveL17 offline
User avatar
Posts: 6744
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Feature Requests

Hi Wilhelm - thanks for your patience. Please try the latest version (v0.4.49) and see if that solves the issue with devices not being taken offline.

https://github.com/IndigoDomotics/GhostXML

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Sun Jan 19, 2020 12:25 pm
Umtauscher offline
User avatar
Posts: 566
Joined: Oct 03, 2014
Location: Cologne, Germany

Re: Feature Requests

Hi Dave,

I just did a quick test with .49
It looks promising, I did only two testruns thought.
I will let it run for some time and will report again.
Thank you very much for your excellent support.
Cheers

Wilhelm

Posted on
Sun Jan 19, 2020 12:32 pm
DaveL17 offline
User avatar
Posts: 6744
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Feature Requests

That's great news. I'll keep my fingers crossed.

Appreciate the help.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Wed Feb 19, 2020 6:58 am
Umtauscher offline
User avatar
Posts: 566
Joined: Oct 03, 2014
Location: Cologne, Germany

Re: Feature Requests

Hi Dave,

unfortunately I have to report another problem today. I don't know, if it's GhostXML's fault or the SQLLogger's but since the problem arrose during updateing a ghostxml device I think it's more likely to be the GhostXML plugin.
Yesterday one of the IndigoPluginHosts consumed all the cpu of my server so I startet to investigate. It turned out, that it was the SQLLogger going to 100% and never stopping.
Then I examined the log an it was an rather odd insert statement running over pages and pages of the log. (I have attached the problematic part.
As you can see, in Line 45, SQLLogger tries to insert hundrets of columns of a GhostXML device into the database which takes for ages and finally fails. In fact it wrote a single line to the log that was more that 1 MB in size.
I am back to 0.4.36 of the GhostXML plugin for now.
Maybe you could have a look at the log.
TIA
Wilhelm

P.S: Sadly that single statement in the log is more that 1.3 MB in size and I cannot attach it. (512 KB, what sizelimit is that, we're in the 21st Century!?!) I have cut most of the statement out. Please tell me where I can send the complete log if you need it.
Attachments
SqlLogger_cut.log
(38.15 KiB) Downloaded 119 times

Posted on
Wed Feb 19, 2020 8:03 am
DaveL17 offline
User avatar
Posts: 6744
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Feature Requests

Hi Wilhelm - I'll need to think about this a little bit more, but on the face of it, I don't believe that this is a problem with the plugin per se. Its purpose is to take the source XML/JSON data and convert 100 percent of that data into device states. Depending on the XML/JSON source, there could be many hundreds of states for a single device. By definition, these states may change over time due to the source data structure changing (new nodes added, existing ones deprecated) so nearly all of the states are handled dynamically.

I know that Matt & Jay have added some new SQL handling, but--I'll be honest--I don't know anything about it because I don't use SQL logging. I'll need to see what the options are, but I suspect (again because of the dynamic nature of the plugin) that it's likely to be an "all or nothing" solution. In other words, either all of the states will be logged or none of them will.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Wed Feb 19, 2020 8:07 am
Umtauscher offline
User avatar
Posts: 566
Joined: Oct 03, 2014
Location: Cologne, Germany

Re: Feature Requests

Maybe so, but the device in question only has 18 custom states shown by GhostXML so it must be a bug at some point.

Posted on
Wed Feb 19, 2020 9:41 am
RogueProeliator offline
User avatar
Posts: 2501
Joined: Nov 13, 2012
Location: Baton Rouge, LA

Re: Feature Requests

unfortunately I have to report another problem today. I don't know, if it's GhostXML's fault or the SQLLogger's but since the problem arrose during updateing a ghostxml device I think it's more likely to be the GhostXML plugin.

I have not investigated this, but based upon a quick viewing of the INSERT statement segment that you posted, it very well could be some SQL Logger related error... Hopefully @Matt or @Jay can take a look but my initial impression is that it looks like something is getting stuck in a loop or recursion -- it looks like like it is continuously adding an extra _ui at the end of the column name.

Could be something in the GhostXML plugin triggering it, of course, just looks suspiciously like something going awry inside the logger.

Adam

Who is online

Users browsing this forum: No registered users and 2 guests

cron