convert html to xml or json

Posted on
Tue Apr 23, 2019 10:53 am
bsp9493 offline
Posts: 153
Joined: Nov 30, 2017
Location: Kelowna, BC

Re: convert html to xml or json

NICE!!!

Thanx for that!!

d

howartp wrote:
bsp9493 wrote:
howartp wrote:
Oh, and once they’re in variables, I launched a new plugin yesterday (forum only yet) that puts variables into a device.



oooohhhh, does it have a name, what section?? I have to look for that.

Sent from my iPhone using Tapatalk Pro

viewtopic.php?t=22336


Sent from my iPhone using Tapatalk Pro

Posted on
Tue Apr 23, 2019 11:11 am
autolog offline
Posts: 3988
Joined: Sep 10, 2013
Location: West Sussex, UK [GMT aka UTC]

Re: convert html to xml or json

bsp9493 wrote:
do you mind sharing? If you can pm me the plugin, I can maybe try to modify for the ambient ws.


It's not in a state that I would want to share at the moment - past experience tells me it is better to get it near correct before releasing. :)

I will make the basic changes to support the Ambient ws and then let you have that as the plugin is in a state of flux at the moment.

What would be useful is if you could run the following python script in a terminal:
Code: Select all
import socket               # Import socket module

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
port = <PORT NUMBER>                 # port specified in your Ambient Ws weather station settings e.g. 9125
ip_address = <IP-ADDRESS>  # ip addresss specified in your Ambient  weather station Server IP/Hostname (customised setting) e.g. 192.168.1.8
s.bind((ip_address, port))  # Bind to the port

s.listen(5)                 # Now wait for client connection.
while True:
   c, addr = s.accept()     # Establish connection with client.
   print 'Got connection from', addr
   print c.recv(1024)
   c.close()                # Close the connection   


You need to modify the above code to set the correct ip-address and port number. IP address will be that of your Indigo server.

To do run this script start a terminal session. Then type python to start the Python interpreter and then paste the code into the terminal and then press enter twice to kick it off.

If it works, you should get a rapidly filling page with content like:
Code: Select all
GET /weatherstation/updateweatherstation.php?ID=weather&PASSWORD=password&tempf=63.1&humidity=56&dewptf=47.3&windchillf=63.1&winddir=78&windspeedmph=4.47&windgustmph=7.61&rainin=0.00&dailyrainin=0.00&weeklyrainin=0.00&monthlyrainin=0.51&yearlyrainin=21.47&solarradiation=95.35&UV=1&indoortempf=72.7&indoorhumidity=49&baromin=29.17&lowbatt=0&dateutc=2019-04-23%2016:57:12&softwaretype=WH2600GEN_V2.2.7&action=updateraw&realtime=1&rtfreq=5 HTTP/1.0
Accept: */*
Host: 192.168.1.8
Connection: Close

GET /weatherstation/up
Got connection from ('192.168.1.199', 30894)
GET /weatherstation/updateweatherstation.php?ID=weather&PASSWORD=password&tempf=63.1&humidity=56&dewptf=47.3&windchillf=63.1&winddir=73&windspeedmph=4.92&windgustmph=4.92&rainin=0.00&dailyrainin=0.00&weeklyrainin=0.00&monthlyrainin=0.51&yearlyrainin=21.47&solarradiation=95.16&UV=1&indoortempf=72.7&indoorhumidity=49&baromin=29.17&lowbatt=0&dateutc=2019-04-23%2016:57:13&softwaretype=WH2600GEN_V2.2.7&action=updateraw&realtime=1&rtfreq=5 HTTP/1.0
Accept: */*
Host: 192.168.1.8
Connection: Close

Just press CTRL-C to stop the session and then copy one of the returned sections starting with GET /weatherstation/up and ending with Connection: Close.

Note: that you will need to stop any other process that is accessing the Ambient while you run the script otherwise you will get an address in use error.

That will tell me how much is involved in getting it working and what changes I might need to make. :)

Posted on
Tue Apr 23, 2019 12:17 pm
bsp9493 offline
Posts: 153
Joined: Nov 30, 2017
Location: Kelowna, BC

Re: convert html to xml or json

nothing running, but get the following

Code: Select all
Python 2.7.10 (default, Feb 22 2019, 21:17:52)
[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket               # Import socket module
>>>
>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> port = 5000                 # port specified in your Ambient Ws weather station settings e.g. 9125
>>> ip_address = 192.168.1.200  # ip addresss specified in your Ambient  weather station Server IP/Hostname (customised setting) e.g. 192.168.1.8
  File "<stdin>", line 1
    ip_address = 192.168.1.200  # ip addresss specified in your Ambient  weather station Server IP/Hostname (customised setting) e.g. 192.168.1.8
                         ^
SyntaxError: invalid syntax
>>> s.bind((ip_address, port))  # Bind to the port
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'ip_address' is not defined
>>>
>>> s.listen(5)                 # Now wait for client connection.
>>> while True:
...    c, addr = s.accept()     # Establish connection with client.
...    print 'Got connection from', addr
...    print c.recv(1024)
...    c.close()                # Close the connection   



autolog wrote:
bsp9493 wrote:
do you mind sharing? If you can pm me the plugin, I can maybe try to modify for the ambient ws.


It's not in a state that I would want to share at the moment - past experience tells me it is better to get it near correct before releasing. :)

I will make the basic changes to support the Ambient ws and then let you have that as the plugin is in a state of flux at the moment.

What would be useful is if you could run the following python script in a terminal:[code]import socket # Import socket module

Posted on
Tue Apr 23, 2019 12:42 pm
autolog offline
Posts: 3988
Joined: Sep 10, 2013
Location: West Sussex, UK [GMT aka UTC]

Re: convert html to xml or json

My bad :oops:

Change
Code: Select all
ip_address = 192.168.1.200
to
Code: Select all
ip_address = '192.168.1.200'
i.e. add in quotes around the ip-address value. :)

Posted on
Tue Apr 23, 2019 12:53 pm
bsp9493 offline
Posts: 153
Joined: Nov 30, 2017
Location: Kelowna, BC

Re: convert html to xml or json

no worries, I had tried that too. Still no luck... now I get...

socket.error: [Errno 49] Can't assign requested address

what exactly do you mean by no other process running... indigo server is shut down, did a clean reboot of system, only thing I can think of is, external data logger is still up, but not processes running on computer.

Code: Select all
>>> import socket               # Import socket module
>>>
>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> port = 5000                 # port specified in your Ambient Ws weather station settings e.g. 9125
>>> ip_address = '192.168.1.200'  # ip addresss specified in your Ambient  weather station Server IP/Hostname (customised setting) e.g. 192.168.1.8
>>> s.bind((ip_address, port))  # Bind to the port
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 49] Can't assign requested address
>>>
>>> s.listen(5)                 # Now wait for client connection.
>>> while True:
...    c, addr = s.accept()     # Establish connection with client.
...    print 'Got connection from', addr
...    print c.recv(1024)
...    c.close()                # Close the connection


autolog wrote:
My bad :oops:

Change
Code: Select all
ip_address = 192.168.1.200
to
Code: Select all
ip_address = '192.168.1.200'
i.e. add in quotes around the ip-address value. :)

Posted on
Tue Apr 23, 2019 1:01 pm
autolog offline
Posts: 3988
Joined: Sep 10, 2013
Location: West Sussex, UK [GMT aka UTC]

Re: convert html to xml or json

Is 192.168.1.200 the address of your Indigo server where you are running the script?

Posted on
Tue Apr 23, 2019 1:05 pm
autolog offline
Posts: 3988
Joined: Sep 10, 2013
Location: West Sussex, UK [GMT aka UTC]

Re: convert html to xml or json

I suspect that is the case as I have just got that error if I change the ip address not to be my Indigo server. You need to point the Ambient WS at the Indigo server not the data logger.

If you need to continue to use the data logger in the future then this probably isn't the solution for you as the plugin makes a direct exclusive connection to the WeatherSleuth / Ambient WS to retrieve the data and doesn't use screen scraping. :)

Posted on
Tue Apr 23, 2019 1:08 pm
bsp9493 offline
Posts: 153
Joined: Nov 30, 2017
Location: Kelowna, BC

Re: convert html to xml or json

192.168.1.200 is IP of ambient device

ie
indigo server is located on 151 (and is currently shut down) , I am running the python shell off a totally separate computer on network

http://192.168.1.200 takes me to device landing page
http://192.168.1.200/livedata.htm - takes me to web interface of live data

just did i-net scan and revealed port 80 is the only open port, so I changed that in the python you send but still get same:(

will switch these to pm so we don't flood this topic



autolog wrote:
Is 192.168.1.200 the address of your Indigo server where you are running the script?

Posted on
Tue Apr 23, 2019 1:12 pm
bsp9493 offline
Posts: 153
Joined: Nov 30, 2017
Location: Kelowna, BC

Re: convert html to xml or json

that may be the problem then, don't think the ambient device supports anything other than a screen scrape. great little inexpensive weather station but limited in functionality beyond what is provided. I'm just trying to do TOO much with it.


bsp9493 wrote:
192.168.1.200 is IP of ambient device

ie
indigo server is located on 151 (and is currently shut down) , I am running the python shell off a totally separate computer on network

http://192.168.1.200 takes me to device landing page
http://192.168.1.200/livedata.htm - takes me to web interface of live data

just did i-net scan and revealed port 80 is the only open port, so I changed that in the python you send but still get same:(

will switch these to pm so we don't flood this topic



autolog wrote:
Is 192.168.1.200 the address of your Indigo server where you are running the script?

Posted on
Tue Apr 23, 2019 1:31 pm
autolog offline
Posts: 3988
Joined: Sep 10, 2013
Location: West Sussex, UK [GMT aka UTC]

Re: convert html to xml or json

I'll await your PM. :)

The IP address needs to be the IP address of the computer you are running the script on. In the Ambient WS you need to set the SERVER IP to be the ip of the computer running the script. The port number defined in the Ambient WS needs to be the same one that you specify in the script.

The tabs I have from weatherSleuth are:
  • Local Network
  • Weather Network
  • Station Settings
  • Live Data
  • Calibration
The settings that need altering are in the Weather Network tab and you need to select a Remote Server of Customized to be able to specify the Server IP/Hostname and Server Port.

My example:
weatherSleuth.png
WeatherSleuth Example
weatherSleuth.png (57.04 KiB) Viewed 3019 times


The Station ID and password are ignored

Who is online

Users browsing this forum: No registered users and 3 guests