Help with JSON Variable Decoding

Posted on
Fri Mar 13, 2020 7:43 pm
lvrouter offline
Posts: 18
Joined: Apr 17, 2016

Help with JSON Variable Decoding

I have a URL (from a temperature sensor) which returns the following:

Code: Select all
{
   "result": "success",
   "server_time": "2020-03-14T01:17:53Z",
   "channels": [{
      "channel_id": "12503",
      "field1": "Temperature",
      "field2": "Humidity",
      "field3": "Light",
      "field4": "Voltage",
      "field5": "WIFI RSSI",
      "field6": "Vibration Index",
      "field7": "Knocks",
      "field8": "External Temperature Probe",
      "field9": "",
      "field10": null,
      "latitude": "36.0097",
      "longitude": "-115.1479",
      "name": "Garage",
      "public_flag": "false",
      "tags": null,
      "url": null,
      "metadata": "{\"fn_th\":300,\"fn_light\":300,\"fn_mag\":0,\"fn_mag_int\":0,\"fn_acc_tap1\":0,\"fn_acc_tap2\":0,\"fn_acc_act\":0,\"fn_acc_min\":10,\"fn_bt\":0,\"fn_ext_t\":0,\"fn_battery\":7200,\"fn_dp\":900,\"cg_data_led\":1,\"wifi_mode\":1,\"no_net_fn\":1}",
      "description": null,
      "traffic_out": "273787",
      "traffic_in": "117292",
      "status": "{\"ssid\":\"cfwap\",\"status\":\"mac=6c:c3:74:eb:7d:77,usb=0\",\"usb\":\"0\",\"mac\":\"6c:c3:74:eb:7d:77\"}",
      "timezone": null,
      "created_at": "2020-03-12T02:09:23Z",
      "updated_at": "2020-03-12T18:28:16Z",
      "usage": "363330",
      "last_entry_id": "1908",
      "last_entry_date": "2020-03-14T01:11:29Z",
      "product_id": "ubibot-ws1",
      "device_id": "cf43a1776ff0218985d7a91a40965b1700819c56",
      "channel_icon": null,
      "last_ip": "70.180.187.74",
      "attached_at": "2020-03-12T02:09:23Z",
      "firmware": "ws1_v2.6.8",
      "full_dump": "0",
      "activated_at": "1583979074",
      "serial": "VBM***WS1",
      "mac_address": "6c:c3:74:eb:7d:77",
      "full_dump_limit": "3",
      "cali": null,
      "size_out": "1073741824",
      "size_storage": "209715200",
      "plan_code": "ubibot_free",
      "plan_start": "2020-03-12T02:09:23Z",
      "plan_end": null,
      "bill_start": "2020-03-12T02:09:23Z",
      "bill_end": "2020-04-11T02:09:23Z",
      "last_values": "{\"field1\":{\"value\":24.754333,\"created_at\":\"2020-03-14T01:11:15Z\",\"net\":\"1\"},\"field3\":{\"value\":22.43,\"created_at\":\"2020-03-14T01:11:15Z\"},\"field4\":{\"value\":3.132439,\"created_at\":\"2020-03-14T00:11:14Z\"},\"field2\":{\"value\":30,\"created_at\":\"2020-03-14T01:11:15Z\",\"net\":\"1\"},\"field5\":{\"value\":-33,\"created_at\":\"2020-03-14T01:11:17Z\"}}",
      "vconfig": "{\"field1\":{\"h\":\"0\",\"u\":\"1\"},\"field2\":{\"h\":\"0\",\"u\":\"3\"},\"field3\":{\"h\":\"0\",\"u\":\"4\"},\"field4\":{\"h\":\"0\",\"u\":\"5\"},\"field5\":{\"h\":\"0\",\"u\":\"6\"},\"field6\":{\"h\":\"0\",\"u\":\"7\"},\"field7\":{\"h\":\"0\",\"u\":\"8\"},\"field8\":{\"h\":\"0\",\"u\":\"1\"},\"field9\":{\"h\":\"0\",\"u\":\"9\"}}",
      "vpref": null,
      "battery": null,
      "vpref_from": "owner",
      "net": "1",
      "c_icon_base": null,
      "full_serial": "VBM39XWS1",
      "triggering_rules": null
   }]
}


If I access it as follows:

Code: Select all
myvar = str(result['channels'][0]['field1'])


I get "Temperature" as the myvar as expected.

But how do I access the values within the "last_values" portion?

What I want is the value (in this case 24.754333) within that string (it's the temperature recorded by the device).

Thank you for any assistance you can provide.

Posted on
Sat Mar 14, 2020 3:11 am
neilk offline
Posts: 714
Joined: Jul 13, 2015
Location: Reading, UK

Re: Help with JSON Variable Decoding

This may be the blind leading the blind but I will give it a go.

"last_values" is returning a string rather than a dictionary, for reasons beyond my knowledge and maybe due to the way the JSON is generated or decoded

Code: Select all
myvar = result['channels'][0]['last_values']


will return a string rather than a dictionary (and it is the same in a JSON viewer)

Code: Select all
{"field1":{"value":24.754333,"created_at":"2020-03-14T01:11:15Z","net":"1"},"field3":{"value":22.43,"created_at":"2020-03-14T01:11:15Z"},"field4":{"value":3.132439,"created_at":"2020-03-14T00:11:14Z"},"field2":{"value":30,"created_at":"2020-03-14T01:11:15Z","net":"1"},"field5":{"value":-33,"created_at":"2020-03-14T01:11:17Z"}}


this does evaluate as a python dictionary so you could use

Code: Select all
last_values_dict = json.loads(result['channels'][0]['last_values'])


So to set your variable you would need

Code: Select all
last_values_dict = json.loads(result['channels'][0]['last_values'])
tempvar = last_values_dict['field1']['value']


I am sure I have missed something and someone better qualified can give a better answer but this does work against your sample.

* Edited to replace the eval with JSON loads as the prior use of eval was not good practice

Neil
Last edited by neilk on Sun Mar 15, 2020 4:55 am, edited 1 time in total.

Posted on
Sat Mar 14, 2020 8:00 am
lvrouter offline
Posts: 18
Joined: Apr 17, 2016

Re: Help with JSON Variable Decoding

There is no better answer, you obviously can "see"! Works perfectly, thank you.

FWIW, this is accessing the temperature and humidity settings on an UbiBOT sensor (on Amazon for $80). Battery powered, wi-fi and has all types of capabilities including a free "portal" to access and record. It's a pretty amazing little device. Been looking for something like this for a while.

Posted on
Sat Mar 14, 2020 8:24 am
DaveL17 offline
User avatar
Posts: 6742
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Help with JSON Variable Decoding

Are there truly escape characters in the data like that? That's weird.

Use extreme caution when using eval as it can expose you to malicious code. If you *always* know the source is safe, you should be okay.

Safer is to use literal_eval.

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Sat Mar 14, 2020 8:29 pm
FlyingDiver offline
User avatar
Posts: 7189
Joined: Jun 07, 2014
Location: Southwest Florida, USA

Re: Help with JSON Variable Decoding

If that’s actually a JSON string, you should use json.loads() to convert it to Python. Using eval() really isn’t safe.

joe (aka FlyingDiver)
my plugins: http://forums.indigodomo.com/viewforum.php?f=177

Posted on
Sun Mar 15, 2020 1:02 am
jay (support) offline
Site Admin
User avatar
Posts: 18199
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Help with JSON Variable Decoding

FlyingDiver wrote:
If that’s actually a JSON string, you should use json.loads() to convert it to Python. Using eval() really isn’t safe.


+1 - it's escaped because it's JSON inside of JSON...

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Posted on
Sun Mar 15, 2020 4:26 am
DaveL17 offline
User avatar
Posts: 6742
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Help with JSON Variable Decoding

jay (support) wrote:
it's escaped because it's JSON inside of JSON...

I don't think I've ever seen that before.

Code: Select all
#! /usr/bin/env python
# -*- coding: utf-8 -*-

import json

result = r'''{
   "result": "success",
   "server_time": "2020-03-14T01:17:53Z",
   "channels": [{
      "channel_id": "12503",
      "field1": "Temperature",
      "field2": "Humidity",
      "field3": "Light",
      "field4": "Voltage",
      "field5": "WIFI RSSI",
      "field6": "Vibration Index",
      "field7": "Knocks",
      "field8": "External Temperature Probe",
      "field9": "",
      "field10": null,
      "latitude": "36.0097",
      "longitude": "-115.1479",
      "name": "Garage",
      "public_flag": "false",
      "tags": null,
      "url": null,
      "metadata": "{\"fn_th\":300,\"fn_light\":300,\"fn_mag\":0,\"fn_mag_int\":0,\"fn_acc_tap1\":0,\"fn_acc_tap2\":0,\"fn_acc_act\":0,\"fn_acc_min\":10,\"fn_bt\":0,\"fn_ext_t\":0,\"fn_battery\":7200,\"fn_dp\":900,\"cg_data_led\":1,\"wifi_mode\":1,\"no_net_fn\":1}",
      "description": null,
      "traffic_out": "273787",
      "traffic_in": "117292",
      "status": "{\"ssid\":\"cfwap\",\"status\":\"mac=6c:c3:74:eb:7d:77,usb=0\",\"usb\":\"0\",\"mac\":\"6c:c3:74:eb:7d:77\"}",
      "timezone": null,
      "created_at": "2020-03-12T02:09:23Z",
      "updated_at": "2020-03-12T18:28:16Z",
      "usage": "363330",
      "last_entry_id": "1908",
      "last_entry_date": "2020-03-14T01:11:29Z",
      "product_id": "ubibot-ws1",
      "device_id": "cf43a1776ff0218985d7a91a40965b1700819c56",
      "channel_icon": null,
      "last_ip": "70.180.187.74",
      "attached_at": "2020-03-12T02:09:23Z",
      "firmware": "ws1_v2.6.8",
      "full_dump": "0",
      "activated_at": "1583979074",
      "serial": "VBM***WS1",
      "mac_address": "6c:c3:74:eb:7d:77",
      "full_dump_limit": "3",
      "cali": null,
      "size_out": "1073741824",
      "size_storage": "209715200",
      "plan_code": "ubibot_free",
      "plan_start": "2020-03-12T02:09:23Z",
      "plan_end": null,
      "bill_start": "2020-03-12T02:09:23Z",
      "bill_end": "2020-04-11T02:09:23Z",
      "last_values": "{\"field1\":{\"value\":24.754333,\"created_at\":\"2020-03-14T01:11:15Z\",\"net\":\"1\"},\"field3\":{\"value\":22.43,\"created_at\":\"2020-03-14T01:11:15Z\"},\"field4\":{\"value\":3.132439,\"created_at\":\"2020-03-14T00:11:14Z\"},\"field2\":{\"value\":30,\"created_at\":\"2020-03-14T01:11:15Z\",\"net\":\"1\"},\"field5\":{\"value\":-33,\"created_at\":\"2020-03-14T01:11:17Z\"}}",
      "vconfig": "{\"field1\":{\"h\":\"0\",\"u\":\"1\"},\"field2\":{\"h\":\"0\",\"u\":\"3\"},\"field3\":{\"h\":\"0\",\"u\":\"4\"},\"field4\":{\"h\":\"0\",\"u\":\"5\"},\"field5\":{\"h\":\"0\",\"u\":\"6\"},\"field6\":{\"h\":\"0\",\"u\":\"7\"},\"field7\":{\"h\":\"0\",\"u\":\"8\"},\"field8\":{\"h\":\"0\",\"u\":\"1\"},\"field9\":{\"h\":\"0\",\"u\":\"9\"}}",
      "vpref": null,
      "battery": null,
      "vpref_from": "owner",
      "net": "1",
      "c_icon_base": null,
      "full_serial": "VBM39XWS1",
      "triggering_rules": null
   }]
}
'''

level_1 = json.loads(result)
level_2 = json.loads(level_1['channels'][0]['last_values'])
print(level_2['field1']['value'])

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Sun Mar 15, 2020 5:00 am
neilk offline
Posts: 714
Joined: Jul 13, 2015
Location: Reading, UK

Re: Help with JSON Variable Decoding

FlyingDiver wrote:
If that’s actually a JSON string, you should use json.loads() to convert it to Python. Using eval() really isn’t safe.


I updated the original post to reflect the "right way". Thanks everyone, every day is a school day :D

BTW I had tried json.load not json.loads as I had figured it might be json in a json but I forgot the string bit. Some of the more esoteric bits of python still catch me out.

Posted on
Sun Mar 15, 2020 6:31 am
DaveL17 offline
User avatar
Posts: 6742
Joined: Aug 20, 2013
Location: Chicago, IL, USA

Re: Help with JSON Variable Decoding

I learned something new in this thread, too. JSON inside JSON? Who knew?

(Jay knew. :D )

I came here to drink milk and kick ass....and I've just finished my milk.

[My Plugins] - [My Forums]

Posted on
Sun Mar 15, 2020 6:36 am
neilk offline
Posts: 714
Joined: Jul 13, 2015
Location: Reading, UK

Re: Help with JSON Variable Decoding

And I should have said some of the simple things catch me out as well :D

Posted on
Sun Mar 15, 2020 9:46 am
jay (support) offline
Site Admin
User avatar
Posts: 18199
Joined: Mar 19, 2008
Location: Austin, Texas

Re: Help with JSON Variable Decoding

I'm not really sure why they'd implement it like that - seems like it would have been easier to just make the value represented by "last_values" the actual dictionary of values rather than the JSON escaped string representation of the dictionary. I can't think of any reason why they did it that way...

Jay (Indigo Support)
Twitter | Facebook | LinkedIn

Page 1 of 1

Who is online

Users browsing this forum: No registered users and 6 guests