store json in self.pluginPrefs.. vs file

I was looking at my plugin prefs files and noticed that one of the files was 10Mbyte
the culprit was a json.dumps entry.
when writing that dict to a file w
f.write(json.dumps(theDict))
instead of
self.pluginPrefs["theDict"] = json.dumps(theDict)
to the indigo plugin prefs file
the file size of the plugin.prefs file went from ~ 10Mbyte to ~30Kbyte and the jsondumps file was just 22 K

It seems that self.pluginPrefs for longer dicts creates really large files ie " --> "
That would explain a factor of 6 but not 200.. ==10,000,000/(30+20)

??

Karl

[matt (support)]

That is strange. Did you look at the plugin prefs XML file to see how it was being stored? json.dumps() just returns a string so I wouldn't expect such a magnitude of change between the two files.

actually the comparison was not fair.
The file written had been initialized == many sub dicts empty.

After redoing it properly the delta was still a factor of ~two: 9.8MB --> 5.6 Mbyte
still not small.

Karl

[matt (support)]

Is it being stored in the XML as a single string node? Or is it being stored as a dict?

I suspect that the XML storage is requiring a lot of escaping of string delimiters, etc, which is causing the expansion.

Here's an example from one of my plugins:

Code: Select all

<aggregators type="string">{&quot;1999390458:shellies/shellyht-F36DC6&quot;: {&quot;brokerID&quot;: &quot;1999390458&quot;, &quot;topic_base&quot;: &quot;shellies/shellyht-F36DC6&quot;, &quot;payload&quot;: {}}, &quot;1867973662:foobar/status&quot;: {&quot;brokerID&quot;: &quot;1867973662&quot;, &quot;topic_base&quot;: &quot;foobar/status&quot;, &quot;payload&quot;: {}}}</aggregators>


saving to prefs:
self.pluginPrefs["theDict"] = json.dumps(theDict)
theDict= {"_background_blocking_": {"lastCPUsub": {}, "evID": 0, "cpuThreshold": 99999999999, "plugData": {"plugName": "_backgr ....

and yes a lot of " in the prefs file

And BBEDIT takes 5 minutes to "unspin" when opening the prefs file. I guess it tries to understand the xml fields
And once it is not busy it is impossible to use as every time I move the cursor it takes >5 secs to actually move

No problems w the json file.

I guess the prefs file is not the place to save larger dicts

Karl