I’ve been struggling to find out where to store various data both user made/edited and set by a plugin.
So far I’ve not been able to find one “blessed answer” from anyone. So I created a list and compiled the suggestions of various authors for where to store data/settings for plugins.
Suggestions for Where to Store Data
@tt_su recommends not saving anything in the Plugins folder, because those files are liable to be deleted unexpectedly when updating/installing plugins:
@DanRathbun recommends storing in %AppData%/SuperGeek/Widget (for a company called SuperGeek) in order to preserve settings if Sketchup version changes.
@jody recommends storing data in Documents on Mac and My Documents on Windows.
Others I’ve seen have recommended svaing things in the Windows Registry, which is a little hairy, and only works on one platform.
So far I feel that @jody’s suggestion is easiest to get working cross-platform, but worry that users may be confused about a new folder appearing in their Documents and may inadvertently delete it. It may be better to save somewhere less frequently accessed.
Here’s my compiled list of the options for storing data
How to Store Settings/Data
Plugin settings that you would like carried across all files:
PStore - Ruby’s blessed method for data storage, but I’ve not heard of anyone using it. Can guarantee against data corruption when reading/writing, and is very fast.
YAML::Store - similar methods to PStore. Less performant but readable and editable from any text editor.
CSV - Easy to read and write, but gives a lot of problems opening/saving with Excel.
JSON
A custom format in a .txt file
File specific settings
AttributeDictionary on the active_model
Component specific data
AttributeDictionary on the component.
So the questions:
Where and how do you store your data? Are there are any I missed? Any suggested best practices?
This seems to have problems on Mac since SketchUp 2018 changed to use JSON by default:
Preferences Location
The majority of preferences are now stored in two json files. There are still some OS controlled preferences that remain in the registry (Win) or plist (Mac). The json files are saved in the following locations:
On Win:
PrivatePreferences.json is saved to C:\Users\AppData\Local\SketchUp\SketchUp 2018\SketchUp SharedPreferences.json is saved to C:\Users\AppData\Roaming\SketchUp\SketchUp 2018\SketchUp
On Mac:
Both PrivatePreferences.json and SharedPreferences.json are saved to /Users//Library/Application Support/SketchUp 2018/SketchUp
Also, I see @DanRathbun recommending against write_default in favor of JSON here:
If the data you store is not from a user point of view a document, that the user might want to double click to open, I would advise against My Documents. It’s quite annoying to have your personal document folder cluttered by things you don’t know what they are for and don’t feel you are in control of. Some users may even delete the files because they don’t remember putting them there (which of course they didn’t do, the plugin did). Storing user settings and other data that are only supposed to be accessed by under the hood by a specific software is exactly what appdata in Windows is for! I don’t know about Mac though.
For storing I think JSON is the simplest, least verbose and most flexible.
For individual values SketchUp’s internal read_default adn save_default methods are quite handy too.
It’s also worth mentioning that some settings are best saved as model attributes. If the settings are project specific this keeps the the settings when the document is moved between machines. This does not however apply to all user settings, only a few. The plugin preferences dialog would also need to clearly state these are Model Settings.
the biggest issue with the generic method is the absence of a delete method…
Sketchup.delete_default([options])
I use system calls to remove items on mac…
the .json file hasn’t made any difference to storing 3rd party preferences…
the only issues I’ve encountered exist in both .plists and .json versions…
they are not due too 3d party extension presences but with SU default values that can’t be accessed…
you will have issues if your data breaks .json formatting convention…
Model/Component level attributes should be kept to a minimum as they ‘travel’ with the component and can ‘bloat’ the file as well as cause issues on machines without the originating extension.
In the last release of one of my extensions I added a one character txt file (either “y” or “n”) to record the user response this dialog which displays on initial load:
This reminder has eliminated countless support emails from customers too lazy to look at the help file after doing an update.
I put the text file in the plugin’s own “resources” subfolder. BUT it seems that my ruby code that writes “n” to the txt file invalidates the extension signature!!! Who knew?
None of the SU or SU API documentation mentions the “invalid signature” status or explains how it might occur and what its implications are for the “loading policy modes”.
As well as the ‘vulnerable’ file-types [RBE/RBS/RB/HTM/HTML/JS/CSS] the powers that be snuck in TXT more recently [with no ‘fanfare’!].
This does cause issues for authors using TXT files to store settings with their extension in its subfolder.
However, there are all of the alternatives already mentioned, OR simply name your settings file as a non-sensitive file-type - e.g. CSV/DAT/LIC/2DXY etc…, OR make your settings file AFTER your extension is installed, just on its first-run - it’s not then considered in the signing-check…
Of course one day ‘someone’ will have the bright-idea [sic] of signing all files in the submitted RBZ, subverting the alternative file-type solution.
An even more far-reaching checking for ANY files added to the subfolder after the signing process, and disallowing them too !
Heaven-forfend…
I have been using simple text files to store external data for many years without any issues. The data is in the format of key value pairs. This makes it very easy to support users since I can open up these files with notepad++ and see the data.
I never use the registry as it is just too painful to support my clients.
I do store my project settings in the model in a single data dictionary record in addition to storing text files. This data in the dictionary record starts out in the exact same format as my text files with the only exception that I substitute the \n for a delimiter which has been the ^ caret symbol. I can then use the exact same methods to parse the dictionary record as I use to parse the text files. I just replace the ^ with a \n before parsing. My project setting record is under 1K and will hardly be noticed let alone considered bloat !!
Generally I use a structure to hold information and I first pass the structure through a method which first sets logical defaults for each element of the structure. This allows me to move forward - adding new items to the structure and still get valid information from older models that do not have all the elements.
Since Trimble’s extension manager may delete the entire extension prior to updating I have settled on a different location for plugin data. Following convention I would store data under my own plugin name under program data. One of the paths that is returned by $LOAD_PATH is:
C:/ProgramData/SketchUp/SketchUp 2018/SketchUp/Plugins
The bold folder names are Company, plugin and user specific
So for windows I would store data under
C:/ProgramData/Company/Plugin_Name/
And with the Mac
/Library/Application Support/Company/Plugin_Name/
However if you have different users on the same machine running the same plugin and wanting their own preferences then I would store data under
C:/Users/User_Name/AppData/Roaming/Company/Plugin_Name/
And with the Mac
/Users/User_Name/Library/Application Support/Company/Plugin_Name/
In my opinion this is NOT a work around. I have chosen to use text files for many many reasons. Even if we get additional functionality I wouldn’t use it.
I see absolutely no benefit in making things more complicated or for that matter fixing something that ain’t broke. My current strategy is simple - scalable - backwards compatible - fast - very readable and very easy to maintain.
The only place I currently use JSON is to communicate with my web service.
So for me The Correct Way to Store is what I am currently doing
I wouldn’t want to invent my own format for saving and parsing data. It may work great at first but scales badly if you in the future need to save strings that include the your control characters, or need nested hashes/arrays. JSON already escapes all special characters, allowing for a JSON string to contain any string data, including a nested JSON object expressed as string. Of course you can escape strings on your own but why reinvent something that already works and is very widespread.
Actually NOT. On MS Windows at least, the name of the user’s "Documents" folder is localized into the user’s system locale language. So you’d need to call a Windows SDK function via either Fiddle or Win32OLE (accessing the Windows Scripting Host Shell and it’s WshSpecialFolders object) to get the proper pathname.
For this and the reasons Christina explained above, it’s actually the worst of the ideas that has been suggested.
%AppData%
This is WHERE I advocate saving plugin settings, regardless of the data format.
On MS Windows normal users will have read and write permissions for these folders.
True, also because users do not understand why their plugin settings are not “brought forward” to the next SketchUp version when they are saved (transparently to the user) in a versioned SketchUp %AppData% sub-folder.
Putting your plugin settings in your own company %AppData% folder path, solves this, and user will see their settings apply from version to version.
The downside, is they’ll begin to expect it from all plugins.
%ProgramData%
Do not have an opinion for Mac (not being a “Macite”,) …
but on MS Windows normal users do not have write permissions for folders in this path.
Data saved here is usually done by installers which run with elevated privileges.
They will have read permissions, so this location is fine for data that will not change after install, and will only change later when the next version is installed.
On Windows, there is also a “Default” user account that a plugin can save the base settings to, and when new user accounts are generated they’ll get “the default’s”. (These defaults are also usually written by installers running under elevated privileges.)
Plugin Subfolder
Agree (with Thomas,) and also for the reasons that Barry and TIG talk about with regard to the ever changing extension “security” headaches.
We avoid this by using our own plugin data directories, beneath our own company data directory, located in the proper place which is %AppData%.
Sketchup::read_default() and Sketchup::write_default() - shared location
This is one of those things where I must disagree with John. Basically the API has failed us, and continues to do so with these methods.
All the historic problems with their use still exist, despite years of complaints by developers.
Future fixes cannot help with plugins running in current or past versions. (Gary avoids this by not using these 2 methods at all, so he and his plugins are all set no matter what happens to these methods.)
These two methods are untrustworthy, just from the fact that their API documentation is incorrect and has been for years. (Add to this, that these methods under went a big change in the last version, but their documentation was not updated.)
They even changed WHEN the settings were written to storage, without any notice.
This is not developer friendly !
I’ve lost patience with regard to these methods. They are just junk to be blunt !
These methods work the way they work, because this is the way that the Trimble team extension’s need them to work for their “shipped” extensions. So fine, I leave these methods to the use of the “shipped” extensions then.
And I really no longer have any care whether they work or not, or get any nifty new overhaul.
With all of the better alternatives … why should I ?
FORMAT
CSV
I do not know what Excel would have to do with plugin settings. It shouldn’t.
Ruby has a CSV library that is part of the standard library (at present.)
If you are afraid of users opening, tweaking and saving your settings with Excel, then code defensively.
Most users will not know how to make the %AppData% path visible as it is hidden by default on Windows.
If they are smart enough to find it, and dumb enough to not make backups when tweaking a file, then it serves them right if they have to re-install the plugin.
Your code having fallback routine that re-creates the settings file regardless of format, is a good idea. (Files get corrupted sometimes.)
JSON
I think I made the best arguments for JSON in that thread, and no need to repeat here. (Interested users can read that thread.)
And I also agree with sentiment here that reinventing a mechanism to save and restore data is kind of a waste (unless plain text is too large for say saving into the model’s attribute dictionary, and you want to compress the data.) The various text formats and their libraries should be used (ie, JSON, CSV, XML, YAML, etc.)
TEXT - Custom
But I won’t poopoo Garry’s use of a custom text format either. If it works … it works.
And he is not reliant on any library at all.
YAML
I also know several developers who prefer YAML files. And that is also fine with me.
They are a bit more terse than JSON, but still human readable.
PStore
The main problem (I think I was told,) with PStore is it uses Marshal.
Also someone else, a Mac guy, either Steve or Andreas explained that it has some pitfalls of it’s own.
(I’ll try and find the link to the quote and put it here.)
Regarding location I’d advise against ProgramData too. This folder is shared between user accounts and being able to have different preferences and settings is much what user accounts are about. If one user signs in they shouldn’t see the settings from another user, and if they change their settings it shouldn’t affect the other user.
Regarding the API methods I find them quite practical for simple information but at the same time I don’t really know what they are doing in the API. The API is (should) be about interacting with the model and the SketchUp application. Saving plugin preferences is in my view outside the scope of the API, and saving the settings elsewhere is in my opinion not a work around, but standard programming.
I totally agree. What one would consider ‘plugin preferences’ is one thing, but ‘plugin data’ could get very complex, and, and as you stated, the API is for SU objects.
Given that Trimble has established a (more or less) read only location for plugins, establishing a location for read/write ‘plugin settings and data’ would certainly make the cross platform issues go away. Maybe just a property that is the plugin_data folder. The plugin creator is responsible for file naming, so they could decide whether it’s user or app wide settings/data…
For general preferences/settings, I would recommend using Sketchup.read_default and Sketchup.write_default. This takes care of cross-platform issues. One catch is that they will be local to that SketchUp version.
If you need more control, then each OS have their own guidelines for where to store data - depending on what they are and whether it needs to be per machine or per user. For this is best to look up the Apple and MSDN information for the freshes information (this between OS versios)
I can’t find the thread now but I think Barry (or someone else) said he saved the character “Y” or “N” using Sketchup.write_default instead of just a boolean because booleans weren’t supported on Mac.
Are ther any such differences between the OSs? I would expect all basic types to work.
