Speeding Up Extension Loading

Developers Ruby API
,
1

When users have many extensions that need to be loaded, developers of SketchUp extensions can help the startup cycle by using conditional require statements.

Since (at least) the SketchUp 2016 release, the "Tools" folder files have been loaded first before any of the files in the "Plugins" folder. This means that "extensions.rb", "langhandler.rb" and "sketchup.rb" are likely already loaded (in this order) by the time any extension begins loading.

If your extension files use plain require statements to ensure that these files are loaded, then SketchUp’s Ruby has to do slow string comparison of the script name argument against every path member in the $LOADED_FEATURES array, each and everytime a non-conditional require statement is executed.

Your code files can instaed use conditional modifiers to only execute the require statements when necessary.

Example of a set of plain require statements used by many extension authors:

require 'sketchup.rb' # internally calls "require 'langhandler.rb'"
require 'extensions.rb'
require 'langhandler.rb' # frivolous

Example of conditional require statements:

require 'sketchup.rb' unless Object.private_method_defined?(:file_loaded)
require 'extensions.rb' unless defined?(SketchupExtension)
require 'langhandler.rb' unless defined?(LanguageHandler)

If you prefer if not rather than unless that will also work fine.

The point is that testing for class and method object existence is way faster than comparing a substring against a growing list of absolute path strings (in the $LOADED_FEATURES array.)

The same sort of thing can be used with an extension that has multiple files to load at startup, if the files are not encrypted or scrambled.

If an extension has an internal indicator of whether it is loading for the first time (ie. an @@loaded variable?,) then Kernel#load could be used instead.

if not @@loaded
  load 'main.rb'
  load 'commands.rb'
  load 'menus.rb'
end

Now if we could also get Sketchup::require changed so that it will skip the $LOADED_FEATURES comparison during the startup cycle, and just load files (similar to a Kernel#load call,) then startups could get considerably faster.

If ALL extensions used conditional require statements then the startup cycle would be faster for those users who load many plugins.

2

That could potentially cause problems if files are loaded without any load guard. global data being re-set and cyclic dependencies not broken.

Yea, I’ve been meaning to dig further into the loading performance. Especially Sketchup.require (Sketchup.require much slower than require · Issue #309 · SketchUp/api-issue-tracker · GitHub)

For un-scrambled extensions there is the option of using autoload.

I’ve also been tempted at doing adding require in-line with code logic. For instance, each menu/toolbar item is an entry point to when the extension is being used. An extension could be crafted to use require or Sketchup.require at the beginning of these command items instead of at a global file level. Lazy loading only what you need.

3

I see no need to ‘require’ any of the ‘shipped’ files, conditionally or not…

that also goes for big ‘library’ items e.g. require 'fileUtils' which SU loads already…

Lazy-loading from menu items is the best solution and should be compulsory…

john

4

Hmm? SketchUp itself doesn’t load fileutils.

I think SketchUp should have required the rb files it ships in the Tools directory before any extensions are loaded. I see little need for extensions to require any of these files directly.

5

want to bet…

rename your Plugins folder, restart and call fileutils

 require 'fileUtils'

you get a lot of warnings because ‘Trimble Connect’ has already loaded it…

warning: already initialized constant FileUtils::VERSION

worst still, require is actually doing a load and re-defining the entire library…

NB: ‘SketchUp itself’ does include all ‘shipped’ files, so any misbehaviour is an SU issue…

john

6

That would mean the Trimble Connect extension, not SketchUp, loads FileUtils.

I disable Trimble Connect whenever I start SketchUp on a new computer along with Dynamic Components and Advanced Camera Tools, not necessarily because it’s bad but because it’s one of SketchUp’s shipped extension. If I install an extension depending on FileUtils but without explicitly requiring it, the extension would fail for me.

7

SU should deal with these issues not the end users nor 3rd party extension developers…

all shipped extensions should be rigorously vetted by the @ChrisFullmer’s ‘team’…

or @SketchupDoug can tell us who is if otherwise…

TC, DC and ACT all misbehave and cause load and runtime errors…

john

8

I only require them if my code files will be using them.

@Aerilius previously argued that he uses the “require 'sketchup.rb'” to get his code editor to load API stubs for autocomplete. (I think I may have retorted that this was for development not runtime, or the like.)

Many coders put them in their code because of the examples that say (or said,) “bring in the API hooks” or some similar erroneous comment.

9

SketchUp doesn’t explicitly reference the shipped extensions. They are extensions like any other and should obey the same rules. Just that a shipped extension happens to require some part of the standard lib doesn’t mean there is an API contract that this content is loaded when your own extension runs.

10

I would argue that extensions.rb, langhandler.rb, sketchup.rb are as they load even with Ruby disabled…

# try this
Sketchup.plugins_disabled = true
# restart, open RC check what Ruby has been explicably loaded...
$"

TC DC ACT aren’t there, but no one else should need to the require those, and they should be LazyLoaded so we don’t have to turn them on/off all the time…

john

11

I believe I also kept it from the examples because I think it can also serve as a “symbolic requirement” because the SketchUp is not part of the Ruby core and to prevent that anyone is mislead to run such scripts in a stand-alone Ruby interpreter. Technically it is redundant since the SketchUp API is not contained in sketchup.rb but injected by the SketchUp runtime.

12

…and because default extensions can be disabled, we should not rely on their features (or loaded dependencies). That caused previously caused enough issues when extension developers relied on API features that were actually added by dynamic components and it was not noticed until a user disabled default extensions.

Shipped extensions may at any time drop or change their dependencies.

13

These files define documented API features. I agree that there really should not be a necessity to explicitly require their loading (anymore) as they are a documented part of the API.

(Julia was meaning the shipped extensions TC DC ACT, not the 'sketchup.rb' file.)

True, but also how often will that happen? The first time the code calls one of the Sketchup module functions there’ll be a NameError exception raised.

Or a coder could stick a test like …

fail(RuntimeError,"Not a SketchUp Ruby Process") unless $PROGRAM_NAME == 'SketchUp'

… at the top of the first file instead of the require statement (that is going to do the slow string compare iteration all through the $LOADED_FEATURES.)

14

As an end user a lot of this is over my head. However I think it touches on a question I’ve been asking myself. I have some powerful extensions and plugins. I only use them occasionally. I have considered disabling them when I don’t need them. For example once I have my roof modeled I may never use “instant roof” again. Would I realize better performance with less extensions loaded? I’m not that concerned with startup time.

15

I probably have more extensions loading than I really need but I have more that only load when I want them. I’ve been managing those with the SCF Tools and it seems to work well. I really ought to add more of the extensions I have to the “don’t load on start” list.

16

Thanks Dave,
I have that tool so I’ll start using it.

17

Most likely not. Unless the extensions have a lot of observers constantly watching what you do, they will not impact performance. But they can have a dramatic effect on SketchUp’s launch time. It takes maybe 20-30 seconds for SketchUp to start on my Mac due to having too many extensions, but since I tend to launch it and leave it running for extended periods that isn’t really much of an issue.

18

Which is easier to get away with on Mac since the same application can have multiple models open. But on Windows you need to start a new SU process per model.

19

Just want to clarify one thing. AFAIK, the algorithm for requiring a file first checks whether the file can be found in $LOAD_PATH, then it checks for whether that file path has been loaded in $LOADED_FEATURES. This is where older Ruby versions can get confused when symlinks are involved.

So, as with many coding issues, the larger portion of time is the directory searching using $LOAD_PATH (disk IO), not the string comparisons against $LOADED_FEATURES…

20

It actually first checks whether the given path resolves to a valid absolute pathname, and if not then begins it’s $LOAD_PATH array iterations.

But you’ve just shown more reason why unnecessary require calls should be avoided.