Handling gem inside extension (yajl-ruby)

Dear SketchUp gurus,

I am working on the update of an old project, originally compatible with SU up to 2016.
At the moment I am having problems with yajl-ruby. I have looked at the forum extensively but before integrating the entire yajl-ruby gem or yajl into my extension, I wanted to know if I tried all possible ways to have it working as a gem.

I am using Sketchup2019 Pro on Windows 10.

In order of events this is what happened:

  • yajl is not found (note that even if we talk about yajl-ruby, the require invokes yajl, but this is normal)

  • Gem.install “yajl-ruby”

  • installation fails and yajl still not found
    (note that there is also a “yajl” gem in rubygems but it doesn’t do what the original yajl does. However it installs without problems and it loads. Problems arrive when one wants to use the yajl original functions)

  • attempt to install into my system Ruby 2.6.6 x64. This would not work at the beginning but would then work with gem install yajl-ruby --platform=ruby

  • install into Ruby 2.5.8 (both x86 and x64, managed with uru ) and copy to Gem64 folder. Then load into path following Dan’s advice

  • when doing this, after requiring the library from Console, I get for blablabla/yajl/yajl.so
    Error: #<LoadError: 193: %1 is not a valid Win32 application.

  • attempted to use rake-compiler and tweak stuff in gemspecs and rakefile but I might have problems for x64 (not sure I even need 64 bit gems, but I understood from the forum that I do). Followed similar procedure to load but no good results.

  • attempted to use rake-compiler-dock and follow similar load procedure. Again problems with the x64 version when trying to set RCD_PLATFORM before calling the dock image as in github Readme. The way to pass this variable is not recognized. If I set it afterwards to RCD_PLATFORM=x64-mingw32 once in the docked environment it seems not to complain but it still builds for i386.

  • found very nice solution to build binaries with gem-compiler by luislavena. After fetching and compiling with gem-compiler I install the gems from local into my rubies and then I copy them from my two ruby versions to Gem64 folder. Attempted loading both of them individually from console, but same error when I require yajl.

I am aware that the advice is to wrap the gems into the extension’s namespace (or rather put the C code in another Ruby C extension) but before doing this I still have some questions.

  • Is there any reason why the custom gem folder for SU2019 is called Gem64? Is there any place for gems with x86-mingw32 binaries inside? Could it be some trouble with the C code not compatible with x64?

  • Can it be that I have to install the gems with binaries also in SU? Not just in my ruby > copy to Gem64 > add to LOAD_PATH temporarily > require? (this is just for testing whether it could work, not final sustainable solution)

  • If none of these can work (for reasons I hope can be clearer), is it better to wrap yajl-ruby as it is with its yajl.so into the extension, or create another C Ruby extension which points to the original yajl ANSI C library (or is it equivalent)? I have managed to successfully require another library in need of Ruby update from 2.0.0 in this way based on the updated template for Ruby 2.5. The case for yajl-ruby might be more complex though.

  • Is there any reason to believe that using gem-compiler from Ruby 2.5.8 would create binaries with problems for Ruby 2.5.5 used by SU2019?

If anyone would like to understand what’s going on with yajl-ruby and SU2019 (and maybe yajl problems), that would be greatly appreciated! If so, I would suggest the gem-compiler since it has the nicest workflow so far and it shouldn’t take long to test.

Regarding the original yajl, I have built it again from source in Visual Studio but I got a yajl.dll and not a .so file. Which one would it better to use if our plugin users are necessarily on windows? Is it possible to use dlls if one goes with the Ruby C SketchUp approach?

Please correct me if I said something wrong and sorry if there might have been other threads with possible solutions. I really looked through many of them! I am here to learn :slightly_smiling_face:

Thank you!

P.S. I am a new user so I am not allowed more than 2 hyperlinks… also the post was hidden when I tried to include the links in the reply! :open_mouth:

This post was flagged by the community and is temporarily hidden.

I don’t know the answer to your question, but I am able to include two links!

https://github.com/lloyd/yajl

https://rubyonwindowsguides.github.io/book/ch02-03.html

1 Like

Firstly, in order to compile binaries for use within SketchUp Ruby, you must use the Ruby version and bitness that you are targeting. You cannot just use any old Ruby system install. This means that you must have a pure Ruby load script that conditionally loads the correct so file.

Early SketchUp 2019 versions used Ruby 2.5.1, … version 2019.2 and later updated to Ruby 2.5.5.
SketchUp 2017 and 2018 use Ruby 2.2.4.
SketchUp 2014 through 2016 use Ruby 2.0.0.

All currently supported versions of SketchUp are 64bit. Extensions devs are increasing dropping support for older versions.


Because of the unnecessary complexity of incorporating gems (especially binary gems) with SketchUp embeddd Ruby, … my advice for SketchUp embedded Ruby is to use the included JSON standard library.

This library is usually preloaded by the SketchUp startup cycle by the time any custom extensions begin loading. But to to be sure I’d add a require "json" statement at the top of your extension module.


Now that said, why is it you feel you need “Yet Another JSON Library” instead of the one supplied with Ruby’s standard library set ?

It is actually named "Gems64" and is for 64bit gems in a 64bit SketchUp install.

ONLY in a 32bit SketchUp install. The name of the folder will be just "Gems".

On each SketchUp install, SketchUp’s copy of the environment will have a path pointing to the gem directory …

ENV['GEM_HOME']

I also notice that the folders are missing from 2019 and 2020 %AppData% paths. Perhaps they are only created if gems are installed ?

Tested …

Attempts to install pure Ruby gems that worked in SketchUp version 2018 are failing in v2020. I get a “unable to resolve dependency error”. It doesn’t matter if I let it default to latest gem version or specify an exact gem version. Same error.

So perhaps gem installs are broken on 2019 and 2020 ?

Dear Dan,

thank you for your reply. I understand that there are different versions of Ruby across the different SketchUps. In fact there was a conditional load that took into account the different Ruby versions (up to 2.0.0) for the old extension that I am trying to upgrade. I wasn’t sure that also the bitness would have been so restrictive in my current case.
The reasons why I am trying to use the yajl-ruby are as follows:

  • the library is used a lot in the project and the methods which are employed have different names from those available in JSON.
  • according to the authors of yajl, it is supposed to be very fast in doing its stuff. Having a different library doing the same things at a slower pace could compromise the functionalities of the extension.

Thank you also for specifying and testing this.
I have successfully installed other old gems from the Ruby Console in the Sketchup 2019 Pro 64 bit I have, through the simple
Gem.install “name-of-the-gem”
command, for example the gem named “color” . Maybe old gems are installed with less trouble than new ones?

since this happened to be in bold I feel obliged to point out that the correct form is ITS…
https://grammarist.com/spelling/its-its/

I use an editor with a powerful search and replace function that makes changes less painful.

Another option is to “fudge it” by creating a fake Yajl that whose methods just call the standard JSON lib’s methods. (Usually called a “wrapper”.)

Are you sure?

Have you actually tested the speed (or slowness) of the standard library as opposed to Yajl ??

It seems to me that you’ve worked yourself into a tight crevasse based upon the subjective opinion of someone promoting their own project. This isn’t really beneficial to your project’s needs.

Especially because the reading of that author’s project introduction implies a falsehood. And that is that the average reader will assume without research that the standard JSON library is not compiled C.

The reality is that it is both.
It has some higher level pure Ruby, but the parser and generator are both compiled C.
(For SketchUp installs, they’re located in "Tools/RubyStdLib/platform_specific/json/ext".)

So, I’m sorry, I just do not believe that a “compromise of functionalities” would happen.


True … and I’ve changed the original to “SketchUp’s” which what I was being lazy and typed “it’s” for.

No, I haven’t, and I didn’t choose to use yajl in the first place… I was just trying to load the original libraries before changing anything or wrapping.
I will consider using the standard JSON in any case, since it will make things surely easier for future maintenance. Thanks for the precious advice. :ok_hand:

Re alleged speed: You are right…from these very old benchmarks for linux and mac the difference is not really remarkable although there seem to be some cases in which the encoding works better for yajl. I am on windows, though… Also, MessagePack seemed to be better anyway!

Old benchmark for linux

Old benchmark for mac

I would surely need to repeat the benchmarks for the setup I am testing to justify the need for yajl-ruby…but considering that it doesn’t work so far, it might be just fine to use JSON…

Anyway, out of curiosity and stubbornness, I would still be interested in knowing if it is actually possible to load this, I haven’t tried so far to use ruby 2.5.5 for the gem-compiler, only 2.5.8, but I don’t think this is the reason for the problem… could it be another form of incompatibility?

Choosing between a bit over 1 tenth of a 10 thousandth of a second and a bit less than a quarter of a 10 thousandth of a second in an interpreted code environment is just a waste of time. (pun intended)

Coders have much more important challenges to worry over and solve.

The recommended way to use Gems in SketchUp extensions is simply to include the Gem in your own codebase wrapped in your own namespace.

https://rubocop-sketchup.readthedocs.io/en/latest/cops_requirements/#sketchuprequirementsgeminstall

I would strongly suggest that, unless using JSON just won’t work. Using gems in SU presents several problems, and extension gems (gems that require compiling) are even messier.

JFYI, I did install yajl-ruby, and it worked in SU, but my SU isn’t standard. I’m also familiar with building Ruby, extension gems, etc. To build extension gems on Windows, you need stand-alone Ruby and correctly installed MSYS (Ruby 2.3 and earlier) or MSYS2 (Ruby 2.4 and later) tools.

Dear MSP_Greg,

thank you for looking into this! I have started changing the code to use the standard JSON. I might try to check the outcome of the method you suggest to clarify any differences between the two, and get back to you in case I need further guidance. Thank you all for your input!