Suggest PLAN A:
You don’t need to take the inverse if you use the following:
center = group.local_bounds.center
scaling = Geom::Transformation.scaling(center, xscale, yscale, zscale)
group.transformation = group.transformation * scaling
The “transformation=” assigns a rigid transformation. With the “scaling” variable last in the multiplication, the center position is in the group’s transformation’s local or intrinsic coordinates.
Note: If you really meant a Sketchup component instance, then the bounding box’s center is would be accessed through “entity.definition.bounds.center”.
Your inverse suggestion would work with an affine transformation:
group.transform! group.transformation * scaling * group.transformation.inverse
After the transformation update, Edit…Undo will undo the simple scaling, and Edit…Redo re-apply it. Since the “transformation=” and “transform!” append Sketchup’s “Edit…Undo” menu option, recommended practice is to wrap them with start and commit operations:
. . . code that affects undo operations
That would set the menu undo option to “Undo Barry’s Scaling”. Additional recommended practice is to include exception handling that calls “.abort_operation” if an error is raised.
The few statements of code do the Pythagoras and trig for you without any trig, as it just involves matrix multiplication.