You can only use he #color= method to set the color of a layer with the limitation that the alpha channel can not be set. ( Sketchup::Layer
alpha channel access ! · Issue #715 · SketchUp/api-issue-tracker · GitHub )
Currently it is not possible to set material. It is requested already many times. E.g.:
Layer.material instead of Layer.color · Issue #7 · SketchUp/api-issue-tracker · GitHub