It is not about plants, it is about the edges and faces within them, whether visible or hidden. Usually a single 3D tree has more in it as the whole other model put together. Check the heaviness of all 3D Warehouse components before you put them in your model. Best is to use 2D face-me type plant components, or simple abstract 3D shapes with a suitable texture, or, if you use a rendering application that supports and supplies them, objects that have a separate detailed model visible only in renders and a simple “proxy” visible in SketchUp. And forget about vector or hybrid rendering in LayOut, use only raster mode.
You should create 2D symbols in their own layer inside the 3D plant component that are on a separate SU scene. Then in LO you are only waiting on the symbol to render which will be very fast if not instant.