This is normal with text and dimensions in SketchUp. One option would be to set the dimension to a height instead or point size.
Or you could eliminate dealing with it altogether by putting your dimensions and other text in LayOut where point size actually means something since that paper space not model space.