Drawing Vs. Display Precision

@DaveR helped me understand the difference back in Jan '20 with this explanation:

FWIW, this isn’t “drawing precision”, it’s display precision. You can model just as precisely with the units set to something coarse but you won’t have the resolution when you are measuring with the Tape Measure tool or adding Dimensions.

Currently when using the tape measure and other tools, I sometimes get a measurement display like ~20.5, other times, 20.5. I am still puzzled by the “~” symbol which seems to occur randomly. Is this what @DaveR was referring to in his comment or caused by some other condition?

And what is inherent drawing precision of SU?

Thanks for your assistance …

the ~ appears when drawing precision exceeds the one you’ve set for your model. for example, if you’ve set in model info the precision to 0,00m and draw a 1,001m long line, taping it will indicate ~1,00.
it will just round it up and warn you about it.

1 Like

The tilde ~ is telling you that the measurement displayed is approximate, not an exact amount to the precision you have set.

For example, if you draw a line exactly (say) 8 5/16” long, but your display precision is set to 1/8” or coarser, then Entity Info, or the tape measure tool, or a dimension, will display either ~ 8 1/4” or ~ 8 3/8” depending on which way it rounds. (Replying from phone, so can’t see which SU will do.)

If you change precision to 1/16”,1/32”, or 1/64”, then you will see 8 5/16” with no tilde in front of it.

Same principle in decimal units. Set length at 8.25 units (inches, mm, cm or m) with precision 2 digits, no tilde. Precision one decimal, display ~ 8.3 or ~ 8.2 units.

NOT random.

1 Like

Thanks Paul/John, for the detailed explanation. Back in Jan '20, I was low on the SU learning curve. I am slowly making progress thanks to you kind folks. This shows the importance of really understanding the basic principles.

My bad, slowly learning my lesson about using terms like, “Random, arbitrary, NO, wrong, etc.”. Will start using less emotive terms like, “unknown, mysterious, etc.”. :wink:

Back to the post. What is the internal drawing accuracy of SU?

And what about “length snapping”? Even off, I still see snapping behavior. Think I remember some folks suggesting it should be turned off.

[Edit in light of @TDahl’s comment below]

Better than a millionth of an inch. [/End edit]

But points closer than about 1/1000 of an inch (0.001 inch) will get merged as SU is working, and treated as ‘the same point’.

It’s highly recommended to turn it off, and to save that setting in your commonly used templates.

The snapping you see when it is off is SU ‘inferencing’ to objects already in the drawing.

There’s a theoretical use for Length snapping, if you are just starting a new drawing, to create lines, rectangles, circles, or arcs that ‘snap’ to a multiple of your chosen snap precision. But that’s a rare use case for most people once past the very new user and new drawing.

Since it can override inference snaps, it may mean that your intended exact match to an existing endpoint, midpoint, circle or arc centre etc. that is NOT a whole number of snap lengths will ‘miss’ and introduce an unintended error. Once those sorts of errors start, they tend to compound and become harder and harder to fix without starting over.

PS. There was quite a long discussion of length snapping back in November.


Guidance duly noted and appreciated. Will modify my template file and keep it turned off.

Extremely small, a millionth of an inch or smaller. Set the display precision to decimal 0.000000 and draw a line that is 0.654321 inches long (by typing in the dimension while using the Line tool). Now measure the length of the edge. It should show as exactly 0.654321 inches long.

@john_mcclenahan the 0.001 inch value is not the granularity of end-point locations in the model. It is the approximate tolerance between nearby end-points that SketchUp applies when “validating” a model.


I’ve never quite fully understood what floating point numbers mean to the model for sure. The first BIM program I used in the 1980’s was Architrion, and it was integer based: you picked what one unit represented (say 1/8") and you couldn’t do anything smaller than that. Also, if the numbers were, say, 16 bit integers, or whatever, that limited the maximum size of the universe.

As I understand it, with floating point numbers, you can model a room or the solar system, but if you move something 1/16" in the room, it’s not a problem, while if you move the earth 1/16" the data is lost being too many digits out? Again, I’m not a programmer, so I’m all ears, but I can see how accuracy could be more complicated than a simple unit of measure, and maybe why SU gets flaky when there are objects very far from the origin?

1 Like

I have no proof and I’m not an OpenGL programmer, but I’ve gradually become convinced that a lot of SketchUp’s limitations both at the small and large ends of the range are due to the passing of values back and forth with the display via OpenGL.

1 Like

That is broadly correct. Floating point values that are close to zero can represent extremely fine granularity. Values that are far away from zero (large positive or negative) cannot represent fine granularity. The floating point format has a continuously-varying trade-off between high precision at the small end vs. huge absolute values at the large end. For example, a value near zero can distinguish between values trillionths of a unit apart, whereas very large values (far from zero) can only distinguish between values that are millions (not millionths) of units apart.

If a unit (numeric value 1.0) is an inch, then a detailed numeric representation of a machine screw could accurately hold points of the #2-56 threads with no problem. Even the locations of the inner parts of a bacterium could be represented no problem, so long as the bacteria was somewhat near the coordinate system origin. Let’s say the coordinate system origin is on Earth. If the bacterium or screw were to be translated (moved) to the Alpha Centauri star system (which is about 10^18 inches or 1000000000000000000 inches away), the details of the object could no longer be numerically distinguished but its overall position could be represented.

Here is a reply I posted a while ago about the behavior of floating-point numbers as used in computers.