So, in my mind, the user is really only interested in using / seeing values as set in the Units panel. So I would present them with values in their selected (expected) precision:
model = Sketchup.active_model
units = model.options["UnitsOptions"]
If I understand what you are saying then I must proceed as follows.
I postfix values in inputbox with value.to_l so they are presented in the unit and precision format of the user. If I want to compare the return value with a comparison variable I must reformat the comparison variable to the users precision because inputbox will have truncated the return value. inputbox cannot return the original value because it converts it to a string to show in the dialog box and thereby losses the original precision. If you convert the return value to a float then you get a number with invalid numbers in the out of precision end of the number, which looks precise but is wrong.
The code snippet below works on my 64bit HP PC. I would point out that it is necessary to temporarily increase the precision to get an accurate comparison if the units are metres and the precision is 2 or less. I increased it to 4 then reset it when the comparison was done in this snippet.
original = 123456789.125456789
shown = original.to_l
options_manager = Sketchup.active_model.options
unitsOptions = options_manager["UnitsOptions"]
lengthPrecision = unitsOptions["LengthPrecision"]
comparison = original.to_l.round(lengthPrecision)
unitsOptions["LengthPrecision"] = 4
puts"original is #{original}. comparison is #{comparison}. shown is #{shown}"
input=UI.inputbox(["Float", "Text"], [shown, "Jim"], "Basic Data")
returned_value = input[0].to_f.round(lengthPrecision)
puts"returned_value is #{returned_value}"
unitsOptions["LengthPrecision"] = lengthPrecision
if comparison== returned_value
then puts("comparison and returned value are effectively equal")
else puts("comparison and returned value are unequal")
end
a. It is not nice to change the userâs precision setting without restoring it when done.
b. If I were doing this I think Iâd leverage the Length classâ ==() method which does comparisons using SketchUpâs internal tolerance. Because basically that is what really matters. (Youâre using Ruby Float classâ comparison method, which brings in the floating point madness into the picture.)
Thank you for your helpful links and advice. I found the articles by Thomas interesting.
a/ I reset the precision with the line quoted below from my example.
Or is there something not right about resetting it in this way?
b/ I tried that but the comparison doesnât work unless I convert it to a string and back into a length. Below is the code I tried first, which doesnât work.
original = 123456789.125456789
shown = original.to_l
comparison = original.to_l
puts"original is #{original}. shown is #{shown}. comparison is #{comparison}."
input=UI.inputbox(["Float", "Text"], [shown, "Jim"], "Basic Data")
returned_value = input[0].to_l
puts"returned_value is #{returned_value}"
if comparison == returned_value
then puts("comparison and returned value are effectively equal")
else puts("comparison and returned value are unequal")
end
Below is the code where the variable comparison is first converted to length, then to string and finally back to length. This comparison seems to work, even if it is a bit of a palaver!
original = 123456789.125456789
shown = original.to_l
comparison = original.to_l.to_s.to_l
puts"original is #{original}. shown is #{shown}. comparison is #{comparison}."
input=UI.inputbox(["Float", "Text"], [shown, "Jim"], "Basic Data")
returned_value = input[0].to_l
puts"returned_value is #{returned_value}"
if comparison == returned_value
then puts("comparison and returned value are effectively equal")
else puts("comparison and returned value are unequal")
end