Math precision

Got a LiveCode personal license? Are you a beginner, hobbyist or educator that's new to LiveCode? This forum is the place to go for help getting started. Welcome!

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller

Post Reply
MaxV
Posts: 1580
Joined: Tue May 28, 2013 2:20 pm
Contact:

Math precision

Post by MaxV » Mon Jun 16, 2014 12:48 pm

Hi,
I'm using WinXp and I get this error:

Code: Select all

1 * 0.000005  = 0.000005
1 * 0.0000005 = 0
:shock:

How can I set math precision on Livecode?
Livecode Wiki: http://livecode.wikia.com
My blog: https://livecode-blogger.blogspot.com
To post code use this: http://tinyurl.com/ogp6d5w

Klaus
Posts: 14205
Joined: Sat Apr 08, 2006 8:41 am
Contact:

Re: Math precision

Post by Klaus » Mon Jun 16, 2014 12:56 pm

Hi Max,

check "numberformat" in the dictionary, the default = 0.######
Which is one digit less than you need in your example :D

Best

Klaus

Post Reply