Comments on: Let’s patch: Fun with Scientific Notation! http://blog.enigmaticthought.com/2009/08/sdk-16241/ import com.enigmaticThought.blog; Thu, 30 Jun 2011 09:13:09 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: Paulhttp://blog.enigmaticthought.com/2009/08/sdk-16241/comment-page-1/#comment-350 Paul Wed, 19 May 2010 21:30:30 +0000 http://enigmaticthought.com/?p=204#comment-350 Is there a bug with NumberValidator as well? For large numbers I think its generating an "invalidChar" result due to an"e" in the exponential notation representation of the number. Is there a bug with NumberValidator as well? For large numbers I think its generating an “invalidChar” result due to an”e” in the exponential notation representation of the number.

]]> By: Timhttp://blog.enigmaticthought.com/2009/08/sdk-16241/comment-page-1/#comment-348 Tim Thu, 06 May 2010 22:19:59 +0000 http://enigmaticthought.com/?p=204#comment-348 I found a similar bug recently with CurrencyFormatter. I have a grid that has a column of currency in it with a sum at the bottom. We recently ran into a case where the sum should have been $0, but was coming out at -$13. After some digging I figured out that floating point error was causing the sum to come out to 2.2737367544323206e-13. That number was then being converted to a string, which was then passed into CurrencyFormatter.format(). Because it's a string it ends up in NumberBase.parseNumberString() which obviously can't handle scientific notation because it returns a -13. Craziness. I found a similar bug recently with CurrencyFormatter. I have a grid that has a column of currency in it with a sum at the bottom. We recently ran into a case where the sum should have been $0, but was coming out at -$13. After some digging I figured out that floating point error was causing the sum to come out to 2.2737367544323206e-13. That number was then being converted to a string, which was then passed into CurrencyFormatter.format(). Because it’s a string it ends up in NumberBase.parseNumberString() which obviously can’t handle scientific notation because it returns a -13. Craziness.

]]> By: Matt Chttp://blog.enigmaticthought.com/2009/08/sdk-16241/comment-page-1/#comment-222 Matt C Sun, 23 Aug 2009 01:42:56 +0000 http://enigmaticthought.com/?p=204#comment-222 FYI, I updated the patch with a fix for both positive and negative exponents FYI, I updated the patch with a fix for both positive and negative exponents

]]> By: Jennhttp://blog.enigmaticthought.com/2009/08/sdk-16241/comment-page-1/#comment-198 Jenn Tue, 18 Aug 2009 19:08:14 +0000 http://enigmaticthought.com/?p=204#comment-198 Yes, the correct expanded notation is .0009870. The 98.70 that i referred to was the end result i wanted to be displayed to the user. Yes, the correct expanded notation is .0009870. The 98.70 that i referred to was the end result i wanted to be displayed to the user.

]]> By: Matt Chttp://blog.enigmaticthought.com/2009/08/sdk-16241/comment-page-1/#comment-197 Matt C Tue, 18 Aug 2009 18:33:54 +0000 http://enigmaticthought.com/?p=204#comment-197 You're right, I missed the decimal cases because I was only looking at large numbers. I'll have to modify the patch to work with lower numbers. Shouldn't the result be .0009870 though? You’re right, I missed the decimal cases because I was only looking at large numbers. I’ll have to modify the patch to work with lower numbers.

Shouldn’t the result be .0009870 though?

]]>