08.11
Flex SDK bug SDK-16241 was interesting for two reasons. It was claimed then abandoned (it shouldn’t take 4 months to patch) and the “Expected Result” was actually incorrect. Lots of bugs are claimed the abandoned. I admit I did it at the BugQuash when I realized I was over my head and should move on rather than flail around for the rest of the day. The expected result threw me though.
The problem lies with Scientific Notation and it’s formatting when converted to string using NumberFormatter, which doesn’t take into account the altered decimal position. Since we’re dealing with strings in the function itself, flex doesn’t have to worry about the size of the number, so I simply built the number up to full size and send it down the line. The same problem was in the NumberBase class when adding the thousands separator via formatThousands since it converts the Object->Number->String which returns my fancy long string to scientific notation. Since I’d already solved the problem, I just copied some code over. Both places needed to be fixed because only the numbers to the right of the decimal are sent to the formatThousands function.
I also learned that when talking precision, flex means number of decimal places, not significant digits. A major distinction mentioned in the NumberBase reference.
So what about the “expected result”? Well the value being formatted is -2.34444444444434e+21 which expands out to -2,344,444,444,444,340,000,000. The expected result is listed as -2,344,444,440,000,000,000,000 for some reason. I assume it’s someone’s calculator rounding off to a numerical precision of 9 since the actual value of 235345-242+99209323-2344444444444444444444 is -2344444444444345000018 which also tells me that flex’s default numerical precision is 15.
My patch is SDK-22643 if you’re interested.
This was a great post to come upon as i’m currently trying to format some scientific notation numbers to round decimal numbers. I knew parts were missing. However, even though I have applied the patch, a problem exists in the line before where \\expand scientific notation code goes in NumberFormatter.as – “var numStr:String = value.toString();”…This actually strips the E-4 off the end of my value. There for I’m left with a large number 9.870177632929455E-4 with out the power at the end. As a result the code does not expand the notation. How can I get the value to remain with the notation when converted to a string. Any suggestions.
Jenn
I believe you should be able to just go with the toString() function of the number, you shouldn’t need the NumberFormatter
Hi Matt,
. The big picture is lets say I have a number 9.870177632929455E-4, eventually after formatting it should be shown as 98.70. So I need to round up, precision 2 and expand notation. I was trying to include the expand notation in the formatter.as. A couple of things I have noticed so far are: The method that returns this number to me from the sends, with an “E”, not an “e”. Also my notation is negative, not positive (for this instance). So i may have to come up with a re-worked version of this notation expansion.
Well I guess this isn’t working well for me at all
You’re right, I missed the decimal cases because I was only looking at large numbers. I’ll have to modify the patch to work with lower numbers.
Shouldn’t the result be .0009870 though?
Yes, the correct expanded notation is .0009870. The 98.70 that i referred to was the end result i wanted to be displayed to the user.
FYI, I updated the patch with a fix for both positive and negative exponents
I found a similar bug recently with CurrencyFormatter. I have a grid that has a column of currency in it with a sum at the bottom. We recently ran into a case where the sum should have been $0, but was coming out at -$13. After some digging I figured out that floating point error was causing the sum to come out to 2.2737367544323206e-13. That number was then being converted to a string, which was then passed into CurrencyFormatter.format(). Because it’s a string it ends up in NumberBase.parseNumberString() which obviously can’t handle scientific notation because it returns a -13. Craziness.
Is there a bug with NumberValidator as well? For large numbers I think its generating an “invalidChar” result due to an”e” in the exponential notation representation of the number.