Calculator: removing scientific notation in results?

This seems like it must be a bug or a corrupted app, but I'd just like to confirm: with the calculator in "basic" mode, dividing two numbers which only have one significant digit returns a result like so: 1/3000 = 0.000333 (the setting for decimal places seems to be irrelevant).
However, changing either number to have more than one significant digit returns the result in scientific notation, like so: 1/3100 = 3.225806e-04.
Can anyone else duplicate this or help me correct it? It's obnoxious to have to keep adjusting the numbers. And yes, I realize that means my math skills suck. Thanks in advance.

Bumpty-bump. Can anyone at least confirm or deny? TIA.

Similar Messages

  • Calculator and Scientific Notation

    My Calculator App seems to randomly switch back to scientific notation instead of displaying results in regular x.xx format.
    I tried deleting the com.apple.calculator.plist file and then relaunching Calculator but that didn't work.
    Any other suggestions on how to get the calculator app back to displaying results in normal (non-scientific notation) mode?
    thanks

    Then, I'm at a loss. Mine only shows scientific notation when the value exceeds whatever's set as the number's max range. You might want to peruse Calculator's help files.

  • Remove scientific notation in report

    I'm calculating a number and placing it in a comment box on a report. This works fine, but one of the numbers has decided to display in scientific notation (1.52666667e-3). The problem is that the comment box is not large enough to display the scientific notation and gives no indication of not fitting into the box. Instead I see 1.52666 with no indication of the exponent. I'd much rather see .001527 however I don't want to force a STR format with "d.dddddd" because most numbers don't need this much precision and I'd probably run into errors for numbers such as 12345 which won't fit into the comment box with forced 6 digit precision. If I can force it out of scientific precision, it'll probably be fine.
    BTW: is there a certain number range where scientific notation becomes the default? If I knew this range I could then force the format with the STR function to display the way I want. e.g. IF x>-001 and x<.001 then STR(X,"d.dddd")
    Either method would probably work for me.
    thanks,
    James
    Solved!
    Go to Solution.

    MyVar is a 2 dimensional pointer (Channel Group, Channel) to numeric channels that meet some criteria.
    change box size: I had to make the box 3 times the width to display the number correctly. It shows 1.27933333333333E-03*. I'm appending the "*" to show the field is calculated instead of unadultered from the .csv. Maybe the problem isn't that the number is small, but that it's infinitely repeating. The number is calculate from the values .001919 - abs(.001919)/3
    Left Justify: no difference from right justify.
    Autoadj: shows 1.28E-
    I'm making a column of channels on the report, that's why the comment text is being set multiple times.
    Humphreyy - thanks for your ideas and help.

  • Result in scientific notation

    Hi,
    We migrated an instance from 7.3 to 7.6 x64 (no unicode) and I get an expression result in scientific notation.
    How can I make the notation use decimal? I cannot find anything in the documentation.
    sqlcli
    EXPRESSION1
    1.3192E7
    1 row selected (449 msec)
    Cheers,
    Jan

    Hi,
    the function FIXED (<your expression>, <number of digits) may help
    Elke

  • Calculator.app Version 4.5.3 (99.2) and Scientific Notation

    I want to enter a number with scientific notation.
    2.71e1 doesn't return 27.1, but rather 0.999...
    It seems that Calculator.app is dividing by e.
    This is not a common operation.
    2.71E1 gives 27.1great! But ...
    2.71E-1 does not return 0.271 neither does
    2.71E(-1)
    I have yet to figure out how to use negative exponents in Scientific notation.
    This seems to be a bug in Calculator.app Version 4.5.3 (99.2)

    All my spaces were deleted.
    Take two.
    My input ======= Calculator
    2 ============ 2
    2. ============ 2.
    2.7 =========== 2.7
    2.7E ========== 2.7 E 0
    2.7E- ========== 2.7 (and the small - subscript indicating a negative number)
    2.7E-1 ========= 1 (and the small - subscript indicating a negative number)
    Now hitting enter === 1.7 (it appears to have subtracted 1 from 2.7)
    Do our version of Calculator match?
    Thanks for being interested.

  • Select Query resulting in Scientific Notation

    Hello all,
    I am running a Select query through a batch file that extracts data from an Oracle database. Several of the fields that I am extracting from contain numbers that are up to 38 digits long. When I extract the data, it converts the numbers into scientific notation and it is important for me to have the entire field. Is there something I can change to my query that will pull the data in its entire form? This is what I'm running now:
    select * FROM ML.APPT where APPTDATE >= to_date('01/1/2010','mm/dd/yyyy'
    I apologize in advance if this has been answered already.
    Thanks!

    >
    When the extractor finishes, it returns the data into a flat file.
    don't quite understand the TO_CHAR function. Does this function mean I need to say something like this: select "TO_CHAR('column name', 99999999999999999999999999999999999999" FROM ML.APPT where APPTDATE >= to_date('01/1/2010','mm/dd/yyyy')
    >
    Yes- if the tool you use to extract the data (your 'extractor') is converting the numeric data to a string then it is responsible for creating the string in the proper format. If the number is an integer that can have as many digits as you have '9's in your sample format string then that is what you need to do.
    Here is how sql*plus (Oracle's tool) will display the data using default settings
    SQL> select 12345678901234567890123456789012345678 no_format,
      2  to_char(12345678901234567890123456789012345678, '99999999999999999999999999
    999999999999') with_format
      3   from dual;
    NO_FORMAT WITH_FORMAT
    1.2346E+37  12345678901234567890123456789012345678
    SQL>
    ----- TOAD will display something similiar but the default uses more decimal digits in the scientific notation data
    NO_FORMAT,WITH_FORMAT
    1.23456789012346E37, 12345678901234567890123456789012345678You can either format the numeric data in the query using TO_CHAR or the 'extractor' can do it when it converts the data to a string.

  • WKT Contains Scientific Notation

    I have a table with an SDO geometry column. Our data is stored in Web Mercator to simplify displaying maps on a web page. My team's preferred way of shuffling geometries around is via its WKT since this is human readable and widely used. So we are fetching the WKT directly from the database using the GET_WKT() method (right term?) on the SDO geoemtry.
    The problem is that when coordinates exceed 10 million in magnitude, those coordinates are represented in E notation (see http://en.wikipedia.org/wiki/Scientific_notation#E_notation). I need to convert this WKT to a .NET object for use with a particular library, and while it supports conversion from WKT, it blows up on the E notation. I'd call it a bug with the library except for the fact that Oracle itself can't parse WKTs with E notation, either. SDO_UTIL.VALIDATE_WKTGEOMETRY returns FALSE for the WKT that GET_WKT() generated, and SDO_UTIL.FROM_WKTGEOMETRY throws an error. I've also tested that SDO_UTIL.SDO_UTIL.TO_WKTGEOMETRY returns the same WKT.
    A large amount of code already depends on the geometry being in WKT format, which means that switching to another format would not be an easy change. For the moment, I'm parsing the WKT using SQL Server's geometry type (see http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.types.sqlgeometry_methods.aspx), and then converting it back to a WKT without E notation using its STAsText() method.
    Is there a way to force Oracle to not return E notation coordinates?
    This is occurring in both of the following versions of Oracle:
    Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Prod
    PL/SQL Release 10.2.0.1.0 - Production
    "CORE     10.2.0.1.0     Production"
    TNS for 32-bit Windows: Version 10.2.0.1.0 - Production
    NLSRTL Version 10.2.0.1.0 - Production
    Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - Production
    PL/SQL Release 11.1.0.6.0 - Production
    "CORE     11.1.0.6.0     Production"
    TNS for 32-bit Windows: Version 11.1.0.6.0 - Production
    NLSRTL Version 11.1.0.6.0 - Production
    Sample SQL:
    The SRID in the following queries does not seem to exist out of the box in version 10 or Oracle. I ran these queries through Oracle SQL Developer.
    Query:
    SELECT MDSYS.SDO_GEOMETRY(2003,3785,NULL,MDSYS.SDO_ELEM_INFO_ARRAY(1,1003,1),MDSYS.SDO_ORDINATE_ARRAY(-13426771.266146,5334024.8870015,-13425624.710722,5326534.0582305,-13412553.978887,5325922.5620044,-13412936.164029,5333719.1388884,-13426771.266146,5334024.8870015)).GET_WKT() FROM DUAL;
    Result:
    POLYGON ((-1.3426771266146E7 5334024.8870015, -1.3425624710722E7 5326534.0582305, -1.3412553978887E7 5325922.5620044, -1.3412936164029E7 5333719.1388884, -1.3426771266146E7 5334024.8870015))
    Query:
    SELECT SDO_UTIL.VALIDATE_WKTGEOMETRY(MDSYS.SDO_GEOMETRY(2003,3857,NULL,MDSYS.SDO_ELEM_INFO_ARRAY(1,1003,1),MDSYS.SDO_ORDINATE_ARRAY(-13426771.266146,5334024.8870015,-13425624.710722,5326534.0582305,-13412553.978887,5325922.5620044,-13412936.164029,5333719.1388884,-13426771.266146,5334024.8870015)).GET_WKT()) FROM DUAL;
    Result:
    FALSE
    Query:
    SELECT SDO_UTIL.FROM_WKTGEOMETRY(MDSYS.SDO_GEOMETRY(2003,3785,NULL,MDSYS.SDO_ELEM_INFO_ARRAY(1,1003,1),MDSYS.SDO_ORDINATE_ARRAY(-13426771.266146,5334024.8870015,-13425624.710722,5326534.0582305,-13412553.978887,5325922.5620044,-13412936.164029,5333719.1388884,-13426771.266146,5334024.8870015)).GET_WKT()) FROM DUAL;
    Result:
    ORA-29532: Java call terminated by uncaught Java exception: java.lang.RuntimeException
    ORA-06512: at "MDSYS.SDO_UTIL", line 172
    29532. 00000 - "Java call terminated by uncaught Java exception: %s"
    *Cause:    A Java exception or error was signaled and could not be
    resolved by the Java code.
    *Action:   Modify Java code, if this behavior is not intended.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       

    Hello jpmc26,
    I am going to guess that most of us live our lives between the 180s and have not really noticed this before. I would call it a bug that needs an SR opened but is the bug on the coming or the going? If you look in the Simple Features 1.2.1 specification on pages 52 and 53 they clearly say that an "approximate numeric literal" of mantissa + E + exponent is valid. However, Oracle Spatial does not support the 1.2.1 spec. Rather they support something akin to the 1.1.0 specification. On pages 28 and 29 of that version there is no mention of approximate numeric literals as it first shows up in the 1.2.0 specification.
    So I would say either:
    1) Oracle Spatial supporting only the 1.1.0 specification and not much interested in updating to current specifications, should remove the output of scientific notation from the SDO_UTIL.to_WKTGEOMETRY procedure (match 1.1.0 spec).
    -or-
    2) Oracle Spatial looking forward to future compatibility with new OGC standards, should add support for parsing scientific notation to the SDO_UTIL.from_WKTGEOMETRY procedure (prepare for 1.2.1 spec).
    I guess its a policy decision on their side. Please update the posting as to what they say as I am curious about the topic.
    Many folks have largely abandoned these java-based, outdated, OGC converters. Feel free to search the forum for complaints, myself being one of the complainers. When you said "a large amount of code already depends" on WKT, my first thought was that must be really slow. Writing your own SDO to WKT converter in PLSQL is really easy and I believe that's what most of us have done. The other direction is more challenging but doable - I need to rewrite mine but its works well enough for straightforward stuff.
    Cheers,
    Paul

  • Int printing out as scientific notation

    maybe doing something stupid here but I can't seem to pick it up.
    I have a Window that calls a subclass to display a calculator, and then returns the final value to the Window, if I input 10 digits it prints on as a 12345678E5
    something like that.
    Anywho here's the two methods that deal with value in the subclass( calculator )
    public int ReturnNumber(){//the method that will return the value from the keyboard
       int final_number = Integer.parseInt(number); 
        return final_number;   // returns value to question screen
      private void NextButtonActionPerformed (java.awt.event.ActionEvent evt) {
       if ( value.length() != allowable_answers[currentQuestionNumber] ){
                     JOptionPane.showMessageDialog(this, "Please make a valid entry.", "Invalid",
                     JOptionPane.WARNING_MESSAGE );
                     value.replace(0,counter,""); 
                     jTextField1.setText( null );
                     return;
        else {
                number = value.toString();
               setVisible(false);
               frame.final_number = ReturnNumber();
               frame.userMakeSelection = true;
               frame.FinalTimer.start();
               frame.ButtonSelected();Code from window that deals with the number
    if(Numeric[currentQuestionNumber]){
                currentAnswers[currentQuestionNumber][1] = final_number;// currentAnswers is a float[][]
                numeric_question_value[currentQuestionNumber][0] = final_number;// used in poll frequency
              }// numeric is a int[]
            else
                currentAnswers[currentQuestionNumber][currentChoice] = currentChoice;Is from trying to jam an int into float?
    Any suggestions
    Jim

    Is from trying to jam an int into float?That's exactly the cause. Here are some solutions:
    - Use java.text.DecimalFormat to format the output or cast the float to an integer type when you want to print it (presicion might become a problem).
    - Keep the number in an int or long all the time. This way you'll not lose any presicion.
    Explanation can be found in the API docs of Float.toString():"If the argument is NaN, the result is the string "NaN".
    Otherwise, the result is a string that represents the sign and magnitude (absolute value) of the argument. If the sign is negative, the first character of the result is '-' ('-'); if the sign is positive, no sign character appears in the result. As for the magnitude m:
    If m is less than 10^-3 or not less than 10^7, then it is represented in so-called "computerized scientific notation." Let n be the unique integer such that 10n<=m<1; then let a be the mathematically exact quotient of m and 10n so that 1<a<10. The magnitude is then represented as the integer part of a, as a single decimal digit, followed by '.' (.), followed by decimal digits representing the fractional part of a, followed by the letter 'E' (E), followed by a representation of n as a decimal integer, as produced by the method Integer.toString(int) of one argument."

  • Powershell / Scientific Notation Woes / Formatting Output to Live Excel Sheet

    I have a WMI query in a script that dumps machine information to a live excel sheet.
    I find when I query the model # of the machine using (this line of code from the script):
    $Sheet.Cells.Item($count,4) = (Get-WmiObject win32_computersystem -ComputerName $computer).Model
    The output almost always changes into Scientific Notation, because this is a typical model number :   "3500-E52"
    Is there a way to modify the formatting of the:
    $Sheet.Cells.Item($count,4)
    so that Excel uses "text" formatting for that cell?  I know you can change the fonts etc... but have not been lucky in finding a powershell > excel reference
    that explains how to change a cell or column's formatting beyond the basics.
    I have seen recommendations to others to dump the results queries to a CSV first, and then import into Excel (which would allow a manual change of that column's format).  I'm just
    hoping to bypass the extra "hands on" and format more directly to a sheet in Excel.
    If .csv is the best way to go, I'll muddle through it and change the code.
    Any help is greatly appreciated.
    Ben

    $sheet.Cells(1,1).NumberFormat
    = "@"
    ¯\_(ツ)_/¯

  • Scientific notation doesn't display correctly

    Hi, this is just a general question but it's about the Calculator app. The problem is with displaying answers in scientific notation. When I'm in scientific calculator mode, answers displayed on the screen only appear in scientific notation when they are larger than the number of digits that can be displayed. However, whenever I use it to calculate something in which the answer is smaller than the set precision for the number of decimal places, it always gives 0 instead of scientific notation. For example, if I type a number such as 1E19 and press =, then the answer displays correctly as 1E19. However, if I type 1/1E19, and press =, then the answer simply shows up as 0 instead of 1E-19, which is the correct answer in scientific notation. So, it seems that the scientific notation in Calculator simply does not display negative exponents in the scientific notation (I know that I can type such numbers but for some reason, it cannot display them as answers). This is very annoying because I often work with very small numbers, not just very large ones. Is there any way that I can get the scientific notation to display negative exponents when I'm working with very small numbers and when the answer is smaller than the set number of decimal places?

    There are tonnes of possibilities and many websites with suggestions. I tried all sorts of these option (clear cache, clear cookies, anti-virus changes, page style options etc) with no success until eventually I accidentally changed my character encoding autodetect (firefox 6) from "off" to "universal" and everything is fine.

  • Scientific Notation to Integer?

    I have a string like 1.780657E7 that I need to convert to an integer. I think that might be too big for Integer, so maybe BigInt would be better.
    Anyway, I can't figure out how to do this! I guess I could write my own method, but I'm sure it's already out there somewhere. I'm pretty new to Java.
    Thanks a lot,
    - Andrew.

    A small exegesis of prometheuzz's perfectly accurate, if slightly terse solution.
    Scientific notation is assumed to be floating-point. You must therefore parse a scientific-notation string using Float.parseFloat() or Double.parseDouble() and "integerise" the result if an integer is what you want.
    The last line is to show that your sample number is easy to fit in an int variable.

  • Scientific Notation in Java

    Hi,
    In the hopes of enforcing a precision of 1.0e-16 for all the doubles in my Java application, I am trying
    to apply a DecimalFormat pattern so that my doubles can be printed in scientific notation if necessary. I am doing:
    DecimalFormat decf = (DecimalFormat)DecimalFormat.getInstance();
    decf.applyPattern("0.################E0");
    as the first step in the execution process but this is not working. Am I missing something?
    Thanks,
    Corrine

    It might help if you explained what "not working" means in this case. You can only use formatting to control how the numbers appear to you, not how they are represented internally for calculation, if that's part of your question.

  • Double.parseDouble(String) - problems when string is in scientific notation

    Hello guys,
    I'm doing some numerical calculations and I wonder whether it is possible for Double.parseDouble(String) to parse string in the scientific notation i.e. 1.0824234234E-10. Is it the notation itself causing the exception : NumberFormatException or the number is just too big/small and double can't hold it ?
    If it's just the notation how can I fix it ?
    Regards

    i'm not quite sure whether double odoes not allow it.
    perhaps consider the api Double.valueOf() and the testing code provided; reproduced below:To avoid calling this method on a invalid string and having a NumberFormatException be thrown, the regular expression below can be used to screen the input string:
            final String Digits     = "(\\p{Digit}+)";
      final String HexDigits  = "(\\p{XDigit}+)";
            // an exponent is 'e' or 'E' followed by an optionally
            // signed decimal integer.
            final String Exp        = "[eE][+-]?"+Digits;
            final String fpRegex    =
                ("[\\x00-\\x20]*"+  // Optional leading "whitespace"
                 "[+-]?(" + // Optional sign character
                 "NaN|" +           // "NaN" string
                 "Infinity|" +      // "Infinity" string
                 // A decimal floating-point string representing a finite positive
                 // number without a leading sign has at most five basic pieces:
                 // Digits . Digits ExponentPart FloatTypeSuffix
                 // Since this method allows integer-only strings as input
                 // in addition to strings of floating-point literals, the
                 // two sub-patterns below are simplifications of the grammar
                 // productions from the Java Language Specification, 2nd
                 // edition, section 3.10.2.
                 // Digits ._opt Digits_opt ExponentPart_opt FloatTypeSuffix_opt
                 "((("+Digits+"(\\.)?("+Digits+"?)("+Exp+")?)|"+
                 // . Digits ExponentPart_opt FloatTypeSuffix_opt
                 "(\\.("+Digits+")("+Exp+")?)|"+
           // Hexadecimal strings
           "((" +
            // 0[xX] HexDigits ._opt BinaryExponent FloatTypeSuffix_opt
            "(0[xX]" + HexDigits + "(\\.)?)|" +
            // 0[xX] HexDigits_opt . HexDigits BinaryExponent FloatTypeSuffix_opt
            "(0[xX]" + HexDigits + "?(\\.)" + HexDigits + ")" +
            ")[pP][+-]?" + Digits + "))" +
                 "[fFdD]?))" +
                 "[\\x00-\\x20]*");// Optional trailing "whitespace"
      if (Pattern.matches(fpRegex, myString))
                Double.valueOf(myString); // Will not throw NumberFormatException
            else {
                // Perform suitable alternative action
    http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Double.html

  • Why Number displayed in  scientific notation

    I have tried to add two numbers and got a result displayed in scientific notation.
    Then I have used the 'decimalformat()' function and result is wrong.
    How can display the correct result. Please help
    Please see the code I have used
    <cfset N1=1>
    <cfset N2=9999999999999899999999>
    <cfset RESULT = N1+ N2>
    <cfoutput>
    #RESULT#
    <br />
    #decimalformat(RESULT)#
    </cfoutput>
    result :
    1E+022
    92,233,720,368,547,760.00
    Thanks in advance

    I think in floating point math adding 1 to 9999999999999899999999 is going to be a meaningless operation as by the time one converts 1 to a floating point of the scale & precision needed to represent 9999999999999899999999, it's going to lose any significance (ie: it's going to basically be represented as zero).
    You'll need to do a search on how to deal with numbers of very high precision & scale.  I don't have any code to hand.  Have you done any of your own investigation on how to deal with this?
    Adam

  • Convert scientific notation value into normal number

    hi,
    I am importing excel(.csv) file into Oracle database, a value in excel file 8.70773E+11 is displaying in scientific notation format, i want to store it like 870772521002.

    i have already run this query, it gave me 871000000000.0000000000 this result.
    Table Structure,
    SQL> desc cdr;
    Name Null? Type
    INVOICENO VARCHAR2(50)
    OCEANREGION NUMBER
    CALLDATE DATE
    CALLTIME VARCHAR2(10)
    ORIGINATORNO                             VARCHAR2(50)
    SUBSCRIBER VARCHAR2(75)
    DESTNO VARCHAR2(50)
    VOLUME FLOAT(50)
    UNIT VARCHAR2(5)
    this above Originatorno field had Number data type, but it wasnt working.

Maybe you are looking for