Simple calculation errors

Does Numbers always make calculation errors?
I have a sheet with 80.9 entered in one cell and 81.1 in another cell
When I do a calculation of 81.1 - 80.9, the result I get is 0.199999999999989
81.1
80.9
0.199999999999989
Why is there a rounding error on numbers with 1 decimal place?
This makes me not trust anything that Numbers does if it can't even do that!

Mason,
this topic has been addressed in many other threads and is a result of the way numbers are stored on a computer.  the difference between the value displayed and the correct value is 0.2 - 0.199999999999989 which is 0.000000000000011 
all computer programs use integer of floating point representations of a number-- see this link for details:
http://en.wikipedia.org/wiki/Single-precision_floating-point_format
you can format the number with two digits using the cell formatter and you will see the "correct" answer:

Similar Messages

  • Flash rookie needs help with error messages when making simple calculator in AS3.

    I am trying to make a simple calculator I have been following this guide:
    http://www.youtube.com/watch?v=5k3h37YKZJI
    this is my code.
    var Hnum:String;
    var PCnum:String;
    var calc:Number = 25;
    var Total:String;
    num1.restrict = "0-9";
    num2.restrict = "0-9";
    Est_btn.addEventListener(MouseEvent.CLICK, calculate);
    function calculate(event:MouseEvent):void{
        Hnum = num1.txt;
        PCnum = num2.txt;
        Total = parseInt(Hnum) * calc;
        Total.toString();
        Total_txt.text = String(Total);
    I am getting these 3 errors:
    Scene 1, Layer 'Actions', Frame 1, Line 14
    1067: Implicit coercion of a value of type flash.text:TextField to an unrelated type String.
    Scene 1, Layer 'Actions', Frame 1, Line 15
    1067: Implicit coercion of a value of type flash.text:TextField to an unrelated type String.
    Scene 1, Layer 'Actions', Frame 1, Line 16
    1067: Implicit coercion of a value of type Number to an unrelated type String.
    I have two input text boxs named num1 and num2 and then one dynamic text box named Total_txt.
    This should be so simple I really don't know what im doing wrong?
    Any help would be really appreciated.
    MrB

    If you look at the tutorial I think you will see that you are using "txt"  where "text" is required.  "text" is a property of a textfield, identifying the text that the textfield holds.  I don't know if you are doing something where the num1 object contains another object named txt, but I think my first sentence catches the problem.

  • Time Machine Backup Size Calculation Error

    This is what I have:
    Internal:
    Mac OS partition: 266GB (257GB Used)
    BootCamp partition: 32GB ( 12GB Used)
    External:
    External USB 2.0 HD: 153GB (142GB Used)
    TM Backup Drive: 466GB ( 0GB Used)
    *What I have excluded:*
    - TM Backup Drive
    - BootCamp partition
    *Therefore what I'm trying to backup are only:*
    - MacOS partition
    - External USB 2.0 HD
    Time Machine Preferences Options Pane reports "Total Included: 398.4GB", and that's totally correct for what I'm intending to backup (i.e. MacOS partition + External USB 2.0 HD = 257GB + 142GB = 399GB). So far so good.
    But when TM kicks off after the countdown, an error dialog pops up saying:
    *Time Machine Error*
    This backup is too large for the backup volume. The backup requires 478.0 GB but only 465.5GB are available.
    To select a large volume, or make the backup smaller by excluding files, open System Preferences and choose Time Machine.
    [Preferences] [OK]
    Now how does TM figure out that 478.0GB number?
    If I add up all used spaces for all drives (excl. TM drive), I only get 411GB.
    If I add up all capacity of all drives (excl. TM drive), I only get 451GB.
    How in the world did TM calculate that 478.0GB number? Why is there a discrepancy between what's reported in the options pane and the error dialog?
    BTW, I have verified all my disks and they are all fine.
    All my drives and partitions (except the bootcamp partition) are formatted properly to Mac OS Extended (Journaled).
    Someone please advice.
    Much appreciated.

    Disregarding the Mac/Win partitions, my internal HD as a whole is 298GB. My external USB drive is 153GB.
    nestea247,
    Your computer info says : iMac Aluminum 24" ....... 320GB HD.
    So, I assumed your internal HD is 320GB and you are currently using 298GB of it. Therefore my simple calculation was not "298GB + 153GB" but "320GB + 153GB(ca.)". Did I misunderstand?
    My point was: For the first-time backup, in order to copy the entire image (including the *free space*) of the volume to backup, TM might think it needs about the same size of backup volume as your internal HD ( + external USB drive, in your case). Not simply adding up the size of the *used space*.
    Leter, however, it will get more realistic and adjust to the actual data size; so, after completing all the hard link setup, the size of its backup seems to "shrink." Your TM could be still calculating.... perhaps struggling to adjust? (I too hope it is not a critical bug.)
    I'm afraid my reply doesn't help solve the mystery of 478.0GB. Hope someone else can.
    Best of luck!

  • Simple calculations not working in acrobat 9

    I have text form fields made in Acrobat 9 from a previously made pdf, where I need a simple calculation, but I cannot get it to work.
    The form is not for submission, only for personal tracking, so I don't even need validation.
    FieldA (a number they enter) x a number I have entered in the calculation = FieldB
    FieldB x FieldC (a number they enter) = FieldD In FieldB I have a simple javascript Calculation in the form field: FieldA * .115, which should automatically calculate answer into FieldB, when tabbed.
    In FieldD I have a Calculation in the form field: FieldB * FieldC, which should automatically calculate answer into FieldD. All fields are set as numbers.
    I have left the Trigger Action as Mouse Up.
    I don't need to validate, as this is personal use, and is not being submitted.
    I have all fields in the correct tab order.
    I have reader enabled the form. When I open the form in Reader, I can enter the numbers, but no calculations are being performed. It reads 0 in FieldB and FieldD. Any assistance would be greatly appreciated.

    Hello Everyone,
    I use the following VBS code, but to tell the truth, I don't know if it works or not. The main idea was to enable commenting from Adobe Reader. But after execution code, saving document and reopening it in Reader, commenting is still 'Not Allowed'. Is showAnnotToolsWhenNoCollab really capable to allow this and there is only error in my code? Or is the functionality different and this way can never be used for what I want?
    There is another issue that application is not closed on Exit, but is still shown.
    scriptPath = "C:\Acrobat\"
    Set gApp = CreateObject("AcroExch.App")  
    Set pdDoc = CreateObject("AcroExch.PDDoc")
    sampleFilePath = scriptPath & "AR_test_orig.pdf"
    pdDoc.Open sampleFilePath
    pdDoc.OpenAVDoc "myFile"
    gApp.Show
    Set formApp = CreateObject("AFormAut.App")
    'formApp.Fields.ExecuteThisJavascript "app.alert(""hello"");"
    formApp.Fields.ExecuteThisJavascript "Collab.showAnnotToolsWhenNoCollab = true;"
    Set formApp = Nothing
    pdDoc.Save 1, scriptPath & "AR_test.pdf"
    'pdDoc.Close
    gApp.CloseAllDocs
    gApp.Exit
    Set pdDoc = Nothing
    Set gApp = Nothing
    Thanks for any ideas.
    Jan
    PS: I use Acrobat 9.0 Pro

  • Simple calculator to use log, successor, predecessor ?

    Hello. I have this simple calculator program, and I am having trouble with the log, successor (if 8 entered returns 9), predecessor (if 8 entered returns 7). The log gets this error:
    Calculator.java:113: log(double) in java.lang.Math cannot be applied to (int,int)
                   result = (int)Math.log(result, stringToInteger(ioField.getText()));
    I did the same thing with power, and it seems to work so I'm a little confused about that.
    If anyone can help me out it would be greatly appreciated, and thanks in advance!
    Here's the code (commented out code is what I need help with):
    import javax.swing.*;
    import java.awt.*;
    import java.awt.event.*;
    public class Calculator extends JFrame implements ActionListener
         public static final int WIDTH = 400;
         public static final int HEIGHT = 200;
         public static final int NUMBER_OF_DIGITS = 30;
         private JTextField ioField;
         private int result = 0;
         public Calculator()
              setSize(WIDTH, HEIGHT);
              setTitle("GUI Calculator");
              setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
              setLayout(new BorderLayout());
              JPanel textPanel = new JPanel();
              textPanel.setLayout(new FlowLayout());
              ioField = new JTextField("Enter numbers here.", NUMBER_OF_DIGITS);
              ioField.setBackground(Color.WHITE);
              textPanel.add(ioField);
              add(textPanel, BorderLayout.NORTH);
              JPanel buttonPanel = new JPanel();
              buttonPanel.setBackground(Color.BLUE);
              buttonPanel.setLayout(new GridLayout(5, 2));
              JButton addButton = new JButton("+");
              addButton.addActionListener(this);
              buttonPanel.add(addButton);
              JButton subButton = new JButton("-");
              subButton.addActionListener(this);
              buttonPanel.add(subButton);
              JButton multButton = new JButton("*");
              multButton.addActionListener(this);
              buttonPanel.add(multButton);
              JButton divButton = new JButton("/");
              divButton.addActionListener(this);
              buttonPanel.add(divButton);
              JButton expButton = new JButton("^");
              expButton.addActionListener(this);
              buttonPanel.add(expButton);
              JButton logButton = new JButton("v");
              logButton.addActionListener(this);
              buttonPanel.add(logButton);
              JButton sucButton = new JButton("s");
              sucButton.addActionListener(this);
              buttonPanel.add(sucButton);
              JButton preButton = new JButton("p");
              preButton.addActionListener(this);
              buttonPanel.add(preButton);
              JButton clearButton = new JButton("C");
              clearButton.addActionListener(this);
              buttonPanel.add(clearButton);
              add(buttonPanel, BorderLayout.CENTER);
         public void actionPerformed(ActionEvent e)
              try
                   assumingCorrectNumberFormats(e);
              catch(NumberFormatException e2)
                   ioField.setText("Error - Reenter number.");
         public void assumingCorrectNumberFormats(ActionEvent e)
              String actionCommand = e.getActionCommand();
              if(actionCommand.equals("+"))
                   result = result + stringToInteger(ioField.getText());
                   ioField.setText(Integer.toString(result));
              else if(actionCommand.equals("-"))
                   result = result - stringToInteger(ioField.getText());
                   ioField.setText(Integer.toString(result));
              else if(actionCommand.equals("*"))
                   result = result * stringToInteger(ioField.getText());
                   ioField.setText(Integer.toString(result));
              else if(actionCommand.equals("/"))
                   result = result / stringToInteger(ioField.getText());
                   ioField.setText(Integer.toString(result));
              else if(actionCommand.equals("^"))
                   result = (int)Math.pow(result, stringToInteger(ioField.getText()));
                   ioField.setText(Integer.toString(result));
    /*          else if(actionCommand.equals("v"))
                   result = (int)Math.log(result, stringToInteger(ioField.getText()));
                   ioField.setText(Integer.toString(result));
              else if(actionCommand.equals("s"))
                   result = result + 1;
              else if(actionCommand.equals("p"))
                   result = result - 1;
    */          else if(actionCommand.equals("C"))
                   result = 0;
                   ioField.setText("0");
              else
                   ioField.setText("Unexpected error.");
         public static int stringToInteger(String stringObject)
              return Integer.parseInt(stringObject.trim());
         public static void main(String[] args)
              Calculator gui = new Calculator();
              gui.setVisible(true);
    }

    A 'double' is not the same as an 'int' and they can't fit the one value-type into the memory of the other.
    Although this is a logical math problem, simple for us, we have better memory than a computer.
    An integer maps a certain amount of memory to fix its maximum value, when you cast, you must cast into a type that is the same or smaller in terms of memory byte size.
    Just like you can't cast some objects as others, you can't cast a double as an int.
    The error is telling you that the result of a method call is incompatiable with the argument of another method call.
    The fault lies at Calculator.java source file at line 113 in the file. This is where you call the method log() which requires an argument of type 'double' which is located in Math class log method where the two arguments are 'int' and 'int'.
    However, in your call, you inflect the reference 'result' as the product of the method call in which the 1st argument is also 'result'
    That in itself is bad design. It's like saying "Joe is the product of Mike (who uses Joe)."
    Memory-wise, this could cause possible leaks.

  • Simple calculator

    Hello,
    I'm new to programming on the Mac and I've been following through a tutorial in the "Cocoa Programing for Mac OS X For Dummies" book. I'm getting a weird calculation error and was wondering if anyone had time to have a look at it for me?
    // Fusebox.h
    // My First Project
    #import <Cocoa/Cocoa.h>
    @interface Fusebox : NSObject {
    IBOutlet id answerField;
    IBOutlet id numberField1;
    IBOutlet id numberField2;
    - (IBAction)calculateAnswer:(id)sender;
    @end
    // Fusebox.m
    // My First Project
    #import "Fusebox.h"
    @implementation Fusebox
    - (IBAction)calculateAnswer:(id)sender
    float num1, num2, answer;
    num1 = [numberField1 floatValue];
    num2 = [numberField2 floatValue];
    answer = num1 + num2;
    [answerField setFloatValue:answer];
    @end
    I've re-read the tutorial and even copy and pasted the code and I can't find any problems. But when I run the simple application the simple addition doesn't return the proper results.
    For example, I just ran the application then and tried entering 1 into the first text field and 1.2 into the second. But when I click "Calculate" it returns "2.200000047683716".
    !http://img23.imageshack.us/img23/9361/picture1fc.png!
    Am I doing something wrong?
    Thanks!

    Just to add my two cents.
    Floating point numbers have always had a representation problem with whatever computer hardware in use, and as noted in many engineering texts, any number is only as accurate as the precision used in the problem.
    Thus, 1.2 is not the same as 1.20 or 1.200 because there is a question about precision. So, when you entered 1 + 1.2, the 1.0 had no fractional component, but a purist engineer will beg to differ. The 1.2 however does have a fractional component and could represent a number between 1.15000 and 1.24999 using traditional mathematical rounding rules. So, you need to either set the display as rounded to the nearest precision used, or work in a specified precision.
    Thus to expect 8 digits of precision, 1 + 1.2 == 1.20000000 you need to enter 1.00000000 + 1.20000000 and unless that wraps the precision of the computer, it should be accurate to the proper number of digits.
    There was a problem with some early pentium chips that adding 2.0+2.0 did not equal 4.0 and it was said to have been closer to 5.0, but I never checked the actual details on this since Macs didn't use pentium processors, so somewhere online there may be a reference to the actual calculation problem.

  • Internal Charges Calculation Error status 03 & Charges disaapearing after FO creation

    Hi Experts,
    I have done following Config & data set up for Internal Settlement (Forwarding Order) while I intent to do Inter Company settlement. (I am using TM9.0 SP01 version)
    In Movement Type Stage sequence set to “No Rule” for internal Settlement.
    Created forwarding order by putting different Planning & Execution Forwarding House on Stage Level.
    Agreement set is done & internal Charges are appearing correctly but with Calculation Error “03”
    I rechecked Internal Agreement, Rate Tables, Calculation Sheet, Everything is fine.
    There are two issues I need your experts advise to resolve following issues:
    Charges are calculated correctly but Why Charge Cal. Status is 03? (No error on Charge Item level & Log analysis is fine)
    Charges are disappearing when I create Freight Order against stages it. (with error shown below)
      (attached Screenshots showing errors)
    Regards,
    Vinod Vikhe

    Hello Vinod,
    Still nothing for me...
    Anyway, have you checked if the assigned of BP to the organization unit? Sometimes this is the reason the internal calculation does not work.
    Best Regards, Marcelo

  • Simple calculations in Universe not working for XI 3.0

    Simple calculations in Universe using key figures are not working in XI 3.0(without any Fix Pack)
    Below is the steps I followed.
    1. Using key figure [Jan] I am trying to round the values by deviding it by 1000.
    <EXPRESSION>@Select([Jan])/1000</EXPRESSION>
    This gives me a null cell value.
    2. I tried addition then
    <EXPRESSION>@Select([Jan])+1000</EXPRESSION> result was all the cell values changed to 1000.
    3. Finally without calculation I tried
    <EXPRESSION>@Select[Jan]</EXPRESSION> this also resulted null value
    But without EXPRESSION tag if I try @Select[Jan] it works fine.
    Am I missing anything in above expressions.
    As of now I am creating variable in report to incorporate the rounding.
    Thanks
    Raghu

    It was my mistake while writing the question. Path I have used is correct one only @Select(Key Figures/.
    If i use the above alone, I am able to se the data. But if I use in betwen <EXPRESSION> then it gives null value.
    Note: Pls dont confuse with flower bracket i used it just to avoid coverting it to a html link and also the backslash it was not showing in preview
    I have followed that pdf and created calculation in universe.
    Edited by: Raghavendra Barekere on Feb 6, 2009 6:42 AM

  • Need help with starting a simple calculator

    I have this simple calculator I just started on, and I'm having just a few problems. First of all, I don't know how to append text to a textfield. Maybe I would have to create a string buffere or something. Then, when the user clicks one of the buttons, the compiler goes crazy, and nothing is displayed.
    import java.awt.*;
    import java.awt.event.*;
    import javax.swing.*;
    public class Calculator extends JFrame implements ActionListener
         JButton[] btnNums;
         JButton btnBack;
         JButton btnClear;
         JButton btnCalculate;
         String[] strNames;
         JTextField txtDisplay;
         public Calculator()
              Container content = getContentPane();
              setSize(210,250);
              setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
              /*  construct a JPanel and the textfield that
               *  will show the inputed values and outputed
               *  results
              JPanel jpDisplay = new JPanel();
              jpDisplay.setLayout(new BorderLayout());
              txtDisplay = new JTextField(15);
              txtDisplay.setHorizontalAlignment(JTextField.RIGHT);
              jpDisplay.add(txtDisplay);
              /*  contstruct a JPanel that will contain a back
               *  button and a clear button.
              JPanel jpRow2 = new JPanel();
              btnBack = new JButton("Backspace");
              btnClear = new JButton("Clear");
              btnClear.addActionListener(this);
              jpRow2.add(btnBack);
              jpRow2.add(btnClear);
              /*  construct a string array with all the names of the
               *  buttons, then in a for loop create the new buttons
               *  and add their name and an actionListener to them.
              String[] strNames = {"7","8", "9","/", "4", "5", "6","*", "1", "2", "3","+", "0", "+/-", ".", "-"};
              btnNums = new JButton[16];
              JPanel jpButtons = new JPanel();
              jpButtons.setLayout(new GridLayout(4,4));
                for (int i = 0; i < 16; i++)
                     btnNums[i] = new JButton(strNames);
                   btnNums[i].addActionListener(this);
                   jpButtons.add(btnNums[i]);
              /* construct the final JPanel and add a
              * calculate button to it.
              JPanel jpLastRow = new JPanel();
              btnCalculate = new JButton("Calculate");
              btnCalculate.addActionListener(this);
              jpLastRow.add(btnCalculate);
              /* set the contentPane and create the layout
              * add the panels to the container
              setContentPane(content);
              setLayout(new FlowLayout());
              setResizable(false);
              content.add(jpDisplay, BorderLayout.NORTH);
              content.add(jpRow2);
              content.add(jpButtons);
              content.add(jpLastRow);
              setTitle("Mini Calculator");
              setVisible(true);
         public void actionPerformed(ActionEvent ae)
              for (int i =0; i < 16; i++)
                   if (ae.getSource() == btnNums[i])
                        txtDisplay.setText(strNames[i]);
         public static void main(String[] args)
              Calculator calc = new Calculator();

    First of all, I don't
    know how to append text to a textfield.
    textField.setText( textField.getText() + TEXT_YOU_WANT_TO_APPEND );
    Then,
    when the user clicks one of the buttons, the compiler
    goes crazy, and nothing is displayed.No, the compiler doesn't go crazy, the compiler has done it's job by then. However, you do get a NullPointerException, and the stacktrace tells you that the problem is on the following line:
    txtDisplay.setText(strNames);
    It turns out that strNames is null. I leave it to you to find out why. ;-)

  • Simple Calculations in BPS Web

    Hi,
    I am using the BPS web layouts (ALV Grid not the Excel Web Component). I would like to add a couple of very simple calculations (difference of columns/percentages etc). Is it possible to do this without using the Excel OWC ?
    Thanks
    Shailesh

    Hi
    the simplier way to achieve yor result is to use FOx formula to calculate your results at runtime .
    You should create a formula function and add FOX code to it .
    Check this link for examples
    http://help.sap.com/saphelp_nw04/helpdata/en/d3/8057f830a911d4b2be0050dadfb23f/frameset.htm
    Hope it helps
    Andr

  • "Item 10     IPC:TTE:Calculation error: Pricing error: Mandatory condition & is

    Hello Experts,
    I need your help to solve this TTE pricing error.
    In Service Order, I am getting this error message for one particular material only
    "Item 10     IPC:TTE:Calculation error: Pricing error: Mandatory condition & is missing".
    Remaining materils I am not getting this error message.
    Please proivide your suggestions to crack this.
    Thanks

    Hi,
      This message come when you caculate 0TTE when doing cross country transactions . Check if this codition needs to be caculated for your order. If it is not required you should not make call .You can try to eliminate this error bu uisng the BADI BADI_TTE_DOCUMENT

  • Has anyone seen a simple calculator pod that will work in Acrobat Connect Pro?

    I need a simple calculator to total up responses from participants in web meetings. Does anyone know of one that can work as a pod where everyone will see the numbers that are pushed and the calculation display?

    If this one works with you: http://connect.wezupport.se/p39142940/
    I've attatched so you can download it from here - cheers

  • Installment cannot be calculated; error in repayment plan Infotype  0045

    Hello Experts.
    I am  getting error in IT0045
    "Installment cannot be calculated; error in repayment plan"
    Please give solution
    Regards
    Ramu

    Hi Ramu
    When you are getting this error? While processing salary.
    Thanks
    Yogesh

  • After Upgrade to PI7.1 getting Simple transformation error

    HI Experts,
    I have an Issue, the scenario which i', working its working properly in DEV, QA, PROD in PI 7.0, We upgraded the system to PI7.1
    Unknown Simple transformation error in converting XML to internal table
    Application error in mapping program , error text: Unknown Simple transformation error in converting XML to internal table An exception has occurred.
    Can any one suggest related notes are suggestion.
    appericate in advance for the solutions which you all provide me
    Thank you
    Venkat Anil

    What kind of mapping is that ? Graphical, Java, XSLT, ABAP ?
    The XML parser in 7.1 contains many subtle changes compared to 3.0
    But this error sounds like a bug. Is there something special in the XML input ? is it wellformed-XML ?
    CSY

  • Simple transformation error in converting XML to internal table

    Hi Team,
    I have an issue, its working in Dev, QA, and Prod in PI 7.0, once we upgrade it to PI 7.1 I'm getting the  below mention error,   
    Unknown Simple transformation error in converting XML to internal table
    Application error in mapping program ZF_INT006_FILE_TO_SAP_FILE, error code: , error text: Unknown Simple transformation error in converting XML to internal table An exception has occurred.
    Can any one suggest related notes are suggestion.
    appericate in advance for the solutions which you all provide me
    Thank you
    Venkat Anil

    Check the flag "use SAP XML toolkit" for the operation mapping and try, if that works then.
    Check the flag for all XSLT and Java mappings which is migrated from PI 7.0

Maybe you are looking for