Widespread calculation errors

I've been having a lot of trouble with accuracy in Microsoft Excel, which I thought was a bug in their software.
I've since tried the same easy formula in Apple's Numbers app and get the same result.
The following formula should give the result of zero (as 9.2x1600 equals 14720)
=((9.21600)-14720)10000000
unfortunately, I get -0.00000001819.... in both applications.
It's a small error, but it doesn't show up (which is why I've multiplied by 10000000) until you need the accuracy - especially if you're using automated comparison/error-checking.
There should never be any such error in such a simple and ordinary multiplication of 0.2 of large numbers
I found that the calculator app that is part of the OS also gives me ridiculously long answers, to several decimal points, for simple sums - try typing 9.2 or 9.3 into it and see the readout on the paper-tape window.
Is this something that can be fixed please. It's very important and makes the Mac unusable for so much when errors like this can introduce errors into every app that calculates anything, not just mathematical ones (from FCP time-stretches to photoshop colour-blending/banding problems).
many thanks
manex

I noticed this too with the OSX calculator app in 10.5 and 10.6, it behaves differently (and incorrectly IMO) than the old one.
Set the calculator for basic view and bring up the tape. (I'm using Version 4.5.3 (99.2) for this) type this:
102.2 + 99.99 =
and the result is 202.19 (correct).
But in the tape you get this:
102.200000000000003+99.989999999999995
=202.189999999999998
So it seems the tape is using different math than the main window (btw, decimals set to 15).
As far as 'internally using binary', well, this seems to be the case and this is a serious error on the part of the programmer. With a calculator, you should always do decimal arithmetic internally so the result is accurate. <flame> This was drilled into my head in engineering school, it's a rookie mistake.</flame> It's a serious problem and should be corrected.
For what it's worth, the old (<= 10.4) calculator didn't have this problem, and it seems this is only in the tape part and not the main calculator window.
Regards,
Marc

Similar Messages

  • Internal Charges Calculation Error status 03 & Charges disaapearing after FO creation

    Hi Experts,
    I have done following Config & data set up for Internal Settlement (Forwarding Order) while I intent to do Inter Company settlement. (I am using TM9.0 SP01 version)
    In Movement Type Stage sequence set to “No Rule” for internal Settlement.
    Created forwarding order by putting different Planning & Execution Forwarding House on Stage Level.
    Agreement set is done & internal Charges are appearing correctly but with Calculation Error “03”
    I rechecked Internal Agreement, Rate Tables, Calculation Sheet, Everything is fine.
    There are two issues I need your experts advise to resolve following issues:
    Charges are calculated correctly but Why Charge Cal. Status is 03? (No error on Charge Item level & Log analysis is fine)
    Charges are disappearing when I create Freight Order against stages it. (with error shown below)
      (attached Screenshots showing errors)
    Regards,
    Vinod Vikhe

    Hello Vinod,
    Still nothing for me...
    Anyway, have you checked if the assigned of BP to the organization unit? Sometimes this is the reason the internal calculation does not work.
    Best Regards, Marcelo

  • "Item 10     IPC:TTE:Calculation error: Pricing error: Mandatory condition & is

    Hello Experts,
    I need your help to solve this TTE pricing error.
    In Service Order, I am getting this error message for one particular material only
    "Item 10     IPC:TTE:Calculation error: Pricing error: Mandatory condition & is missing".
    Remaining materils I am not getting this error message.
    Please proivide your suggestions to crack this.
    Thanks

    Hi,
      This message come when you caculate 0TTE when doing cross country transactions . Check if this codition needs to be caculated for your order. If it is not required you should not make call .You can try to eliminate this error bu uisng the BADI BADI_TTE_DOCUMENT

  • Installment cannot be calculated; error in repayment plan Infotype  0045

    Hello Experts.
    I am  getting error in IT0045
    "Installment cannot be calculated; error in repayment plan"
    Please give solution
    Regards
    Ramu

    Hi Ramu
    When you are getting this error? While processing salary.
    Thanks
    Yogesh

  • Simple calculation errors

    Does Numbers always make calculation errors?
    I have a sheet with 80.9 entered in one cell and 81.1 in another cell
    When I do a calculation of 81.1 - 80.9, the result I get is 0.199999999999989
    81.1
    80.9
    0.199999999999989
    Why is there a rounding error on numbers with 1 decimal place?
    This makes me not trust anything that Numbers does if it can't even do that!

    Mason,
    this topic has been addressed in many other threads and is a result of the way numbers are stored on a computer.  the difference between the value displayed and the correct value is 0.2 - 0.199999999999989 which is 0.000000000000011 
    all computer programs use integer of floating point representations of a number-- see this link for details:
    http://en.wikipedia.org/wiki/Single-precision_floating-point_format
    you can format the number with two digits using the cell formatter and you will see the "correct" answer:

  • Notice days Calculation Error.....

    Dear Experts.....
    I am facing the problem that when i run the leaving action and give the notice period salary to employee system will generate wrong calculation. If i didnot run the leaving action then amount is rightly calculated. Plz find the below RX table entries...
    /101 Total gross                                                    23,885.00
    /110 Net payment                                                        23.00-
    /550 Statutory n                                                    23,885.00
    /557 Cash Paymen                                                    23,862.00
    /560 Payment Amo                                                    23,862.00
    /700 Wage/salary                                                    23,885.00
    /840 Diff. to av01                                   9.00
    /840 Diff. to av02                                   9.00
    3 /001 Valuation b01                                                   6,970.00
    3 /001 Valuation b02                                                   6,970.00
    3 0001 Basic Pay  01                                                   1,441.00
    3 0002 House Rent 01                                                     649.00
    3 0003 Utility    01                                                      72.00
    3 0004 Medical    01                                                      86.00
    3 1001 Attendance 01                                                      30.00
    3 6001 EOBI Eploye01                           10.00                      23.00-
    3 6002 EOBI Employ01                           10.00                     114.00
    3 7002 Notice Peri01                                  31.00           21,607.00
    Employee salary is 7000/-, and notice days are 31 mean one month salary but see the above calculation amount. I have tried PRCL 20 also, but no effect.
    Below check the PCR calculation in Payroll, here system is not calculating anything.
    3 /001 Valuation b01                                                   6,970.00
    3 /001 Valuation b02                                                   6,970.00
    3 0001 Basic Pay  01                                                   4,468.00
    3 0001 Basic Pay  02                                                   4,468.00
    3 0002 House Rent 01                                                   2,011.00
    3 0002 House Rent 02                                                   2,011.00
    3 0003 Utility    01                                                     223.00
    3 0003 Utility    02                                                     223.00
    3 0004 Medical    01                                                     268.00
    3 0004 Medical    02                                                     268.00
    3 1001 Attendance 01                                                      30.00
    3 6001 EOBI Eploye01                                                      23.00-
    3 6002 EOBI Employ01                                                     113.00
    3 7002 Notice Peri01                                  31.00
    Please help me, how to remove this error.

    Abhihash -
    Your suggestion worked good, thanks! Thank you too Naga.
    One more question please. I have grouped on {Asgnmnt.GroupName} for the different departments and have added this code to the details section and suppressed it to remove the duplicates...
    Here's the code....{CallLog.CallID} = Next( {CallLog.CallID})
    My question: Is this the best way/method to remove duplicates?
    I also going to add a Summary to count the days for each group - should I use 'Count' or 'Distinct Count' to get an accurate count of the days?
    Thx.
    G.

  • Get "Error in Net Price Calculation" error while creating a Purchase Order

    I am getting a error when i try to create a PO. Says "Error in Net Price Calculation item 00010(Please correct)"
    I have defined conditon records with the price for the material  and have also created info record.However the price in the info record (including currency i.e USDN is different than the one defined in the conditon records currency is USD).
    However the price is different from the one defined in info record and in material master.
    Where is it picking the price from also is there any way to edit the price in material master or info record ?
    Also when i view the condition records by using Tcode MEK1 i get the following error
    "Access Pricing Purchasing Z00q 20 contains no fields"
    I know Z001 is the access sequence
    Can any one tell me wht is this error message about ?
    Edited by: Dspsac on Jan 29, 2008 4:31 AM

    how can u maintain the condition record as well as info record for a vendor -material in the same time frame
    it is not possibe
    as info record is also 1 of the condition table
    if u have maintained the info record then system is going to fetch the data from there but point top notice is it depends on the validity of record
    system fetch the price from info record
    and the currency used is fom vendor master

  • EBSTax calculation error at the time of PO creation

    At the time of any purchasing document creation, like PO or any type of requisition, system is showing error telling, an error occured during tax calculation.
    Now, here we are not using EBSTax, so no setup related to tax has benn touched upon. I am currently configuring the Test instance R12. Can anybody suggest
    me the solution?

    Posting the exact error message is a lot more helpful.
    Check Unexpected Error Occured During Tax Calculation [ID 472911.1]
    Sandeep Gandhi

  • Cost calculation error while saving the activity in network

    Hi Experts,
    We have created internal activty in the network at the time of saving it is giving an error.
    "Error while calcualting Cost,log is deleted when saving do you want to save?".
    When I checked the log through extraslogscost calculation in CJ20N,it is showing following reasons
    1) Missing formula in work center
    2) Operation 0020/suboperation(or cost comp):activity type not entered.
    3) No control record for activity type in version 000/2009 activity planning/qty planning.
    could you please help me in this?
    Regards,
    Abdul Rehman

    In your work center, use "SAP008" formula in the costing tab.
    Copy the activity type you have used in the work center (again in costing tab) & use this activity type in your project internal activity. Use the same unit of measure in your project internal activity, that you have defined in the activity type.
    Define correct rates in KP26 for your activity type.
    It should work now.
    Good luck.

  • Time Machine Backup Size Calculation Error

    This is what I have:
    Internal:
    Mac OS partition: 266GB (257GB Used)
    BootCamp partition: 32GB ( 12GB Used)
    External:
    External USB 2.0 HD: 153GB (142GB Used)
    TM Backup Drive: 466GB ( 0GB Used)
    *What I have excluded:*
    - TM Backup Drive
    - BootCamp partition
    *Therefore what I'm trying to backup are only:*
    - MacOS partition
    - External USB 2.0 HD
    Time Machine Preferences Options Pane reports "Total Included: 398.4GB", and that's totally correct for what I'm intending to backup (i.e. MacOS partition + External USB 2.0 HD = 257GB + 142GB = 399GB). So far so good.
    But when TM kicks off after the countdown, an error dialog pops up saying:
    *Time Machine Error*
    This backup is too large for the backup volume. The backup requires 478.0 GB but only 465.5GB are available.
    To select a large volume, or make the backup smaller by excluding files, open System Preferences and choose Time Machine.
    [Preferences] [OK]
    Now how does TM figure out that 478.0GB number?
    If I add up all used spaces for all drives (excl. TM drive), I only get 411GB.
    If I add up all capacity of all drives (excl. TM drive), I only get 451GB.
    How in the world did TM calculate that 478.0GB number? Why is there a discrepancy between what's reported in the options pane and the error dialog?
    BTW, I have verified all my disks and they are all fine.
    All my drives and partitions (except the bootcamp partition) are formatted properly to Mac OS Extended (Journaled).
    Someone please advice.
    Much appreciated.

    Disregarding the Mac/Win partitions, my internal HD as a whole is 298GB. My external USB drive is 153GB.
    nestea247,
    Your computer info says : iMac Aluminum 24" ....... 320GB HD.
    So, I assumed your internal HD is 320GB and you are currently using 298GB of it. Therefore my simple calculation was not "298GB + 153GB" but "320GB + 153GB(ca.)". Did I misunderstand?
    My point was: For the first-time backup, in order to copy the entire image (including the *free space*) of the volume to backup, TM might think it needs about the same size of backup volume as your internal HD ( + external USB drive, in your case). Not simply adding up the size of the *used space*.
    Leter, however, it will get more realistic and adjust to the actual data size; so, after completing all the hard link setup, the size of its backup seems to "shrink." Your TM could be still calculating.... perhaps struggling to adjust? (I too hope it is not a critical bug.)
    I'm afraid my reply doesn't help solve the mystery of 478.0GB. Hope someone else can.
    Best of luck!

  • Shipment costs not yet calculated  error in vt01n

    When i try to create individual shipment in vt01n to check estimate cost there are shown error like this : shipment costs not yet calculated.
    Note : I have set TK11
    Please Help Me.
    Edited by: Dani Herdian on Jun 24, 2010 9:54 AM
    Edited by: Dani Herdian on Jun 24, 2010 10:11 AM

    Yes, this quesiton is answered

  • Production order Target cost Zero - Varience calculation error KV151

    Hi Experts,
    I am having an issue on KKS2 variance calculation, system throwing error KV151 Target cost is 'Zero'
    All pre-requisites are ok including BOM, Rates, Routings etc..
    Process:
    Production order with FERT - valuated material,
    All BOM components in production order are Non valuated stock items, Non-stock items
    Note: Cost Estimat e is for Labor and Overhead only.
    After production order confirmation CO11N, production order Cost analysis displays - Target value ("Zero") and actual value.
    Due to above reason in KSS2 we are unable to calculate variance, system displayed error KV151.
    I cannot settle the production order, guide if I am missing some thing in config. ?
    I appreciate you quick response.
    Thanks,
    RK

    Thanks for your reply Swapvik & Takashi, I have no issues on configuration side very thing looks good.
    Cost estimate - Good
    Variance key - Good
    Production order - FERT (Valuated) - good
    BOM components - UNBW, NLGS only - Good
    Labor rates - good
    Periods - good
    Production order confirmation - Good
    Actual cost - Good
    My Issue is Target cost is zero on cost analysis in CO03 but if I can see target cost in KKBC_ORD report.
    I am unable to calculate variance and settlement.
    I appreciate your response.
    Thanks,
    Raj

  • COFC - Actual cost calculation error

    Hi Friends,
    I am a PP consultant and i have a issue with cost calculation at the time of confirmation.
    There are production orders that were confirmed in year 2010 and year 2011. For year 2010 costing documents were created but not no costing docuemnts for confirmations in year 2011.
    I checked tcode COFC and found error log as : A master record for 1111/ABCD exists only in 2010.(where 1111 is cost center and ABCD is activity type.)
    I have checked the validaty of both cost center and activity type & both are valid.
    can anyone please suggest what has to be maintained for year 2011 from controlling side.
    Thanks In Advance.
    Rohit

    Hi Rohit,
    The similar error i have faced  just couple of hours back.
    I have checked up the plan prices in KP26 for the activity type and could see that they were not maintained for 2011.
    I have maintained the same for the current year and the error got resolved.
    Pls see that you are choosing the correct cost centre and activity type combination alongwith the costing version for 2011.
    Regards
    Sachin

  • Hp10b11+ calculator error message "running" when using INPUT and Shifted AMORT keys- why?

    above describes the issue. Any ideas why? I've tried resetting methods, batteries removed,etc.

    Thanks, Yes I understand that the calculator message "running" should be a brief event. It's definitely not. Many minutes can go by and no change.  And the display does not show what my course manual states should be coming up.
    Its a mortgage calc. using the following sequence:
    KEYS USED:                                        DISPLAY:
    6 I/YR                                         6              
    12  shift P/YR                          12
    60000 PV                                  60,000
    240 N                                          240
    0 FV                                             0
    PMT                                           -429.858635087
    then rounding payment up:
    429.86 +/- PMT                      -429.86
    N                                                    239.998529245
    36INPUT shift AMORT          PER 36-36                     *This is what is supposed to display, but this is where "running"
                                                                                                        displays and does not solve.  I left the calc. alone for a very long time.
    So I have removed both batteries, and tried to run different clearing options that the manual outlines. Nothing is mentioned about the functon of the INPUT key or what to do with this error. The calculator had done numerous other functions without a problem, so I am wondering if there is some value stored that creates a long running calculation, or if there is indeed a glitch with the actual calculator that presents itself with these calculations. 

  • Net Price Calculation Error in LSMW(Direct Input)

    Hi,
    i am using LSMW-Direct Input method for Creating Purchase Order .i am facing Problem like Quantity Conversion error in Net Price Calculation.
    Any one please help me out.
    Regards,
    Shobha Henry

    Best practice would be to use BAPI/ALE method instead of direct input but if you've already done with the mapping etc. try putting a break point in the generated LSMW program where the net price is calculated and transferred to SAP field and then run it to see in the debugger what's going on. I reckon its possibly a type compatibility error.
    Hope this helps,
    Cheers,
    Sougata.

Maybe you are looking for