Simple Math Calculation is incorrect

I have seen this topic here before but cant find it. When I
do the following script it returns an incorrect value;
var myNumber = 999.43
trace(myNumber-999) //returns: 0.42999999999995
So how can I retrieve the value .43? I suppose I can do this;
var myNumber = 999.43
var hundred = myNumber*100;
var whole = hundred-99900;
trace(whole*.01) // returns: .43
But is there a better way? And why is it that Flash is
calculating like this?

the reason for the apparent error is because flash is
performing arithmetic on a binary computer. within the limits of
flash'es accuracy (16 digits),
0.42999999999995 = .43
to eliminate those errors, you can use integer arithmetic or
the flash string methods. integer arithmetic is probably preferable
if you're doing arithmetic operations.

Similar Messages

  • Simple math calculation in numbers

    Hi I'm trying to do a simple calculation of 46.66*2 in numbers and i get a red triangle saying syntax error. Any light on this as what is wrong.

    Hi Fuija,
    Happy to help, and strangely enough, I just got a fresh cup of coffee.
    Most issues are easier to solve when enough information is presented. Keep that in mind when you have another question.
    As to using , as the decimal separator, it's not a Mac thing. That's dependent on where you live and what regional settings you've made in System Preferences. If you'd normally write ten thousand four hundred twenty-three and five hundredths like this:
    10,423.05
    Then your regional settings for numbers should show 1,234.56
    and you should use a period as the decimal separator.
    If you'd normally write that number this way:
    10.423,05
    Then your regional settings for numbers should show 1.234,56
    and you should use a comma as the decimal separator.
    Pick whichever region is set to use the same separators as you would use when writing the number with pencil on paper.
    Regards,
    Barry

  • Dashboard Calculator suddenly incorrect (simple division)!

    I've relied on the Dashboard calculator for years, and now suddenly it's spitting out incorrect errors.  I've seen posts on the boards regarding order of operations, but this is messing up simple math.  I'll give you some examples of the answers it's spitting out:
    15 / 1 = 55,808
    15 / 2 = 27,904
    15 / 3 = 18,602.6667
    3 / 1 = 55,808
    3 / 2 = 27,904
    3 / 3 = 18,602.6667
    I've cleared it out (C key) to no avail.  There seems to be a pattern to the answers above.  Everything works fine for addition, subtraction and multiplication.  I only get the erroneous answers with division.  Can anyone shed some light on this issue?  Thanks!

    I've relied on the Dashboard calculator for years, and now suddenly it's spitting out incorrect errors.  I've seen posts on the boards regarding order of operations, but this is messing up simple math.  I'll give you some examples of the answers it's spitting out:
    15 / 1 = 55,808
    15 / 2 = 27,904
    15 / 3 = 18,602.6667
    3 / 1 = 55,808
    3 / 2 = 27,904
    3 / 3 = 18,602.6667
    I've cleared it out (C key) to no avail.  There seems to be a pattern to the answers above.  Everything works fine for addition, subtraction and multiplication.  I only get the erroneous answers with division.  Can anyone shed some light on this issue?  Thanks!

  • Simple Graphing Calculator with 2D array

    Hello,
    I am building a simple graphing calculator which reads in coefficients from a txt file. All is going well except when I plot the points, the graph is at the corner of the screen and sideways. As we all know, the origin of the array is at the top right corner, so how do I transform/slide it so that it is at the xy origin? Code is posted below, thanks!
    import javax.swing.*;
    import java.awt.*;
    public class baseClass extends JFrame{
         private shapesPanel panel;
         public baseClass(String title){
              panel = new shapesPanel();
              setTitle(title);
              setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
              getContentPane().add(panel);
              pack();
              setVisible(true);
         public void drawLine(int x1, int y1, int x2, int y2){
              panel.addLine((short)x1,(short)y1,(short)x2,(short)y2);
              panel.repaint();
         public void drawDot(int x, int y){
              panel.addOval((short)x,(short)y,(short)1,(short)1);
              panel.repaint();
         private class shapesPanel extends JPanel{
              final private short max = 10000;
              private short[][] lines, ovals;
              private int numLines, numOvals;
              public shapesPanel(){
                   setBackground(Color.white);
                   setPreferredSize (new Dimension(500,500));
                   lines = new short[max][4];
                   ovals = new short[max][4];
                   numLines = 0;
                   numOvals = 0;
              public void paintComponent(Graphics g){
                   super.paintComponent(g);
                   g.setColor(new Color(0,0,255));
                   for(int i = 0; i < numLines;++i){
                        g.drawLine(lines[0],lines[i][1],lines[i][2],lines[i][3]);
                   g.setColor(new Color(255,0,0));
                   for(int i = 0; i < numOvals;++i){
                        g.drawOval(ovals[i][0],ovals[i][1],ovals[i][2],ovals[i][3]);
              public void addLine(short x1, short y1, short x2, short y2){
                   lines[numLines][0] = x1;
                   lines[numLines][1] = y1;
                   lines[numLines][2] = x2;
                   lines[numLines][3] = y2;
                   ++numLines;
              public void addOval(short x, short y, short w, short h){
                   ovals[numOvals][0] = x;
                   ovals[numOvals][1] = y;
                   ovals[numOvals][2] = w;
                   ovals[numOvals][3] = h;
                   ++numOvals;
    import java.util.Scanner;
    import java.io.*;
    public class graphingCalculator extends baseClass{
         // CONTAINS THE FOUR FUNCTION COEFFICIENTS
         private double[] theFunction;
    // CONTAINS THE DISCRETE FUNCTION PLOT
    private boolean[][] grid;
    // SIZE OF THE COORDINATE SYSTEM
    private final int columns = 500, rows = 500;
    public graphingCalculator(String filename) throws IOException{
         // INITIALIZATIONS ///////////////////////////////////////
    super("GRAPHING CALCULATOR");
    theFunction = new double[4];
         grid = new boolean[rows][columns];
    for(int i = 0;i<rows;++i){
    for(int j = 0;j<columns;++j){
    grid[i][j] = false;
    // DRAW THE COORDINATE SYSTEM ON THE SCREEN /////////////
    drawLine(0,250,500,250);
    drawLine(250,0,250,500);
    for(int i = 0; i < columns;i+=20){
    drawLine(i,247,i,253);
    for(int i = 0; i < rows;i+=20){
    drawLine(247,i,253,i);
    // GET THE FUNCTION COEFFICIENTS ///////////////////////
    // theFunction[0] will have the x^3 coefficient
    // theFunction[1] will have the x^2 coefficient
    // theFunction[2] will have the x^1 coefficient
    // theFunction[3] will have the x^0 coefficient
    Scanner scan1 = new Scanner(new File(filename));
    for(int i=0;i<4;++i){
         theFunction[i] = scan1.nextDouble();
    scan1.close();
    // DRAW THE FUNCTION ////////////////////////////////////
    computeGrid();
    drawDiscrete();
    drawContinuous();
    private double computeFunction(int x)
         double a=theFunction[0];
         double b=theFunction[1];
         double c=theFunction[2];
         double d=theFunction[3];
         double answer=(a*Math.pow(x,3.0)+b*Math.pow(x,2.0)+c*Math.pow(x,1.0)+d*Math.pow(x,0.0));
         return answer;
    private void computeGrid()
              // Populate the 'grid' array with true values that correspond
              // to the discrete function.
              // The 'grid' array is the internal representation of the function
         for(int x=0; x<columns; x++)
                   if(computeFunction(x)>=0&&computeFunction(x)<500)
                   grid[(int)computeFunction(x)][x]=true;
    private void drawDiscrete()
              // Use drawDot(x,y) to draw the discrete version of the function
         for(int x=0;x<columns;x++)
              for(int j=0;j<rows;j++)
         if(grid[x][j])
              drawDot(x,j);
    private void drawContinuous()
              // Use drawLine(x1,y1,x2,y2) to draw the continuous version of the function

    Rest of code:
    import java.util.Scanner;
    import java.io.*;
    public class Main {
        public static void main(String[] args) throws IOException{
        String infile;
        Scanner scan = new Scanner(System.in);
        // The input file should have 4 doubles in it
        // corresponding to the 4 coefficients of a cubic function.
        // A GOOD EXAMPLE:
        //                  0.01
        //                  0.0
        //                  -5.0
        //                          50.0
        // CORRESPONDS TO FUNCTION: 0.01*X^3 - 5.0*X + 50.0       
        // ANOTHER GOOD EXAMPLE:
        //                  0.0
        //                  0.01
        //                  -1.0
        //                          -100.0
        // CORRESPONDS TO FUNCTION: 0.01*X^2 - X - 100.0       
        System.out.println("Please enter filename of input file: ");
        infile = scan.next();
        graphingCalculator cal = new graphingCalculator(infile);
    }

  • PO with Total Price amount calculated is incorrect at the time of printing.

    Hi Experts,
    I have one PO with Total Price amount calculated is incorrect at the time of printing.
    When my  user took the first print out the PO shows the incorrect Total amount.
    After 10 mins my user taken the printout again,that time it shows the Toatal Price value is correct amount.
    How i can analysis this type of BUG..!!!
    I am thinking it might be BUG...???
    Please advice with you valuable inputs.
    Best Regards,
    RK

    Hi,
    Are you using the standard programm and print form or are you using your own?. Most of the printing issues happens because of own programms. Try with the standard and then you will be able to see why is this happening.
    Best Regards,
    Arminda Jack

  • I am using Numbers on my iPhone5 and cannot get the app to do a simple (SUM) calculation.  It shows the formula correctly in the cell, but all I get for my long list of numbers to add is 0.  How can I get this to work?

    I am using Numbers on my iPhone5 and cannot get the app to do a simple (SUM) calculation.  It shows the formula correctly in the cell, but all I get for my long list of numbers to add is 0.  How can I get this to work?

    Oaky, at least we got that far.  Next step is to determine if all those "numbers" are really numbers.  Changing the format to "number" doesn't necessarily change the string to a number. The string has to be in the form of a number.  Some may appear to you and me as numbers but will not turn into "numbers" when you change the formatting from Text to Number. Unless you've manually formatted the cells to be right justified, one way to tell if it is text or a number is the justification. Text will be justified to the left, numbers will be justified to the right.
    Here are some that will remain as strings:
    +123
    123 with a space after it.
    .123

  • Creating a form with a field that contains a simple math problem.

    I am creating a form in Acrobat 9 Standard that contains a field that requires a simple math function (divide).
    if (QuantityRow1 > 0)
    ExtensionRow1 / QuantityRow1
    Take the value input in ExtensionRow1 and divide it by the value in QuantityRow1. I have included an IF statement to prevent an error that occur if QuantityRow1 were equal to zero.
    After inputting the data into the two fields I tab past the field where I expect to see my result and the field remains blank. I don't receive any error messages.
    Any ideas? Thanks - jb

    This is a duplicate of your question on Acrobatusers.com (
    http://answers.acrobatusers.com/Need-simple-math-operation-function-Acrobat-9-Standard-cre ating-form-q142800.aspx
    Did the information provided there not help you with your problem?
    Karl Heinz Kremer
    PDF Acrobatics Without a Net
    PDF Software Development, Training and More...
    [email protected]
    http://www.khkonsulting.com

  • Simple  Fact Calculation Help

    Hi Team
    I need help in creating a simple fact calculation. I need to create a fact column that should give the lastest transaction amount , if the type of the transacion is "DEBIT". for example if there 100 transactions are there, and out of that only 25 are DEBIT Transaction, I need the amount of the last (Most recent date's) DEBIT Transaction.
    Please help me, what should be the calculation and the aggrecate setting of that fact column.
    Our version is OBIEE 10.1.3.4.1
    Thanks a lot for the help
    With Regards
    Bas

    The analytics function that yo search is last.
    Check here:
    http://gerardnico.com/wiki/dat/obiee/last_first
    Try to apply the last on only one dimension (ie time dimension) otherwise it can take a lot of time. Check the bad performance section of the article.
    Cheers
    Nico

  • About math calculation in form command

    i want to know  if  we can do math calculation in script form's command line?i try it by using define command ,but cann't get the result,i have to make sure if the command line have the function of math calculation?

    do ur calculations inside a subroutine and call ur subroutine in ur script.
    check this wiki on how to use subroutines inside scripts
    https://www.sdn.sap.com/irj/sdn/wiki?path=/display/profile/external%2bsubroutines%2bin%2bsap%2bscript

  • Calculated Column with simple math for price comparisons

    Hi all,
    I want to calculate a price difference from today's price versus the price of 5 days ago.
    Please help me with a best practice for that. Is the use of calculated column in a calculation view the right approach? How can the price field of 5 days ago be addressed? Ideally 5 should be a constant which should be able to be changed to 6 or 7 for further simulations.
    Thanks for any help.
    Guenther

    Hi Patter ,
    For the above requirement we don't need a calculation view to be created. Check the code below, I am re creating the scenario with an INNER JOIN.
      CREATE COLUMN TABLE "METALS_ONE_GEN"."TEST_TAB" (ID INTEGER,DATES DATE,PRICE INTEGER);
      INSERT INTO "METALS_ONE_GEN"."TEST_TAB" VALUES ( 1 , '2014-03-10',50);
        INSERT INTO "METALS_ONE_GEN"."TEST_TAB" VALUES ( 1 , '2014-03-15',50);
        INSERT INTO "METALS_ONE_GEN"."TEST_TAB" VALUES ( 1 , '2014-03-25',100);
        INSERT INTO "METALS_ONE_GEN"."TEST_TAB" VALUES ( 1 , '2014-03-20',80);
        INSERT INTO "METALS_ONE_GEN"."TEST_TAB" VALUES ( 1 , '2014-03-30',75);
        INSERT INTO "METALS_ONE_GEN"."TEST_TAB" VALUES ( 1 , '2014-04-04',150);
        INSERT INTO "METALS_ONE_GEN"."TEST_TAB" VALUES ( 1 , '2014-04-9',450);
    select A.DATES,
           A.PRICE,
          ( B.PRICE - A.PRICE )
           from "METALS_ONE_GEN"."TEST_TAB" A
           INNER JOIN "METALS_ONE_GEN"."TEST_TAB" B
           ON
        B.DATES = ADD_DAYS(A."DATES",5) ;
    Looks like, the JOIN operation is more optimized than the Calculation view in this case . Depends on the data , JOIN type may very .
    Sreehari
    Message was edited by: Sreehari V Pillai

  • Simple Math Actions ( float, double... )

    Hi guys, does anybody could help me with those
    simple calculations ?
    Take a look at next code:
    public class CMainApp {
    public static void main(String[] args) {
    float f1, f2 ;
    f1 = 1.2f ;
    f2 = 3f ;
    f1 = f1 * f2 ;
    // Why returned value couldn't be
    // 3.6 ???
    System.out.println( f1 ); // PAY ATTENTION !!! : 3.6000001
    }//main
    }//CMainApp
    For some reason I got strange answers 3.6000001. It should be 3.6.
    Is there any way to deal with this problem ?
    Thanks a lot, Mark. If you could, please mail me at
    mailto:[email protected]

    JohanUP, first thanks for respond.
    But you solution dosn't serve my needs.
    // This code line will print you 4
    // but I wants my 3.6
    System.out.println( Math.round( 1.2 * 3 ) );
    Now I'm trying some tricks with NumberFormat, If I'll
    success I'll let you guys to know.

  • Simple Math in SQL Query or cfset

    Calculated fields in the DB vs <cfset>?
    It would seem simple, but:
    ACCESS DB ;>(
    CF9
    IIS
    <cfquery name="get_total" datasource="#Request.BaseDSN#">
    SELECT SUM((current_charge+penalty)-credit) AS total
    FROM moorings
    WHERE mooring_ID = #URL.mooring_ID#
    </cfquery>
    <cfdump var="#total#"> 
    Variable TOTAL is undefined?
    or
    <cfset subtotal = current_charge+penalty>
    <cfset total = subtotal-credit>
    Variable CURRENT_CHARGE is undefined?
    Missing ',", (, #'s?
    Thanks

    Run the query without using SUM to see what values are stored in the database:
    <cfquery name="get_total" datasource="#Request.BaseDSN#">
    SELECT current_charge, penalty, credit
    FROM moorings
    WHERE mooring_ID = #URL.mooring_ID#
    </cfquery>
    <cfoutput>
    charge: #get_total.current_charge#<br />
    penalty: #get_total.penalty#<br />
    credit: #get_total.credit#<br />
    </cfoutput>
    or
    <cfdump var="#get_total#">
    Also, it's a good idea to get into the habit of using cfqueryparam in your SQL queries. It helps sanitize your inputs and prevents SQL injection attacks:
    <cfquery name="get_total" datasource="#Request.BaseDSN#">
    SELECT current_charge, penalty, credit
    FROM moorings
    WHERE mooring_ID = <cfqueryparam value="#URL.mooring_ID#" cfsqltype="cf_sql_integer">
    </cfquery>

  • Can someone help me with a simple duration calculation?

    I created a simple workbook for calculating hours worked. I am entering the start time, end time, and using the DUR2HOURS formula to calculate the duration. For some reason the calculation is wrong. I am sure it is somethign simple, but I could not find an answer and it has been bothering me for a couple of days now. Here is an example of what is happening:
    Start Time     |     End Time     |     Duration
    7:15 AM               5:30 PM               10.25  (formula: =DUR2HOURS(ENDTIME - STARTTIME)
    The duration in this example should be 10.15?
    Thanks,
    Nate

    Hi NorthrailDrive,
    DUR2HOURS gives the value in decimal hours, not hours:minutes.
    Your answer (in decimal hours) is correct (10.25, or ten and a quarter hours).
    Cell C2 contains the formula you need to calculate a duration:
    =B2-A2
    C2 is formatted as Duration h (hours) m (minutes). You can change the display to show 10:15
    Regards,
    Ian.

  • Simple percentage calculation

    Hello,
    i am new to Numbers...i need to do something i think is very simple, but still can't figure out how to...
    Basically i need to have a calculation of percentage.
    Say in cell B2 i have a certain value, i would like to have cell C2 that calculate B2 value + the 20% of B2 value.
    Would it be possible? Which formula should i use?
    Or would i need to set C2 only as percentage value (20%), then set a SUM (B2+C2) formula in D2?
    I've actually already tried all this, but i must make some formula editing errors...
    Appreciate your help.
    Max

    Either if these will do the job:
    =B+B*0.2
    =B*1.2
    "20 percent" is just a 'fancy' way of saying "20/100", and "20/100" expressed as a decimal is "0.20"
    =B+B*20%
    This will also work, but you will need to set the cell's format to display the result as a number. Set to Automatic, the cell will choose Percent to display the result.
    Regards,
    Barry

  • Simple Sum Calculation in Forms

    I have a seriously simple situation, alas when I try to set it up, I am only able to select one field to calculate?  I am working in Adobe X.  I simply want to calculate the sum of 5 fields into a "Total" Text Field... 

    Hi miaff,
    I think this tutorial is exactly what you're looking for: http://acrobatusers.com/tutorials/how-do-i-use-basic-calculations-in-a-form
    Please let us know if you still have trouble!
    -David

Maybe you are looking for

  • Can't sign up for play now

    Everytime I enter in my phone number it tells me Invalid phone number. Please try again and remember the country code. When I select Canada as my country it says Example: +1613123456789 I enter in my phone number with +1 then my city area code then t

  • I have a iq820 uk touch smart is it possable to upgrade the processor

    is it possable to upgrade my processor if so what will i need please

  • Getting back original image

    Hello. I am using Labview 2010 version to develop a program to load an image and process them using threshold.  I would like to compare the original image with the threshold image. Can anybody help me to show me some guides how to find back the origi

  • Where to find WebLogic Server credentials

    Hi I have installed Oracle WebLogic Server Version 10.3.1.0 and it is running fine. I'd like to know where (in which DB table) can I find usernames and passwords in the DATABASE. On the tab Security Realms I can see that I have users listed. Rrgds Ko

  • HT201210 Restore issues iphone 3G

    Hello I followed the advice of this article http://support.apple.com/kb/HT1808 and I managed to restore my iphone via iTunes without an error. Once the restore was done the apply logo came on the screen, some progress bars appeared and then nothing,