Stumped on basic problem with floating point math

I can't figure this out! It should be sooooo simple.
Here is the challenge:
I have an incoming time array. For example: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
I want to scale this array by a constant (e.g. multiply by 0.1). So the resulting array should be: 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9
Then I want to calculate the difference between each and every subsequent element in the array. In this example, the difference should 0.1 between every element. But my comparison fails.
See the example below. As far as I can see, the resulting boolean should always be TRUE. But its not.
But if I remove the scaling operation, then it works ok!
Please help!
Solved!
Go to Solution.

It has been awhile since smercurio has had a contribution to his retirement fund, but once again you have "discovered" that there is no exact binary representation for 0.1.
I'd use one of the current "almost equals" comparisons described here:
http://forums.ni.com/t5/LabVIEW/Darin-s-Weakly-Nugget-2-8-11/m-p/1444262
And vote for this if you haven't already:
http://forums.ni.com/t5/LabVIEW-Idea-Exchange/quot-Almost-Equal-quot-functions-for-Float-comparisons...

Similar Messages

Problem with floating point numbers

Hi Experts,
I am accepting two float values thru a method :- amount and tenure.
Now wat i want to do is
target = amount / ( tenure * 12 );
but result target is not what it supposed to be. For example:-
if amount = Rs. 15 00 000.0 and tenure = 3 yrs, target amount should be 5 00 000. but it is coming 41666 sumthing.
How should I code this?
Pls help,
regards,
Amey

Hi,
I am accepting two float values thru a method :- amount and tenure.
Now wat i want to do is
target = amount / ( tenure * 12 );
but result target is not what it supposed to be. For example:-
if amount = Rs. 15 00 000.0 and tenure = 3 yrs, target amount should be 5 00 000. but it is coming 41666 sumthing.
How should I code this?
I dont see any error in calculation,
15 00 000.0 / 36 is perfecctly 41666 sumthing
how can you expect this to be 5 00 000?
If you want that change your calculation from
target = amount / tenure ;
Regards
Ayyapparaj

Problem with floating point precision

Hello!
I need to store in a variable a number like this 1234512345.123412345.
But when I store it on a double or long double... it gets truncated to 1234512345.123412... only six decimals?
What can I do??
Thanks!

If you are going to use the BigDecimal class, it is worth bearing in mind a tip that was passed on to me from georgemc. If you create the BigDecimal class instances using the constructor that accepts a String as input, then you can easily specify how many digits should follow the decimal point.

Problem with decimal point in Korea and Japan

Hi all,
I have one problem with decimal points.
I need to change in my out put cuurency values as per japan or korea standards.
Please help me is there any function module for this one.
program using alv grid display.
i use write statement but it is not working. Because
data: L_AMT type glpca-hsl,
         G_AMT TYPE CHAR15.
CURR this value is 'JPY'.
WRITE L_AMT CURRENCY CURR TO G_AMT.
here i need to do
L_AMT = G_AMT.   "" going to runtime error.
if i taken like
data: L_AMT TYPE CHAR15,
         G_AMT TYPE CHAR15.
CURR this value is 'JPY'.
WRITE L_AMT CURRENCY CURR TO G_AMT.   "" it is not working
Regards
margani

Hello,
This is pretty basic, whenever you've currency & quantity fields you need to pass the corres. reference fields.
In your case you need to populate the cfieldname & ctabname fields of the FieldCat.
BR,
Suhas

So... if there's no floating point math, how do all those calculators work?

I see there's no floating point math support in J2ME.
So how do those calculators work? There's lots of 'em for download.
I have an app that's almost done; all I need to do is finish the calculations, but I need to read Strings from TextArea's and compute values based on them.
What am I missing?

You can use third party libraries
(MathFP http://www.jscience.net)
or
implement them with classical algorithms.
Carlos Sanchez
[Intesys]

Problem with Vanishing Point

hello!
I've got a problem using Vanishing Point filter in Photoshop CS4. When I load this filter an error message pops saying "The operation could not be completed"
I attach a screen cap of the error. Somebody can help? thanks.

I run into this problem with Vanishing Point on about a quarter of the files I run the filter on. I can initially create a grid and do one transform, but the next time I try to run the filter I get the error, "The operation could not be completed". Then the error seems to stay attached to the file. I can run the filter on another file, but even if I close and reopen the file, I still can't run the filter. Even doing Save As and renaming the file doesn't rid it of the error.
This has been the case since VP was introduced. I'm now on CS4. MacPro Dual-Core, 8GB Ram.
I've tried changing the memory allocation to 55% with no change.
I see that this has been addressed before in the Forum with no solution, Chris Cox care to wade in?
This is a potentially valuable tool if it were implemented better.
Craig

Inconsistent Floating Point Math and NaNs on Windows Laptops?

All -
I am seeing some very strange inconsistent floating point calculations on Windows Laptops, and I am wondering if anyone has any ideas. Read on, as (to me!) it's very interesting....
I have attached a segment of our code, along with some sample output. Looking at the code and the output, it seems like it's totally impossible.
With supposedly non-NaN and non-infinite double values, I am seeing unrepeatable and inconsistent math - the below example only illustrates one such case where we're seeing this behavior.
If you look at the code below, you will see that I do things like:
double rhoYo = ... // some math
double rho = ... // exact same mathStrangely enough, I might get rhoYo = 1.51231231 etc and rho = NaN.
If I reverse those lines (vertically), then again, rhoYo comes out good and rho comes out NaN; however, this is unpredictable and inconsistent. If I project a source point, get a destination point with NaNs as a result, and project the source again, the second destination point may be just fine. Matter of fact, i can put a loop in the code such as:
      double rho = Double.NaN;
      for( int i = 0; i < 10; i++ )
        rho = my_earthRad * my_F / Math.pow(Math.tan(Math.PI/4.0 + latRad/2.0), my_n);
        if( ! Double.isNaN( rho ) )
          break;
        System.out.println("NaN'ed rho");
      }and rho will eventually become non-NaN (random # of iterations)!!
How's that possible? Our code might be tromping on memory somewhere, but this sure seems crazy to me, especially considering that
we're only using local variables. Anyone know of floating point errors on Windows Laptops?
With the exact same codebase, this behavior ONLY happens on Windows Laptops, including brand new Dells, old Dells, IBM, Intel and AMD chips (I've tried several ;-). It does NOT happen on Mac or Linux, including the Linux side of a Linux/Windows dual-boot (where it does happen with the Windows side). Even more strangely, it does NOT happen with Windows desktops. I have tried several 1.5.x JVMs, webstart vs no webstart, etc, to no avail. Always and only on Windows Laptops.
Please help.... ;-) and thanks in advance.
Sample code:
public class Projection
protected Point2D.Double _project(Point2D.Double srcPt, Point2D.Double dstPt) {
    final double my_degToRad = Math.PI / 180.0;
    final double my_originLon = -95.0;
    final double my_originLonRad = my_originLon * my_degToRad;
    final double my_originLat = 25.0;
    final double my_originLatRad = my_originLat * my_degToRad;;
    final double my_stdLat1 = 25.0;
    final double my_stdLat1Rad = my_stdLat1 * my_degToRad;
    final double my_earthRad = 6371.2;
    final double my_n = Math.sin( my_stdLat1Rad );
    final double my_F = Math.cos( my_stdLat1Rad ) * Math.pow( Math.tan( Math.PI / 4.0 + my_stdLat1Rad / 2.0 ), my_n ) / my_n;
    final double my_rhoZero = my_earthRad * my_F / Math.pow( Math.tan( Math.PI / 4.0 + my_originLatRad / 2.0 ), my_n );
    if ( Double.isNaN( my_n ) || Double.isNaN( my_F ) || Double.isNaN( my_rhoZero )) {
      return new Point2D.Double(Double.NaN, Double.NaN);
    if( Double.isNaN( srcPt.x ) || Double.isNaN( srcPt.y ) )
        System.out.println("======= _project received a srcPt with NaNs. Returning NaN point.");
        Point2D.Double nanPoint = new Point2D.Double();
        nanPoint.x = nanPoint.y = Double.NaN;
        return nanPoint;
    if( Double.isInfinite( srcPt.x ) || Double.isInfinite( srcPt.y ) )
        System.out.println("======= _project received a srcPt with isInfinite. Returning NaN point.");
        Point2D.Double nanPoint = new Point2D.Double();
        nanPoint.x = nanPoint.y = Double.NaN;
        return nanPoint;
    // Inputs are lon, lat degrees.
    final double lonRad = srcPt.x * my_degToRad;
    final double latRad = srcPt.y * my_degToRad;
    final double theta = my_n * (lonRad - my_originLonRad);
    // One Std lat -- tangential cone.
    final double rhoYo = my_earthRad * my_F / Math.pow(Math.tan(Math.PI/4.0 + latRad/2.0), my_n);
    final double rho   = my_earthRad * my_F / Math.pow(Math.tan(Math.PI/4.0 + latRad/2.0), my_n);
    // Computes kilometers in lambert space.
    dstPt.x = rho * Math.sin(theta);
    dstPt.y = my_rhoZero - (rho * Math.cos(theta));
    // WANK - Here's the problem! These values shouldnt be NaN!
    if( Double.isNaN( dstPt.x ) || Double.isNaN( dstPt.y ) )
        System.out.println("======= A _projected dstPt has NaNs. Dumping...vvvvvvvvvvvvvvvvvvvvvvvvvvvv");
        if( Double.isNaN( dstPt.x ) )
            System.out.println("======= dstPt.x is NaN");
        if( Double.isNaN( dstPt.y ) )
            System.out.println("======= dstPt.y is NaN");
        System.out.println("======= my_stdLat1 = " + my_stdLat1 );
        System.out.println("======= my_n = " + my_n );
        System.out.println("======= my_originLonRad = " + my_originLonRad );
        System.out.println("======= my_F = " + my_F );
        System.out.println("======= my_earthRad = " + my_earthRad );
        System.out.println("======= lonRad = " + lonRad );
        System.out.println("======= latRad = " + latRad );
        System.out.println("======= theta = " + theta );
        System.out.println("======= Math.tan(Math.PI/4.0 + latRad/2.0) = " + Math.tan(Math.PI/4.0 + latRad/2.0) );
        System.out.println("======= Math.pow(Math.tan(Math.PI/4.0 + latRad/2.0), my_n) = " + Math.pow(Math.tan(Math.PI/4.0 + latRad/2.0), my_n) );
        System.out.println("======= rho = " + rho );
        System.out.println("======= rhoYo = " + rhoYo );
        System.out.println("======= Math.sin(theta) = " + Math.sin(theta) );
        System.out.println("======= dstPt.x = " + dstPt.x );
        System.out.println("======= Math.cos(theta) = " + Math.cos(theta) );
        System.out.println("======= my_rhoZero = " + my_rhoZero );
        System.out.println("======= (rhoYo * Math.cos(theta)) = " + (rho * Math.cos(theta)) );
        System.out.println("======= my_rhoZero - (rhoYo * Math.cos(theta)) = " + (my_rhoZero - (rho * Math.cos(theta)) ));
        System.out.println("======= dstPt.y = " + dstPt.y );
        System.out.println("======= A _projected dstPt had NaNs. Done dumping. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^");
    return dstPt;
}And here's the sample output:
======= A _projected dstPt has NaNs. Dumping...vvvvvvvvvvvvvvvvvvvvvvvvvvvv
======= dstPt.x is NaN
======= dstPt.y is NaN
======= my_stdLat1 = 25.0
======= my_n = 0.42261826174069944
======= my_originLonRad = -1.6580627893946132
======= my_F = 2.5946660025799146
======= my_earthRad = 6371.2
======= lonRad = -2.7564670759053924
======= latRad = 0.3730758324037379
======= theta = -0.4642057102537187
======= Math.tan(Math.PI/4.0 + latRad/2.0) = 1.4652768116539785
======= Math.pow(Math.tan(Math.PI/4.0 + latRad/2.0), my_n) = 1.175224090766834
======= rho = NaN
======= rhoYo = 14066.369269924155
======= Math.sin(theta) = -0.44771270676160557
======= dstPt.x = NaN
======= Math.cos(theta) = 0.8941774612481554
======= my_rhoZero = 13663.082491950498
======= (rhoYo * Math.cos(theta)) = NaN
======= my_rhoZero - (rhoYo * Math.cos(theta)) = NaN
======= dstPt.y = NaN
======= A _projected dstPt had NaNs. Done dumping. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

HI JSchell (and others?) -
I have created a simple example attached below, that when run repeatedly, does indeed generate spurious NaNs. I have made it as simple as possible. In the code, I use my own lon/lat binary data file, though I am sure any will do. Let me know if anyone wants that file.
So the deal is that (with my data at least) this program should never generate NaN results. And if one runs it 25432 (eg: random #) times, it wont, but then the 25433th time, it will create NaNs, etc. ie: inconsistent NaN math results.
As I said before, I have run this on old and new Dell laptops under Windows XP, using java 1.5_02, 1.5_04 and 1.5_08. The latest run was on a brand new Dell with a Intel Core Duo T2600 processor, running XP. If this is a result of the Pentium bug, one would think that would be fixed by now. I have NOT yet tested on AMD, though I will do that this afternoon.
Remember, this ONLY happens with Windows Laptops.
Any ideas anyone? Thanks in advance ;-)
Here's the code that produces spurious NaNs:
import java.awt.geom.Point2D;
import java.io.DataInputStream;
import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
public class FloatingPointTest2 implements Runnable
private static final int NUM_ITERATIONS = 100000;
private double _degToRad = Math.PI / 180.0;
private double _originLon = -95.0;
private double _originLat = 25.0;
private double _originLonRad = _originLon * _degToRad;
private double _originLatRad = _originLat * _degToRad;;
private double _stdLat1 = 25.0;
private double _stdLat1Rad = _stdLat1 * _degToRad;
private double _earthRad = 6371.2;
private double _n = _n = Math.sin( _stdLat1Rad );
private double _F = Math.cos( _stdLat1Rad ) * Math.pow( Math.tan( Math.PI / 4.0 + _stdLat1Rad / 2.0 ), _n ) / _n;
private double _rhoZero = _earthRad * _F / Math.pow( Math.tan( Math.PI / 4.0 + _originLatRad / 2.0 ), _n );
private Point2D.Double _project( Point2D.Double srcPt, Point2D.Double dstPt )
    if( Double.isNaN( srcPt.x ) || Double.isNaN( srcPt.y ) )
      System.out.println( "FloatingPointTest2: received a NaN srcPt. Skipping." );
      return new Point2D.Double( Double.NaN, Double.NaN );
    // Inputs are lon, lat degrees.
    final double lonRad = srcPt.x * _degToRad;
    final double latRad = srcPt.y * _degToRad;
    final double theta = _n * ( lonRad - _originLonRad );
    double rho = _earthRad * _F / Math.pow( Math.tan( Math.PI / 4.0 + latRad / 2.0 ), _n );
    dstPt.x = rho * Math.sin( theta );
    dstPt.y = _rhoZero - ( rho * Math.cos( theta ) );
    return dstPt;
public void doTest()
    DataInputStream instream = null;
    int thisRunNaNCount = 0;
    Point2D.Double tempPt = new Point2D.Double();
    Point2D.Double dstPt = new Point2D.Double();
    try
      instream = new DataInputStream( new FileInputStream( System.getProperty(
        "user.home" ) + File.separatorChar + "lonLatBinaryData.bin" ) );
      try
        while( true )
          double lon = instream.readDouble();
          double lat = instream.readDouble();
          if( Double.isNaN( lon ) || Double.isNaN( lat ) )
            continue;
          tempPt.x = lon;
          tempPt.y = lat;
          dstPt = _project( tempPt, dstPt );
          if( Double.isNaN( dstPt.x ) || Double.isNaN( dstPt.y ) )
            thisRunNaNCount++;
      catch( EOFException e )
//        System.out.println( "End of file" );
      if( thisRunNaNCount > 0 )
        System.out.println( "thisRunNaNCount = " + thisRunNaNCount );
      instream.close();
    catch( Exception e )
      e.printStackTrace();
      System.exit( 1 );
public void run()
    doTest();
public static void main( String args[] )
    System.out.println( "Executing FloatingPointTest2." );
    for( int i = 0; i < NUM_ITERATIONS; i++ )
      FloatingPointTest2 test = new FloatingPointTest2();
      test.doTest();
}

Problem with Float.toString

Hi,
I have a small problem with wrapper class Float.
For example
float f = 99999999999999999f;
f.toString(); returns a mentioned in the api doc: 9.9999999999999999E16
while I wanted the original 17 times a 9,
how can I achieve this??
THX

I tried to use the DecimalFormat class with the patter 17 * #, but the string it prints is no longer the same as in the original Float (17 * 9), now I get 17 chars but they are no all 9

Problem with Black Point adjustment

When making Black Point adjustments (with hot and cold highlights enabled) my photos suddenly turn completely blue. If I hold click the slider button, I can see the photo again as expected.
I also notice this histogram is empty.
I have the same problem with both my Nvidia and Radeon graphics cards.
This problem has started since my upgrade to 10.5.7
Has anyone else experience this? ...am I doing something wrong?

I think I have found the problem. For some reason the Black Point values were incrementing in much higher values than before.

Problem with Access Point app

Yesterday I experienced a problem with my internet sharing where it wouldn't start and gave me an error: Connection not shared, Internet sharing isn't currently available
. I ran across an article on wpcentral.com where they suggested installing the Access Point app from this thread:
http://discussions.nokia.com/t5/Pool-of-Knowledge/Nokia-Windows-Phone-8-App-Updates/td-p/1640082
Now I've done that and I see 3 profiles in the app, only 2 are LTE. Activating either of those LTE profiles now gets me an error when trying to internet share that looks like this: Connection not shared, To enable tethering on this device, go to att.com/mywireless or dial 611. When I use the 3g access point profile I get the same message I was getting yesterday.
At this point I don't think this app is doing anything for me. I dont' use multiple sims, I'm not a world traveller. I also don't know which profile I should be on. I'm clearly in over my head and would like to just take a step back. However, I can't see a way to uninstall the app as it's now in settings. How do I get myself back to where I was yesterday?
And for bonus points, does anybody know what the original error I was getting means?
Thanks,
Joel

Hi Adrian,
Yes, I've tried a soft reset to no avail. I've gone to both the AT&T store and the Microsoft store and neither of them have any thoughts on what's going on. At this point I think I will need to replace the hardware since I've not found a solution from anyone.
Thanks for you help.
Joel

Generic delta has problem with delts pointer time stamp

Hi All,
We have generic delta data source with Delte pointer as Time stamp. We have not used any safety limits. We are loading the data to DSO and using Additive delta. But the problem is when delta flag is set, there might be a case where some records still being inserted to table with lower time stamp. These records are not being fetched in next delta.
We are planning to keep a safety upper limit of 5 min so that those records can not be missed. But we have a doubt as it is a Additive delta duplicate records will be loaded. If we make the delta to 'new status to changed records', it is going to work?? Any other things we have to check when we make this changes??
Thanks
Haaris

Hi
If you had selected additive delta then you can load data to DSo with addition mode only. No overwrite possible.
If you want to select new or changed status then you can load to DSO with overwrite mode only.
Additive or changed delta depends on your requirement because if you want to add data to the DSO then you can go ahead with additive otherwise you have to use DSO with overwrite mode only.
You can either give safety upper limit or lower limit but if you give safety lower limit then you can only load data to DSO with overwrite mode because there could be some duplicate records.
Hope it clarifies.
Regards
Sadeesh

Why do we run into stupid basic problems with firefox?

I have jre1.6_u31 installed on a Windows 7 and the brilliant
firefox keep tell me that I've to install the jre_1.6_u29 missing plugin when I try to run an applet...a simple applet!! The plugin doesn't appear on add-ons manager then i use "check your plugins" and the stupid response is: "For your safety, Firefox has disabled your outdated version of Java. Please upgrade to the latest version."
Running an applet is basic! Why are u making new versions when older ones are better? I' m sick of running into this stupid problems with firefox! Everything works on ie, chrome and safari but not on Firefox!
I' m sick of trying to solve firefox basic problems!
We have a site that has one thousand ore more visits per day...and we only have troubles with firefox! What do I tell the costumers? I' m not going to give them solutions difficult and confusing as usually your solutions are! They are common people not developers as me...they just want the things to work! I gave up and the solution that I give to them now is to stop using firefox and uninstall it. And people ask I chrome is becoming the second most used browser...here is your answer...
I' m a developer and I've to develop I don't have the time patiente to trying solving basic firefox problems. If you don't know how to solve the problems then stop developing this and save us the headaches.

Have you updated Java lately? You say you have jre1.6_u31 installed, but "Java(TM) Platform SE 6 U31" is not showing in the Plugins submitted with your question (click "More system details" to the right of your original question post). To enable, Add-ons > Plugins, and enable the item.
If your OS is 64-bit and you have 32-bit browsers (Firefox) and 64-bit browsers installed, then you need to install both the 32-bit and 64-bit versions of Java.
*See --> https://www.java.com/en/download/faq/java_win64bit.xml
'''Older versions of Java blocked in Firefox''' --> https://addons.mozilla.org/en-US/firefox/blocked/p80 <br />
'''Using/Updating Java''' --> [https://support.mozilla.org/en-US/kb/Using%20the%20Java%20plugin%20with%20Firefox#w_installing-or-updating-java Installing or Updating Java in Firefox] <br />
'''Unblocking Java''' --> https://support.mozilla.org/en-US/kb/unblocking-java-plugin
'''If this reply solves your problem, please click "Solved It" next to this reply when <u>signed-in</u> to the forum.'''

Problem with floated elements in IE5.5

I've apparently got a problem with a right sidebar float
being dropped in a primitive version of IE and I can't figure out a
remedy, as none are suggested in dwcs3's browser compatibility
function. The site is lindafarronknapp.com... if anyone has any
suggestions I would more than appreciate it...

Thank you for the replies. What was happening is that the other audio files were being onverted by PPPro. This one was not. I moved the file to the desktop and once again placed it into the project. This time the yellow conversion line on the bottom right corner went to work and it works fine. It was some type of glitch with that one file. Thanks again. Roman

Can I implement advanced control algorithm with floating-point computations in Ni 7831R ?

Hi,
I plan to use a Ni 7831R to control a MIMO nano-positioning stage with servo rate above 20kHz. The control algorithm is based on robust control design and it is much more complex than PID. it also includes floating-point caculations. Can I implement such algorithm with Ni 7831R?
By the way, is there any way to expand the FPGA gates number for Ni 7831R? Suppose I run out of the FPGA gates (1M), can I add more FPGA gates by buying some different hardware?
Thanks
Jingyan
Message Edited by Jingyan on 08-22-2006 01:45 PM

Jingyan,
as long as there is no GPU core implemented on the FPGA these devices only support integer arithmetic. NI's FPGA targets currently don't contain a GPU core so there is no native floating point arithmetic available.
Still there are several options to implement floating point arithmetic on your own or to work around this target specific limitation. Here are some links that might help:
Floating-Point Addition in LabVIEW FPGA
Multiplying, Dividing and Scaling in LabVIEW FPGA
The NI 7831R uses an 1M FPGA. If your application requires more gates the NI 7833R (3M) is a good solution.
I hope that helps,
Jochen Klier
National Instruments Germany

Intermittent problem with access point (WAP321)

We hane an intermittent problem with our Wifi access point WAP321. Suddenly, all clients are unable to access Internet, server and local network. The signal is 100%. The message is no internet connection or limited connection (on all device, Iphone, computer, etc.). When I check to Ip configuration, the ip adress have change for loca 169.x.x.x. The only way to resolve is to reboot the access point. (POE).
It seem that access point don't relay information to rest of network. Strange.
The access point is connected to a simple unmanaged switch. This switch is connect to router and server. The serveur (Win2008) is the DHCP Server and DNS Server.
Can you help me to find a solution. ?

This may be the problem;
Devices: WAP discovers devices with ASSOC method that don't have an IP
CSCtz20891
Description
Symptom:
The OnPlus topology will show nodes with no IP address. Devices associated to WAP121/321 VAP/SSID not on the management VLAN will display topology nodes with with no IP address.
Conditions:
This occurs when the client associated to the WAP121/321 but is on a VLAN other than the management VLAN. In this case the WAP client information can be collected on the management VLAN, but the clients cannot be ARP/ping to validate.
Workaround:
None

Stumped on basic problem with floating point math

Similar Messages

Maybe you are looking for