Floating point problem in method

When I got a method, like moveTo(float,float), originally, it should be, for example moveTo(30.45f,100.894f). But if I passed an array which id of type float but don't have the 'f' at the back. Does this matter? An Example:
float x1Points[]=new float[ix.length];
float y1Points[]=new float[iy.length];
     for(int i=0;i<ix.length;i++){
     x1Points=ix[i]*100000-299900;
     y1Points[i]=iy[i]*1000-101710;
GeneralPath polygon = new GeneralPath(GeneralPath.WIND_EVEN_ODD);
polygon.moveTo(x1Points[0], y1Points[0]);//problem here
for ( int i = 1; i < x1Points.length; i++ ) {
polygon.lineTo(x1Points[i], y1Points[i]);
g2d.draw(polygon);

No, this does not matter.
The "f" is only for numeric literal values, to indicate that the value is a float, instead of an int or double (which it is by default).
For variables, there is no need to specify the type with a postfix letter like the "f", because the type of variables is always known; for example, the compiler already knows that you have an array of floats, so there is no need to tell the compiler again that you want to pass floats to the method.
Jesper

Similar Messages

  • Floating Point Problem

    Hi,
    I wish to know how can I get the floating point to two significant digit?
    like 0.012234455 to 0.01?

    public class TaDa {
        public static void main(String[] args) {
            double percent = 2 / Math.PI * 100;
            System.out.printf("%.2f", percent);
    }The string starting wth % is a format string. The ".2" means you want 2 decimal
    places. The "f" means you are dealing with a floating point thing.
    There is also String.format("%.2f", percent) which will return a String - the same
    one as is printed in the example. These methods became available with Java
    1.5 and are documented here:
    http://java.sun.com/j2se/1.5.0/docs/api/java/util/Formatter.html
    (A similar printf is available in PHP).

  • 32bit floating point is the audio sync problem

    Hi,
    just thought I post this solution.
    We are working on a HD project (1h), ProRes422(HQ) setting, 48kHz. The footage was shot with EX-3.
    When we got the sound back from the mix (WAV, 48kHz) is showed up in the finder as correct duration, but once placed into the timeline is shrunk and ended up too short (about 3sec) and hence the sync was lost.
    Next step was getting a AIFF bounced out from the Sound Guys, but same problem.
    Changing to other sequence settings didn't help. We tried Sony EX, uncompressed etc. Also looked at all render settings but our system is MacPro with 10GB Ram, so speak isn't the issue.
    I then ended up importing it into Soundtrack Pro and output another AIFF 2ith 32bit floating point - problem solved. No its on duration and in Sync.
    Cheers Alex

    I'll run through it again along the lines you suggest. Hardware should be good enough. 15" Retina, i7, 16GB RAM, discrete graphics, etc, but I like the idea of copying the card completely. The drive I use is a G-RAID Thunderbolt striped as a 0, so yes, it's an external. Optimized media as Apple Prores 422 behaves just the same. When you view the waveform and the picture, single-framing through the clip, you see and hear they are not in sync.
    Thanks, Ben. I'll report back. If your friend's works, it may be the camera. I have a case number with Panasonic in case that's possible, and I'll try an import into Premiere CC, though I don't use that personally.
    (The DVX100 Audio Sync Tool was a thing, but it was a long time ago in a galaxy buried underneath our own. It came as an optional install on the DVD for FCP 4 for sure, but I may have used it with 3.0.)

  • Inline functions in C, gcc optimization and floating point arithmetic issues

    For several days I really have become a fan of Alchemy. But after intensive testing I have found several issues which I'd like to solve but I can't without any help.
    So...I'm porting an old game console emulator written by me in ANSI C. The code is working on both gcc and VisualStudio without any modification or crosscompile macros. The only platform code is the audio and video output which is out of scope, because I have ported audio and video witin AS3.
    Here are the issues:
    1. Inline functions - Having only a single inline function makes the code working incorrectly (although not crashing) even if any optimization is enabled or not (-O0 or O3). My current workarround is converting the inline functions to macros which achieves the same effect. Any ideas why inline functions break the code?
    2. Compiler optimizations - well, my project consists of many C files one of which is called flash.c and it contains the main and exported functions. I build the project as follows:
    gcc -c flash.c -O0 -o flash.o     //Please note the -O0 option!!!
    gcc -c file1.c -O3 -o file1.o
    gcc -c file2.c -O3 -o file2.o
    ... and so on
    gcc *.o -swc -O0 -o emu.swc   //Please note the -O0 option again!!!
    mxmlc.exe -library-path+=emu.swc --target-player=10.0.0 Emu.as
    or file in $( ls *.o ) //Removes the obj files
        do
            rm $file
        done
    If I define any option different from -O0 in gcc -c flash.c -O0 -o flash.o the program stops working correctly exactly as in the inline funtions code (but still does not crash or prints any errors in debug). flash has 4 static functions to be exported to AS3 and the main function. Do you know why?
    If I define any option different from -O0 in gcc *.o -swc -O0 -o emu.swc  the program stops working correctly exactly as above, but if I specify -O1, -O2 or O3 the SWC file gets smaller up to 2x for O3. Why? Is there a method to optimize all the obj files except flash.o because I suspect a similar issue as when compilling it?
    3. Flating point issues - this is the worst one. My code is mainly based on integer arithmetic but on 1-2 places it requires flating point arithmetic. One of them is the conversion of 16-bit 44.1 Khz sound buffer to a float buffer with same sample rate but with samples in the range from -1.0 to 1.0.
    My code:
    void audio_prepare_as()
        uint32 i;
        for(i=0;i<audioSamples;i+=2)
            audiobuffer[i] = (float)snd.buffer[i]/32768;
            audiobuffer[i+1] = (float)snd.buffer[i+1]/32768;
    My audio playback is working perfectly. But not if using the above conversion and I have inspected the float numbers - all incorrect and invalid. I tried other code with simple floats - same story. As if alchemy refuses to work with floats. What is wrong? I have another lace whre I must resize the framebuffer and there I have a float involved - same crap. Please help me?
    Found the floating point problem: audiobuffer is written to a ByteArray and then used in AS. But C floats are obviously not the same as those in AS3. Now the floating point is resolved.
    The optimization issues remain! I really need to speed up my code.
    Thank you in advice!

    Dear Bernd,
    I am still unable to run the optimizations and turn on the inline functions. None of the inline functions contain any stdli function just pure asignments, reads, simple arithmetic and bitwise operations.
    In fact, the file containing the main function and those functions for export in AS3 did have memset and memcpy. I tried your suggestion and put the code above the functions calling memset and memcpy. It did not work soe I put the code in a header which is included topmost in each C file. The only system header I use is malloc.h and it is included topmost. In other C file I use pow, sin and log10 from math.h but I removed it and made the same thing:
    //shared.h
    #ifndef _SHARED_H_
    #define _SHARED_H_
    #include <malloc.h>
    static void * custom_memmove( void * destination, const void * source, unsigned int num ) {
      void *result; 
      __asm__("%0 memmove(%1, %2, %3)\n" : "=r"(result) : "r"(destination), "r"(source), "r"(num)); 
      return result; 
    static void * custom_memcpy ( void * destination, const void * source, unsigned int num ) { 
      void *result; 
      __asm__("%0 memcpy(%1, %2, %3)\n" : "=r"(result) : "r"(destination), "r"(source), "r"(num)); 
      return result; 
    static void * custom_memset ( void * ptr, int value, unsigned int num ) { 
      void *result; 
      __asm__("%0 memset(%1, %2, %3)\n" : "=r"(result) : "r"(ptr), "r"(value), "r"(num)); 
      return result; 
    static float custom_pow(float x, int y) {
        float result;
      __asm__("%0 pow(%1, %2)\n" : "=r"(result) : "r"(x), "r"(y));
      return result;
    static double custom_sin(double x) {
        double result;
      __asm__("%0 sin(%1)\n" : "=r"(result) : "r"(x));
      return result;
    static double custom_log10(double x) {
        double result;
      __asm__("%0 log10(%1)\n" : "=r"(result) : "r"(x));
      return result;
    #define memmove custom_memmove
    #define memcpy custom_memcpy
    #define memset custom_memset
    #define pow custom_pow
    #define sin custom_sin
    #define log10 custom_log10 
    #include "types.h"
    #include "macros.h"
    #include "m68k.h"
    #include "z80.h"
    #include "genesis.h"
    #include "vdp.h"
    #include "render.h"
    #include "mem68k.h"
    #include "memz80.h"
    #include "membnk.h"
    #include "memvdp.h"
    #include "system.h"
    #include "loadrom.h"
    #include "input.h"
    #include "io.h"
    #include "sound.h"
    #include "fm.h"
    #include "sn76496.h" 
    #endif /* _SHARED_H_ */ 
    It still behave the same way as if nothing was changed (works incorrectly - displays jerk which does not move, whereby the image is supposed to move)
    As I am porting an emulator (Sega Mega Drive) I use manu arrays of function pointers for implementing the opcodes of the CPU's. Could this be an issue?
    I did a workaround for the floating point problem but processing is very slow so I hear only bzzt bzzt but this is for now out of scope. The emulator compiled with gcc runs at 300 fps on a 1.3 GHz machine, whereby my non optimized AVM2 code compiled by alchemy produces 14 fps. The pure rendering is super fast and the problem lies in the computational power of AVM. The frame buffer and the enulation are generated in the C code and only the pixels are copied to AS3, where they are plotted in a BitmapData. On 2.0 GHz Dual core I achieved only 21 fps. Goal is 60 fps to have smooth audio and video. But this is offtopic. After all everything works (slow) without optimization, and I would somehow turn it on. Suggestions?
    Here is the file with the main function:
    #include "shared.h"
    #include "AS3.h"
    #define FRAMEBUFFER_LENGTH    (320*240*4)
    static uint8* framebuffer;
    static uint32  audioSamples;
    AS3_Val sega_rom(void* self, AS3_Val args)
        int size, offset, i;
        uint8 hardware;
        uint8 country;
        uint8 header[0x200];
        uint8 *ptr;
        AS3_Val length;
        AS3_Val ba;
        AS3_ArrayValue(args, "AS3ValType", &ba);
        country = 0;
        offset = 0;
        length = AS3_GetS(ba, "length");
        size = AS3_IntValue(length);
        ptr = (uint8*)malloc(size);
        AS3_SetS(ba, "position", AS3_Int(0));
        AS3_ByteArray_readBytes(ptr, ba, size);
        //FILE* f = fopen("boris_dump.bin", "wb");
        //fwrite(ptr, size, 1, f);
        //fclose(f);
        if((size / 512) & 1)
            size -= 512;
            offset += 512;
            memcpy(header, ptr, 512);
            for(i = 0; i < (size / 0x4000); i += 1)
                deinterleave_block(ptr + offset + (i * 0x4000));
        memset(cart_rom, 0, 0x400000);
        if(size > 0x400000) size = 0x400000;
        memcpy(cart_rom, ptr + offset, size);
        /* Free allocated file data */
        free(ptr);
        hardware = 0;
        for (i = 0x1f0; i < 0x1ff; i++)
            switch (cart_rom[i]) {
         case 'U':
             hardware |= 4;
             break;
         case 'J':
             hardware |= 1;
             break;
         case 'E':
             hardware |= 8;
             break;
        if (cart_rom[0x1f0] >= '1' && cart_rom[0x1f0] <= '9') {
            hardware = cart_rom[0x1f0] - '0';
        } else if (cart_rom[0x1f0] >= 'A' && cart_rom[0x1f0] <= 'F') {
            hardware = cart_rom[0x1f0] - 'A' + 10;
        if (country) hardware=country; //simple autodetect override
        //From PicoDrive
        if (hardware&8)        
            hw=0xc0; vdp_pal=1;
        } // Europe
        else if (hardware&4)    
            hw=0x80; vdp_pal=0;
        } // USA
        else if (hardware&2)    
            hw=0x40; vdp_pal=1;
        } // Japan PAL
        else if (hardware&1)      
            hw=0x00; vdp_pal=0;
        } // Japan NTSC
        else
            hw=0x80; // USA
        if (vdp_pal) {
            vdp_rate = 50;
            lines_per_frame = 312;
        } else {
            vdp_rate = 60;
            lines_per_frame = 262;
        /*SRAM*/   
        if(cart_rom[0x1b1] == 'A' && cart_rom[0x1b0] == 'R')
            save_start = cart_rom[0x1b4] << 24 | cart_rom[0x1b5] << 16 |
                cart_rom[0x1b6] << 8  | cart_rom[0x1b7] << 0;
            save_len = cart_rom[0x1b8] << 24 | cart_rom[0x1b9] << 16 |
                cart_rom[0x1ba] << 8  | cart_rom[0x1bb] << 0;
            // Make sure start is even, end is odd, for alignment
            // A ROM that I came across had the start and end bytes of
            // the save ram the same and wouldn't work.  Fix this as seen
            // fit, I know it could probably use some work. [PKH]
            if(save_start != save_len)
                if(save_start & 1) --save_start;
                if(!(save_len & 1)) ++save_len;
                save_len -= (save_start - 1);
                saveram = (unsigned char*)malloc(save_len);
                // If save RAM does not overlap main ROM, set it active by default since
                // a few games can't manage to properly switch it on/off.
                if(save_start >= (unsigned)size)
                    save_active = 1;
            else
                save_start = save_len = 0;
                saveram = NULL;
        else
            save_start = save_len = 0;
            saveram = NULL;
        return AS3_Int(0);
    AS3_Val sega_init(void* self, AS3_Val args)
        system_init();
        audioSamples = (44100 / vdp_rate)*2;
        framebuffer = (uint8*)malloc(FRAMEBUFFER_LENGTH);
        return AS3_Int(vdp_rate);
    AS3_Val sega_reset(void* self, AS3_Val args)
        system_reset();
        return AS3_Int(0);
    AS3_Val sega_frame(void* self, AS3_Val args)
        uint32 width;
        uint32 height;
        uint32 x, y;
        uint32 di, si, r;
        uint16 p;
        AS3_Val fb_ba;
        AS3_ArrayValue(args, "AS3ValType", &fb_ba);
        system_frame(0);
        AS3_SetS(fb_ba, "position", AS3_Int(0));
        width = (reg[12] & 1) ? 320 : 256;
        height = (reg[1] & 8) ? 240 : 224;
        for(y=0;y<240;y++)
            for(x=0;x<320;x++)
                di = 1280*y + x<<2;
                si = (y << 10) + ((x + bitmap.viewport.x) << 1);
                p = *((uint16*)(bitmap.data + si));
                framebuffer[di + 3] = (uint8)((p & 0x1f) << 3);
                framebuffer[di + 2] = (uint8)(((p >> 5) & 0x1f) << 3);
                framebuffer[di + 1] = (uint8)(((p >> 10) & 0x1f) << 3);
        AS3_ByteArray_writeBytes(fb_ba, framebuffer, FRAMEBUFFER_LENGTH);
        AS3_SetS(fb_ba, "position", AS3_Int(0));
        r = (width << 16) | height;
        return AS3_Int(r);
    AS3_Val sega_audio(void* self, AS3_Val args)
        AS3_Val ab_ba;
        AS3_ArrayValue(args, "AS3ValType", &ab_ba);
        AS3_SetS(ab_ba, "position", AS3_Int(0));
        AS3_ByteArray_writeBytes(ab_ba, snd.buffer, audioSamples*sizeof(int16));
        AS3_SetS(ab_ba, "position", AS3_Int(0));
        return AS3_Int(0);
    int main()
        AS3_Val romMethod = AS3_Function(NULL, sega_rom);
        AS3_Val initMethod = AS3_Function(NULL, sega_init);
        AS3_Val resetMethod = AS3_Function(NULL, sega_reset);
        AS3_Val frameMethod = AS3_Function(NULL, sega_frame);
        AS3_Val audioMethod = AS3_Function(NULL, sega_audio);
        // construct an object that holds references to the functions
        AS3_Val result = AS3_Object("sega_rom: AS3ValType, sega_init: AS3ValType, sega_reset: AS3ValType, sega_frame: AS3ValType, sega_audio: AS3ValType",
            romMethod, initMethod, resetMethod, frameMethod, audioMethod);
        // Release
        AS3_Release(romMethod);
        AS3_Release(initMethod);
        AS3_Release(resetMethod);
        AS3_Release(frameMethod);
        AS3_Release(audioMethod);
        // notify that we initialized -- THIS DOES NOT RETURN!
        AS3_LibInit(result);
        // should never get here!
        return 0;

  • Problem with floating point numbers

    Hi Experts,
    I am accepting two float values thru a method :- amount and tenure.
    Now wat i want to do is
    target = amount / ( tenure * 12 );
    but result target is not what it supposed to be. For example:-
    if amount = Rs. 15 00 000.0 and tenure = 3 yrs, target amount should be 5 00 000. but it is coming 41666 sumthing.
    How should I code this?
    Pls help,
    regards,
    Amey

    Hi,
    I am accepting two float values thru a method :- amount and tenure.
    Now wat i want to do is
    target = amount / ( tenure * 12 );
    but result target is not what it supposed to be. For example:-
    if amount = Rs. 15 00 000.0 and tenure = 3 yrs, target amount should be 5 00 000. but it is coming 41666 sumthing.
    How should I code this?
    I dont see any error in calculation,
    15 00 000.0 / 36 is perfecctly 41666 sumthing
    how can you expect this to be 5 00 000?
    If you want that change your calculation from
    target = amount / tenure ;
    Regards
    Ayyapparaj

  • SQL Loader and Floating Point Numbers

    Hi
    I have a problem loading floating point numbers using SQL Loader. If the number has more than 8 significant digits SQL Loader rounds the number i.e. 1100000.69 becomes 1100000.7. The CTL file looks as follows
    LOAD DATA
    INFILE '../data/test.csv' "str X'0A'"
    BADFILE '../bad/test.bad'
    APPEND
    INTO TABLE test
    FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY '"'
    Amount CHAR
    and the data file as follows
    "100.15 "
    "100100.57 "
    "1100000.69 "
    "-2000000.33"
    "-100000.43 "
    the table defined as follows
    CREATE TABLE test
    Amount number(15,4)
    ) TABLESPACE NNUT050M1;
    after loading a select returns the following
    100.15
    100100.57
    1100000.7
    -2000000
    -100000.4
    Thanks in advance
    Russell

    Actually if you format the field to display as (say) 999,999,999.99, you will see the correct numbers loaded via SQL Loader.
    null

  • R6002 - floating point not loaded / C++ Runtime Error

    Good Evening,
    I have been having this problem with Adobe Bridge for a while and tonight sat down to try and solve it. Fresh version of Windows XP reinstalled all programs and this is still happening! Any Ideas?
    This error only occurs with Adobe Bridge, I can load Photoshop etc.. all fine.
    Error:
    "Runtime Error!
    Program: C:\ProgramFiles\Adobe\Adobe Bridge CS3\Bridge.exe
    R6002
    - floating point not loaded"
    Here is a print screen..
    http://images.unbrokenphotography.co.uk/BridgeError.jpg

    Is there are answer to this problem?  This error message is appearing on an entire lab full of computers.  We are running Web Premium CS4
    I have tried to reset the Bridge Preferences:
    Hold down the Ctrl key and click on Bridge icon to start.
    Should get a reset window with 3 options.
    Choose the first option
    I still get "Runtime Error!   Program: C:\Prgram Files\Adobe\Adobe Bridge CS4\Bridge.exe  R6002 -floating point support not loaded"

  • Check Floating Point Number

    Hello All,
    I am having some trouble checking the value of a field with Key Figure type Number with 8 byte floating point. I want to read that field and populate another field with an X if true. For example if that field is equal to 5,0000000000000000E+07 then i want to mark the other field with an 'X'.
    The problem is in my code, how do i read that number in the fltp field, such as the number above. my code reads as follows for the 'X' field.
        if SOURCE_FIELDS-abc123 eq 5000000.
          RESULT = 'X'.
        endif.
    Thanks everyone in advance

    You don't need to worry about converting the code into standard format or floating, just implement your code as you want and it will automatically take care of the conversion. Basically 5,0000000000000000E+07 = 50,000,000.
    thanks.
    Wond

  • Reading Floating Point PLC5 Registers Kills my ability to write to any register

    When ever I create an object that reads a PLC5 Floating Point Register(F8:5),it kills my ability to write to any other PLC register. The problem only shows up with Floating Point registers. If I use integer registers (N7:121 etc.)I can read/write fine. It also severely slows down my normal driver scan update time. Even though it is set to scan every 30 seconds, I may only get an update every few minutes. Anyone else ever run into this? Thanks in advance!

    Hi,
    It might be helpful creating a log file to monitor com port activity.
    Here's the Knowledgebase article about it.
    http://digital.ni.com/public.nsf/3efedde4322fef198​62567740067f3cc/cac576863ff648a186256b5900794761?O​penDocument
    Another way to check the serial port activity would be using "portmon"
    software available at
    http://www.sysinternals.com/ntw2k/freeware/portmon​.shtml
    I hope this helps.
    Remzi A.
    Applications Engineering
    National Instruments

  • How to gather a set of floating point numbers from a web page form?

    I am pretty new to Java and I am working on a project to apply Benfords law to find the probability of digits submitted by the user. My first step is to gather set of floating point numbers from a web page. How do I go about doing this? Any suggestion or a proper site where I can learn this stuff will be highly appreciated.

    I am using NetBeans IDE 5.5.1 and for this project. I have realized that the first question was not well phrased.
    I created a web project with 2 jsp files and a class file in it. When my jsp file runs I ask the user to enter a number for finding the probablility based on Benfords law.
    This is what I got so far:
    This is input.jsp
    <h1></h1>
    Please enter a number to be checked
    <form action="result.jsp" method="post">
    <input type="number" name="number" >
    <input type="submit" value="Check number">
    </form>
    </body>
    </html>
    This is result.jsp
    String number=request.getParameter("number");

  • Floating Point Arithmatic Error

    Hi,
    I know actionscript represents numbers and double precision
    floating point values. I'm having a problem where double arithmatic
    in actionscript doesn't match the results of the same double
    arithmatic in C++ / C#.
    EXAMPLE:
    In C++ / C#:
    double x, y, x1, y1;
    x = 209.4;
    y = 148.8;
    x1 = 203.0;
    y1 = 145.0;
    double ddx = x - x1;
    double ddy = y - y1;
    RESULT
    ddx: 6.4000000000000057
    ddy: 3.8000000000000114
    In Flash ActionScipt 2:
    var x, y, x1, y1;
    x = 209.4;
    y = 148.8;
    x1 = 203.0;
    y1 = 145.0;
    var ddx = x - x1;
    var ddy = y - y1;
    RESULT
    ddx: 6.39999999999992
    ddy: 3.80000000000024
    After researching Flash / Actionscript "var" stores numerical
    values as doubles ( 8 bytes ) just like doubles are stored in C++ /
    C# ( 8 bytes ). Why would there be a difference between the results
    of ddx and ddy? Are there different implementations of double
    floating point math? If so, Is there a way I can mimic the Flash /
    Actionscript version in C++ / C#?
    Any help would be great!
    Thanks!

    Hmmm, so you're saying the actual binary representation is
    the same but they're just displayed differently?

  • Floating point arithmetic

    Hi everybody,
    This line:
    System.out.println((0.1+0.7)*10);outputs 7.999999999999999
    This is due to how floating point numbers are stored. When writing
    a code, sometimes it behaves in an intended way, sometimes it doesn't
    (like the one above). Is there a way to "predict" when the code is ok and
    when isn't ? Are there any tips to be aware of to get around that kind
    of problems ?
    Cheers,
    Adrian

    No. Using BigDecimal just because you don't understand how floating-point numbers work would be... um... short-sighted. And it wouldn't help, either. As soon as you divide 1 by 3 then you have to know how decimal numbers work, which is essentially the same problem.
    Edit: I forgot the forum hasn't been automated to provide the mandatory link for people who ask this question. We still have to do it by hand.
    http://docs.sun.com/source/806-3568/ncg_goldberg.html
    Edited by: DrClap on Oct 11, 2007 3:02 PM

  • Floating point precision of "Flatten to XML"

    It appears that the "Flatten to XML" function (LV 7.1.1) truncates floating point numbers to 5 decimal places.  This is a rather annoying limitation, since I'm trying to store a relative time in hours, accurate to the second (chosen by a previous coder that I have to be compatible with - otherwise I'd just use seconds).  Is there a workaround to this?  (other than multiplying by some power of 10 before flattening, and dividing after unflattening)
    Jaegen

    Hi Paul and Jaegen,
    I checked our databases and found entries of product suggestions and
    corrective action requests for the behavior of the limited precision
    when flattening to XML. I found an interesting reply from a LabVIEW
    developer on the request for further precision:
    The Flatten To XML primitive puposefully cuts off all numbers at 5
    digits after the decimal. There are 3 main reasons for this:
    Information regarding precision is not propagated on the wire.
    Therefore, there is no real way to know how many significant digits or
    even places past the decimal point is appropriate when data is
    flattened to XML.
    Bloat. If all floating point values printed all of the possible
    decimal digits all of the time, this would provide for some very large
    blocks of XML code.
    Given the arbitrarily complex nature of LabVIEW data, it is
    difficult to provide a method for specifying precision. For example, if
    a user has a cluster of clusters, each of which contain a single,
    double and extended representing various measurements of differing
    accuracy, how can one precision setting be applied to each of these
    values? The user would have to unbundle (and index if an array was
    involved), flatten, concatenate, and then the reverse on the unflatten
    side.
    I suggest that you go ahead and file a new product suggestion by using the "feedback" link on www.ni.com/contact.
    It would be best if you could give some detailed information on how you
    would like LabVIEW to handle different scenarios while getting around
    the above issues.
    Thanks for the feedback!
    - Philip Courtois, Thinkbot Solutions

  • Floating point formats: Java/C/C++, PPC and Intel platforms

    Hi everyone
    Where can I find out about the various bit formats used for 32 bit floating numbers in Java and C/C++ for both Mac hardware platforms?
    I'm developing a Java audio application which needs to convert vast quantities of variable width integer audio samples to canonical float audio format. I've discovered that a floating point divide by the maximum integer value gives the correct answer but takes too much processor time, so I'm trying out bit-twiddling in C via JNI to carve out my own floating point bit patterns. This is very fast, however, I need to take into account the various float formats used on the different platforms so my app can be universal. Can anyone point me to the information?
    Thanks in advance.
    Bob

    I am not sure that Rosetta floating point works the same as PPC floating point. I was using RealBasic (a PPC basic compiler) and moved one of the my compiled applications to a MacBook Pro and floating point comparisons that had been exact on the PPC stopped working under Rosetta. I changed the code to do an approximate comparison (i.e. abs(a -b) < tolerance) and this fixed things.
    I reported the problem to the RealBasic people and thought nothing more of it until I fired up Adobe's InDesign and not being used to working with picas, changed the units of measurement to inches. The default letter paper size was suddenly 8.5000500050005 inches instead of the more usual 8.5! This was not a big problem, but it appears that all of InDesign's page math is running into some kind of rounding errors.
    The floating point format is almost certainly IEEE, and I cannot imagine Rosetta doing anything other than using native hardware Intel floating point. On the other hand, there is a subtle difference in behavior.
    I am posting this here as a follow up, but I am also going to post this as a proper question in the forum. If you have to delete one or the other of these duplicate posts, please zap the reply, not the question.

  • BUG: Large floating point numbers convert to the wrong integer

    Hi,
    When using the conversion "bullets" to convert SGL, DBL and EXT to integers there are some values which convert wrong. One example is the integer 9223370937343148030, which can be represented exactly as a SGL (and thus exactly as DBL and EXT as well). If you convert this to I64 you get 9223370937343148032 instead, even though the correct integer is within the range of an I64. There are many similar cases, all (I've noticed) within the large end of the ranges.
    This has nothing to do with which integers can be represented exactly as a floating point value or not. This is a genuine conversion bug mind you.
    Cheers,
    Steen
    CLA, CTA, CLED & LabVIEW Champion
    Solved!
    Go to Solution.

    Yes, I understand the implications involved, and there definetely is a limit to how many significant digits that can be displayed in the numeric controls and constants today. I think that either this limit should be lifted or a cap should be put onto the configuration page when setting the display format.
    I ran into this problem as I'm developing a new toolset that lets you convert all the numeric formats into any other numeric format, just like the current "conversion bullets". My conversion bullets have outputs for overflow and exact conversion as well, since I need that functionality myself for a Math toolset (GPMath) I'm also developing. Eventually I'll maybe include underflow as well, but for now just those two outputs are available. Example:
    I do of course pay close attention to the binary representation of the numbers to calculate the Exact conversion? output correctly for each conversion variation (there are hundreds of VIs in polymorphic wrappers), but I relied in some cases on the ability of the numeric indicator to show a true number when configured appropriately - that was when I discovered this bug, which I at first mistook for a conversion error in LabVIEW.
    Is there a compliancy issue with EXT?
    While doing this work I've discovered that the EXT format is somewhat misleadingly labelled as "80-bit IEEE compliant" (it says so here), but that statement should be read with some suspicion IMO. The LabVIEW EXT is not simply IEEE 754-1985 compliant anyways, as that format would imply the x87 80-bit extended format. An x87 IEEE 754 extended precision float only has 63-bit fraction and a 1-bit integer part. That 1-bit integer part is implicit in single and double precision IEEE 754 numbers, but it is explicit in x87 extended precision numbers. LabVIEW EXT seems to have an implicit integer part and 64-bit fraction, thus not straight IEEE 754 compliant. Instead I'd say that the LabVIEW EXT is an IEEE 754r extended format, but still a proprietary one that should deserve a bit more detail in the available documentation. Since it's mentioned several places in the LabVIEW documentation that the EXT is platform independent, your suspicion should already be high though. It didn't take me many minutes to verify the apparent format of the EXT in any case, so no real problem here.
    Is there a genuine conversion error from EXT to U64?
    The integer 18446744073709549568 can be represented exactly as EXT using this binary representation (mind you that the numeric indicators won't display the value correctly, but instead show 18446744073709549600):
    EXT-exponent: 0x100000000111110b
    EXT-fraction: 0x1111111111111111111111111111111111111111111111111111000000000000b
    --> Decimal: 18446744073709549568
    The above EXT value converts exactly to U64 using the To Unsigned Quad Integer "bullet". But then let's try to flip the blue bit from 0 to 1 in the fraction part of the EXT, making this value:
    EXT-exponent: 0x100000000111110b
    EXT-fraction: 0x1111111111111111111111111111111111111111111111111111100000000000b
    --> Decimal: 18446744073709550592
    The above EXT value is still within U64 range, but the To Unsigned Quad Integer "bullet" converts it to U64_max which is 18446744073709551615. Unless I've missed something this must be a genuine conversion error from EXT to U64?
    /Steen
    CLA, CTA, CLED & LabVIEW Champion

Maybe you are looking for

  • How do i save my photos in my iphone to icloud?

    how do i save my photos in my iphone to icloud?

  • Beware: new file format and the consequences

    Yesterday I updated both Pages and Numbers to 5.0. I played a little with a few documents and found out that way too many features are missing to get it working for me, of which the most important one is mail merge. I use that very heavily and it isn

  • Permissions on control directory startup error

    Messaging Server is exiting with the following error: Startup Problem:, The permissions on directory f:/suitespot/MailServer/spool/control are incorrect., Netscape Messaging Server Exiting! What is going on? <P> To fix this check the directory permis

  • RAW support for Lumix GX1?

    When will Aperture start supporting RAW files from Panasonic Lumix GX1? Do I really have to switch to Lightroom?

  • Deployment of a BPEL Process usng ANT with new version

    Hi Experts, How to deploy a bpel process using ANT command into weblogic server with a newer version when already a bpel process is existing there in weblogic server with 1.0 version. how to create a new jar file with a newer version. your help is hi