Floating Point Problem

Hi,
I wish to know how can I get the floating point to two significant digit?
like 0.012234455 to 0.01?

public class TaDa {
    public static void main(String[] args) {
        double percent = 2 / Math.PI * 100;
        System.out.printf("%.2f", percent);
}The string starting wth % is a format string. The ".2" means you want 2 decimal
places. The "f" means you are dealing with a floating point thing.
There is also String.format("%.2f", percent) which will return a String - the same
one as is printed in the example. These methods became available with Java
1.5 and are documented here:
http://java.sun.com/j2se/1.5.0/docs/api/java/util/Formatter.html
(A similar printf is available in PHP).

Similar Messages

  • Floating point problem in method

    When I got a method, like moveTo(float,float), originally, it should be, for example moveTo(30.45f,100.894f). But if I passed an array which id of type float but don't have the 'f' at the back. Does this matter? An Example:
    float x1Points[]=new float[ix.length];
    float y1Points[]=new float[iy.length];
         for(int i=0;i<ix.length;i++){
         x1Points=ix[i]*100000-299900;
         y1Points[i]=iy[i]*1000-101710;
    GeneralPath polygon = new GeneralPath(GeneralPath.WIND_EVEN_ODD);
    polygon.moveTo(x1Points[0], y1Points[0]);//problem here
    for ( int i = 1; i < x1Points.length; i++ ) {
    polygon.lineTo(x1Points[i], y1Points[i]);
    g2d.draw(polygon);

    No, this does not matter.
    The "f" is only for numeric literal values, to indicate that the value is a float, instead of an int or double (which it is by default).
    For variables, there is no need to specify the type with a postfix letter like the "f", because the type of variables is always known; for example, the compiler already knows that you have an array of floats, so there is no need to tell the compiler again that you want to pass floats to the method.
    Jesper

  • 32bit floating point is the audio sync problem

    Hi,
    just thought I post this solution.
    We are working on a HD project (1h), ProRes422(HQ) setting, 48kHz. The footage was shot with EX-3.
    When we got the sound back from the mix (WAV, 48kHz) is showed up in the finder as correct duration, but once placed into the timeline is shrunk and ended up too short (about 3sec) and hence the sync was lost.
    Next step was getting a AIFF bounced out from the Sound Guys, but same problem.
    Changing to other sequence settings didn't help. We tried Sony EX, uncompressed etc. Also looked at all render settings but our system is MacPro with 10GB Ram, so speak isn't the issue.
    I then ended up importing it into Soundtrack Pro and output another AIFF 2ith 32bit floating point - problem solved. No its on duration and in Sync.
    Cheers Alex

    I'll run through it again along the lines you suggest. Hardware should be good enough. 15" Retina, i7, 16GB RAM, discrete graphics, etc, but I like the idea of copying the card completely. The drive I use is a G-RAID Thunderbolt striped as a 0, so yes, it's an external. Optimized media as Apple Prores 422 behaves just the same. When you view the waveform and the picture, single-framing through the clip, you see and hear they are not in sync.
    Thanks, Ben. I'll report back. If your friend's works, it may be the camera. I have a case number with Panasonic in case that's possible, and I'll try an import into Premiere CC, though I don't use that personally.
    (The DVX100 Audio Sync Tool was a thing, but it was a long time ago in a galaxy buried underneath our own. It came as an optional install on the DVD for FCP 4 for sure, but I may have used it with 3.0.)

  • Inline functions in C, gcc optimization and floating point arithmetic issues

    For several days I really have become a fan of Alchemy. But after intensive testing I have found several issues which I'd like to solve but I can't without any help.
    So...I'm porting an old game console emulator written by me in ANSI C. The code is working on both gcc and VisualStudio without any modification or crosscompile macros. The only platform code is the audio and video output which is out of scope, because I have ported audio and video witin AS3.
    Here are the issues:
    1. Inline functions - Having only a single inline function makes the code working incorrectly (although not crashing) even if any optimization is enabled or not (-O0 or O3). My current workarround is converting the inline functions to macros which achieves the same effect. Any ideas why inline functions break the code?
    2. Compiler optimizations - well, my project consists of many C files one of which is called flash.c and it contains the main and exported functions. I build the project as follows:
    gcc -c flash.c -O0 -o flash.o     //Please note the -O0 option!!!
    gcc -c file1.c -O3 -o file1.o
    gcc -c file2.c -O3 -o file2.o
    ... and so on
    gcc *.o -swc -O0 -o emu.swc   //Please note the -O0 option again!!!
    mxmlc.exe -library-path+=emu.swc --target-player=10.0.0 Emu.as
    or file in $( ls *.o ) //Removes the obj files
        do
            rm $file
        done
    If I define any option different from -O0 in gcc -c flash.c -O0 -o flash.o the program stops working correctly exactly as in the inline funtions code (but still does not crash or prints any errors in debug). flash has 4 static functions to be exported to AS3 and the main function. Do you know why?
    If I define any option different from -O0 in gcc *.o -swc -O0 -o emu.swc  the program stops working correctly exactly as above, but if I specify -O1, -O2 or O3 the SWC file gets smaller up to 2x for O3. Why? Is there a method to optimize all the obj files except flash.o because I suspect a similar issue as when compilling it?
    3. Flating point issues - this is the worst one. My code is mainly based on integer arithmetic but on 1-2 places it requires flating point arithmetic. One of them is the conversion of 16-bit 44.1 Khz sound buffer to a float buffer with same sample rate but with samples in the range from -1.0 to 1.0.
    My code:
    void audio_prepare_as()
        uint32 i;
        for(i=0;i<audioSamples;i+=2)
            audiobuffer[i] = (float)snd.buffer[i]/32768;
            audiobuffer[i+1] = (float)snd.buffer[i+1]/32768;
    My audio playback is working perfectly. But not if using the above conversion and I have inspected the float numbers - all incorrect and invalid. I tried other code with simple floats - same story. As if alchemy refuses to work with floats. What is wrong? I have another lace whre I must resize the framebuffer and there I have a float involved - same crap. Please help me?
    Found the floating point problem: audiobuffer is written to a ByteArray and then used in AS. But C floats are obviously not the same as those in AS3. Now the floating point is resolved.
    The optimization issues remain! I really need to speed up my code.
    Thank you in advice!

    Dear Bernd,
    I am still unable to run the optimizations and turn on the inline functions. None of the inline functions contain any stdli function just pure asignments, reads, simple arithmetic and bitwise operations.
    In fact, the file containing the main function and those functions for export in AS3 did have memset and memcpy. I tried your suggestion and put the code above the functions calling memset and memcpy. It did not work soe I put the code in a header which is included topmost in each C file. The only system header I use is malloc.h and it is included topmost. In other C file I use pow, sin and log10 from math.h but I removed it and made the same thing:
    //shared.h
    #ifndef _SHARED_H_
    #define _SHARED_H_
    #include <malloc.h>
    static void * custom_memmove( void * destination, const void * source, unsigned int num ) {
      void *result; 
      __asm__("%0 memmove(%1, %2, %3)\n" : "=r"(result) : "r"(destination), "r"(source), "r"(num)); 
      return result; 
    static void * custom_memcpy ( void * destination, const void * source, unsigned int num ) { 
      void *result; 
      __asm__("%0 memcpy(%1, %2, %3)\n" : "=r"(result) : "r"(destination), "r"(source), "r"(num)); 
      return result; 
    static void * custom_memset ( void * ptr, int value, unsigned int num ) { 
      void *result; 
      __asm__("%0 memset(%1, %2, %3)\n" : "=r"(result) : "r"(ptr), "r"(value), "r"(num)); 
      return result; 
    static float custom_pow(float x, int y) {
        float result;
      __asm__("%0 pow(%1, %2)\n" : "=r"(result) : "r"(x), "r"(y));
      return result;
    static double custom_sin(double x) {
        double result;
      __asm__("%0 sin(%1)\n" : "=r"(result) : "r"(x));
      return result;
    static double custom_log10(double x) {
        double result;
      __asm__("%0 log10(%1)\n" : "=r"(result) : "r"(x));
      return result;
    #define memmove custom_memmove
    #define memcpy custom_memcpy
    #define memset custom_memset
    #define pow custom_pow
    #define sin custom_sin
    #define log10 custom_log10 
    #include "types.h"
    #include "macros.h"
    #include "m68k.h"
    #include "z80.h"
    #include "genesis.h"
    #include "vdp.h"
    #include "render.h"
    #include "mem68k.h"
    #include "memz80.h"
    #include "membnk.h"
    #include "memvdp.h"
    #include "system.h"
    #include "loadrom.h"
    #include "input.h"
    #include "io.h"
    #include "sound.h"
    #include "fm.h"
    #include "sn76496.h" 
    #endif /* _SHARED_H_ */ 
    It still behave the same way as if nothing was changed (works incorrectly - displays jerk which does not move, whereby the image is supposed to move)
    As I am porting an emulator (Sega Mega Drive) I use manu arrays of function pointers for implementing the opcodes of the CPU's. Could this be an issue?
    I did a workaround for the floating point problem but processing is very slow so I hear only bzzt bzzt but this is for now out of scope. The emulator compiled with gcc runs at 300 fps on a 1.3 GHz machine, whereby my non optimized AVM2 code compiled by alchemy produces 14 fps. The pure rendering is super fast and the problem lies in the computational power of AVM. The frame buffer and the enulation are generated in the C code and only the pixels are copied to AS3, where they are plotted in a BitmapData. On 2.0 GHz Dual core I achieved only 21 fps. Goal is 60 fps to have smooth audio and video. But this is offtopic. After all everything works (slow) without optimization, and I would somehow turn it on. Suggestions?
    Here is the file with the main function:
    #include "shared.h"
    #include "AS3.h"
    #define FRAMEBUFFER_LENGTH    (320*240*4)
    static uint8* framebuffer;
    static uint32  audioSamples;
    AS3_Val sega_rom(void* self, AS3_Val args)
        int size, offset, i;
        uint8 hardware;
        uint8 country;
        uint8 header[0x200];
        uint8 *ptr;
        AS3_Val length;
        AS3_Val ba;
        AS3_ArrayValue(args, "AS3ValType", &ba);
        country = 0;
        offset = 0;
        length = AS3_GetS(ba, "length");
        size = AS3_IntValue(length);
        ptr = (uint8*)malloc(size);
        AS3_SetS(ba, "position", AS3_Int(0));
        AS3_ByteArray_readBytes(ptr, ba, size);
        //FILE* f = fopen("boris_dump.bin", "wb");
        //fwrite(ptr, size, 1, f);
        //fclose(f);
        if((size / 512) & 1)
            size -= 512;
            offset += 512;
            memcpy(header, ptr, 512);
            for(i = 0; i < (size / 0x4000); i += 1)
                deinterleave_block(ptr + offset + (i * 0x4000));
        memset(cart_rom, 0, 0x400000);
        if(size > 0x400000) size = 0x400000;
        memcpy(cart_rom, ptr + offset, size);
        /* Free allocated file data */
        free(ptr);
        hardware = 0;
        for (i = 0x1f0; i < 0x1ff; i++)
            switch (cart_rom[i]) {
         case 'U':
             hardware |= 4;
             break;
         case 'J':
             hardware |= 1;
             break;
         case 'E':
             hardware |= 8;
             break;
        if (cart_rom[0x1f0] >= '1' && cart_rom[0x1f0] <= '9') {
            hardware = cart_rom[0x1f0] - '0';
        } else if (cart_rom[0x1f0] >= 'A' && cart_rom[0x1f0] <= 'F') {
            hardware = cart_rom[0x1f0] - 'A' + 10;
        if (country) hardware=country; //simple autodetect override
        //From PicoDrive
        if (hardware&8)        
            hw=0xc0; vdp_pal=1;
        } // Europe
        else if (hardware&4)    
            hw=0x80; vdp_pal=0;
        } // USA
        else if (hardware&2)    
            hw=0x40; vdp_pal=1;
        } // Japan PAL
        else if (hardware&1)      
            hw=0x00; vdp_pal=0;
        } // Japan NTSC
        else
            hw=0x80; // USA
        if (vdp_pal) {
            vdp_rate = 50;
            lines_per_frame = 312;
        } else {
            vdp_rate = 60;
            lines_per_frame = 262;
        /*SRAM*/   
        if(cart_rom[0x1b1] == 'A' && cart_rom[0x1b0] == 'R')
            save_start = cart_rom[0x1b4] << 24 | cart_rom[0x1b5] << 16 |
                cart_rom[0x1b6] << 8  | cart_rom[0x1b7] << 0;
            save_len = cart_rom[0x1b8] << 24 | cart_rom[0x1b9] << 16 |
                cart_rom[0x1ba] << 8  | cart_rom[0x1bb] << 0;
            // Make sure start is even, end is odd, for alignment
            // A ROM that I came across had the start and end bytes of
            // the save ram the same and wouldn't work.  Fix this as seen
            // fit, I know it could probably use some work. [PKH]
            if(save_start != save_len)
                if(save_start & 1) --save_start;
                if(!(save_len & 1)) ++save_len;
                save_len -= (save_start - 1);
                saveram = (unsigned char*)malloc(save_len);
                // If save RAM does not overlap main ROM, set it active by default since
                // a few games can't manage to properly switch it on/off.
                if(save_start >= (unsigned)size)
                    save_active = 1;
            else
                save_start = save_len = 0;
                saveram = NULL;
        else
            save_start = save_len = 0;
            saveram = NULL;
        return AS3_Int(0);
    AS3_Val sega_init(void* self, AS3_Val args)
        system_init();
        audioSamples = (44100 / vdp_rate)*2;
        framebuffer = (uint8*)malloc(FRAMEBUFFER_LENGTH);
        return AS3_Int(vdp_rate);
    AS3_Val sega_reset(void* self, AS3_Val args)
        system_reset();
        return AS3_Int(0);
    AS3_Val sega_frame(void* self, AS3_Val args)
        uint32 width;
        uint32 height;
        uint32 x, y;
        uint32 di, si, r;
        uint16 p;
        AS3_Val fb_ba;
        AS3_ArrayValue(args, "AS3ValType", &fb_ba);
        system_frame(0);
        AS3_SetS(fb_ba, "position", AS3_Int(0));
        width = (reg[12] & 1) ? 320 : 256;
        height = (reg[1] & 8) ? 240 : 224;
        for(y=0;y<240;y++)
            for(x=0;x<320;x++)
                di = 1280*y + x<<2;
                si = (y << 10) + ((x + bitmap.viewport.x) << 1);
                p = *((uint16*)(bitmap.data + si));
                framebuffer[di + 3] = (uint8)((p & 0x1f) << 3);
                framebuffer[di + 2] = (uint8)(((p >> 5) & 0x1f) << 3);
                framebuffer[di + 1] = (uint8)(((p >> 10) & 0x1f) << 3);
        AS3_ByteArray_writeBytes(fb_ba, framebuffer, FRAMEBUFFER_LENGTH);
        AS3_SetS(fb_ba, "position", AS3_Int(0));
        r = (width << 16) | height;
        return AS3_Int(r);
    AS3_Val sega_audio(void* self, AS3_Val args)
        AS3_Val ab_ba;
        AS3_ArrayValue(args, "AS3ValType", &ab_ba);
        AS3_SetS(ab_ba, "position", AS3_Int(0));
        AS3_ByteArray_writeBytes(ab_ba, snd.buffer, audioSamples*sizeof(int16));
        AS3_SetS(ab_ba, "position", AS3_Int(0));
        return AS3_Int(0);
    int main()
        AS3_Val romMethod = AS3_Function(NULL, sega_rom);
        AS3_Val initMethod = AS3_Function(NULL, sega_init);
        AS3_Val resetMethod = AS3_Function(NULL, sega_reset);
        AS3_Val frameMethod = AS3_Function(NULL, sega_frame);
        AS3_Val audioMethod = AS3_Function(NULL, sega_audio);
        // construct an object that holds references to the functions
        AS3_Val result = AS3_Object("sega_rom: AS3ValType, sega_init: AS3ValType, sega_reset: AS3ValType, sega_frame: AS3ValType, sega_audio: AS3ValType",
            romMethod, initMethod, resetMethod, frameMethod, audioMethod);
        // Release
        AS3_Release(romMethod);
        AS3_Release(initMethod);
        AS3_Release(resetMethod);
        AS3_Release(frameMethod);
        AS3_Release(audioMethod);
        // notify that we initialized -- THIS DOES NOT RETURN!
        AS3_LibInit(result);
        // should never get here!
        return 0;

  • SQL Loader and Floating Point Numbers

    Hi
    I have a problem loading floating point numbers using SQL Loader. If the number has more than 8 significant digits SQL Loader rounds the number i.e. 1100000.69 becomes 1100000.7. The CTL file looks as follows
    LOAD DATA
    INFILE '../data/test.csv' "str X'0A'"
    BADFILE '../bad/test.bad'
    APPEND
    INTO TABLE test
    FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY '"'
    Amount CHAR
    and the data file as follows
    "100.15 "
    "100100.57 "
    "1100000.69 "
    "-2000000.33"
    "-100000.43 "
    the table defined as follows
    CREATE TABLE test
    Amount number(15,4)
    ) TABLESPACE NNUT050M1;
    after loading a select returns the following
    100.15
    100100.57
    1100000.7
    -2000000
    -100000.4
    Thanks in advance
    Russell

    Actually if you format the field to display as (say) 999,999,999.99, you will see the correct numbers loaded via SQL Loader.
    null

  • R6002 - floating point not loaded / C++ Runtime Error

    Good Evening,
    I have been having this problem with Adobe Bridge for a while and tonight sat down to try and solve it. Fresh version of Windows XP reinstalled all programs and this is still happening! Any Ideas?
    This error only occurs with Adobe Bridge, I can load Photoshop etc.. all fine.
    Error:
    "Runtime Error!
    Program: C:\ProgramFiles\Adobe\Adobe Bridge CS3\Bridge.exe
    R6002
    - floating point not loaded"
    Here is a print screen..
    http://images.unbrokenphotography.co.uk/BridgeError.jpg

    Is there are answer to this problem?  This error message is appearing on an entire lab full of computers.  We are running Web Premium CS4
    I have tried to reset the Bridge Preferences:
    Hold down the Ctrl key and click on Bridge icon to start.
    Should get a reset window with 3 options.
    Choose the first option
    I still get "Runtime Error!   Program: C:\Prgram Files\Adobe\Adobe Bridge CS4\Bridge.exe  R6002 -floating point support not loaded"

  • Check Floating Point Number

    Hello All,
    I am having some trouble checking the value of a field with Key Figure type Number with 8 byte floating point. I want to read that field and populate another field with an X if true. For example if that field is equal to 5,0000000000000000E+07 then i want to mark the other field with an 'X'.
    The problem is in my code, how do i read that number in the fltp field, such as the number above. my code reads as follows for the 'X' field.
        if SOURCE_FIELDS-abc123 eq 5000000.
          RESULT = 'X'.
        endif.
    Thanks everyone in advance

    You don't need to worry about converting the code into standard format or floating, just implement your code as you want and it will automatically take care of the conversion. Basically 5,0000000000000000E+07 = 50,000,000.
    thanks.
    Wond

  • Reading Floating Point PLC5 Registers Kills my ability to write to any register

    When ever I create an object that reads a PLC5 Floating Point Register(F8:5),it kills my ability to write to any other PLC register. The problem only shows up with Floating Point registers. If I use integer registers (N7:121 etc.)I can read/write fine. It also severely slows down my normal driver scan update time. Even though it is set to scan every 30 seconds, I may only get an update every few minutes. Anyone else ever run into this? Thanks in advance!

    Hi,
    It might be helpful creating a log file to monitor com port activity.
    Here's the Knowledgebase article about it.
    http://digital.ni.com/public.nsf/3efedde4322fef198​62567740067f3cc/cac576863ff648a186256b5900794761?O​penDocument
    Another way to check the serial port activity would be using "portmon"
    software available at
    http://www.sysinternals.com/ntw2k/freeware/portmon​.shtml
    I hope this helps.
    Remzi A.
    Applications Engineering
    National Instruments

  • Floating Point Arithmatic Error

    Hi,
    I know actionscript represents numbers and double precision
    floating point values. I'm having a problem where double arithmatic
    in actionscript doesn't match the results of the same double
    arithmatic in C++ / C#.
    EXAMPLE:
    In C++ / C#:
    double x, y, x1, y1;
    x = 209.4;
    y = 148.8;
    x1 = 203.0;
    y1 = 145.0;
    double ddx = x - x1;
    double ddy = y - y1;
    RESULT
    ddx: 6.4000000000000057
    ddy: 3.8000000000000114
    In Flash ActionScipt 2:
    var x, y, x1, y1;
    x = 209.4;
    y = 148.8;
    x1 = 203.0;
    y1 = 145.0;
    var ddx = x - x1;
    var ddy = y - y1;
    RESULT
    ddx: 6.39999999999992
    ddy: 3.80000000000024
    After researching Flash / Actionscript "var" stores numerical
    values as doubles ( 8 bytes ) just like doubles are stored in C++ /
    C# ( 8 bytes ). Why would there be a difference between the results
    of ddx and ddy? Are there different implementations of double
    floating point math? If so, Is there a way I can mimic the Flash /
    Actionscript version in C++ / C#?
    Any help would be great!
    Thanks!

    Hmmm, so you're saying the actual binary representation is
    the same but they're just displayed differently?

  • Floating point arithmetic

    Hi everybody,
    This line:
    System.out.println((0.1+0.7)*10);outputs 7.999999999999999
    This is due to how floating point numbers are stored. When writing
    a code, sometimes it behaves in an intended way, sometimes it doesn't
    (like the one above). Is there a way to "predict" when the code is ok and
    when isn't ? Are there any tips to be aware of to get around that kind
    of problems ?
    Cheers,
    Adrian

    No. Using BigDecimal just because you don't understand how floating-point numbers work would be... um... short-sighted. And it wouldn't help, either. As soon as you divide 1 by 3 then you have to know how decimal numbers work, which is essentially the same problem.
    Edit: I forgot the forum hasn't been automated to provide the mandatory link for people who ask this question. We still have to do it by hand.
    http://docs.sun.com/source/806-3568/ncg_goldberg.html
    Edited by: DrClap on Oct 11, 2007 3:02 PM

  • Floating point formats: Java/C/C++, PPC and Intel platforms

    Hi everyone
    Where can I find out about the various bit formats used for 32 bit floating numbers in Java and C/C++ for both Mac hardware platforms?
    I'm developing a Java audio application which needs to convert vast quantities of variable width integer audio samples to canonical float audio format. I've discovered that a floating point divide by the maximum integer value gives the correct answer but takes too much processor time, so I'm trying out bit-twiddling in C via JNI to carve out my own floating point bit patterns. This is very fast, however, I need to take into account the various float formats used on the different platforms so my app can be universal. Can anyone point me to the information?
    Thanks in advance.
    Bob

    I am not sure that Rosetta floating point works the same as PPC floating point. I was using RealBasic (a PPC basic compiler) and moved one of the my compiled applications to a MacBook Pro and floating point comparisons that had been exact on the PPC stopped working under Rosetta. I changed the code to do an approximate comparison (i.e. abs(a -b) < tolerance) and this fixed things.
    I reported the problem to the RealBasic people and thought nothing more of it until I fired up Adobe's InDesign and not being used to working with picas, changed the units of measurement to inches. The default letter paper size was suddenly 8.5000500050005 inches instead of the more usual 8.5! This was not a big problem, but it appears that all of InDesign's page math is running into some kind of rounding errors.
    The floating point format is almost certainly IEEE, and I cannot imagine Rosetta doing anything other than using native hardware Intel floating point. On the other hand, there is a subtle difference in behavior.
    I am posting this here as a follow up, but I am also going to post this as a proper question in the forum. If you have to delete one or the other of these duplicate posts, please zap the reply, not the question.

  • BUG: Large floating point numbers convert to the wrong integer

    Hi,
    When using the conversion "bullets" to convert SGL, DBL and EXT to integers there are some values which convert wrong. One example is the integer 9223370937343148030, which can be represented exactly as a SGL (and thus exactly as DBL and EXT as well). If you convert this to I64 you get 9223370937343148032 instead, even though the correct integer is within the range of an I64. There are many similar cases, all (I've noticed) within the large end of the ranges.
    This has nothing to do with which integers can be represented exactly as a floating point value or not. This is a genuine conversion bug mind you.
    Cheers,
    Steen
    CLA, CTA, CLED & LabVIEW Champion
    Solved!
    Go to Solution.

    Yes, I understand the implications involved, and there definetely is a limit to how many significant digits that can be displayed in the numeric controls and constants today. I think that either this limit should be lifted or a cap should be put onto the configuration page when setting the display format.
    I ran into this problem as I'm developing a new toolset that lets you convert all the numeric formats into any other numeric format, just like the current "conversion bullets". My conversion bullets have outputs for overflow and exact conversion as well, since I need that functionality myself for a Math toolset (GPMath) I'm also developing. Eventually I'll maybe include underflow as well, but for now just those two outputs are available. Example:
    I do of course pay close attention to the binary representation of the numbers to calculate the Exact conversion? output correctly for each conversion variation (there are hundreds of VIs in polymorphic wrappers), but I relied in some cases on the ability of the numeric indicator to show a true number when configured appropriately - that was when I discovered this bug, which I at first mistook for a conversion error in LabVIEW.
    Is there a compliancy issue with EXT?
    While doing this work I've discovered that the EXT format is somewhat misleadingly labelled as "80-bit IEEE compliant" (it says so here), but that statement should be read with some suspicion IMO. The LabVIEW EXT is not simply IEEE 754-1985 compliant anyways, as that format would imply the x87 80-bit extended format. An x87 IEEE 754 extended precision float only has 63-bit fraction and a 1-bit integer part. That 1-bit integer part is implicit in single and double precision IEEE 754 numbers, but it is explicit in x87 extended precision numbers. LabVIEW EXT seems to have an implicit integer part and 64-bit fraction, thus not straight IEEE 754 compliant. Instead I'd say that the LabVIEW EXT is an IEEE 754r extended format, but still a proprietary one that should deserve a bit more detail in the available documentation. Since it's mentioned several places in the LabVIEW documentation that the EXT is platform independent, your suspicion should already be high though. It didn't take me many minutes to verify the apparent format of the EXT in any case, so no real problem here.
    Is there a genuine conversion error from EXT to U64?
    The integer 18446744073709549568 can be represented exactly as EXT using this binary representation (mind you that the numeric indicators won't display the value correctly, but instead show 18446744073709549600):
    EXT-exponent: 0x100000000111110b
    EXT-fraction: 0x1111111111111111111111111111111111111111111111111111000000000000b
    --> Decimal: 18446744073709549568
    The above EXT value converts exactly to U64 using the To Unsigned Quad Integer "bullet". But then let's try to flip the blue bit from 0 to 1 in the fraction part of the EXT, making this value:
    EXT-exponent: 0x100000000111110b
    EXT-fraction: 0x1111111111111111111111111111111111111111111111111111100000000000b
    --> Decimal: 18446744073709550592
    The above EXT value is still within U64 range, but the To Unsigned Quad Integer "bullet" converts it to U64_max which is 18446744073709551615. Unless I've missed something this must be a genuine conversion error from EXT to U64?
    /Steen
    CLA, CTA, CLED & LabVIEW Champion

  • Separator for floating point numbers

    Hello,
    I work with oracle 9 and have a problem with the entry of floating point numbers.
    The separator for floating point numbers in my data is a point (5.60).
    The pre-setting of oracle is a comma (5,60).
    By inserting I get the error message:
    01722. 00000 - "invalid number"
    How can i change this setting
    Thanks for Help
    F.

    Hi,
    I'm not sure if I understood your problem, however the NLS_NUMERIC_CHARACTERS variable specifies the characters to use as the group separator and decimal character.
    SQL> create table t1 (val number);
    Table created.
    SQL> select * from nls_session_parameters;
    PARAMETER                      VALUE
    NLS_LANGUAGE                   AMERICAN
    NLS_TERRITORY                  BRAZIL
    NLS_CURRENCY                   R$
    NLS_ISO_CURRENCY               BRAZIL
    NLS_NUMERIC_CHARACTERS ,.
    NLS_CALENDAR                   GREGORIAN
    NLS_DATE_FORMAT                DD/MM/YYYY
    NLS_DATE_LANGUAGE              AMERICAN
    NLS_SORT                       BINARY
    NLS_TIME_FORMAT                HH24:MI:SSXFF
    NLS_TIMESTAMP_FORMAT           DD/MM/RR HH24:MI:SSXFF
    NLS_TIME_TZ_FORMAT             HH24:MI:SSXFF TZR
    NLS_TIMESTAMP_TZ_FORMAT        DD/MM/RR HH24:MI:SSXFF TZR
    NLS_DUAL_CURRENCY              Cr$
    NLS_COMP                       BINARY
    NLS_LENGTH_SEMANTICS           BYTE
    NLS_NCHAR_CONV_EXCP            FALSE
    17 rows selected.
    SQL> insert into t1 values (1.50);
    1 row created.
    SQL> select * from t1;
           VAL
           1,5
    SQL> alter session set nls_numeric_characters='.,';
    Session altered.
    SQL> select * from t1;
           VAL
           1.5Cheers
    Legatti

  • Floating point numbers into XML file

    Hi,
    I am a learner in Labview and I am using Labview 8.5 version.
    when I use FlattentoXML component to convert floating point numbers ( having more than 5 decimal points) into XML, the output contains alway 5 decimal points.
    But, I want exact decimal number to be displayed as XML. i.e 0.263746 should not be displayed as 0.26375 in XML.
    Do you have any suggestions ?
    Attachments:
    Float_to_XML.vi ‏7 KB

    I tested it and could not see your problem in Labview 2009. So it is perhaps a bug in your Labview version. You can use this VI as a workaround.
    Besides which, my opinion is that Express VIs Carthage must be destroyed deleted
    (Sorry no Labview "brag list" so far)
    Attachments:
    Float_to_XML[2].vi ‏8 KB

  • MIDP and floating-point numbers

    Hiya,
    I need to use floating numbers in a few occasions in a MIDP app that I'm programming at the moment. Whenever I do so though I get a nervous-breaking error at the pre-verification stage of the build. I am working with Wireless Toolkit 2.0 and the rror I get is:
    Building "MyMIDlet"
    Error preverifying class <ClassName>
    ERROR: floating-point constants should not appear
    com.sun.kvem.ktools.ExecutionException: Preverifier returned 1
    Build failedI get this error whenever I use a floating-point varible or even whenever I compare something to a floating-point value.
    It's got me really puzzled and I can't think of any way around the problem. Any help would be very much appreciated...
    Thanks!

    also WTK 2.1 (and CLDC 1.1I think ) does support FP. You think right, CLDC 1.1, that's when they added the floats (in theory a device could be MIDP 1.0 on top of CLDC 1.1, and it would also have floats). There's only one CLDC 1.1 device I've heard of: Nokia 6320.
    shmoove

Maybe you are looking for

  • How to use NVL Function

    hi, i want to use nvl Function in that Quary select f_words(SUM(a.AMOUNT)+sum(a.vat_amount) + nvl( (select sum(b.LABOUR_AMT)+sum(b.service_tax_amt) from LAB_WORK_DTL b where a.bill_no =b.bill_no), 0) )) as t from TRANSACTION_DETAILS a where a.bill_no

  • Special characters in Lion

    The ease of use of special characters has been a major strength of the Mac OS since the very foundation. Now I find in Lion I hvae to do a very Windoze-like thing and open a Special Characters dialog and search for the character I want from a confusi

  • Re-routing of SNMP traps from port 162 to port greater than 1024

    Hello, I have to re-route SNMP traps received at port 162 to some other port greater that 1024 (say 2041). There is an application which sends SNMP traps to port 162 and our application running on Windows listens for SNMP traps on port 2041, so we wa

  • Flash music player HELP on Safari

    Trying to listen to the music stream on http://www.abercrombie.co.uk/anf/flash/music/music.html However it wont stream the music..any ideas why? was working a few weeks back but now wont stream...not sure if its a safari or flash player issue...maybe

  • Steps to create workbook and need to insert 10 query's in the workbook

    Hi, Please let me know Steps to create workbook and need to insert 10 query's in the workbook. Thanks, rajesh b