Converting Floating Point

I am receiving numbers from a Tinius Olsen Model 290 Universal Testing System.  I made a little program to send commands to the machine and receive the responses back as strings.  For the decimal value 3.7620 I get the floating point number EDC57040.  This is obviously not correct.  The correct floating point representation is 4070C5ED, which is accomplished by swapping all the bits.  I need to know how to do this in labview, basically go from the string representation of the floating point to the correct swapped floating point.

Here is a simple example:
Be aware that if you have an ancient version of LabVIEW, this option (byte order) is not available and you need to swap the bytes manually. No big deal, really.
Message Edited by altenbach on 05-08-2007 01:15 PM
LabVIEW Champion . Do more with less code and in less time .
Attachments:
LittleEndianSGL.png ‏7 KB

Similar Messages

  • Convert Floating Point Decimal to Hex

    In my application I make some calculations using floating point format DBL,and need to write these values to a file in IEEE 754 Floating Point Hex format. Is there any way to do this using LabVIEW?

    Mike,
    Good news. LabVIEW has a function that does exactly what you want. It is well hidden though...
    In the Advanced/Data manipulation palette there is a function called Flatten to String. If you feed this funtion with your DBL precision digital value you get the IEEE-754 hexadecimal floating point representation (64 bit) at the data string terminal (as a text string).
    I attached a simple example that shows how it works.
    Hope this helps. /Mikael Garcia
    Attachments:
    ieee754converter.vi ‏10 KB

  • Profibus data type converting to floating point

    Hi, 
    Is there an efficient way to convert  the incoming data to the floating point in the NI cRIO Profibus system? 
    1) The system is : NI cRIO 9068 Controller with Comsoft profibus slave module. The lavview we are using is 2013. 
    2) We are using profibus slave example and are able to see a array of unsigned 8 bit data. Please see the Output data shown in the attachement.
    3) On the other side of profibus system, thrid party profibus master are converting floating points and tranmit the converted data to NI profibus slave. 
    Attachments:
    Screenshoot.png ‏7 KB

    If it's just a matter of converting data types once you have the data in LabVIEW, you can always manually scale and convert the data using the arithmetic functions and the "To Double Precision Float" or "To Single Precision Float" functions. You just have to know what floating point value the unsigned byte integer corresponds to. Is that what you're asking?
    If you're asking for a way to this inherently with the Profibus functions, I'm afraid I can't be of much help...
    Ryan K.

  • Floating point numbers into XML file

    Hi,
    I am a learner in Labview and I am using Labview 8.5 version.
    when I use FlattentoXML component to convert floating point numbers ( having more than 5 decimal points) into XML, the output contains alway 5 decimal points.
    But, I want exact decimal number to be displayed as XML. i.e 0.263746 should not be displayed as 0.26375 in XML.
    Do you have any suggestions ?
    Attachments:
    Float_to_XML.vi ‏7 KB

    I tested it and could not see your problem in Labview 2009. So it is perhaps a bug in your Labview version. You can use this VI as a workaround.
    Besides which, my opinion is that Express VIs Carthage must be destroyed deleted
    (Sorry no Labview "brag list" so far)
    Attachments:
    Float_to_XML[2].vi ‏8 KB

  • Floating point conversation

    Dear all,
        In a particular filed i entered value as 60.800 kg. But when i try to pick the value   i found that the value is stored in floating pont format ie 6.080000000000e+00 . Actually i need it in integer format .Is any function module available to convert floating point to integer.
    Regards
    Mahesh V

    Why use a function module?
    DATA: l_val_int type i,
          l_val_flt type f,
          l_val_pck type p decimals 2.
    l_val_flt = '6.080000000000e+00'.
    l_val_int = l_val_flt.
    l_val_pck = l_val_pck.
    matt

  • Floating point to time conversion

    Hi all,
      I need to convert floating point value to time.Any help would be gr8

    Hai,
       Define two variables,one as float & another one as time take a & b.After assigning values to a,assign a to b.Then write b.The float value will be converted into time format.
    Regards,
    Padmam.

  • BUG: Large floating point numbers convert to the wrong integer

    Hi,
    When using the conversion "bullets" to convert SGL, DBL and EXT to integers there are some values which convert wrong. One example is the integer 9223370937343148030, which can be represented exactly as a SGL (and thus exactly as DBL and EXT as well). If you convert this to I64 you get 9223370937343148032 instead, even though the correct integer is within the range of an I64. There are many similar cases, all (I've noticed) within the large end of the ranges.
    This has nothing to do with which integers can be represented exactly as a floating point value or not. This is a genuine conversion bug mind you.
    Cheers,
    Steen
    CLA, CTA, CLED & LabVIEW Champion
    Solved!
    Go to Solution.

    Yes, I understand the implications involved, and there definetely is a limit to how many significant digits that can be displayed in the numeric controls and constants today. I think that either this limit should be lifted or a cap should be put onto the configuration page when setting the display format.
    I ran into this problem as I'm developing a new toolset that lets you convert all the numeric formats into any other numeric format, just like the current "conversion bullets". My conversion bullets have outputs for overflow and exact conversion as well, since I need that functionality myself for a Math toolset (GPMath) I'm also developing. Eventually I'll maybe include underflow as well, but for now just those two outputs are available. Example:
    I do of course pay close attention to the binary representation of the numbers to calculate the Exact conversion? output correctly for each conversion variation (there are hundreds of VIs in polymorphic wrappers), but I relied in some cases on the ability of the numeric indicator to show a true number when configured appropriately - that was when I discovered this bug, which I at first mistook for a conversion error in LabVIEW.
    Is there a compliancy issue with EXT?
    While doing this work I've discovered that the EXT format is somewhat misleadingly labelled as "80-bit IEEE compliant" (it says so here), but that statement should be read with some suspicion IMO. The LabVIEW EXT is not simply IEEE 754-1985 compliant anyways, as that format would imply the x87 80-bit extended format. An x87 IEEE 754 extended precision float only has 63-bit fraction and a 1-bit integer part. That 1-bit integer part is implicit in single and double precision IEEE 754 numbers, but it is explicit in x87 extended precision numbers. LabVIEW EXT seems to have an implicit integer part and 64-bit fraction, thus not straight IEEE 754 compliant. Instead I'd say that the LabVIEW EXT is an IEEE 754r extended format, but still a proprietary one that should deserve a bit more detail in the available documentation. Since it's mentioned several places in the LabVIEW documentation that the EXT is platform independent, your suspicion should already be high though. It didn't take me many minutes to verify the apparent format of the EXT in any case, so no real problem here.
    Is there a genuine conversion error from EXT to U64?
    The integer 18446744073709549568 can be represented exactly as EXT using this binary representation (mind you that the numeric indicators won't display the value correctly, but instead show 18446744073709549600):
    EXT-exponent: 0x100000000111110b
    EXT-fraction: 0x1111111111111111111111111111111111111111111111111111000000000000b
    --> Decimal: 18446744073709549568
    The above EXT value converts exactly to U64 using the To Unsigned Quad Integer "bullet". But then let's try to flip the blue bit from 0 to 1 in the fraction part of the EXT, making this value:
    EXT-exponent: 0x100000000111110b
    EXT-fraction: 0x1111111111111111111111111111111111111111111111111111100000000000b
    --> Decimal: 18446744073709550592
    The above EXT value is still within U64 range, but the To Unsigned Quad Integer "bullet" converts it to U64_max which is 18446744073709551615. Unless I've missed something this must be a genuine conversion error from EXT to U64?
    /Steen
    CLA, CTA, CLED & LabVIEW Champion

  • Convert string to floating-point

    Hi all,
    ..very basic question, but I tryed it for hours and only received short-dumps
    <b>How can I convert a string into a floating-point number?</b>
    Kind regards,
    Stefan

    hi
    try this
    to convert  string to float.
    data : a type f,
    s type string value '1.023'.
    a = s.
    write :/ a.
    to convert float to string.
    data : a type f value '1.023',
    s type string.
    s = a.
    write : s.

  • Converting binary to floating point?

    ok, i am programming a little java program at the consulter
    the main idea of the program is to convert a decimal number to IEEE (floating point)
    i am almost done except the exponent part
    i dont understand the logic behind it
    so, for example:
    -6.625 = 110.101 (binary)
    after normalization it will be -1.10101 * 2^2
    so the IEEE will be
    1 10000001 10101000000000000000000
    i understand the sign part and the fraction part
    but i have no idea how the exponent part came like this
    the book say that 129-127=+2 so the exponent is 10000001
    da, where it came from ???
    i will appreciate it if someone explain this part for me step by step
    thank you,
    Edited by: abdoh2010 on Jan 26, 2008 2:37 AM

    got it
    thank you for viewing my question

  • Convertion Hex String to 32bits Decimal floating point??

    Hi,
    I would like to know how to convert hexa like: 416b0ac3 in decimal 32bits floating-point. The result of this string is suppose to be 14.690127.
    So i must be able to do:
    From 32-bit Hexadecimal Representation To Decimal Floating-Point
    Thanks for your support
    RiderMerlin

    RiderMerlin
    You can use the typecast function to do this.
    David
    Message Edited by David Crawford on 09-06-2006 03:31 PM
    Attachments:
    Typecast to Single.jpg ‏6 KB

  • Conversion of a floating point type field

    Hi,
    I'm fetching field ATFLV from table AUSP for a particular value of ATINN.ATFLV is a floating point type field.
    Can anyone please guide me as to how to convert this field(ATFLV) from floating point no. to a simple no.?
    Helpful answers will be rewarded.
    Regards,
    Sipra

    hi,
    Do like this,
    float f = 234.33;
    int i = (int) f; // i has value 234.
    reward points if helpful..

  • Check Floating Point Number

    Hello All,
    I am having some trouble checking the value of a field with Key Figure type Number with 8 byte floating point. I want to read that field and populate another field with an X if true. For example if that field is equal to 5,0000000000000000E+07 then i want to mark the other field with an 'X'.
    The problem is in my code, how do i read that number in the fltp field, such as the number above. my code reads as follows for the 'X' field.
        if SOURCE_FIELDS-abc123 eq 5000000.
          RESULT = 'X'.
        endif.
    Thanks everyone in advance

    You don't need to worry about converting the code into standard format or floating, just implement your code as you want and it will automatically take care of the conversion. Basically 5,0000000000000000E+07 = 50,000,000.
    thanks.
    Wond

  • Floating point formats: Java/C/C++, PPC and Intel platforms

    Hi everyone
    Where can I find out about the various bit formats used for 32 bit floating numbers in Java and C/C++ for both Mac hardware platforms?
    I'm developing a Java audio application which needs to convert vast quantities of variable width integer audio samples to canonical float audio format. I've discovered that a floating point divide by the maximum integer value gives the correct answer but takes too much processor time, so I'm trying out bit-twiddling in C via JNI to carve out my own floating point bit patterns. This is very fast, however, I need to take into account the various float formats used on the different platforms so my app can be universal. Can anyone point me to the information?
    Thanks in advance.
    Bob

    I am not sure that Rosetta floating point works the same as PPC floating point. I was using RealBasic (a PPC basic compiler) and moved one of the my compiled applications to a MacBook Pro and floating point comparisons that had been exact on the PPC stopped working under Rosetta. I changed the code to do an approximate comparison (i.e. abs(a -b) < tolerance) and this fixed things.
    I reported the problem to the RealBasic people and thought nothing more of it until I fired up Adobe's InDesign and not being used to working with picas, changed the units of measurement to inches. The default letter paper size was suddenly 8.5000500050005 inches instead of the more usual 8.5! This was not a big problem, but it appears that all of InDesign's page math is running into some kind of rounding errors.
    The floating point format is almost certainly IEEE, and I cannot imagine Rosetta doing anything other than using native hardware Intel floating point. On the other hand, there is a subtle difference in behavior.
    I am posting this here as a follow up, but I am also going to post this as a proper question in the forum. If you have to delete one or the other of these duplicate posts, please zap the reply, not the question.

  • Inline functions in C, gcc optimization and floating point arithmetic issues

    For several days I really have become a fan of Alchemy. But after intensive testing I have found several issues which I'd like to solve but I can't without any help.
    So...I'm porting an old game console emulator written by me in ANSI C. The code is working on both gcc and VisualStudio without any modification or crosscompile macros. The only platform code is the audio and video output which is out of scope, because I have ported audio and video witin AS3.
    Here are the issues:
    1. Inline functions - Having only a single inline function makes the code working incorrectly (although not crashing) even if any optimization is enabled or not (-O0 or O3). My current workarround is converting the inline functions to macros which achieves the same effect. Any ideas why inline functions break the code?
    2. Compiler optimizations - well, my project consists of many C files one of which is called flash.c and it contains the main and exported functions. I build the project as follows:
    gcc -c flash.c -O0 -o flash.o     //Please note the -O0 option!!!
    gcc -c file1.c -O3 -o file1.o
    gcc -c file2.c -O3 -o file2.o
    ... and so on
    gcc *.o -swc -O0 -o emu.swc   //Please note the -O0 option again!!!
    mxmlc.exe -library-path+=emu.swc --target-player=10.0.0 Emu.as
    or file in $( ls *.o ) //Removes the obj files
        do
            rm $file
        done
    If I define any option different from -O0 in gcc -c flash.c -O0 -o flash.o the program stops working correctly exactly as in the inline funtions code (but still does not crash or prints any errors in debug). flash has 4 static functions to be exported to AS3 and the main function. Do you know why?
    If I define any option different from -O0 in gcc *.o -swc -O0 -o emu.swc  the program stops working correctly exactly as above, but if I specify -O1, -O2 or O3 the SWC file gets smaller up to 2x for O3. Why? Is there a method to optimize all the obj files except flash.o because I suspect a similar issue as when compilling it?
    3. Flating point issues - this is the worst one. My code is mainly based on integer arithmetic but on 1-2 places it requires flating point arithmetic. One of them is the conversion of 16-bit 44.1 Khz sound buffer to a float buffer with same sample rate but with samples in the range from -1.0 to 1.0.
    My code:
    void audio_prepare_as()
        uint32 i;
        for(i=0;i<audioSamples;i+=2)
            audiobuffer[i] = (float)snd.buffer[i]/32768;
            audiobuffer[i+1] = (float)snd.buffer[i+1]/32768;
    My audio playback is working perfectly. But not if using the above conversion and I have inspected the float numbers - all incorrect and invalid. I tried other code with simple floats - same story. As if alchemy refuses to work with floats. What is wrong? I have another lace whre I must resize the framebuffer and there I have a float involved - same crap. Please help me?
    Found the floating point problem: audiobuffer is written to a ByteArray and then used in AS. But C floats are obviously not the same as those in AS3. Now the floating point is resolved.
    The optimization issues remain! I really need to speed up my code.
    Thank you in advice!

    Dear Bernd,
    I am still unable to run the optimizations and turn on the inline functions. None of the inline functions contain any stdli function just pure asignments, reads, simple arithmetic and bitwise operations.
    In fact, the file containing the main function and those functions for export in AS3 did have memset and memcpy. I tried your suggestion and put the code above the functions calling memset and memcpy. It did not work soe I put the code in a header which is included topmost in each C file. The only system header I use is malloc.h and it is included topmost. In other C file I use pow, sin and log10 from math.h but I removed it and made the same thing:
    //shared.h
    #ifndef _SHARED_H_
    #define _SHARED_H_
    #include <malloc.h>
    static void * custom_memmove( void * destination, const void * source, unsigned int num ) {
      void *result; 
      __asm__("%0 memmove(%1, %2, %3)\n" : "=r"(result) : "r"(destination), "r"(source), "r"(num)); 
      return result; 
    static void * custom_memcpy ( void * destination, const void * source, unsigned int num ) { 
      void *result; 
      __asm__("%0 memcpy(%1, %2, %3)\n" : "=r"(result) : "r"(destination), "r"(source), "r"(num)); 
      return result; 
    static void * custom_memset ( void * ptr, int value, unsigned int num ) { 
      void *result; 
      __asm__("%0 memset(%1, %2, %3)\n" : "=r"(result) : "r"(ptr), "r"(value), "r"(num)); 
      return result; 
    static float custom_pow(float x, int y) {
        float result;
      __asm__("%0 pow(%1, %2)\n" : "=r"(result) : "r"(x), "r"(y));
      return result;
    static double custom_sin(double x) {
        double result;
      __asm__("%0 sin(%1)\n" : "=r"(result) : "r"(x));
      return result;
    static double custom_log10(double x) {
        double result;
      __asm__("%0 log10(%1)\n" : "=r"(result) : "r"(x));
      return result;
    #define memmove custom_memmove
    #define memcpy custom_memcpy
    #define memset custom_memset
    #define pow custom_pow
    #define sin custom_sin
    #define log10 custom_log10 
    #include "types.h"
    #include "macros.h"
    #include "m68k.h"
    #include "z80.h"
    #include "genesis.h"
    #include "vdp.h"
    #include "render.h"
    #include "mem68k.h"
    #include "memz80.h"
    #include "membnk.h"
    #include "memvdp.h"
    #include "system.h"
    #include "loadrom.h"
    #include "input.h"
    #include "io.h"
    #include "sound.h"
    #include "fm.h"
    #include "sn76496.h" 
    #endif /* _SHARED_H_ */ 
    It still behave the same way as if nothing was changed (works incorrectly - displays jerk which does not move, whereby the image is supposed to move)
    As I am porting an emulator (Sega Mega Drive) I use manu arrays of function pointers for implementing the opcodes of the CPU's. Could this be an issue?
    I did a workaround for the floating point problem but processing is very slow so I hear only bzzt bzzt but this is for now out of scope. The emulator compiled with gcc runs at 300 fps on a 1.3 GHz machine, whereby my non optimized AVM2 code compiled by alchemy produces 14 fps. The pure rendering is super fast and the problem lies in the computational power of AVM. The frame buffer and the enulation are generated in the C code and only the pixels are copied to AS3, where they are plotted in a BitmapData. On 2.0 GHz Dual core I achieved only 21 fps. Goal is 60 fps to have smooth audio and video. But this is offtopic. After all everything works (slow) without optimization, and I would somehow turn it on. Suggestions?
    Here is the file with the main function:
    #include "shared.h"
    #include "AS3.h"
    #define FRAMEBUFFER_LENGTH    (320*240*4)
    static uint8* framebuffer;
    static uint32  audioSamples;
    AS3_Val sega_rom(void* self, AS3_Val args)
        int size, offset, i;
        uint8 hardware;
        uint8 country;
        uint8 header[0x200];
        uint8 *ptr;
        AS3_Val length;
        AS3_Val ba;
        AS3_ArrayValue(args, "AS3ValType", &ba);
        country = 0;
        offset = 0;
        length = AS3_GetS(ba, "length");
        size = AS3_IntValue(length);
        ptr = (uint8*)malloc(size);
        AS3_SetS(ba, "position", AS3_Int(0));
        AS3_ByteArray_readBytes(ptr, ba, size);
        //FILE* f = fopen("boris_dump.bin", "wb");
        //fwrite(ptr, size, 1, f);
        //fclose(f);
        if((size / 512) & 1)
            size -= 512;
            offset += 512;
            memcpy(header, ptr, 512);
            for(i = 0; i < (size / 0x4000); i += 1)
                deinterleave_block(ptr + offset + (i * 0x4000));
        memset(cart_rom, 0, 0x400000);
        if(size > 0x400000) size = 0x400000;
        memcpy(cart_rom, ptr + offset, size);
        /* Free allocated file data */
        free(ptr);
        hardware = 0;
        for (i = 0x1f0; i < 0x1ff; i++)
            switch (cart_rom[i]) {
         case 'U':
             hardware |= 4;
             break;
         case 'J':
             hardware |= 1;
             break;
         case 'E':
             hardware |= 8;
             break;
        if (cart_rom[0x1f0] >= '1' && cart_rom[0x1f0] <= '9') {
            hardware = cart_rom[0x1f0] - '0';
        } else if (cart_rom[0x1f0] >= 'A' && cart_rom[0x1f0] <= 'F') {
            hardware = cart_rom[0x1f0] - 'A' + 10;
        if (country) hardware=country; //simple autodetect override
        //From PicoDrive
        if (hardware&8)        
            hw=0xc0; vdp_pal=1;
        } // Europe
        else if (hardware&4)    
            hw=0x80; vdp_pal=0;
        } // USA
        else if (hardware&2)    
            hw=0x40; vdp_pal=1;
        } // Japan PAL
        else if (hardware&1)      
            hw=0x00; vdp_pal=0;
        } // Japan NTSC
        else
            hw=0x80; // USA
        if (vdp_pal) {
            vdp_rate = 50;
            lines_per_frame = 312;
        } else {
            vdp_rate = 60;
            lines_per_frame = 262;
        /*SRAM*/   
        if(cart_rom[0x1b1] == 'A' && cart_rom[0x1b0] == 'R')
            save_start = cart_rom[0x1b4] << 24 | cart_rom[0x1b5] << 16 |
                cart_rom[0x1b6] << 8  | cart_rom[0x1b7] << 0;
            save_len = cart_rom[0x1b8] << 24 | cart_rom[0x1b9] << 16 |
                cart_rom[0x1ba] << 8  | cart_rom[0x1bb] << 0;
            // Make sure start is even, end is odd, for alignment
            // A ROM that I came across had the start and end bytes of
            // the save ram the same and wouldn't work.  Fix this as seen
            // fit, I know it could probably use some work. [PKH]
            if(save_start != save_len)
                if(save_start & 1) --save_start;
                if(!(save_len & 1)) ++save_len;
                save_len -= (save_start - 1);
                saveram = (unsigned char*)malloc(save_len);
                // If save RAM does not overlap main ROM, set it active by default since
                // a few games can't manage to properly switch it on/off.
                if(save_start >= (unsigned)size)
                    save_active = 1;
            else
                save_start = save_len = 0;
                saveram = NULL;
        else
            save_start = save_len = 0;
            saveram = NULL;
        return AS3_Int(0);
    AS3_Val sega_init(void* self, AS3_Val args)
        system_init();
        audioSamples = (44100 / vdp_rate)*2;
        framebuffer = (uint8*)malloc(FRAMEBUFFER_LENGTH);
        return AS3_Int(vdp_rate);
    AS3_Val sega_reset(void* self, AS3_Val args)
        system_reset();
        return AS3_Int(0);
    AS3_Val sega_frame(void* self, AS3_Val args)
        uint32 width;
        uint32 height;
        uint32 x, y;
        uint32 di, si, r;
        uint16 p;
        AS3_Val fb_ba;
        AS3_ArrayValue(args, "AS3ValType", &fb_ba);
        system_frame(0);
        AS3_SetS(fb_ba, "position", AS3_Int(0));
        width = (reg[12] & 1) ? 320 : 256;
        height = (reg[1] & 8) ? 240 : 224;
        for(y=0;y<240;y++)
            for(x=0;x<320;x++)
                di = 1280*y + x<<2;
                si = (y << 10) + ((x + bitmap.viewport.x) << 1);
                p = *((uint16*)(bitmap.data + si));
                framebuffer[di + 3] = (uint8)((p & 0x1f) << 3);
                framebuffer[di + 2] = (uint8)(((p >> 5) & 0x1f) << 3);
                framebuffer[di + 1] = (uint8)(((p >> 10) & 0x1f) << 3);
        AS3_ByteArray_writeBytes(fb_ba, framebuffer, FRAMEBUFFER_LENGTH);
        AS3_SetS(fb_ba, "position", AS3_Int(0));
        r = (width << 16) | height;
        return AS3_Int(r);
    AS3_Val sega_audio(void* self, AS3_Val args)
        AS3_Val ab_ba;
        AS3_ArrayValue(args, "AS3ValType", &ab_ba);
        AS3_SetS(ab_ba, "position", AS3_Int(0));
        AS3_ByteArray_writeBytes(ab_ba, snd.buffer, audioSamples*sizeof(int16));
        AS3_SetS(ab_ba, "position", AS3_Int(0));
        return AS3_Int(0);
    int main()
        AS3_Val romMethod = AS3_Function(NULL, sega_rom);
        AS3_Val initMethod = AS3_Function(NULL, sega_init);
        AS3_Val resetMethod = AS3_Function(NULL, sega_reset);
        AS3_Val frameMethod = AS3_Function(NULL, sega_frame);
        AS3_Val audioMethod = AS3_Function(NULL, sega_audio);
        // construct an object that holds references to the functions
        AS3_Val result = AS3_Object("sega_rom: AS3ValType, sega_init: AS3ValType, sega_reset: AS3ValType, sega_frame: AS3ValType, sega_audio: AS3ValType",
            romMethod, initMethod, resetMethod, frameMethod, audioMethod);
        // Release
        AS3_Release(romMethod);
        AS3_Release(initMethod);
        AS3_Release(resetMethod);
        AS3_Release(frameMethod);
        AS3_Release(audioMethod);
        // notify that we initialized -- THIS DOES NOT RETURN!
        AS3_LibInit(result);
        // should never get here!
        return 0;

  • Precision loss - conversions between exact values and floating point values

    Hi!
    I read this in your SQL Reference manual, but I don't quite get it.
    Conversions between exact numeric values (TT_TINYINT, TT_SMALLINT, TT_INTEGER, TT_BIGINT, NUMBER) and floating-point values (BINARY_FLOAT, BINARY_DOUBLE) can be inexact because the exact numeric values use decimal precision whereas the floating-point numbers use binary precision.
    Could you please give two examples: one where a TT_TINYINT is converted to a BINARY_DOUBLE and one when a TT_BIGINT is converted into a DOUBLE, both cases give examples on lost precision? This would be very helpful.
    Thanks!
    Sune

    chokpa wrote:
    Public Example (float... values){}
    new Example (1, 1e2, 3.0, 4.754);It accepts it if I just use 1,2,3,4 as the values being passed in, but doesn't like it if I use actual float values.Those are double literals, try
    new Example (1f, 1e2f, 3.0f, 4.754f);

Maybe you are looking for

  • Error run time  DYNPRO_FIELD_CONVERSION

    I have a dump: Errore run-time        DYNPRO_FIELD_CONVERSION        Verific. il     28.11.2007 h  02:12:16 A conversion error occurred. Causa dell'errore The current screen processing was terminated, since a situation occurred from which it could no

  • No Data in Downtown Los Angeles

    For the last 6 months or so, I have been experiencing 0 data in Downtown Los Angeles.  Specifically on 9th and Grand.  I have reported it three times and have not heard of any solution yet.  I did my own researching here in our building and I found o

  • Why can't constants of a package not be use in view definition?

    Hi, maybe the answer is obvious, but I don't understand why this is not possible. We have a large numer of key tables which define an ID-number (PK, generated by sequence), a short and long description as well as a numeric ID (AK1, in most cases auto

  • SOAP To File (Attachment swapping with Payload using Payload Swap Bean)

    Hi, I am working on the subject line. The scenario works well and good when I use the bean in Receiver File Communication Channel. The same bean configuration if I use on the sender SOAP the swapping of payload and attachment is not happening. Any Id

  • Missing icons and unformated web query

    Hi everyone. When i run a query with BEx, all icons are missing and text is not formated. What is happening? Kind regards, Inigo.