Unexpected floating part value

Hi,
I came across some really strange issue.
I’ve got column in my table declared as following:
VALUE numeric(20,5)
If I execute following update:
update [MyTable] set
VALUE = 8030769230769.25
where [SomeCondition]
and run select afterwards as following:
select VALUE from [MyTable]
where [SomeCondition]
I am getting back the following result:
8030769230769.25056
I could not figure out where *056* is coming from.
Please advice.
Thank you

In order to answer your question, we need the DB version, the platform and some exact SQL that produces this result. I.e., SQL that shows the table creation, the data update, the select statement and the result, cut and pasted from a SQL*Plus window.

Similar Messages

Easiest way to split a floating point value

Hello, I am trying to split a floating-point value (double) into its fractional and integer parts? I have been trying to find some sort of method in BigDecimal or Double that would do it but so far I haven't come up with anything. I know it could be done by searching through it as a String but that sounds like a very inefficent way to do it. I would appreciate any help you could give. Thanks

float f;
int i = (int) f;
float frac = f-i;

How to fix 'Unexpected error: Incorrect value: Location URL' in WSDL

Hi All,
Using SOAMANAGER transaction, I created a service ECC_CUSTOMERQUOTEERPCRTRC and its binding 'ECC_CUSTOMERQUOTEERPCRTRC_binding' using the Create Service option under the tab Configurations and using the web service definition 'ECC_CUSTOMERQUOTEERPCRTRC'. When I click on the link "Open WSDL document for selected binding" under the tab 'Overview', I get the following XML code in the popup window for the WSDL.
- <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
- <soap:Body>
- <soap:Fault>
<faultcode>soap:Server</faultcode>
<faultstring>Unexpected error: Incorrect value: Location URL for subject IF {http</faultstring>
- <detail xmlns:slibfault="http://xml.sap.com/2005/11/esi/slib/fault/">
<slibfault:timestamp>Wed, 17 Nov 2010 22:29:02 GMT</slibfault:timestamp>
- <slibfault:exception>
<slibfault:text>Incorrect value: Location URL for subject IF {http://sap.com/xi/APPL/Global2}:CustomerQuoteERPCreateRequestConfirmation_In not found</slibfault:text>
<slibfault:position program="CL_SIDL_SUBJECT_ADAPTER=======CP" include="CL_SIDL_SUBJECT_ADAPTER=======CM006" line="47" />
</slibfault:exception>
</detail>
</soap:Fault>
</soap:Body>
</soap:Envelope>
The XML code for nthe WSDL displays an error 'Unexpected error: Incorrect value: Location URL for subject IF '. It looks like the code could find the required URL. Please let me know how to fix this error.
Thanks in advance.
--R D

Hello.
In SOAMANAGER delete your "unexpected" service (endpoint).
Then create it again.
If same error will occurs - try to delete all definitions of this service (include sicf-transaction) and then create it completely again.
Hope it helps.

Precision loss - conversions between exact values and floating point values

Hi!
I read this in your SQL Reference manual, but I don't quite get it.
Conversions between exact numeric values (TT_TINYINT, TT_SMALLINT, TT_INTEGER, TT_BIGINT, NUMBER) and floating-point values (BINARY_FLOAT, BINARY_DOUBLE) can be inexact because the exact numeric values use decimal precision whereas the floating-point numbers use binary precision.
Could you please give two examples: one where a TT_TINYINT is converted to a BINARY_DOUBLE and one when a TT_BIGINT is converted into a DOUBLE, both cases give examples on lost precision? This would be very helpful.
Thanks!
Sune

chokpa wrote:
Public Example (float... values){}
new Example (1, 1e2, 3.0, 4.754);It accepts it if I just use 1,2,3,4 as the values being passed in, but doesn't like it if I use actual float values.Those are double literals, try
new Example (1f, 1e2f, 3.0f, 4.754f);

Floating point values

Hi all;
Please help.
I am getting floating point values in my source IDoc like 1.234- but while inserting into the ODS system i need the negative sign before 1.234.
How to achive this out.
Nalin

Hi
You can achive this by using graphical mapping UUser defined function.
Mapping source fieldremove context UDF --- target field
UDF
for(int i=0;i<Float.length;i++)
if(Float<i>.endsWith("-"))
result.addValue("-"+Float<i>.substring(0,Float<i>.length()-1));
else
result.addValue(Float<i>);
Mudit
Award points if it helps

Passing floating point values, but being recognised as doubles and int's?

Hi,
I was wondering how to differentiate between floating point values and ints etc
I am passing values into a public class with the parameters (float... values)
Public Example (float... values){}new Example (1, 1e2, 3.0, 4.754);
It accepts it if I just use 1,2,3,4 as the values being passed in, but doesn't like it if I use actual float values.
How come?
Cheers

chokpa wrote:
Public Example (float... values){}
new Example (1, 1e2, 3.0, 4.754);It accepts it if I just use 1,2,3,4 as the values being passed in, but doesn't like it if I use actual float values.Those are double literals, try
new Example (1f, 1e2f, 3.0f, 4.754f);

JSlider with Floating point values

I desperatly need to create a JSlider with floating/decimal values. For example between 0 and 1 and ticks within an interval of 0.1.
The default behaviour of JSlider doesn't allow me to do so, hence is there any other way around ? I know it's possible to programmatically handle the required calculation by divding the values by 10, 100 or anything, but i want to display the actual values (floating/decimal) on the JSlider bar.
Thanks in advance

this might do you for the display
import javax.swing.*;
import java.awt.*;
class JSliderLabels extends JFrame
public JSliderLabels()
    setLocation(400,200);
    setDefaultCloseOperation(EXIT_ON_CLOSE);
    JSlider slider = new JSlider(0, 100, 1);
    slider.setMajorTickSpacing(25);
    slider.setPaintTicks(true);
    java.util.Hashtable labelTable = new java.util.Hashtable();
    labelTable.put(new Integer(100), new JLabel("1.0"));
    labelTable.put(new Integer(75), new JLabel("0.75"));
    labelTable.put(new Integer(50), new JLabel("0.50"));
    labelTable.put(new Integer(25), new JLabel("0.25"));
    labelTable.put(new Integer(0), new JLabel("0.0"));
    slider.setLabelTable( labelTable );
    slider.setPaintLabels(true);
    JPanel jp = new JPanel();
    jp.add(slider);
    getContentPane().add(jp);
    pack();
public static void main(String[] args){new JSliderLabels().setVisible(true);}
}

Right justification of floating point values

Hi all
i need to convert a incoming floating point value into (8.2) format i.e. (33678.9956 into 33678.99) i.e. spaces should be padded before if number of digits before decimal is less than 8 and i need to right justify all the incoming values..
pls help me in doing that through graphical mapping.

Hi Barry,
sorry, the format given in the previous post is incorrect...
the required format is just opposite to that..i.e. all DOTS in the same column.
hope my requirement is clear..
Regards,
Manohar

Putting float/int values in xml document

I am using Oracles XML Parser for C (ver 9i). I have a float or int value which i want to put in the xml document that I am creating. The part of the code is as below..
char str_l[50];
float height_l;
elem_l = createElement(ctx_i, (oratext *)"HEIGHT");
if (!appendChild(ctx, parent_l, elem_l))
printf("Error Creating Element\n");
return FAILURE;
parent_l = elem_l;
height_l = 15.7;
sprintf(str_l, "%f", height_l);
elem_l = createTextNode(ctx_i, str_l);
if (!appendChild(ctx, parent_l, elem_l) )
printf("Error Creating Element\n");
return FAILURE;
This put 0.00000 as the value of height in the document. ie the o/p is
<HEIGHT>0.00000</HEIGHT>
Can anyone help me out with this.
Thanks and regards
Umesh

I am using Oracles XML Parser for C (ver 9i). I have a float or int value which i want to put in the xml document that I am creating. The part of the code is as below..
char str_l[50];
float height_l;
elem_l = createElement(ctx_i, (oratext *)"HEIGHT");
if (!appendChild(ctx, parent_l, elem_l))
printf("Error Creating Element\n");
return FAILURE;
parent_l = elem_l;
height_l = 15.7;
sprintf(str_l, "%f", height_l);
elem_l = createTextNode(ctx_i, str_l);
if (!appendChild(ctx, parent_l, elem_l) )
printf("Error Creating Element\n");
return FAILURE;
This put 0.00000 as the value of height in the document. ie the o/p is
<HEIGHT>0.00000</HEIGHT>
Can anyone help me out with this.
Thanks and regards
Umesh

Float Data Value and Scientific Notation

Hello All,
SQL Server converts some of the float value into scientific notation for example 0.00001 will be stored as
1E-05.
Is there any simple method\formula\threshold to precisely tell when some particular value will be converted to scientific notation?
Also I think oracle store same value above i.e. 0.00001 as it is with float data type but not the case with SQL Server. Any reason for the same?
Thanks in Advance.
-Malkesh

Thanks Erland.
I got your point that internal storage as you mentioned will be 53 bit and 11 bit exponent of 2. However my question remains the same that is there any method\threshold from which we can say that after this numeric value stored data will be represented in
scientific notation or any rule of thumb? For example
drop
table #t
CREATE
TABLE #T (T1
FLOAT)
INSERT
INTO #T (T1)
SELECT 0.000001
INSERT
INTO #T (T1)
SELECT 0.00001
INSERT
INTO #T (T1)
SELECT 0.0001
INSERT
INTO #T (T1)
SELECT 0.001
INSERT
INTO #T (T1)
SELECT 0.01
INSERT
INTO #T (T1)
SELECT 0.1
INSERT
INTO #T (T1)
SELECT 1.
INSERT
INTO #T (T1)
SELECT 1.0
INSERT
INTO #T (T1)
SELECT 1.01
INSERT
INTO #T (T1)
SELECT 1.001
INSERT
INTO #T (T1)
SELECT 1.0001
INSERT
INTO #T (T1)
SELECT 1.00001
INSERT
INTO #T (T1)
SELECT 1.000001
INSERT
INTO #T (T1)
SELECT 1.0000001
INSERT
INTO #T (T1)
SELECT 1.00000001
INSERT
INTO #T (T1)
SELECT 1.000000001
INSERT
INTO #T (T1)
SELECT 1.0000000001
INSERT
INTO #T (T1)
SELECT 1.00000000001
INSERT
INTO #T (T1)
SELECT 1.000000000001
INSERT
INTO #T (T1)
SELECT 1.1
INSERT
INTO #T (T1)
SELECT 10.1
INSERT
INTO #T (T1)
SELECT 100.1
INSERT
INTO #T (T1)
SELECT 1000.1
INSERT
INTO #T (T1)
SELECT 10000.1
INSERT
INTO #T (T1)
SELECT 100000.1
INSERT
INTO #T (T1)
SELECT 1000000.1
INSERT
INTO #T (T1)
SELECT 10000000.1
INSERT
INTO #T (T1)
SELECT 100000000.1
INSERT
INTO #T (T1)
SELECT 1000000000.1
INSERT
INTO #T (T1)
SELECT 10000000000.1
INSERT
INTO #T (T1)
SELECT 100000000000.1
INSERT
INTO #T (T1)
SELECT 1000000000000.1
SELECT T1
FROM #T
Here you can see value < 0.0001 i.e. 0.00001 and 0.000001 both are in scientific notation. So which rule or formula drives this behavior that is what my question is. Sorry I am somewhat poor in mathematics fundamentals So if you can make it simple.
Thanks Again.

KF of type Floating Point Values

Hi
i had a KF of type Nubmer and data type FLTP-Floating point.
When i see the data loaded in the cube for that KF..
Sample values are '8,8310000000000000E-01' and '1,1690000000000000E-01'
What is E-01 mean and How can i intrept their actual values...
Thaks

Still confused..
For better understanding...can please convert '8,8310000000000000E-01' and '1,1690000000000000E-01' for a number with 5 Decimal places or update me how to convert
Thanks

How to read binary floating point values from TCP/IP

I am attempting to use a LabView application to read an array of binary single precision floating point numbers transferred through a TCP/IP connection from a Windows C++ program. The endianization occurs before the values are sent to the Labview application. When I read the values in LabView, some values are interpreted correctly, some are not. For instance, the C program is sending a 6 as one element, and 7 as another. Labview interprets both as 6. The difference between 6 and 7 in binary format is 1 bit (bit #21 if counting from 0 in LabView format). There are 2 other values that show the same error- 459.67 is being sent, 395.67 is read by labview (1 bit difference, exact same bit... 7.5 is being sent, 6.5 is being read by LabView (1 bit difference, exact same bit). LabView reads that bit as 0, when it should be 1.
This seems very odd to me because most values are being read correctly, including other values with that same bit on. There are values being read correctly that are both before and after the incorrect values in the array, so it's not just an issue with an offset or something in the bit stream. Additionally, when attempting to read an array of values with all bits on, I get a strange pattern of 111110011111100111111001111110.
We have also verified that the binary representation for the values is the same on both machines, once you account for the byte swapping. What am I missing here? How in the world can some values come across correctly and others incorrectly? Any help would be greatly appreciated! Our fallback is to transfer everything in ASCII, which is going to greatly increase packet sizes.
Thanks,
Jason

Update:
Problem fixed. I say fixed and not solved because I don't know why it's fixed. I had been storing the TCP/IP read in a string and passing it through a shift register after each read. I would then concatenate it with the TCP/IP read result in the next loop. After each read, it would search the concatenated string for ASCII flags. When it found them, it would strip off the flags and then type cast the rest as single precision floating points.
I knew I would be getting the same number of bytes each time, so I ditched the ASCII flags and had just the binary values sent. This way, I expect to get all of the values in one TCP/IP read. No values are passed through a shift register to the next loop and there is no concatenation of the string outputs from TCP/IP read.
I'm not sure if it was the ASCII flags being included or something with the way I was manipulating the string that was causing the binary values to be interpreted incorrectly. Hope this helps someone else.
Jason

Visual Web Part Value not Selecting in Page Redirection

Hi..
I have a Visual Web Part having DropDown Controls and adding Webpart in 3 Web Part Pages .. When we redirect from 1 page to another, the selected dropdown values are not coming in 2nd and 3rd page..
Ravindranath

You have added visual web part in three different pages, drop down value will not be selected when you redirect to page 2 or page 3 because they are separate pages and a separate web parts added on each page. Each page has a separate page life cycle.
You can only set dropdown value by passing query string parameter.
Adnan Amin MCT, SharePoint Architect | If you find this post useful kindly please mark it as an answer.

To change the defaults headings/parts' values in a FSG Report

Hi,
I m making FSG reports in Oracle 11.5.10.2.
The default headings Like "Corporate set of books" on the top of the reports page or the "no company requested" at the upper left position are required to be removed or changed. But i m not finding a way to do this.
Is there any idea????
thanks.

Yeah Sunil !!
you can't make reports without columns and column headings are the must in most of the cases. But i m talking about the fix and defaults parts that are there for each and every type of report. So any help in this regard is required.
Thanks.

Unexpected "numeric or value error" when using CAST COLLECT

I am having trouble with string aggregation using CAST / COLLECT and the to_string function described on various sites around the net including AskTom and http://www.oracle-developer.net/display.php?id=306.
I am getting "numeric or value error: character string buffer too small" but cannot see which limit I am exceeding.
I have put together a simple test case to highlight this problem which I have pasted below.
The error does not seem to be coming from the to_string function itself (else I expect we would see "TO_STRING raised an exception" in the returned error message).
Any thoughts much appreciated,
Thanks, Andy
SQL*Plus: Release 10.1.0.4.2 - Production on Tue Jun 15 09:56:53 2010
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL> CREATE TYPE table_of_varchar2 AS TABLE OF VARCHAR2(32000);
2 /
Type created.
SQL> CREATE OR REPLACE FUNCTION to_string (
2              nt_in IN   table_of_varchar2
3      ,       delimiter_in    IN VARCHAR2 DEFAULT ',')
4      RETURN VARCHAR2
5      IS
6          l_idx   PLS_INTEGER;
7          l_str   VARCHAR2(32767);
8          l_dlm   VARCHAR2(10);
9
10      BEGIN
11
12          l_idx := nt_in.FIRST;
13          WHILE l_idx IS NOT NULL LOOP
14              l_str := l_str || l_dlm || nt_in(l_idx);
15              l_dlm := delimiter_in;
16              l_idx := nt_in.NEXT(l_idx);
17          END LOOP;
18
19          RETURN l_str;
20      EXCEPTION
21          WHEN OTHERS THEN
22              raise_application_error(-20000
23                                  ,   'TO_STRING raised an exception. '||
24                                      'The reported error was: '||sqlerrm);
25     END to_string;
26 /
Function created.
SQL> DECLARE
2      l_longstring varchar2(32000);
3 BEGIN
4      SELECT to_string(CAST( COLLECT( substr(object_name,1,1) ) AS table_of_varchar2 ) )
5      INTO    l_longstring
6      FROM    all_objects
7      WHERE   rownum < 2001;
8
9 EXCEPTION
10      WHEN OTHERS THEN
11          raise_application_error(-20001
12                ,   'The anonymous block raised an exception: '||
13                    sqlerrm||'. '||DBMS_UTILITY.format_error_backtrace);
14 END;
15 /
PL/SQL procedure successfully completed.
SQL> DECLARE
2      l_longstring varchar2(32000);
3 BEGIN
4      SELECT to_string(CAST( COLLECT( substr(object_name,1,1) ) AS table_of_varchar2 ) )
5      INTO    l_longstring
6      FROM    all_objects
7      WHERE   rownum < 2002;
8
9 EXCEPTION
10      WHEN OTHERS THEN
11          raise_application_error(-20001
12                ,   'The anonymous block raised an exception: '||
13                    sqlerrm||'. '||DBMS_UTILITY.format_error_backtrace);
14 END;
15 /
DECLARE
ERROR at line 1:
ORA-20001: The anonymous block raised an exception: ORA-06502: PL/SQL: numeric
or value error: character string buffer too small
ORA-06512: at line 1. ORA-06512: at line 1
ORA-06512: at line 4
ORA-06512: at line 11

Aha, of course.
I was aware of the 4000 character SQL VARCHAR2 limit but didn't think it would apply here since we are calling a PLSQL function and trying to assign the value it returns into a PLSQL varchar2(32000) variable. BUT... we are of course doing this via a SELECT statement and hence via SQL. Therefore the SQL 4000 limit applies.
With this in mind, I changed the RETURN type of the to_string function to be CLOB. This solved the problem.
Thank you,
Andy

Unexpected floating part value

Similar Messages

Maybe you are looking for