Rounding ist wrong for float value (MS SQL 2005)

Hello
I have a simple report with a command:
select num = convert(float, 4.145)
Field round over "Format Field" or ToText( num, 2) the result is 4.14 -> Wrong
if i use Round(num, 2) result is 4.15 --> OK
In CR 8.5 result is always correct
Thx

I know the problematic of the floating values.
My big trouble is the inconsistency in the report!
It can't be that the formula function Round(x,y) show another result than the integrated field-rounding function.
Also in the formula that ToText(x, y) is different to Round(x,y).
Following another stupid difference:
Command.num = 4.145 Result: TRUE
Command.num - 4.145 = 0 Result: FALSE
Mathematical it's the same.
CR8 was consistence, after migration we have different result on our reports!

Similar Messages

Rounding up of a float value

i need some help in rounding up the float value to two places after the decimal point
i have a value like 25.34446789 and i need to round it up like 25.34

i tried this function but i am not getting the result
correctly .if i am using the round () function i am
getting a value like 55.0 when the original value is
55.085673.
i want to get it like 55.09. round to two places
after the decimal point .is there any other way to do
it ?How about doing what Chuck said you should do? Give it a try and think about what he wrote. It may actually work.

Different results for decimal values between SQL and B1 RecordSet

Hi Experts,
This is really a strange one. I have the query below:
SELECT Amount, TaxPrcnt FROM [@Table] WHERE ORDER# = '20246' AND ItemCode = '112488'
The same query gives me 0.085 for TaxPrcnt if I run it in SQL Management Studio, but gives me 8.5 if I run it in B1 Query Generator, and the same if I use RecordSet object in DI. The actual value in the table is 0.085. Anyone know why B1 is multiplying the value by 100?

Hi Ronnie,
sp_prepexec is a system stored procedure used to generate to prepare and execute a parameterized SQL statement. Have a look at:
http://jtds.sourceforge.net/apiCursors.html#_sp_prepexec
Basically, it's another way of passing a query to SQL rather than using a stored procedure. This command shouldn't be converting the data in any way, unless there's a specific cast or calculation in the query itself.
Can you give simple details of the SQL table you created and I'll see if I can replicate the issue.
Kind Regards,
Owen
P.S. Sorry about the query, I wrote that down this morning before all my brain cells were awake Referencing a field by index only works when grouping and sorting.

Sun implementation of WebRowSet fails for FLOAT values

I have been attempting to use the sun reference implementation of WebRowSet class to query a vendor table and serialize the rowset to XML.
Oracle's proprietary RowSet package does not include an implementation of WebRowSet.
For simple tables everything works ok. However, a few of the vendor's more robust tables define numeric columns as FLOAT(126) in the DDL. These columns return thru the Oracle driver as type NUMBER (numeric Type 2) with driver recommended java class java.math.BigDecimal. The Precision is correctly returned as 126.
The problem seems to be that the driver considers these columns to have a Scale of -127 which is nonsensical. When populating the WebRowSet from a ResultSet an exception is thrown which indicates that a negative scale has been encountered.
Since WebRowSet can only be populated from a ResultSet and the scale property is returned by the driver, this means that WebRowSet is effectively unavailable for Oracle databases.
BTW, our Oracle database is v 9.2.0.4.0 and the Oracle JDBC driver is 9.2.0.5. The WebRowSetImpl class was downloaded from javasoft this month - it is version 1.01.
Has anyone else observed FLOAT columns with negative scale ? Is there any way to coerce the driver into returning a more accurate scale ?

I have been attempting to use the sun reference implementation of WebRowSet class to query a vendor table and serialize the rowset to XML.
Oracle's proprietary RowSet package does not include an implementation of WebRowSet.
For simple tables everything works ok. However, a few of the vendor's more robust tables define numeric columns as FLOAT(126) in the DDL. These columns return thru the Oracle driver as type NUMBER (numeric Type 2) with driver recommended java class java.math.BigDecimal. The Precision is correctly returned as 126.
The problem seems to be that the driver considers these columns to have a Scale of -127 which is nonsensical. When populating the WebRowSet from a ResultSet an exception is thrown which indicates that a negative scale has been encountered.
Since WebRowSet can only be populated from a ResultSet and the scale property is returned by the driver, this means that WebRowSet is effectively unavailable for Oracle databases.
BTW, our Oracle database is v 9.2.0.4.0 and the Oracle JDBC driver is 9.2.0.5. The WebRowSetImpl class was downloaded from javasoft this month - it is version 1.01.
Has anyone else observed FLOAT columns with negative scale ? Is there any way to coerce the driver into returning a more accurate scale ?

Options for loading parent-child SQL 2005 tables

I was reading about a few examples of using Table types and a stored procedure to load tables that are joined by a foreign key (parent-child). Seems straight forward enough. However, that requires at least SQL 2008 and the system we are using
is on 2005 still for the time being. As much as I would like to upgrade, there is no time for it, even though we have other 2008 and 2012 instances. So with that said, what are my options for getting data from BTS to SQL where the destination is 2 tables
that have relations.
There are probably a dozen or more, but my first thought is a stored procedure with an xml parameter, then use xquery and xpath statements to pull the data out and insert. TRY/CATCH and RAISEERROR to return error codes back to the BTS application.
Thoughts?

First, how many records are you dealing with? I don't even think of terms of 'bulk' until 10,000 or so. And once that threshold is crossed, the first tool out of the box is SSIS.
Second, are you debatching or is there a triple confirmed requirement that all records be loaded in a single transaction?
Finally, the 'DB round-trip' issue is often misunderstood in real world settings. In a typical db transaction, the most expensive operation is establishing the connection. Once that's done, the SQL Client is very efficient in transporting data
to and from the database. Since the wcfSqlBinding uses a Connection Pool, that setup is only done once to some high-water mark.
Consider, the data has to be sent no matter what and in every conseivable scneario*, the byte count of a stored procedure over TDS would be less than sending the same data wrapped in Xml. The network adds considerable latency to the conversation.
This is why I'm skeptical of using OPENXML() without a provable benefit.
So, don't get caught up in theoritical 'performance' issues without understanding the entire scenario first.
*SQL Code being the primary exception.

ORA-01426: numeric overflow, while migrating "float" values

When I try to insert a value of *2.58612749363472E302* (originally a "float" in SQL DB) into an Oracle column of type float(126), I get this error "ORA-01426: numeric overflow"
I tried changing the column type to BINARY_FLOAT, BINARY_DOUBLE, but I still get same error.
I'm using Oracle SQL Developer (2.1.1.64.45) to migrate a SQL Server 2005 Database to Oracle 11g.
Do you have any thoughts on this? Or workarounds?
Thanks in advance.
Arul

Arul,
I have checked the Migration Workbench documentation where it says for float data -
Oracle® SQL Developer
Supplementary Information for Microsoft SQL Server and
Sybase Adaptive Server Migrations
Release 2.1
Table 2–2 Data Types in Oracle and Microsoft SQL Server or Sybase Adaptive Server
NOTE: If you try to migrate
floating point data greater
than or equal to 1.0E+126
then SQL Developer will fail
to insert this data in the
Oracle database and1 will
return an error.This also
applies to negative values
less than or equal to
-1.0E+126.
So, if I understand it correctly your value of 2.58612749363472E302 is outside this range.
Regards,
Mike

Can SQL 2000 and SQL 2005 drivers co-exist in PI?

Hi,
Currently in our PI environment, we have JDBC Adapter connections to SQL 2000 database servers for which the Driver has already been installed.
For connecting to new SQL 2005 database servers, I understand that we have to install the SQL 2005 drivers. I also came to know that SQL 2005 Driver is backward compatible and supports JDBC connection to SQL 2000 server.
But the question I have is whether both the SQL 2000 and SQL 2005 drivers can be installed together in a way that connection to SQL 2000 use the 2000 driver and the connection to 2005 uses 2005 Driver.
(They have different connection strings)
Many thanks in advance,
Regards,
Rosie Sasidharan

The SQL Server 2005 JDBC driver is far better than the SQL Server 2000 JDBC driver. The 2005 JDBC driver primarily consists for one file, sqljdbc.jar. ( if you want to use windows authentication there is another file you would need based on your platform x64 vs x86 called sqljdbc_auth.dll. If in the connect string you list ;integratedSecurity=true which means integrated security would be used and this second file would be required. The 2000 driver did not support integrated security.
The SQL 2000 JDBC driver was made up of the 3 jar files:
msbase.jar,mssqlserver.jar and msutil.jar.
If you want to use a Microsoft JDBC driver to connect to SQL Server 2005, than you should use the 2005 driver, sqljdbc.jar.
Follow the instructions in point 1 of the note #831162 - FAQ: XI 3.0 / PI 7.0 / PI 7.1 JDBC Adapter questions 1 and 13.
Since the jar are different I assume you can use the both drivers, but it doesnt make sense.

Upgrade Process from SQL 2005 to SQL 2012

Hi All,
I have gone thru multiple threads and would like to understand some steps in detail for me to upgrade SQL 2005 to SQL 2012.
Below is my understanding and questions around it, I would be setting up a new environment with OS 2007/2008 with SQL 2012:
Environment Readiness - Ensure new environment is ready with Win 2007/2008 with SQL 2012 Server
Run Upgrade Advisor on the new environment and point to SQL 2005 instance
Find the breaking/discontinued changes to be done before or After Upgrade
<Question>: Over here I would like to know what should be our approach to make the required changes
<Question>: Obviously I should not be making changes directly on the Production SQL 2005 server, then in that case should I take a backup/restore on the new SQL Server 2012 and then make the required changes to the database objects
like (Table schema(changing column names/datatypes etc.), Stored Procedure, Functions & Views)
<Question>: When I do a backup/restore I would also get the data along with it, so is this a recommended approach or should we only run/use the table schema creation scripts, fix the issues identified by UA and then do an Table to Table import,
any thoughts on this ?
<Question>: If I have to do a table to table Import then I should ensure there's a downtime during the Import or would I have any other better approach
So, basically my questions are around the approach of fixing the issues identified by UA and moving the database to SQL Server 2012 environment, I have around 300 databased to me upgraded.
Please do let me know what should be a good/proven approach ? Pl. do let me know if you would need any more details from me.
---Umesh

Thanks all for the replies.
I am trying summarize what I have understood here:
Document SQL Server 2005 environment details, Is there any tool which could give us all relevant details
I would take the backup from the SQL Server 2005 Production environment and then restore the same in a test environment which again has SQL Server 2005 and Run Upgrade Advisor on that
Probbaly run SQL Profiler as well to find out the deprecated changes to be done
I should fix all the breaking, discontinued changes on the Test system identified by UA
I should fix all the deprecated changes on the Test system identified by SQL Profiler
Steps 3 & 4 should give me the changed scripts for all database objects (i.e. Table Schema, Stored Procedures, Functions & Views)
<Question> Perform test, I understand this acitivity test can be captured by having SQL profiler running in the background on the SQL Server 2005 production environment where we can identify some days in a week or a month where maximum activity is
expected to ensure we cover most the scenarios, let me know is this approach looks fine or is there a better approach
Now, with regards to acutal upgrade I need to identify a time window where in less activity is done during which I could have a downtime for the actual SQL Server 2005 production environment
Post downtine, I should do a BackUp for all databases from SQL Server 2005 production environment
Test on environment readiness of SQL Server 2012 can be compared against the documentation done as part of the first Step
<Question> Ideally I should a restore first in the new environment which has SQL Server 2012
<Question> Then run the Scripted changes where we fixed all the breaking, discontinued & deprecated changes
<Question> Does above sequence of Step 9 & 10 make sense or is there a better approach of doing it
<Question> Post Upgrade and running of changed scripts, I should run the Upgrade Advisor again and see if everything looks fine
<Question> Perform test again which we ran on SQL Server 2005 Test environment
<Question> These steps should complete the Upgrade process.
Please do let me know if the above sequence look ok to all of you and does it really work in a practical scenario, do let me know your feedback or any change in sequence/process or any other tools that can be used here.
Thanks Again ...

Logging float values in a MySQL : wrong fromatting

Hi,
I'm using TS with standard DB schema for MySQL to log results from my tests.
Some of the test are standard numeric limit tests (no adapter). Limits and data source are float numbers.
Most of the time everything is well logged in the MySQL DB, but sometimes the floating value is rounded !
You can see attached the result in the database and the value from the result which has been logged. In the resultList the data is weel formatted but in MySQL it's rounded.
The screen copy shows several numeric limit tests, these numeric limits are in fact the same step which is called at several moments in my sequence. So it's not a problem of result formatting for specific test. However it seems that it happens when the limits (coming from a sequence parameters) are set below 1.
What can cause this behavior ?
BTW, I'm using TS2012 and MySQL 5.2 (ODBC 5.2.5.0 ANSI Driver).
Attachments:
MySQLResults.PNG ‏50 KB
ResultList.PNG ‏21 KB

Hi zyl7,
The issue is most likely related to the way floating point values are stored in MySQL. To quote MySQL documentation, "Floating-point numbers sometimes cause confusion because they are approximate and not stored as exact values. A floating-point value as written in an SQL statement may not be the same as the value represented internally".
There are more details regarding this in the following link:
http://dev.mysql.com/doc/refman/5.1/en/problems-with-float.html.
I have tried to reproduce your scenario using TS 2012 sp1 and MySQL 5.2.5.0 ANSI ODBC Driver but I was unsuccessful even after trying a range of similar values. So the issue you faced is most likely dependent on the machine in use and might not be reproducible in a different set-up.
If the reason you raised this concern is because any comparisions on the floating point values stored in the database might be incorrect (because of the differences in the value passed from TestStand and the actual value stored), please follow the guidelines specified in the link above (especially the comments in the end of the page) to ensure your comparision operations always use a tolerence when calculating the results.
TestStand does float comparisions using a tolerence, so the pass/fail status generated by TestStand should be accurate.
Let me know if you have any concerns.

T-SQL and CLR types for return value do not match

Hi I am trying to create a CLR function to call a webservice, the CLR function return data type is double, whether I try
to create this as a table valued funcion or a scalar to return a distance travelled value I am receiving the error below. I've tried changing data types around in the CLR side and the SQL side but keep receiving the same error message, any help would be appreciated,
Thank you,
[Microsoft.SqlServer.Server.SqlFunction(Name = "DistanceCalc")]
public static Double DistanceCalc(Double SrcLat, Double SrcLong,
Double DestLat, Double DestLong)
MileageWS ws = new MileageWS();
ws.Url = "http://test.isp.ca/Distance.asmx";
int intUom = 0; // 0 = Mile, 1 = KM
RouteType RouteMethod = RouteType.Practical;
Requester RequestedFrom = Requester.LinkRoute;
Double distance;
distance = ws.GetDistanceInfoForLonLat(SrcLat, SrcLong, DestLat, DestLong, intUom, RouteMethod, RequestedFrom);
return distance;
CREATE FUNCTION DistanceCalc
@SrcLat as float, @SrcLong as float,
@DestLat as float, @DestLong as float
RETURNS TABLE (Distance float)
External NAME CLRfunctions.RIFunctions.DistanceCalc
GO
Error received when try to Create function ...
1, Level 16, State 2, Procedure pcMiler, Line 6
CREATE FUNCTION for "pcMiler" failed because T-SQL and CLR types for return value do not match.

You defined at table-valued CLR function, but I think you meant to define a scalar CLR function. That might be the cause of the error.
RETURNS TABLE (Distance float)
should be
RETURNS (Distance float)

SQL Command code for multiple value string parameter

Hi,
I'm using crystal 2008 and there is a check box for multiple value SQL Command I need some help in writing the SQL Command code for oracle (or sql server) for a multiple value STRING parameter.
Thanks in advance,
Marilyn

I could be wrong here, but I do not believe you can pass a multiple valued parameter to an SQL Command data source. How I have gotten around this in the past is to put the "real" report into a subreport. In the main report, create a formula field (basic syntax):
formula = join({?parameter}, "|")
Then, use this to pass the selected values to the subreport's parameter (call it {?sr-parm}). The SQL Command in the subreport can then use that (MS SQL):
select *
from table
where charindex(table.field, '{?sr-parm}') > 0
HTH,
Carl

SQL query to store float values in french culture.

Here is how my requirement goes..
I have an application where users can store float values into the SQL server. Here float values are period(.) based. I have now started supporting the users who use the french culture where the float values are Comma(,) based. Example: 13.75 is represented
as 13,75. Instead of making changes to my code i.e the way i am inserting/retrieving the date now, is there a way i can handle this requirement based on culture. Like, if I specify the culture, will I be able to insert float inputs as comma(,) based values
without disturbing my existing code?
Any help is appreciated
Thanks,
Chandrahas.
-Chandrahas

is there a way i can handle this requirement based on culture.
Hello Chandrahas,
SQL Server don't store any culture info for numeric values; they are plain numeric.
It's part of the frontend to represent / accept values depending on the culture setting of the client.
Olaf Helper
[ Blog] [ Xing] [ MVP]

Wrong depreciation calculated for base value 04 + EA-FIN activated

Hi,
after activating EA-FIN, depreciation for assets using base value 04 (= half of replacement value) is calculated wrong.
While AW01 for a particular asset shows depreciation values based on the FULL replacement value (like it would be base value 03) AW01_AFAR (the u201Coldu201D asset explorer) shows the correct depreciation for base value 04.
The wrong base value can also be seen using the "display depreciation calculation" button in both transactions.
There are no enhancements for base value determination active.
I didnu2019t find any SAPNET notes nor SCN entries regarding this case, any ideas?
Best regards, Christian

For what it's worth, I did a search and didn't find anything either.
It sounds like you've already tried this, but if the base value (the actual $$ value) is different in the trace on both tcodes then you have a problem. I would be sure to recalculate the values when you enter AW01 and AW01_AFAR to ensure that it's correct before looking at the trace.
The formula for the replacement value is:
( u_ganlc-kansw + u_ganlc-answl +
               u_ganlc-kmafa + u_ganlc-mafav + u_ganlc-zusma +
               u_ganlc-mafam + u_ganlc-mafal +
               u_ganlc-kaufw + u_ganlc-aufwv +
               u_ganlc-aufwp + u_ganlc-aufwl ) / 2 .
I'd look at ANLC for the asset and do the calculation yourself to confirm the value.
You've probably done all of that so I predict an OSS message in your future !
-nathan

Insert float value into ms sql server

Hi.
How do I insert a float value into my ms sql db.
The Code:
select.setFloat( 4, theForm.getVaerdi() );ERROR:
"MakeClapetInsertAction.java": Error #: 300 : method setFloat(int, java.lang.Float) not found in interface java.sql.PreparedStatement at line 89, column 12

setFloat takes a float primitive, not a Float object.
select.setFloat( 4, theForm.getVaerdi().floatValue() );

Mainframe data loaded into Oracle tables - Test for low values using PL/SQL

Mainframe legacy data has been copied straight from the legacy tables into mirrored tables in Oracle. Some columns from the mainframe data had 'low values' in them. These columns were defined on the Oracle tables as varchar2 types. In looking at the data, some of these columns appear to have data that looks like little square boxes, not sure but maybe that is the way Oracle interprets the 'low values' in the original data into varchar. When I run a select to find all rows where this column is not null, it selects these columns. In the results of the select statement, the columns appear to be blank, however, in looking at the data in the column using SQL Developer, I can see the odd 'square boxes'. My guess is that the select statement is detecting that something exists in this column. Long story short, some how I am going to have to test this legacy data on the Oracle table using Pl/Sql to test for 'low values'. Does anyone have any suggestions on how I could do this????? Help! The mainframe data we are loading into these tables is loaded with columns with low values.
I am using Oracle 11i.
Thanks
Edited by: ncsthbell on Nov 2, 2009 8:38 AM

ncsthbell wrote:
Mainframe legacy data has been copied straight from the legacy tables into mirrored tables in Oracle. Not a wise thing to do. Mainframe operating systems typically use EBCDIC and Unix and Windows servers use ASCII. The endian is also different (big endian vs little endian).
Does anyone have any suggestions on how I could do this????? As suggested, use the SQL function called DUMP() to see the actual contents (in hex) of these columns.

Rounding ist wrong for float value (MS SQL 2005)

Similar Messages

Maybe you are looking for