Rounding a floating point value
How can I set the precision of a float, I want to express a dollar amount using a float, so I need to set the precision to two places past the decimal point, but am unsure of how to do so... hopefully this isn't a stupid question.
thanks!
would this be accomplished by calling the toString of a Float? I have a primitive type float, and I don't see any methods or constructors in the DecimalFormat class that can handle a float as a parameter...
Similar Messages
-
Precision loss - conversions between exact values and floating point values
Hi!
I read this in your SQL Reference manual, but I don't quite get it.
Conversions between exact numeric values (TT_TINYINT, TT_SMALLINT, TT_INTEGER, TT_BIGINT, NUMBER) and floating-point values (BINARY_FLOAT, BINARY_DOUBLE) can be inexact because the exact numeric values use decimal precision whereas the floating-point numbers use binary precision.
Could you please give two examples: one where a TT_TINYINT is converted to a BINARY_DOUBLE and one when a TT_BIGINT is converted into a DOUBLE, both cases give examples on lost precision? This would be very helpful.
Thanks!
Sunechokpa wrote:
Public Example (float... values){}
new Example (1, 1e2, 3.0, 4.754);It accepts it if I just use 1,2,3,4 as the values being passed in, but doesn't like it if I use actual float values.Those are double literals, try
new Example (1f, 1e2f, 3.0f, 4.754f); -
Hi all;
Please help.
I am getting floating point values in my source IDoc like 1.234- but while inserting into the ODS system i need the negative sign before 1.234.
How to achive this out.
NalinHi
You can achive this by using graphical mapping UUser defined function.
Mapping source fieldremove context UDF --- target field
UDF
for(int i=0;i<Float.length;i++)
if(Float<i>.endsWith("-"))
result.addValue("-"+Float<i>.substring(0,Float<i>.length()-1));
else
result.addValue(Float<i>);
Mudit
Award points if it helps -
Passing floating point values, but being recognised as doubles and int's?
Hi,
I was wondering how to differentiate between floating point values and ints etc
I am passing values into a public class with the parameters (float... values)
Public Example (float... values){}new Example (1, 1e2, 3.0, 4.754);
It accepts it if I just use 1,2,3,4 as the values being passed in, but doesn't like it if I use actual float values.
How come?
Cheerschokpa wrote:
Public Example (float... values){}
new Example (1, 1e2, 3.0, 4.754);It accepts it if I just use 1,2,3,4 as the values being passed in, but doesn't like it if I use actual float values.Those are double literals, try
new Example (1f, 1e2f, 3.0f, 4.754f); -
Right justification of floating point values
Hi all
i need to convert a incoming floating point value into (8.2) format i.e. (33678.9956 into 33678.99) i.e. spaces should be padded before if number of digits before decimal is less than 8 and i need to right justify all the incoming values..
pls help me in doing that through graphical mapping.Hi Barry,
sorry, the format given in the previous post is incorrect...
the required format is just opposite to that..i.e. all DOTS in the same column.
hope my requirement is clear..
Regards,
Manohar -
Easiest way to split a floating point value
Hello, I am trying to split a floating-point value (double) into its fractional and integer parts? I have been trying to find some sort of method in BigDecimal or Double that would do it but so far I haven't come up with anything. I know it could be done by searching through it as a String but that sounds like a very inefficent way to do it. I would appreciate any help you could give. Thanks
float f;
int i = (int) f;
float frac = f-i; -
KF of type Floating Point Values
Hi
i had a KF of type Nubmer and data type FLTP-Floating point.
When i see the data loaded in the cube for that KF..
Sample values are '8,8310000000000000E-01' and '1,1690000000000000E-01'
What is E-01 mean and How can i intrept their actual values...
ThaksStill confused..
For better understanding...can please convert '8,8310000000000000E-01' and '1,1690000000000000E-01' for a number with 5 Decimal places or update me how to convert
Thanks -
How to read binary floating point values from TCP/IP
I am attempting to use a LabView application to read an array of binary single precision floating point numbers transferred through a TCP/IP connection from a Windows C++ program. The endianization occurs before the values are sent to the Labview application. When I read the values in LabView, some values are interpreted correctly, some are not. For instance, the C program is sending a 6 as one element, and 7 as another. Labview interprets both as 6. The difference between 6 and 7 in binary format is 1 bit (bit #21 if counting from 0 in LabView format). There are 2 other values that show the same error- 459.67 is being sent, 395.67 is read by labview (1 bit difference, exact same bit... 7.5 is being sent, 6.5 is being read by LabView (1 bit difference, exact same bit). LabView reads that bit as 0, when it should be 1.
This seems very odd to me because most values are being read correctly, including other values with that same bit on. There are values being read correctly that are both before and after the incorrect values in the array, so it's not just an issue with an offset or something in the bit stream. Additionally, when attempting to read an array of values with all bits on, I get a strange pattern of 111110011111100111111001111110.
We have also verified that the binary representation for the values is the same on both machines, once you account for the byte swapping. What am I missing here? How in the world can some values come across correctly and others incorrectly? Any help would be greatly appreciated! Our fallback is to transfer everything in ASCII, which is going to greatly increase packet sizes.
Thanks,
JasonUpdate:
Problem fixed. I say fixed and not solved because I don't know why it's fixed. I had been storing the TCP/IP read in a string and passing it through a shift register after each read. I would then concatenate it with the TCP/IP read result in the next loop. After each read, it would search the concatenated string for ASCII flags. When it found them, it would strip off the flags and then type cast the rest as single precision floating points.
I knew I would be getting the same number of bytes each time, so I ditched the ASCII flags and had just the binary values sent. This way, I expect to get all of the values in one TCP/IP read. No values are passed through a shift register to the next loop and there is no concatenation of the string outputs from TCP/IP read.
I'm not sure if it was the ASCII flags being included or something with the way I was manipulating the string that was causing the binary values to be interpreted incorrectly. Hope this helps someone else.
Jason -
JSlider with Floating point values
I desperatly need to create a JSlider with floating/decimal values. For example between 0 and 1 and ticks within an interval of 0.1.
The default behaviour of JSlider doesn't allow me to do so, hence is there any other way around ? I know it's possible to programmatically handle the required calculation by divding the values by 10, 100 or anything, but i want to display the actual values (floating/decimal) on the JSlider bar.
Thanks in advancethis might do you for the display
import javax.swing.*;
import java.awt.*;
class JSliderLabels extends JFrame
public JSliderLabels()
setLocation(400,200);
setDefaultCloseOperation(EXIT_ON_CLOSE);
JSlider slider = new JSlider(0, 100, 1);
slider.setMajorTickSpacing(25);
slider.setPaintTicks(true);
java.util.Hashtable labelTable = new java.util.Hashtable();
labelTable.put(new Integer(100), new JLabel("1.0"));
labelTable.put(new Integer(75), new JLabel("0.75"));
labelTable.put(new Integer(50), new JLabel("0.50"));
labelTable.put(new Integer(25), new JLabel("0.25"));
labelTable.put(new Integer(0), new JLabel("0.0"));
slider.setLabelTable( labelTable );
slider.setPaintLabels(true);
JPanel jp = new JPanel();
jp.add(slider);
getContentPane().add(jp);
pack();
public static void main(String[] args){new JSliderLabels().setVisible(true);}
} -
Compare floating point values in a column
I have a column defined as NUMBER. The values are say 1.2345 and 1.23443. Visually i can say they are almost the same but if i do select count(distinct colA) from table it gives two values, which is perfectly right. What i want to do is say if they are less than 0.0001 then count them as same value.
You could argue if i use ROUND or TRUNCATE to the 'n' decimal point but i see this as an issue even even to two decimal places. If the 3rd digit after the decimal is greater than 5 it would round the 2nd digit. If i use TRUNCATE to 'n' places then the last digit might be smaller than the previous value so again the same issue.
However, my problem is how do i do if its in 2 or 3 different rows for the same column. If they are in two columns, i am able to do (COLA - COLB) and if the value is < 0.00001 then ignore or filter from the query result.
Example:
ColA , ColB
Variable1, 1.23453876
Variable1, 1.2342736
Variable1, 1.23420087
Variable2, 10.250
Variable2, 15.775
Variable3, -1.93815994262695
Variable3, -1.93815803527832
Output: Count of unique values for each variable in columnA
Variable1, 1
Variable2, 2
Variable3, 1
OR
Output: Display on those variables whose values is more than 0.0001 (4 digits after the decimal)
Variable2, 2
I am using oracle 10.2 Enterprise Edition and accessing data via TOAD
ThanksPeter,
Thank you for the sql, i had not tried lag function before so it was new to me. I modified it slightly to lag by 'partition query'. The issue i see is it does delta between only two consecutive values so if there are 5 values in 5 rows then they can be off by 0.0001 (some small number) and we will never catch those variables because the case statement would evaluate to '0' based on the if condition.
However, for my case it would work perfectly fine since i wanted to separate small values from the real obvious big deltas. As in the case below where the delta was 5.XX which narrows down to Variable2.
with
tab as
(select 'Variable1' cola, 1.23453876 colb from dual union all
select 'Variable3', -1.93815994262695 from dual union all
select 'Variable1', 1.2342736 from dual union all
select 'Variable2', 10.250 from dual union all
select 'Variable2', 15.775 from dual union all
select 'Variable1', 1.23420087 from dual union all
select 'Variable3', -1.93815803527832 from dual union all
select 'Variable4', 47.41209 from dual union all
select 'Variable4', 47.41207 from dual
select *
from
select
cola,colb,lag(colb) over (partition by cola order by colb)prev_value, round(abs(colb - lag(colb) over (partition by cola order by colb)),4)prev_delta
,case when colb - nvl(lag(colb) over (partition by cola order by colb ), colb) < 0.0001 then 0
else 1 end dif_count
from
tab
--order by cola
--group by cola
COLA COLB PREV_VALUE PREV_DELTA DIF_COUNT
Variable1 1.23420087 0
Variable1 1.2342736 1.23420087 0.0001 0
Variable1 1.23453876 1.2342736 0.0003 1
Variable2 10.25 0
Variable2 15.775 10.25 5.525 1
Variable3 -1.938159943 0
Variable3 -1.938158035 -1.938159943 0 0
Variable4 47.41207 0
Variable4 47.41209 47.41207 0 0
You guys rock! thank you. -
How to convert PM Module Breakdown Duration (AUSZT) floating point value into Minutes
Hi Frnds,
My requirement is to display a normal alv report. Out of which one of the filed is from VIQMEL-AUSZT (Breakdown Duration).
Which is of Data type FLTP.
Details:
AUZTV
AUZTV
TIMS
6
Start of Malfunction (Time)
12:39:19
AUZTB
AUZTB
TIMS
6
End of Malfunction (Time)
12:42:00
AUSZT
AUSZT
FLTP
16
Breakdown Duration
1.6100000000000000E+02
Now I want to convert the value '1.6100000000000000E+02' into Minutes.
Kindly let me know the solution.
Note: I tried with standard FM 'PM_TIME_CONVERSION' and converted the break down value into hours.
For example: 1.6100000000000000E+02 to 4,472222222222222E-02.
But i want to convert the break down duration value into perfect minutes.
With Regards,
Sudhir.Simply equate to a variable of type P decimals zero. It should give you value in seconds. Divide it by 60 to have it in minutes.
-
Re-FLOATING POINT VALUE CONVERSION.
Hi,
I have value lile this : 1.000000000000000E03, or 1.090000000000000E02 , is there
any function module to convert it into actual value like 1,00.00 or 1,099.00 etc.
Thanks in advance.
Shyam.hi
call fm CONVERT_WITH_OVERFLOW_CHECK.
data lv_num type f .
data: p10_4(10) type p decimals 4.
lv_num = 27.
CALL FUNCTION 'CONVERT_WITH_OVERFLOW_CHECK'
EXPORTING
i_value = lv_num
I_UNIT = ' '
I_KRUND = ' '
IMPORTING
E_VALUE = p10_4
EXCEPTIONS
UNIT_CONVERSION_ERROR = 1
WRONG_E_VALUE_TYPE = 2
OTHERS = 3
IF sy-subrc <> 0.
MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
ENDIF.
the variable p10_4 will be having your result. its format will be driven by the type you have taken.
regards
Vishal Kapoor -
How can I get a floating point value from Labview 6 into VB 6
I need a numeric value from Labview and use it in my Visual Basic program. How can I do that...
ThanksHello jardinpaul,
You can use Visual Basic to call a LabVIEW DLL that you create from your LabVIEW VI. Take a look at this example on our website:
Using Microsoft Visual Basic to Call LabVIEW DLLs That Pass Numeric Data Types
David McClelland
NI Applications Engineering -
Hi all,
I have a field <field> of type FLTP and length 16. This field has a value 4.650000000000000E+01. I want to print this value as 46.5. How can I do this.
Waiting for replies. ThanksI have the same problem like you, but cannot solve it with these answers in this thread here.
My <fs> is assigned to a field of a table (dynamicly and direktly). To copy the value in <fs> to an P-Variable, it doesn't helped me...
See my code:
DATA:
grfk_ok_code TYPE sy-ucomm,
grfk_values TYPE TABLE OF GPRVAL WITH HEADER LINE,
wa_ztab TYPE zqm_chq,
l_index TYPE n,
l_field TYPE string,
l_fs2p TYPE p DECIMALS 3
FIELD-SYMBOLS: <fs_field> TYPE ANY.
LOOP AT ztab INTO wa_ztab.
UNASSIGN <fs_field>.
l_index = sy-tabix.
CONCATENATE 'grfk_values-val' l_index INTO l_field.
ASSIGN (l_field) TO <fs_field>.
IF sy-subrc <> 0.
RETURN.
ENDIF.
<fs_field> = wa_ztab-cf_dlp.
l_fs2p = <fs_field>.
<fs_field> = l_fs2p.
ENDLOOP.
And I cannot use the floating point values in the grfk_value table...
Regards,
Steffen -
[Solved] Bash and Floating point arithmetic
I didn't realize how troublesome floating point numbers can be until now.
What I want to do should be simple I dare say:
properRounding( ( currentTime - downloadTime ) / ( dueTime - downloadTime ) * 100 )
however best I've been able to achieve so far is this:
echo "($currentTime-$downloadTime)/($dueTime-$downloadTime)*100" | bc -l
Which prints the correct floating point value.
I've tried to put the result in a variable, but I must be doing it wrong as I get the most peculiar error. I can live without it, but it would make life easier.
As for the rounding, that is a must. I've read that if you remove the -l param from bc, then it will round, but in my case something goes wrong as I just get the value 0 in return and besides, concluded after a simple test, bc always rounds down as in integer division, which I can not use.
So of course I'll continue reading and hopefully someday arrive at a solution, but I would very much appreciate if someone could lend me a hand.
This after all is not just a learning experience, I'm trying to create something useful.
Best regards.
edit:
nb:
all variables are integers.
Last edited by zacariaz (2012-09-09 14:50:18)Just for the fun of it, here's my progress thus far... Well, there really isn't much more to do. The rest is a conky thing.
#!/bin/bash
# Variables from unitinfo.txt - date as unix timestamps.
dueTime="$(date +%s -d "$(grep 'Due time: ' ~/unitinfo.txt | cut -c11-)")"
if [ "$1" = "end" ]
then echo $dueTime
fi
downloadTime="$(date +%s -d "$(grep 'Download time: ' ~/unitinfo.txt | cut -c16-)")"
if [ "$1" = "start" ]
then echo $downloadTime
fi
progress="$(grep 'Progress: ' ~/unitinfo.txt | cut -c11-12 | sed 's/ *$//')"
if [ "$1" = "prog1" ]
then echo $progress
fi
# The rest
#progress valued 0-1 for use with conky proress bars
progress2=$( echo 2k $progress 100 / f | dc )
if [ "$1" = "prog2" ]
then echo $progress2
fi
# Current time - unix timestamp.
currentTime="$(date +%s)"
# Remaining time - unix timestamp
remainingTime=$(( dueTime-$currentTime ))
if [ "$1" = "remain" ]
then echo $remainingTime
fi
# Elapsed time - unix timestamp
elapsedTime=$(( currentTime-downloadTime ))
if [ "$1" = "elap1" ]
then echo $elapsedTime
fi
# Total amount of time available - unix timestamp
totalTime=$(( dueTime-downloadTime ))
if [ "$1" = "total" ]
then echo $totalTime
fi
# How much time has elapsed in percent
progress3="$(echo 3k $elapsedTime $totalTime / 100 \* 0k 0.5 + 1 / f | dc)"
if [ "$1" = "elap2" ]
then echo $progress3
fi
# Like the above bur 0-1
progress4=$( echo 2k $progress3 100 / f | dc )
if [ "$1" = "elap3" ]
then echo $progress4
fi
# In percent, expected completion vs $dueTime - less than 100 is better.
expectedCompletion="$( echo 3k $elapsedTime 10000 \* $progress / $totalTime / 0k 0.5 + 1 / f | dc )"
if [ "$1" = "exp1" ]
then echo $expectedCompletion
fi
# Same as above, but unix timestamp
expectedCompletion2=$(( downloadTime+(expectedCompletion*totalTime/100) ))
if [ "$1" = "exp2" ]
then echo $expectedCompletion2
fi
#efficiency
if [ "$1" = "ef1" ]; then
if [ $progress -lt $progress3 ]
then echo "you're behind schedule."
elif [ $progress -eq $progress3 ]
then echo "You're right on schedule."
else
echo "You're ahead of schedule."
fi
fi
if [ "$1" = "ef2" ]; then
if [ $expectedCompletion -gt 100 ]
then echo "You're not going to make it!"
else
echo "You're going to make it!"
fi
fi
Maybe you are looking for
-
How can I copy and then insert a column in Numbers 09
I have a column in a Table that I would like to copy and then insert this column into another existing table on the same sheet. I don't want to copy and paste this, but rather insert it between existing columns. Thanks Eric
-
need help with edtiing pdf doc?
-
How long is the trial for Lightroom 5?
I am taking photos at my niece's wedding in June and I want to play with the program some before I purchase it. I was hoping the free trial is for 30 days, but I can't find it anywhere. Thanks!
-
How to make dependent choice lists grow dynamically?
Is it possible to have dependent choice list which grow as the new content is added for choice list item? E.g If we have countries, states, cities as dependent choice lists. If contributor adds a new country the value should appear in the drop down n
-
my pc with windows 7 was working up until just before x-mas now i can not use firefox to get my email or any thing else that requires a password. the screen that comes up does not allow or have a place to put in the password most other pages are ok