Precision of a variable

how can I do to modify the precision of a variable.
In ex: I have an "extended precision" variable in the output of a "formula
node" and before I put it into another formula node, I wish to round it to
two digit of precision (x,xx).
How can I do?
Can you help me?
Thanks

There are several ways to do this, e.g. format to string and back. I probably would multiply by 100, round to nearest integer, then divide by 100. (see attached code image).
LabVIEW Champion . Do more with less code and in less time .
Attachments:
Keep2Digits.gif ‏2 KB

Similar Messages

Setting precision format for variables or in STATS...

I need to be able to format variables so that they only have a certain number of decimal places. I know you can use a rounding function but it is for about 90 variables so that would not be an efficient way to accomplish this. Is there a way when global dimensioning variables to set a precision factor or can you set while doing STATS like for average or stdev?? Help appreciated.

Hi BW_Cummins,
There is no way to set an internal precision property of a DIAdem variable or a VBScript variable. You can of course format the display of that variable with the "str()" function, which you can include as an embedded expresion in a REPORT layout (text box, Axis lable, legend, etc.). You can also declare DIAdem real variables as either DBL or SGL, so that would set a certain maximum precision, but that doesn't sound like what you want.
Let me know if you need something other than display formatting,
Brad Turpin
DIAdem Product Support Engineer
National Instruments

Authorization (rsecadmin) with customer exit variable

Hello,
I need to maintain authorization on 0CALMONTH with a customer exit variable.
0CALMONTH is "authorization relevant"
I created a variable of type "customer exit" : ZVAR001 (this variable is OK, I checked its value in a query)
I created a new authorization object with 0CALMONTH = $ZVAR001.
When I run my query I have a message due to authorization error.
If I change my authorization object by replacing my variable ($ZVAR001) by a constant value I have no authorization problem.
I don't understand why...
Error logs don't help me to solve my problem : I have the following message "Message EYE007: You do not have sufficient authorization" and system just says I have "0CALMONTH I EQ $ZVAR001 " but doesn't precise values under variable ZVAR001
Thanks for your help

Indeed problem was in costumer exit because I used condition with "I_STEP". Since I have delete my condition I have no authorization problem with my variable....

J2RE + Windows + J2SE Adapter Engine = Dump Xmx128m

Hi,
I want to use a decentralized J2SE Adapter Engine in order to data to XI.
When I install it on a Windows 2000 Professional, AE running is good and all are perfect... But when I install it on a Windows Server 2003, I have the following dump when I use the file "run_adapter.bat":
"-Xmx128m: illegal argument"
Here is the instruction when my tool uses for its running :
java -Xmx128m -classpath aii_msg_adapter.jar;aii_msg_runtime.jar;httpclient.jar;aii_af_trace.jar;aii_rfcadapter.jar;pmi.jar;exception.jar;jperflib.jar;guidgenerator.jar;aii_utilxi_misc.jar;aii_util_cimaccess.jar;sapxmltoolkit.jar;logging.jar;lcrclient.jar;tc_sec_core.jar;tc_sec_api.jar;jms.jar;servlet.jar;jarm.jar;%CLASSPATH%; com.sap.aii.messaging.adapter.AdapterImpl
Notes:
1. J2RE is j2re-1_4_2_06-windows-i586-p.exe)
2. I have replace 128 by 256 or 512, but it's the same issue.
So,
1. Anybody could explain why I have that with Windows Server 2003 ?
2. is it a problem with a Java Virtual Machine ?
3. Must I do something inside Windows : Environment variable, register key or other ?
Any helps are welcome.
Mickael

No. (neither PATH or JAVA_HOME, or J2EE_HOME)
What folder must I precise in this variable :
C:\Program Files\Java\j2re1.4.2_06\javaws ?
or
C:\Program Files\Java\j2re1.4.2_06\bin ?
Regards.
Mickael

Simple/easy, test your knowledge!

Hi All,
Coming from a C/C++ background, please assist us with the following:
1) Is there an equivalent in java for "memset" and "memcpy". I have an array of bytes:
byte[] bTxBuffer = new byte(512);
and I hava a need to set all the array elements with '0' using one command as opposed to set a (for i=0; i <5122; i++) ...
In C++ we do memset(&bTxBuffer, 0 , 512);
Likewise I need to copy a chunck of data to the byte array using one statement, like memcpy.
2) How can I cast integer to byte. I have the following:
int iAddress = 12;
byte[] bTxBuffer = new byte(512);
I've tried:
bTxBuffer[4] = (byte)iAddress;
but the compiler complains about loss of precision. The variable iAddress value will never be more than 255, but it needs to be type integer for other reasons;
Thanks in advance
Regards,
Trajano

http://java.sun.com/j2se/1.4.1/docs/api/java/util/Arrays.html
public static void fill(int[] a, int fromIndex, int toIndex, int val)
should take care of that memset problem.
memcpy ? i didn't find exact method, but search the api, maybe System has somethint (i didn't look there)
if you know that iAddress is never more than 255, then you actually don't lose any precision, but the compiler doesn't know that, thats why it is warning you.
BTW:
java.lang.System:
arraycopy(Object src, int srcPos, Object dest, int destPos, int length)
you wellcome

Showing error

hi
why it is showing
1 create or replace procedure ven21( eno IN number,sname OUT varchar2) as
2 begin
3 select ename into sname from emp where empno=eno;
4 dbms_output.put_line(sname);
5* end ven21;
SQL> /
Procedure created.
variable sa varchar2;
SQL> execute ven21(7900,:sa);
BEGIN ven21(7900,:sa); END;
ERROR at line 1:
ORA-06502: PL/SQL: numeric or value error
ORA-06512: at "SCOTT.VEN21", line 3
ORA-06512: at line 1

797525 wrote:
hi
why it is showing
1 create or replace procedure ven21( eno IN number,sname OUT varchar2) as
2 begin
3 select ename into sname from emp where empno=eno;
4 dbms_output.put_line(sname);
5* end ven21;
SQL> /
Procedure created.
variable sa varchar2;
SQL> execute ven21(7900,:sa);
BEGIN ven21(7900,:sa); END;
ERROR at line 1:
ORA-06502: PL/SQL: numeric or value error
ORA-06512: at "SCOTT.VEN21", line 3
ORA-06512: at line 1Because you never specified a precision for the variable.
ME_XE?variable sa varchar2;
ME_XE?exec :sa := 'hi';
BEGIN :sa := 'hi'; END;
ERROR at line 1:
ORA-06502: PL/SQL: numeric or value error: character string buffer too small
ORA-06512: at line 1And now....
ME_XE?variable sa varchar2(2);
ME_XE?exec :sa := 'hi';
PL/SQL procedure successfully completed.

Static class containing scanner throws java.util.NoSuchElementException

I use the scanner class pretty often in my coding and recently i have been trying to create a static method i can put in an external class to do the scanning for me and trap any errors.
I seem to be able to create the class successfully but i have a slight problem in that i cannot use the method twice in the same program without having it throw a java.util.NoSuchElementException
If anyone could figure out what error i have made and suggest a way to fix it i would be very grateful
code is as follows:
Scan.java:
// name information removed for privacy
// class info removed for privacy
Assignment: 4.3,
File Name: /ASSIGNMENTS/ASSIGNMENT_4/Part3/Scan.java
import java.util.Scanner;
public class Scan
public static double nextDouble (String prompt)
    // variable to hold the result
    double result = 0.0;
    // variable to determine if input was ok
    Boolean inputOK = false;
    // do-while loop to get some input
    do
      // print the prompt. what do you want to ask for?
      System.out.print ( prompt );
      // Try to scan and capture the input and catch the exception
      try
        // create the scanner object
        Scanner scan = new Scanner (System.in);
        // parse the input`
        result = scan.nextDouble();
        // if we reach this step then good. we got good input let's exit the loop
        inputOK = true;
        // close the scanner
        scan.close();
      // oops we got an error!
      catch (java.util.InputMismatchException e)
        // inform the user of the error
        System.out.println ("That is not a number! Try again. \n");
        // input was not ok so we can't exit the loop yet
        inputOK = false;
    // keep looping as long as inputOK is false
    } while (!inputOK);
    // return out result
    return result;
}Mathematics.java:
// name information removed for privacy
// class information removed for privacy
* Assignment: 4.3
* File Name: ASSIGNMENTS/ASSIGNMENT_4/Part3/Mathematics.java
// Import Scanner class
import java.util.Scanner;
//Class Body
public class Mathematics{
// Main method. AUTO EXECUTE
public static void main(String[] args)
// additional variable declarations removed for brevity
    // Create double precision floating point variables
      double coeff4 = 0.0, coeff3 = 0.0, coeff2 = 0.0, coeff1 = 0.0, cons = 0.0;
    coeff4 = Scan.nextDouble("Enter the coefficient of x^4 (0.0 if none): ");
    coeff3 = Scan.nextDouble("Enter the coefficient of x^3 (0.0 if none): ");
    coeff2 = Scan.nextDouble("Enter the coefficient of x^2 (0.0 if none): ");
    coeff1 = Scan.nextDouble("Enter the coefficient of x (0.0 if none): ");
    cons = Scan.nextDouble("Enter the constant (0.0 if none): ");
// do fun things with input (code removed for brevity)
}output looks like this
dragonfly@Home:~/ASSIGNMENTS/ASSIGNMENT_4/Part3$ java Mathematics
Enter the coefficient of x^4 (0.0 if none): hello boys!
That is not a number! Try again.
Enter the coefficient of x^4 (0.0 if none): 1.2.
That is not a number! Try again.
Enter the coefficient of x^4 (0.0 if none): d
That is not a number! Try again.
Enter the coefficient of x^4 (0.0 if none): 1
Enter the coefficient of x^3 (0.0 if none): Exception in thread "main" java.util.NoSuchElementException
        at java.util.Scanner.throwFor(Scanner.java:817)
        at java.util.Scanner.next(Scanner.java:1431)
        at java.util.Scanner.nextDouble(Scanner.java:2335)
        at Scan.nextDouble(Scan.java:23)
        at Mathematics.main(Mathematics.java:34)
dragonfly@Home:~/ASSIGNMENTS/ASSIGNMENT_4/Part3$

Through trial and error i seem to have hit upon a solution to my problem. i am stress testing it now but i think it will hold up admirably
import java.util.Scanner;
public class Scan
// scan in a double without a prompt. catch and handle any exceptions
public static double nextDouble ()
    return nextDouble("");
// give the user a prompt and then scan in a double. catch and handle any exceptions
public static double nextDouble (String prompt)
    // variable to hold the result
    double result = 0.0;
    // variable to determine if input was ok
    boolean inputOK = false;
    // do-while loop to get some input
    do
      // print the prompt. what do you want to ask for?
      System.out.print ( prompt );
      // Try to scan and capture the input and catch the exception
      try
      // Call private method to do the actual scanning
        result = scanDouble();
        // if we reach this step then good. we got good input let's exit the loop
        inputOK = true;
      // oops we got an error!
      catch (java.util.InputMismatchException e)
        // inform the user of the error
        System.out.println ("That is not a number! Try again. \n");
        // input was not ok so we can't exit the loop yet
        inputOK = false;
    // keep looping as long as inputOK is false
    } while (!inputOK);
    // return out result
    return result;
// Helper method to nextDouble
private static double scanDouble()
// hold the result
    double result=0.0;
// create the scanner object
    Scanner scan = new Scanner (System.in);
// parse the input
      // If input was not of proper type this call will fail with :
      // java.util.InputMismatchException
      // and return control to calling method
    result = scan.nextDouble();
// Return our findings
    return result ;
}

Streaming data from disk to analog output (using the PCI-6713)

I have a question about the PCI-6713. I want to stream data from the hard disk to all of the card's 8 outputs at its maximum update rate of 1 MS/s. Using single precision (32-bit) variables, this works out to a 32 MB/s transfer rate. That seems to be pushing the limits of most hard disks.
However, the output card only has 12-bit resolution. Is there a way - using LabView or some other software - to store the data on the disk using only 12-bit numbers? This would drastically decrease the transfer rate required.
Thanks for your help.

You could use 8 or 16 bit integers in LabVIEW rather than using 32-bit single precision numbers. I'm sure integers don't have the precision you're wanting though. That is the only way to reduce the size of the numbers you are writing. I am fairly positive there is no way in LabVIEW to do what you are wanting.
Regards,
Todd D.

Update and replace a index via script

Hi everybody,
I created a book with 30 chapters, each of them having a local (or chapter) index at the last pages.
I would like to update these indexes and replace them with the new, updated ones (just as the command "Generate Index..."
do with the option "Replace the Existing Index" on).
I'm able to generate the index via script, but I don't know how to make it replace the existing one.
I dont want the Place the Index, as its precise location is variable and it can run for more than one page and frame.
So even I could get the coordinates of an existing index to place the new one,
I would have the re-layout them, what would defeat my automation goal altogether.
I just want to replace the existing indexes via script, as I can do using the interface.
Is it possible?
Thank you all in advance.
Edson Furman
[email protected]

I'd like to re-ask the same question. I am able to generate an index for the first time, but am at a loss how to do the scripting equivalent of pressing the "OK" button in the following dialog:
I'm guessing I should proceed something like this:
myDocument.indexGenerationOptions.replaceExistingIndex = true;
myIndex = myDocument.indexes[0].update();
myIndex.generate( ? DO I REALLY HAVE TO SPECIFY A PAGE ETC. HERE ? );

Extended precision variable declaration in formula node?

I'm looking for a way to define my variables inside my formula node as extended precision but everytime I attempt this, I get an error. To my understanding, the declaration should be "long double" but it does not seem to work. Anyone have any ideas?
Greycat

Thanks Dennis ... I guess sometimes it seems quicker to ask instead of looking... but then I found what you were referencing very quickly. Just being lazy I suppose!!! Anyway, thanks for the help.
Greycat

Formula Variable decimal digits precision

Hi All,
I am using a formual variable checked for cumulated and digits to be 0.00. But with report output displays, decimal places upto 7 digits.
When I uncheck cumulated, it displays properly with two decimal digits.
Can you please throw some light on how to bring 2 decimal digit precision with checked for cumulated ?
Thanks,
Sri Arun Prian

Hi Arun,
As suggested by Akhan,Click on the KF and go to the display of KF and make the Decimals as 0.00.
Rgds
SVU

Precision of Variables Window

Hello,
I thought to remember that it is possible to set the number of digits that are displayed in the variable view. Right now, doubles show with five decimal places... where can I change this setting to show 'all' digits? Or has this feature gone with the new Variables Window in CVI2013?
Solved!
Go to Solution.

May be I wasn't clear enough...
Right now, there are three possibilities to see a variable value during debugging: as data tooltip or in variable view; the latter provides the option to display the value in floating point format or in scientific format. Floating point provides a precision of 5 digits, scientific format provides a precision of 6 digits, see the figures below.
Once in a while I need to see all digits - where can I set the precision of the displayed variables? I knew how to accomplish it prior to CVI2013...
Thanks.
In this test case, p has been defined as 0.31830988618379067153776752675.

Printf() : Variable precision field for a double

Hello,
First of all, thank you for reading my post and trying to help me, I really appreciate it.
My issue is that I am trying to print a double but the precision is being change by the user while the program is running. Printf() in C/C++ allows an asterisk to put a variable width or precision. It is unclear on my search results if this works with Java' printf() method as well.
I have the following statement: System.out.printf("%.*f", max, val); with max being an integer. It prints the following error message: java.util.UnknownFormatConversionException: Conversion = '*'
Am I doing it wrong please, or is it simply not available in Java?
If it is not available, would you please have any idea for that I can print this double with a different precision at run time, by using my value max?
Thank you a lot for your help.

It appears that C++ does things differently. I think that you're going to have to build your format String at the time that the user gives your program their precision desires or the time of printf call.
e.g.,
String fromatString = "%." + max + "f" ;
System.out.printf(formatString, val);If you program does this a lot, then perhaps you want to create a static method to handle it.
Edited by: Encephalopathic on Jan 13, 2010 5:15 PM

Port of Giac [Longfloat] Library to HP Prime allowing [Variable Precision] Floating Point Arithmetic

HP Prime CAS is based on Giac, but [ misses ] some of its Special Purpose Libraries like the Giac [ Longfloat ] Library, which if [ Ported ] would allow HP Prime to be the First ( handheld ) Calculator to provide [ Variable Precision ] Floating Point Arithmetic routines ( fully integrated at its CAS Kernel level ). HP Prime already have internal calls to [ Longfloat ] library, but resulting in [ Error Messages ], like when selecting more than 14 Digits in [ evalf ] Numerical evaluation, as for example: evalf( 1/7, 14 ) producing 0.142857142857 and evalf( 1/7, 15 ) resulting in "Longfloat library not available Error: Bad Argument Value" The same happens when one tries to Extend the [ Digits ] variable to a value greater than 13, like Digits := 50 which returns Digits := 13 as output ( from any specified value higher than 13 ). The porting of [ Longfloat ] library to HP Prime, would open many New opportunities in [ handheld ] Numerical Computation, usually available only on Top Level Computer Algebra Systems, like Maple, Mathematica or Maxima, and also on Giac/XCas. Its worth mentioning that Any [ Smartphone ] with Xcas/Giac App installed, can fully explore [ Variable Precision ] Floating Point Arithmetic, on current ARM based architectures, which means that a Port of [ Longfloat ] Library from Giac to HP Prime, although requiring some considerable amount of labor, is Not an impossible task. The Benefits of such Longfloat [ Porting ] to a handheld Calculator like HP Prime, would put it several levels Up on the list of Top current Calculator Features, miles and miles away from competitors like TI Nspire CX CAS and Casio ClassPad II fx-CP 400 ... Even HP 49/50g have third party developed routines with limited Variable Precision floating point support, while such feature is Not fully integrated to their native CAS Kernel. For those who do not see "plenty" reason for a [ Longfloat ] Porting to HP Prime its needless to say that the PRIMARY reason for ANY [ CALCULATOR ] is to CALCULATE ! and besides Symbolic Computation ( already implemented on all contemporaries top calculator models ), Arbitrary / [ Variable Precision ] Floating Point Arithmetic is simply The TOP of the TOP ( of the IceCream ) in [ Numerical ] Computation ! ( and beside Computer Algebra Manipulation routines, one of the Main reasons for the initial development of the major packages like Maple, Mathematica or Maxima ).

Thanks for the Link to [ HPMuseum.org ] Page with Valuable Details about the Internal Floating Point implementations both on Home and CAS environments of HP Prime. Its interesting to point to the fact that HP 49/50g has a [ Longfloat ] Version 3.93 package implementation ( with the Same Name but Distinct Code from the Giac Library ) available at [ http://www.hpcalc.org/details.php?id=5363 ] Also its worth mentioning [ Wikipedia ] pages on Arbitrary Precision Arithmetic like [ https://en.wikipedia.org/wiki/Arbitrary-precision_arithmetic ], [ https://en.wikipedia.org/wiki/List_of_arbitrary-precision_arithmetic_software ] and [ https://en.wikipedia.org/wiki/List_of_computer_algebra_systems ] and the Xcas/Giac project at [ https://en.wikipedia.org/wiki/Xcas#Giac ] and Official Site at [ http://www-fourier.ujf-grenoble.fr/~parisse/giac.html ] It would be a Dream come True when a Fully Integrated Variable Precision Floting Point Arithmetic package where definetively incorporated to HP Prime CAS Kernel, like the Giac [ Longfloat ] Library, allowing the Prime to be the First calculator with such Resource trully incorporated at its [ Kernel ] level ( and not like an optional third party module as the HP 49/50g one, which lacks complete integration with their respective Kernel, since HP 49/50g does not have native support for Longfloats ).

Controlling precision in java float variable

Hi,
I need a small clarification. In java, is there any way to control precision of float values.
For ex, when I perform the below
float f1 = 0.0F;
f1 = 200/11;
The result will be = 18.181818
But i want only *18.18*. Is there any way to perform this truncation? Please help me on this.
Thanks

There is no way to control the precision except on printout where you can use java.text.DecimalFormat or String.format() or java.util.Formatter.
P.S. The result of
f1 = 200/11;
will be 18 not 18.181818... because the division will be done as integers.
Edited by: sabre150 on Dec 24, 2007 8:44 AM

Precision of a variable

Similar Messages

Maybe you are looking for