String Tokenizer Helps

I need a little help
i am having trouble removing ALL the white space from my string using a tokenizer
here is my code so far:
    String str = "klj f a sklj fdaskldf"
    String deLimiters = " ";
    StringTokenizer tokens = new StringTokenizer(str, deLimiters);
    while(tokens.hasMoreTokens())
}how would i remove all the white space...and then display the string w/out white space???

DrLaszloJamf wrote:
<quote>
StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.
</quote>
It feels quite comfortable to use StringTokenizer over split. And in schools, and the way that I was tough at my time
instructors mainly used StringTokenizer.

Similar Messages

Need help for string tokenizer!

I need help in writing out the string tokenizer.. i've got all of the functions written except for the nexttokken function which i am stuck in. Can anyone help me out in writing the function?

An example of splitting string by comma
String str"A,B,C";
StringTokenizer st = new StringTokenizer(str,",");
while (st.hasMoreTokens()) {
System.out.println(st.nextToken());
This is just an example of usage; i would prefer to use split() function instead

Help!! i dont know how to make use of token from string tokenizer :(

Hi,
I would like to ask how to manipulate the token that are taken from string tokenizer. For example :
input = a b 1 f 986
supposed output:
a
b
1
f
986
How to make the output look like this?
Output = (a,b,f),(1,986)
Below is the source code of Algebra.java
import java.io.*;
import java.util.*;
* Read and print, using BufferedReader from System.in, onto System.out
public class Algebra {
public static void main(String[] av) {
try {
BufferedReader is = new BufferedReader(new InputStreamReader(System.in));
String inputLine;
System.out.print("Input =");
while ((inputLine = is.readLine()) != null) {
System.out.println("Output="+inputLine);
StringTokenizer st =
new StringTokenizer(inputLine, ", |");
while (st.hasMoreElements())
System.out.println("Token: " + st.nextElement());
break;
is.close();
} catch (IOException e) {
System.out.println("IOException: " + e);
Thank You

You can modify this to serve your goal.
BufferedReader is=null;
try {
is = new BufferedReader(new InputStreamReader(new FileInputStream("Data.txt")));
StreamTokenizer st = new StreamTokenizer(is);
int n=0,k=0,q=0;
String[] s=new String[20];
String[] s0=new String[10];
while(st.nextToken() != StreamTokenizer.TT_EOF) {
        switch(st.ttype) {
          case StreamTokenizer.TT_EOL:
            s[k] = new String("EOL");k++;
            break;
          case StreamTokenizer.TT_NUMBER:
            s0[q] = Double.toString(st.nval);q++;
            break;
          case StreamTokenizer.TT_WORD:
            s[k] = st.sval; // Already a String
             k++;
            break;
          default: // single character in ttype
            s[k] = String.valueOf((char)st.ttype); k++;
        } //end of while
   //   Some output processing here
}catch(IOException ioe){
   ioe.printStackTrace();
}catch(ArrayIndexOutOfBoundsException bx){
   System.err.println(bx);
}finally{
       try{      if(is !=null)     is.close();}catch(IOException ioe){;}
}

Problems with string tokenizer

okay heres my problem, i have a textarea displaying a program listing and i have to extract variables i.e public int something;
i have tried many different approaches but still the output is not what i was looking for. Can someone have a look at my code and check for mistakes that i cant find
public void createDataDictionary() {
          if(fullText.equals("")) return;//do nothing if no text exists
          String checkText = fileContents.getText();//store text area contents into string
          String dataType = "";
          String variable = "";
          String accessModifier = "";
          StringTokenizer str = new StringTokenizer(checkText," \n", true);
          fileCheckBox.setText("");//clear file check box
          while(str.hasMoreTokens()) {//loop while there are more tokens to tokenize
               try{
                    checkText = str.nextToken();
                    //check for comments
                    if((checkText.startsWith("//")) || (checkText.equals("//")) ||
                       (checkText.startsWith("/**")) || (checkText.equals("/**")) ||
                       (checkText.startsWith("*")) || (checkText.equals("*"))) {
                         isComment = true;
                    if((checkText.equals("\n"))) isComment = false;
                    if(!isComment)
                         //check for access modifiers
                         if((checkText.equals("public")) || (checkText.equals("private")) ||
                            (checkText.equals("protected"))) {
                                      accessModifier = checkText;
                                 }else {
                                      accessModifier = "";
                         //check for data types
                         if((checkText.equals("boolean")) || (checkText.equals("char")) ||
                                 (checkText.startsWith("String")) || (checkText.equals("int"))) {
                                   dataType = checkText;
                                   variable = str.nextToken();//get variable expression
                                   System.out.println(accessModifier + " " + dataType + " " + variable);
               }catch(NoSuchElementException nsee) {//not enough tokens to complete operation
                    JOptionPane.showMessageDialog(null, "End of file");
                    return;//break from method
     }here is sample text
private String seven = "help";
char five[];
// String here
//int found
public boolean okay
and here is the output
String
char
String
boolean
//note the space before each output
hope someone can help
thanx in advance

1. Why do you check to see if the token starts with
//, /*, etc, etc. Later you check if the token
equals private, public, protected. It couldn't
equal if it started with a comment.if the token starts with a comment i dont want to read the rest of the line, so i set the isComment flag to true
2. I strongly suggest that you do it line by line.
Perhaps a string tokenizer that only tokenizes by
lines, then within that loop, another that tokenizes
by whitespace. i take it you mean putting the text into a bufferedreader and then using readLine()?? Bit new to string tokenization as you can possibly see
i managed to get the small test text to work more or less as i wanted but when ever i load a large code listing, the results are erratic to say the least
heres a section of this code
private int textNum = 0;/**Integer used to hold the text position within any given line*/
     private int lineNum = 0;/**Integer used to hold the line number within any given file*/
     static boolean application = false;/**Used to track if applet is ran from a browser or JAR file*/
     static boolean fileOpened = false;/**Used to track if file has already been opened*/
     static boolean isComment = false;
     private char lCurve = '(';
     private char rCurve = ')';
     private char lCurly = '{';
     private char rCurly = '}';
     private char lSquare = '[';
     private char rSquare = ']';
     String fullText = "";and heres the output
public int textNum //should be private!!!!
int lineNum //missing private
boolean application //missing static
boolean fileOpened //missing static
boolean isComment //missing static
//all below missing private
char lCurve
char rCurve
char lCurly
char rCurly
char lSquare
char rSquare
String fullText //not there at all

How to retrieve IndividualStrings from a txt file using String Tokenizer.

hello can any one help me to retrieve the individual strings from a txt file using string tokenizer or some thing like that.
the data in my txt file looks like this way.
Data1;
abc; cder; efu; frg;
abc1; cder2; efu3; frg4;
Data2
sdfabc; sdfcder; hvhefu; fgfrg;
uhfhabc; gffjcder; yugefu; hhfufrg;
Data3
val1; val2; val3; val4; val5; val6;
val1; val2; val3; val4; val5; val6;
val1; val2; val3; val4; val5; val6;
val1; val2; val3; val4; val5; val6;
i need to read the data as an individual strings and i need to pass those values to diffarent labels,the dat in Data3 i have to read those values and add to an table datamodel as 6 columns and rows depends on the data.
i try to retrieve data using buffered reader and inputstream reader,but only the way i am retrieving data as an big string of entire line ,i tried with stringtokenizer but some how i was failed to retrive the data in a way i want,any help would be appreciated.
Regards,

Hmmm... looks like the file format isn't even very consistent... why the semicolon after Data1 but not after Data2 or Data3??
Your algorithm is reading character-by-character, and most of the time it's easier to let a StringTokenizer or StreamTokenizer do the work of lexical analysis and let you focus on the parsing.
I am also going to assume your format is very rigid. E.g. section Data1 will ALWAYS come before section Data2, which will come before section Data3, etc... and you might even make the assumption there can never be a Data4, 5, 6, etc... (this is why its nice to have some exact specification, like a grammar, so you know exactly what is and is not allowed.) I will also assume that the section names will always be the same, namely "DataX" where X is a decimal digit.
I tend to like to use StreamTokenizer for this sort of thing, but the additional power and flexibility it gives comes at the price of a steeper learning curve (and it's a little buggy too). So I will ignore this class and focus on StringTokenizer.
I would suggest something like this general framework:
//make a BufferedReader up here...
do
String line = myBufferedReader.readLine();
if (line!=null && line.trim().length()>0)
line = line.trim();
//do some processing on the line
while (line!=null);So what processing to do inside the if statement?
Well, you can recognize the DataX lines easily enough - just do something like a line.startsWith("Data") and check that the last char is a digit... you can even ignore the digit if you know the sections come in a certain order (simplifying assumptions can simplify the code).
Once you figure out which section you're in, you can parse the succeeding lines appropriately. You might instantiate a StringTokenizer, i.e. StringTokenizer strtok = new StringTokenizer(line, ";, "); and then read out the tokens into some Collection, based on the section #. E.g.
strtok = new StringTokenizer(line, ";, ");
if (sectionNo==0)
//read the tokens into the Labels1 collection
else if (sectionNo==1)
//read the tokens into the Labels2 collection
else //sectionNo must be 2
//create a new line in your table model and populate it with the token values...
}I don't think the delimiters are necessary if you are using end-of-line's as delimiters (which is implicit in the fact that you are reading the text out line-by-line). So the original file format you listed looks fine (except you might want to get rid of that rogue semicolon).
Good luck.

Regarding string tokenizer

Hello all,
Can anybody help me to get the file name which is specified in the path like C:/programs/folder/example.txt" or "C:/programs/folder/test.txt" or "C:/programs/example/verify.txt" . The path name will change dynamically in my program.
What I want to do is to get the filename from the path using string tokenizer.
Thanks in advance.

I think I do. Why don't you try that code snippet out
and see that when you invoke File.getName it returns
you the file name with the path part stripped out.
Isn't that what you said you wanted?Even if that ability did not exist (but clearly a programmer should always use it if it is available) then it would be simply to substring the full path starting at last index of the current platform's file separator, and ending at the end of the path string.

Do i need to use a string tokenizer here?

I am doing a program that reads a text file and write it to another new file. Before, writing out the required text lines to a new file, the program should be able to extract certain words from the text file that had been read and the user can able to put some semicolons after certain word character. Do i need to use substring or string tokenizer in order to extarct the words and add some chracters to it??? i dunt understand how to do it. Can anyone help me out here?..thanks.

sure, uncle. here i give u the input and the sample out put it should be. Another thing, is that my data length for certain characters are not same , so how would i determine the appropriate length to put the semicolon....the example input :
470615-03-5039058004TA SECURITIES HOLDINGS BERHAD 000891226AB KARIM BIN MAT HUSSIN LOT 2761 DEPAN ESSO TG MAS JALAN PENGKALAN CHEPA15400 KOTA BHARU 15400DMYSMYSBI 09-7735699 8516 000000080000000100 812974
this is stored in a file n the program will read it
then, it shud put semicolon at certain parts n write it out back. i try to use the insert code but it only works for string buffer as im using buffer reader here.
the output should be.
470615-03-5039 ; 004; SECURITIES HOLDINGS BERHAD ;00891226AB ; KARIM BIN MAT HUSSIN; LOT 2761 DEPAN ESSO TG MAS ; JALAN PENGKALAN CHEPA;15400 KOTA BHARU ; 15400;D;MY;SMYSBI ;09-7735699 8516 ;000000080000000100; 2812974 ...the semicolons shud be put here...but there are different numbers for this part ( 470615-03-5039 )....for example...(978T)...so, how am i goin to determine the correct part to put the semicolon????..plz help me...thanks

Quick string tokenizer question

HELP!
I'm quite new to using string tokenizers in Java.
I'm trying to use a string tokenizer to read some data into a multi dimensional int array and then print it to the screen.
What I'm using so far is:
FileReader file = new FileReader("test.txt");
BufferedReader inputFile = new BufferedReader(file);
StringTokenizer data;
for(int i=0; i<LEARN; i++)
     data = new StringTokenizer(inputFile.readLine());
     for (int j=0; j<NEURONS; j++)
                             examples[i][j] = new int(data.nextToken().intValue);
for(int row=0; row<LEARN; row++)
     for (int collumn=0; collumn<NEURONS; collumn++)
                         screen.print(examples[row][collumn] + " ");
                         screen.flush();
}However, I'm getting errors!!
Could anyone help at all??
Cheers
Ross

However, I'm getting errors!!The messages that come with errors usually play a significant role in resolving them. Would you mind posting those messages?
Where do you define the variable "examples" ?
new int(data.nextToken().intValue);This is definitely wrong.

Sort a string :: Please help.

Hello Everyone,
I am having this very simple problem of sorting a String. please help.
     static String sortString(String str){
          List list = Arrays.asList(str);
          Collections.sort(list);
          for(int i=0, n=list.size();i<n;i++){
               System.out.println(","+ list.get(i));
The function is supposed to take a String and sort it & print it out. This should be simple. Where am I making mistakes? Please help!!

Hello Everyone,
I am having this very simple problem of sorting a String. please help.
     static String sortString(String str){
          List list = Arrays.asList(str);
          Collections.sort(list);
          for(int i=0, n=list.size();i<n;i++){
               System.out.println(","+ list.get(i));
               return str;
The function is supposed to take a String and sort it & print it out. This should be simple. Where am I making mistakes? Please help!!

Pass username and password ADFS without using query string, Please help.

pass username and password ADFS without using query string, Please help.
I used query string , but it is unsecured to pass credentials over url, with simple tool like httpwatch , anyone can easily get the password and decrypt it.

Hi,
According to your post, my understanding is that you had an issue about the ADFS.
As this issue is related to ADFS, I recommend you post your issue to the forum for ADFS.
http://social.msdn.microsoft.com/Forums/vstudio/en-US/home?forum=Geneva
The reason why we recommend posting appropriately is you will get the most qualified pool of respondents, and other partners who read the forums regularly can either share their knowledge or learn from your interaction with us.
Thank you for your understanding and support.
Thanks,
Jason
Forum Support
Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Subscriber Support, contact
[email protected]
Jason Guo
TechNet Community Support

String tokenizer question

I need to parse strings of the form "token A ops token B" where ops can be <, >, =, != and output "tokenA ops tokenB"....(remove spaces in the tokens).
I can use all the ops as delimiters (though I cant specify '!='), and get the tokens and remove the spaces in them. Using StringTokenizer, I doubt if I can know the exact delimiter I encountered so that I can rebuild the string quickly. Sure, I can do an indexOf() for each of the ops to find the one used in original string. But I am curious to know if there is any better approach to do this (doesnt have to be string tokenizer approach)?

If the tokens always consist of "word characters" (letters, digits, and underscores) and the operators always consist of punctuation characters, you can do this: str = str.replaceAll("\\b\\s+\\b", "");

String Tokenizer/ Multiple Client app.

Hi all,
I am developing a multi user pure Java App.
My situation:
1. client request server for connection. > server accepted
2. client send his details (name*|*group*|*ID)
3. server receive the string, update itself and other client accordingly
My problem:
1. to send client detail (name*|*group*|*ID), String tokenizer is being used? and *|" serve as delimeter ?
2. how do create an statement that count the total user connected to server or leave the server
3. and how to assign an identifier to each connected client, thus server can perform differ operation based on the identifier.
Thanks Ya, if someone can provide me some suggestions
Happy Life

To answer your problems directly
1. Client Info: Yes, read all information from the client using an inputstream. convert that into a String, and use StringTokenizer (str, "|") to parse the information
2. # of clients: After a client has connected to your server, they will be automatically shunted to another port that that user alone communicates on. You should set up a new thread to listen to data being sent on that port. Keep these listeners in an array/vector/arraylist and use getSize()
3. The identifier could just be their port #

String tokenizer manipulation

Hi,
I would like to ask how to manipulate the token that are taken from string tokenizer. For example :
input = a b 1 f 986
supposed output:
a
b
1
f
986
How to make the output look like this?
Output = (a,b,f),(1,986)
How to make the program will be able to recognize and arrange the data into the output that i want.
Below is the source code of Algebra.java
import java.io.*;
import java.util.*;
* Read and print, using BufferedReader from System.in, onto System.out
public class Algebra {
public static void main(String[] av) {
try {
BufferedReader is = new BufferedReader(new InputStreamReader(System.in));
String inputLine;
System.out.print("Input =");
while ((inputLine = is.readLine()) != null) {
System.out.println("Output="+inputLine);
StringTokenizer st =
new StringTokenizer(inputLine, ", |");
while (st.hasMoreElements())
System.out.println("Token: " + st.nextElement());
break;
is.close();
} catch (IOException e) {
System.out.println("IOException: " + e);
Thank You

I'd make two arrays and fill them separately depending on whether you can parse the token as an int or not. Try running this code:
import java.io.*;
import java.util.*;
* Read and print, using BufferedReader from System.in, onto System.out
public class Main {
public static void main(String[] av) {
    int numbers[] = new int[20];
    String variables[] = new String[20];
    int numberPosition = 0;
    int variablePosition = 0;
    try {
        BufferedReader is = new BufferedReader(new InputStreamReader(System.in));
        String inputLine;
        System.out.print("Input =");
        while ((inputLine = is.readLine()) != null) {
            StringTokenizer st =
            new StringTokenizer(inputLine, ", |");
            while (st.hasMoreElements()) {
                String token = st.nextToken();
                try {
                    int i = Integer.parseInt(token);
                    numbers[numberPosition++] = i;
                } catch (NumberFormatException e) {
                    variables[variablePosition++] = token;
            System.out.print("(");
            for (int x = 0; x < variablePosition -1; x++)
                System.out.print(variables[x]+",");
            System.out.print(variables[variablePosition-1]+")");
            System.out.print("(");
            for (int x = 0; x < numberPosition -1; x++)
                System.out.print(numbers[x]+",");
            System.out.print(numbers[numberPosition-1]+")");
            break;
    } catch (IOException e) {
        System.out.println("IOException: " + e);
}Sample Output:
Input =a b 1 f 986
(a,b,f)(1,986)
press any key to exit...

String Tokenizer in JSP

What is the best way to write a string tokenizer is a JSP page??
I am passing a vector to the JSP page which is being outputted on to the page and now I want to edit the contents of the vector. I need to tokenize the vector into a series of strings but I am making a hash of it so far.
Is there a better way to do this?

I'm not sure what your exact question is. StringTokenizer works with String classes not Vectors. Do you want to change the data in the Vector and leave the Vector in tact, or do you want to retrieve the data from the Vector and do something with it?
You can retrive the contents of the Vector into an Object array, or you can enumerate through the elements, or you can use the get method itself. It depends on what you need to do.

Parsing and String tokenizer

I am trying to use String Tokenizer to extract fields from a file then do a calculation on some of the fields.
My problem is when I try to parse the strings that need the calculations I get error codes as shown at the end of post. I don't see why the parsing shouldn't work.( error codes are class expected and incompatable types)
// ParsingTextFile.java: Process text file using StringTokenizer
import java.io.*;
import java.util.StringTokenizer;
public class ParsingTextFile1 {
/** Main method */
public static void main(String[] args) {
   // Declare buffered file reader and writer streams
   BufferedReader brs = null;
   BufferedWriter bws = null;
   //declare tokenizer
   StringTokenizer tokenizer;
   // Declare a print stream
   PrintWriter out = null;
    // Five input file fields:total string, student name, midterm1,
    // midterm2, and final exam score
    String line=null;
    String sname = null;
   String mid1= null;
    String mid2 = null;
    String finalSc = null;
/ double midterm1 = 0;
    double midterm2 = 0;
    double finalScore = 0;
    // Computed total score
    double total = 0;
    try {
      // Create file input and output streams
      brs = new BufferedReader(new FileReader("in.dat"));
      bws = new BufferedWriter(new FileWriter("out.dat"));
     while((line= brs.readLine())!=null){
     tokenizer =new StringTokenizer(line);
      sname = tokenizer.nextToken();
      mid1 = tokenizer.nextToken();
      mid2 = tokenizer.nextToken();
      finalSc= tokenizer.nextToken();
      midterm1 = double.parseDouble(mid1)//this code not working
      midterm2 = double.parseDouble(mid2);)//this code not working
      finalScore = double.parseDouble(finalScore);//this code not working
      out = new PrintWriter(bws);
      total = midterm1*0.3 + midterm2*0.3 + finalScore*0.4;
      out.println(sname + " " + total);
    catch (FileNotFoundException ex) {
      System.out.println("File not found: in.dat");
    catch (IOException ex) {
      System.out.println(ex.getMessage());
    finally {
      try {
        if (brs != null) brs.close();
        if (bws != null) bws.close();
      catch (IOException ex) {
        System.out.println(ex);
}[\code]
errorsC:\j2sdk1.4.1_06\bin\ParsingTextFile1.java:43: class expected
    midterm1 = double.parseDouble(mid1);
                      ^
C:\j2sdk1.4.1_06\bin\ParsingTextFile1.java:44: class expected
    midterm2 = double.parseDouble(mid2);
                      ^
C:\j2sdk1.4.1_06\bin\ParsingTextFile1.java:45: class expected
    finalScore = double.parseDouble(finalScore);
                        ^
C:\j2sdk1.4.1_06\bin\ParsingTextFile1.java:43: incompatible types
found   : java.lang.Class
required: double
    midterm1 = double.parseDouble(mid1);
                     ^
C:\j2sdk1.4.1_06\bin\ParsingTextFile1.java:44: incompatible types
found   : java.lang.Class
required: double
    midterm2 = double.parseDouble(mid2);
                     ^
C:\j2sdk1.4.1_06\bin\ParsingTextFile1.java:45: incompatible types
found   : java.lang.Class
required: double
    finalScore = double.parseDouble(finalScore);
                       ^
6 errors
Process completed.

while(line.hasMoreTokens())
sname = tokenizer.nextToken();
   mid1 = tokenizer.nextToken();
mid2 = tokenizer.nextToken();
   finalSc= tokenizer.nextToken();
    midterm1 = double.parseDouble(mid1)//this code not working
     midterm2 = double.parseDouble(mid2);)//this code not working
   finalScore = double.parseDouble(finalScore);//this code not working
}if it doesn't work then you need to add delimiter Or the token that you are trying to convert into is not valid.

String Tokenizer Helps

Similar Messages

Maybe you are looking for