The StringTokenizer class allows an application to break a string into tokens. Consider that you have a string which has delimiters in it and you want to break it into tokens. You can achieve this using String Tokenizer class, just in few lines of code. The same thing can also be achieved by standard string functions like indexOf() and subString() but it involves a lot of coding.
StringTokenizer provides following methods for performing operations:
int countTokens()
boolean hasMoreElements()
boolean hasMoreTokens()
Object nextElement()
String nextToken()
String nextToken(String delim)
For example we have a string which contains various names with semi colon as delimiter.
We want to get person names from that String.
In the example below, we will use StringTokenizer for this purpose:
String names = “Jeffrey;Moses;Rahul;Sachin;Dravid”;
StringTokenizer Tok = new StringTokenizer(names,”;”);
System.out.println(“Number of tokens: ” + Tok.countTokens());
while (Tok.hasMoreElements())
System.out.println(Tok.nextElement());
Ok. Let me know how it is better to String.split() method. Also let me know the cons of split().
No offenses , i am just trying to make this more interesting
Raameshwar.
Nice question Raameshwar. Below i have mentioned 6 difference between them. Need ur comments and suggestions for that.
# 1
We are spliting string as delimiter. If there is multiple space in a word. Then we will get zero length strings in split, but we wont get in Tokenizer.
string: the cat mat
Split:
String str =”the cat sat on the mat”;
String stra[] = str.split(” “);
System.out.println(“Split length “+stra.length);
for(int i=0; i<stra.length;i++){
System.out.println(” split “+stra[i]);
}
O/P:
split length 5
split the
split
split
split cat
split mat
StringTokenizer:
StringTokenizer tok = new StringTokenizer(str,” “);
while (tok.hasMoreElements())
System.out.println(“token “+tok.nextElement());
O/P:
token the
token cat
token mat
# 2
split( String )
split( String, int )
1. There is an option to stop at particular chop in split. For e.g If we want to stop after 3 chop then
String spiting = ” This,is,we,want,to,do”
String arrSplit [] = split.spliting(“,”,3);
for(int i=0; i<arrSplit.length;i++){
System.out.println(” split “+stra[i]);
}
o/p:
split This
split is
split we,want,to,do
But this option is not available in StringTokenizer.
# 3
split will taker as regular experssion. It may cause problem in split. For e.g if we want to split the word with “.”. As of regular expression “.” means single character. But it is entirely different for StringTokenizer.
For e.g
Split :
String str =”This.is.we.want.to,do”;
String stra[] = str.split(“.”);
System.out.println(“Split length “+stra.length);
for(int i=0; i<stra.length;i++){
System.out.println(” split “+stra[i]);
}
O/P:
Split length 0
StringTokenizer:
StringTokenizer tok = new StringTokenizer(str,”.”);
while (tok.hasMoreElements())
System.out.println(“token “+tok.nextElement());
O/P:
token This
token is
token we
token want
token to,do
So there is a special character in regular expressions, which means “treat the following special character as a normal character”.
That character is \\
we can use like this String stra[] = str.split(“\\.”);
# 4
StringTokenizer is capable of returning delimiters whereas split() cannot.
Ex :
StringTokenizer st = new StringTokenizer(“Java|StringTokenizer|Example 1″, “|”, true);
//iterate through tokens
while(st.hasMoreTokens())
System.out.println(st.nextToken(“|”));
}
Output: Java | StringTokenizer | Example 1
# 5
StringTokenizer is very much faster than split().
# 6
split() is very flexible than StringTokenizer.
useful link
http://www.codeguru.com/forum/showthread.php?s=&threadid=292394
http://java.sun.com/j2se/1.4.2/docs/api/java/util/StringTokenizer.html
useful links
http://java.sun.com/j2se/1.4.2/docs/api/java/util/StringTokenizer.html
http://www.codeguru.com/forum/showthread.php?s=&threadid=292394
#1
String str = “the cat sat on the mat”;
String stra[] = str.split(” “);
System.out.println(“Split length ” + stra.length);
for (int i = 0; i < stra.length; i++) {
System.out.println(” split ” + stra[i]);
Having space as delimitter gives the following output.
Split length 6
split the
split cat
split sat
split on
split the
split mat
Which is the expected one. so difference between split and tokenizer in terms of the result.
#2 Agreed
#3 Yes for dot (.) delimitter you have to use the escape sequence. But still it is possible with split.
#4 Agreed
#5 Agreed
#6 I think it the other way round. StiringTokenizer is more flexible.
The most important con of the split is that it creates a String[] in the memory. When you parse huge data, say multiple paragraphs, all the splits are stored in the memory which is a big memory issue and performance hit.
Conclusion:
Use split for smaller texts such as date validation or similar simple types. Tokenizer is a full fledged api which is used to handle huge data. so use it for that purpose.
Regards,
Raameshwar.
i struck up with a problem while establishing JDBC-ODBC connectivity.
i am sending the code as well as the compiler error log. pls help me out.
following is the compiler messages.:
Generated servlet error:
[javac] Since fork is true, ignoring compiler setting.
[javac] Compiling 1 source file
[javac] Since fork is true, ignoring compiler setting.
[javac] D:\Program Files\Apache Group\Tomcat 4.1\work\Standalone\localhost\_\jagan_jsp.java:92: variable Db might not have been initialized
[javac] DataRequest = Db.createStatement();
[javac] ^
[javac] D:\Program Files\Apache Group\Tomcat 4.1\work\Standalone\localhost\_\jagan_jsp.java:101: variable Results might not have been initialized
[javac] Records = Results.next();
[javac] ^
[javac] D:\Program Files\Apache Group\Tomcat 4.1\work\Standalone\localhost\_\jagan_jsp.java:115: variable DataRequest might not have been initialized
[javac] DataRequest.close();
[javac] ^
[javac] 3 errors
the following is the script
Hi Jagan,
Send across your code. We will try to help you. Seems like syntax issue.
Regards,
Raameshwar.