C:\> java TokenTester Enter a string: val = 12+8 val 12 8
You probably regard the "=" and the "+" symbols as important,
so they should be regarded as tokens.
But "+" also needs to work as a delimiter so that "12+8"
is broken into the tokens "12", "+" and "8".
This is done by using true as the third parameter
in the constructor:
Now individual delimiters (as well as tokens) are returned
by nextToken()
.
import java.io.*; import java.util.*; public class TokenTester { public static void main ( String[] args ) throws IOException { BufferedReader stdin = new BufferedReader(new InputStreamReader(System.in)); System.out.print("Enter a string:"); String data = stdin.readLine(); StringTokenizer tok = new StringTokenizer( data, " =+-", true ); // note the space before = while ( tok.hasMoreTokens() ) System.out.println( tok.nextToken() ); } }