ammasmohan

 Profile Answers by ammasmohan 

• Oct 4th, 2005

 

The StreamTokenizer class takes an input stream and parses it into "tokens", allowing the tokens to be read one at a time. The parsing process is controlled by a table and a number of flags that can be set to various states. The stream tokenizer can recognize identifiers, numbers, quoted strings, and various comment styles. Each byte read from the input stream is regarded as a character in the range '\u0000' through '\u00FF'. The character value is used to look up five possible attributes of the character: white space, alphabetic, numeric, string quote, and comment character. Each character can have zero or more of these attributes.Source:http://java.sun.com/j2se/1.4.2/docs/api/java/io/StreamTokenizer.html

Thamizhselvi

• Jul 23rd, 2007

 

1. String Tokenizing is the process of taking big pieces of source data, breaking them into little pieces.

2. Storing the little pieces in variables.

3. We have two classes in the API that provide tokenizing capabilities,

1. String [ Using split() ].
2. Scanner has Next(), hasNextInt(), hasNextBoolean(), etc.,