O'Reilly logo

Building Parsers with Java™ by Steven John Metsker

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

9.10. The TokenStringSource Utility

The TokenStringSource class in sjm.parse.tokens is a utility that helps divide an input stream into lines that a parser can parse one at a time. Figure 9.16 shows this class.

Figure 9.16. The TokenStringSource class. The TokenStringSource nextTokenString() method returns all the tokens up to the specified delimiter as a TokenString.

This class provides an enumeration over the tokens of a tokenizer. Each call to nextTokenString() results in a TokenString that contains all the tokens up to a specified delimiter. This is useful for languages that consist of statements that are separated by a delimiter, such as “ ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required