Domain-specific languages (DSLs) are encountered in many aspects of a software engineer’s life: configuration file formats, data transfer protocols, model schemas, application extensions, interface definition languages, and many others. Because of the nature of such languages, the language expression needs to be straightforward and easy to understand.
Given a math expression as a string, this is the series of processing applied to that string:
The string is split into a stream of tokens.
The tokens are used to construct the syntax tree.
The syntax tree is traversed to evaluate the expression.
Each step will be described in the following sections.
The first important thing to do to a string representing a math expression is lexical analysis—that is, splitting the string into a stream of tokens. Quite expectedly, a function that does this is often called a tokenizer. Alternatively, it is also known as a lexer or a scanner.
We first need to define the types of the tokens. Since we’ll be dealing with simple math expressions, all we really need are ...