Mastering Windows PowerShell Scripting - Third Edition

Tokenizer

In addition to the AST, PowerShell can also convert a script into a series of tokens, each representing an element of a script.

In PowerShell 2, the Tokenize static method of System.Management.Automation.PSParser may be used; an example is as follows:

$script = @'# A short scriptif ($true) { Write-Host 'Hello world'}'@$errors = @()$tokens = [System.Management.Automation.PSParser]::Tokenize($script, [Ref]$errors)

The tokens array contains objects describing each part of the script. The first of these is shown as follows; it describes the comment at the start of the script:

Content     : # A short scriptType        : CommentStart       : 0Length      : 16StartLine   : 1StartColumn : 1EndLine     : 1EndColumn   : 17

With PowerShell 3, two static methods on the ...