The token Module
The token
module, shown in Example 13-21, contains a list of all tokens used by the standard Python
tokenizer.
Example 13-21. Using the token Module
File: token-example-1.py import token print "NUMBER", token.NUMBER print "PLUS", token.STAR print "STRING", token.STRINGNUMBER 2
PLUS 16
STRING 3
Get Python Standard Library now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.