book

lex & yacc, 2nd Edition

Name: lex & yacc, 2nd Edition
ISBN: 9781565920002

by John Levine, Doug Brown, Tony Mason

October 1992

Intermediate to advanced

384 pages

7h 8m

English

O'Reilly Media, Inc.

Read now

Unlock full access

lex & yacc, 2nd Edition
A Note Regarding Supplemental Files
Preface
What’s New in the Second EditionScope of This BookAvailability of Lex and YaccSample ProgramsConventions Used in This HandbookAcknowledgments
1. Lex and Yacc
The Simplest Lex ProgramRecognizing Words with LexSymbol TablesGrammarsParser-Lexer CommunicationThe Parts of Speech LexerA Yacc ParserThe Rules SectionRunning Lex and YaccLex vs. Hand-written LexersExercises
2. Using Lex
Regular ExpressionsExamples of Regular ExpressionsA Word Counting ProgramParsing a Command LineStart StatesA C Source Code AnalyzerSummaryExercises
3. Using Yacc
GrammarsRecursive RulesShift/Reduce ParsingWhat Yacc Cannot ParseA Yacc ParserThe Definition SectionThe Rules SectionSymbol Values and ActionsThe LexerCompiling and Running a Simple ParserArithmetic Expressions and AmbiguityWhen Not to Use Precedence RulesVariables and Typed TokensSymbol Values and %unionSymbol TablesFunctions and Reserved WordsReserved Words in the Symbol TableInterchangeable Function and Variable NamesBuilding Parsers with MakeSummaryExercises
4. A Menu Generation Language
Overview of the MGLDeveloping the MGLBuilding the MGLInitializationScreen ProcessingTerminationSample MGL CodeExercises
5. Parsing SQL
A Quick Overview of SQLRelational Data BasesManipulating RelationsThree Ways to Use SQLThe Syntax CheckerThe LexerError and Main RoutinesThe ParserDefinitionsTop Level RulesThe Schema SublanguageBase TablesView DefinitionPrivilege DefinitionsThe Module SublanguageCursor DefinitionsThe Manipulation SublanguageSimple StatementsFETCH StatementsINSERT StatementsDELETE StatementsUPDATE StatementsScalar ExpressionsSELECT StatementsTable ExpressionsSearch ConditionsOdds and EndsUsing the Syntax CheckerEmbedded SQLChanges to the LexerChanges to the ParserAuxiliary RoutinesUsing the PreprocessorExercises
6. A Reference for Lex Specifications
Structure of a Lex SpecificationDefinition SectionRules SectionUser SubroutinesBEGINBugsAmbiguous LookaheadAT&T LexFlexCharacter TranslationsContext SensitivityLeft ContextRight ContextDefinitions (Substitutions)ECHOInclude Operations (Logical Nesting of Files)File Chaining with yywrap()File NestingAT&T LexFlexMKS LexAbraxas PclexPOSIX LexInput from StringsAT&T LexFlexAbraxas PclexMKS LexPOSIX Lexinput()Internal Tables (%N Declarations)lex Librarymain()Other Library RoutinesLine Numbers and yylinenoLiteral BlockMultiple Lexers in One ProgramCombined LexersMultiple LexersUsing the p FlagFaking Itoutput()Portability of Lex LexersPorting Lex SpecificationsPorting Generated C LexersLibrariesBuffer SizesCharacter SetsRegular Expression SyntaxMetacharactersPOSIX ExtensionsREJECTReturning Values from yylex()Start Statesunput()yyinput(), yyoutput(), yyunput()yylengyyless()yylex()User Code in yylex()yymore()yytextEnlarging yytextAT&T and MKS LexFlexPclexyywrap()
7. A Reference for Yacc Grammars
Structure of a Yacc GrammarSymbolsDefinition SectionRules SectionUser Subroutines SectionActionsEmbedded ActionsSymbol Types for Embedded ActionsObsolescent FeatureAmbiguity and ConflictsTypes of ConflictsShift/Reduce ConflictsReduce/Reduce ConflictsBugs in YaccReal BugsError HandlingDeclaring Literal TokensInfinite RecursionUnreal BugsInterchanging PrecedencesEmbedded ActionsEnd MarkerError Token and Error Recovery%ident DeclarationInherited Attributes ($0)Symbol Types for Inherited AttributesLexical FeedbackLiteral BlockLiteral TokensPortability of Yacc ParsersPorting Yacc GrammarsPorting Generated C ParsersLibrariesCharacter CodesPrecedence, Associativity, and Operator DeclarationsPrecedence and AssociativityPrecedenceAssociativityOperator DeclarationsUsing Precedence and Associativity to Resolve ConflictsTypical Uses of PrecedenceRecursive RulesLeft and Right RecursionRulesSpecial CharactersStart DeclarationSymbol ValuesDeclaring Symbol TypesCalculator ExampleExplicit Symbol TypesTokensToken NumbersToken Values%type Declaration%union DeclarationVariant and Multiple GrammarsCombined ParsersMultiple ParsersUsing the -p FlagFaking ItRecursive ParsingLexers for Multiple Parsersy.output FilesYacc Librarymain()yyerror()YYABORTYYACCEPTYYBACKUPyyclearinyydebug and YYDEBUGYYDEBUGyydebugyyerrokYYERRORyyerror()yyparse()YYRECOVERING()

8. Yacc Ambiguities and Conflicts
The Pointer Model and ConflictsTypes of ConflictsParser StatesContents of y.outputReduce/Reduce ConflictsShift/Reduce ConflictsReview of Conflicts in y.outputCommon Examples of ConflictsExpression GrammarsIF—THEN—ELSENested List GrammerHow Do I Fix the Conflict?IF—THEN—ELSE (Shift/Reduce)Loop Within a Loop (Shift/Reduce)Expression Precedence (Shift/Reduce)Limited Lookahead (Shift/Reduce or Reduce/Reduce)Overlap of Alternatives (Reduce/Reduce)SummaryExercises
9. Error Reporting and Recovery
Error ReportingBetter Lex Error ReportsError RecoveryYacc Error RecoveryWhere to Put Error TokensCompiler Error RecoveryExercises
A. AT&T Lex
Error Messages
B. AT&T Yacc
OptionsError Messages
C. Berkeley Yacc
OptionsError MessagesFatal ErrorsRegular ErrorsWarningsInformative Messages
D. GNU Bison
Differences
E. Flex
Flex DifferencesOptionsError MessagesFlex Versions of Lexer Examples
F. MKS lex and yacc
DifferencesNew Features
G. Abraxas lex and yacc
DifferencesNew Features
H. POSIX lex and yacc
OptionsDifferences
I. MGL Compiler Code
MGL Yacc SourceMGL Lex SourceSupporting C Code
J. SQL Parser Code
Yacc ParserCross-reference
K. SQL Parser Code
Lex ScannerSupporting Code
Glossary
Bibliography
Index
About the Authors
Copyright

Overview

This book shows you how to use two Unix utilities, lex andyacc, in program development. These tools help programmers build compilers and interpreters, but they also have a wider range of applications.The second edition contains completely revised tutorial sections for novice users and reference sections for advanced users. This edition is twice the size of the first and has an expanded index.The following material has been added:

Each utility is explained in a chapter that covers basic usage and simple, stand-alone applications
How to implement a full SQL grammar, with full sample code
Major MS-DOS and Unix versions of lex and yacc are explored in depth, including AT&T lex and yacc, Berkeley yacc, Berkeley/GNU Flex, GNU Bison, MKS lex andyacc, and Abraxas PCYACC

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781565920002Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills