Languages and Machines:

An Introduction to the Theory of Computer Science




Addison-Wesley Publishing Co.




The primary objective of the book Languages and Machines is to give a mathematically sound presentation of the theory of computing at a level suitable for junior and senior level computer science majors. The topics covered include the theory of formal languages and automata, computability, computational complexity, and the deterministic parsing of context-free languages. To make these topics accessible to the undergraduate student, no special mathematical prerequisites are assumed. Rather, the mathematical tools of the theory of computing, naive set theory, recursive definitions, and proof by mathematical induction, are introduced in the course of the presentation.


The presentation of formal language theory and automata develops the relationships between the grammars and abstract machines of the Chomsky hierarchy. Parsing context-free languages is introduced via standard graph-searching algorithms to make it accessible to students having taken a data structures course.


Finite-state automata and Turing machines provide the framework for the study of effective computation. Topics covered include decidability, the Church-Turing thesis, and the equivalence of Turing computability and mu-recursive functions. The classes P and NP of solvable decision problems and the theory of NP-completeness are introduced by analyzing the time complexity of Turing machines.

Chapters on LL and LR grammars are included to permit a presentation of formal language theory that will provide the foundation for a course in compiler design.


Solutions Manual: I have written a solutions manual with worked out solutions to approximately one third of the problems in the book. Because some instructors do not want students to have solutions worked out by others (I am not one of them), Addison-Wesley choose not to publish the manual but will make these solutions available to instructors. Of course, the instructors are then free to make them available to whomever they wish. For information about obtaining the solutions, please contact Addison-Wesley at aw.cse@awl.com.


Errata: The book is currently in the third edition. Technical errors and other typos that may cause confusion in this printing can be found in ERRATA.





Table of Contents


Part I: Foundations

Chapter 1: Mathematical Preliminaries

1.1 Set Theory

1.2 Cartesian Product, Relations, and Functions

1.3 Equivalence Relations

1.4 Countable and Uncountable Sets

1.5 Diagonalization and Self-Reference

1.6 Recursive Definitions

1.7 Mathematical Induction

1.8 Directed Graphs

Chapter 2: Languages

2.1 Strings and Languages

2.2 Finite Specification of Languages

2.3 Regular Sets and Expressions

2.4 Regular Expressions and Text Searching


Part II: Grammars, Automata, and Languages

Chapter 3: Context-Free Grammars

3.1 Context-Free Grammars and Languages

3.2 Examples of Grammars and Languages

3.3 Regular Grammars

3.4 Verifying Grammars

3.5 Leftmost Derivations and Ambiguity

3.6 Context-Free Grammars and Programming Language Definition

Chapter 4: Normal Forms for Context-Free Grammars

4.1 Grammar Transformations

4.2 Elimination of Lambda-Rules

4.3 Elimination of Chain Rules

4.4 Useless Symbols

4.5 Chomsky Normal Form

4.6 The CYK Agorithm

4.7 Removal of Direct Left Recursion

4.8 Greibach Normal Form

Chapter 5: Finite Automata

5.1 A Finite-State Machine

5.2 Deterministic Finite Automata

5.3 State Diagrams and Examples

5.4 Nondeterministic Finite Automata

5.5 Lambda-Transitions

5.6 Removing Nondeterminism

5.7 DFA Minimization

Chapter 6: Properties of Regular Languages

6.1 Finite-State Acceptance and Regular Languages

6.2 Expression Graphs

6.3 Regular Grammars and Finite Automata

6.4 Closure Properties of Regular Languages

6.5 A Nonregular Language

6.6 The Pumping Lemma for Regular Languages

6.7 The Myhill-Nerode Theorem

Chapter 7: Pushdown Automata and Context-Free Languages

7.1 Pushdown Automata

7.2 Variations on the PDA Theme

7.3 Pushdown Automata and Context-Free Languages

7.4 The Pumping Lemma for Context-Free Languages

7.5 Closure Properties of Context-Free Languages


Part III Computability

Chapter 8: Turing Machines

8.1 The Standard Turing Machine

8.2 Turing Machines as Language Acceptors

8.3 Alternative Acceptance Criteria

8.4 Multitrack Machines

8.5 Two-Way Tape Machines

8.6 Multitape Machines

8.7 Nondeterministic Turing Machines

8.8 Turing Machines as Language Enumerators

Chapter 9: Turing Computable Functions

9.1 Computation of Functions

9.2 Numeric Computation

9.3 Sequential Operation of Turing Machines

9.4 Composition of Functions

9.5 Uncomputable Functions

9.6 Toward a Programming Language

Chapter 10: The Chomsky Hierarchy

10.1 Unrestricted Grammars

10.2 Context-Sensitive Grammars

10.3 Linear-Bounded Automata

10.4 The Chomsky Hierarchy

Chapter 11: Decision Problems and the Church-Turing Thesis

1.1 Representation of Decision Problems

11.2 Decision Problems and Recursive Languages

11.3 Problem Reduction

11.4 The Church-Turing Thesis

11.5 A Universal Turing Machine

Chapter 12: Undecidability

12.1 The Halting Problem for Turing Machines

12.2 Problem Reduction and Undecidability

12.3 Additional Halting Problem Reductions

12.4 Rice’s Theorem

12.5 An Unsolvable Word Problem

12.6 The Post Correspondence Problem

12.7 Undecidable Problems in Context-Free Grammars

Chapter 13: Mu-Recursive Functions

13.1 Primitive Recursive Functions

13.2 Some Primitive Recursive Functions

13.3 Bounded Operators

13.4 Division Functions

13.5 Godel Numbering and Course-of-Values Recursion

13.6 Computable Partial Functions

13.7 Turing Computability and Mu-Recursive Functions

13.8 The Church-Turing Thesis Revisited


Part IV: Computational Complexity

Chapter 14: Time Complexity

14.1 Measurement of Complexity

14.2 Rates of Growth

14.3 Time Complexity of a Turing Machine

14.4 Complexity and Turing Machine Variations

14.5 Linear Speedup

14.6 Properties of Time Complexity of Languages

14.7 Simulation of Computer Computations

Chapter 15: Tractability and Decision Problems

15.1 Time Complexity of Nondeterministic Turing Machines

15.2 The Classes P and NP

15.3 Complexity of Decision Problems

15.4 Decision Problems and Complexity Classes

15.5 The Hamiltonian Circuit Problem

15.6 Polynomial-Time Reduction

15.7 P = NP?

15.8 The Satisfiability Problem

15.9 Complexity Class Relations

Chapter 16: NP-Complete Problems

16.1 Reduction and NP-Complete Problems

16.2 The 3-Satisfiability Problm

16.3 Reductions from 3-Satisfiability

16.4 Reduction and Subproblems

16.5 Optimization Problems

16.6 Approximation Algorithms

16.7 Approximation Schemes

Chapter 17: Additional Complexity Classes

17.1 Derivative Complexity Classes

17.2 Space Complexity

17.3 Relations between Time and Space Complexity

17.3 P-Space, NP-Space, and Savitch’s Theorem

17.4 P-Space Completeness

17.5 An Intractable Problem


Part VI: Deterministic Parsing

Chapter 18: Parsing: An Introduction

18.1 The Graph of a Grammar

18.2 A Top-Down Parser

18.3 Reductions and Bottom-Up Parsing

18.4 A Bottom-Up Parser

18.5 Parsing and Compiling

Chapter 19: LL(k) Grammars

19.1 Lookahead in Context-Free Grammars

19.2 FIRST, FOLLOW, and Lookahead Sets

19.3 Strong LL(k) Grammars

19.4 Construction of FIRSTk Sets

19.5 Construction of FOLLOWk Sets

19.6 A Strong LL(l) Grammar

19.7 A Strong LL(k) Parser

19.8 LL(k) Grammars

Chapter 20: LR(k) Grammars

20.1 LR(0) Contexts

20.2 An LR(0) Parser

20.3 The LR(0) Machine

20.4 Acceptance by the LR(0) Machine

20.5 LR(1) Grammars


Appendix I: Index of Notation

Appendix II: The Greek Alphabet

Appendix III: The ASCII Character Set

Appendix IV: Backus-Naur Form Definition of Java