Languages and Machines:

An Introduction to the Theory of Computer Science




Addison-Wesley Publishing Co.




The primary objective of the book Languages and Machines is to give a mathematically sound presentation of the theory of computing at a level suitable for junior and senior level computer science majors. The topics covered include the theory of formal languages and automata, computability, computational complexity, and the deterministic parsing of context-free languages. To make these topics accessible to the undergraduate student, no special mathematical prerequisites are assumed.  Rather, the mathematical tools of the theory of computing, naive set theory, recursive definitions, and proof by mathematical induction, are introduced in the course of the presentation.

The presentation of formal language theory and automata develops the relationships between the grammars and abstract machines of the Chomsky hierarchy. Parsing context-free languages is introduced via standard graph-searching algorithms to make it accessible to students having taken a data structures course.

Finite-state automata and Turing machines provide the framework for the study of effective computation. Topics covered include decidability, the Church-Turing thesis, and the equivalence of Turing computability and mu-recursive functions. The classes P and NP of solvable decision problems and the theory of NP-completeness are introduced by analyzing the time complexity of Turing machines.

Chapters on LL and LR grammars are included to permit a presentation of formal language theory that will provide the foundation for a course in compiler design.


Solutions Manual: I have written a solutions manual with worked out solutions to approximately one third of the problems in the book. Because some instructors do not want students to have solutions worked out by others (I am not one of them), Addison-Wesley choose not to publish the manual but will make these solutions available to instructors.  Of course, the instructors are then free to make them available to whomever they wish.  For information about obtaining the solutions, please contact Addison-Wesley at aw.cse@awl.com.


Comments and Suggestions: If you have any general comments and suggestions to improve or expand the presentation or have found errors in the text please feel free to contact me.  I appreciate the help and will attempt to incorporate your suggestions into any subsequent editions.


Errata:  The book is currently in the first printing of the third edition.  Technical errors and other typos that may cause confusion in this printing can be found in ERRATA.





Table of Contents


Part I: Foundations

Chapter 1: Mathematical Preliminaries

            1.1 Set Theory

            1.2 Cartesian Product, Relations, and Functions

            1.3 Equivalence Relations

            1.4 Countable and Uncountable Sets

            1.5 Diagonalization and Self-Reference

            1.6 Recursive Definitions

            1.7 Mathematical Induction

            1.8 Directed Graphs

Chapter 2: Languages

            2.1 Strings and Languages

            2.2 Finite Specification of Languages

            2.3 Regular Sets and Expressions

            2.4 Regular Expressions and Text Searching


Part II: Grammars, Automata, and Languages

Chapter 3: Context-Free Grammars

            3.1 Context-Free Grammars and Languages

            3.2 Examples of Grammars and Languages

            3.3 Regular Grammars

            3.4 Verifying Grammars

            3.5 Leftmost Derivations and Ambiguity

            3.6 Context-Free Grammars and Programming Language Definition

Chapter 4: Normal Forms for Context-Free Grammars

            4.1 Grammar Transformations

            4.2 Elimination of Lambda-Rules

            4.3 Elimination of Chain Rules

            4.4 Useless Symbols

            4.5 Chomsky Normal Form

            4.6 The CYK Agorithm

            4.7 Removal of Direct Left Recursion

            4.8 Greibach Normal Form

Chapter 5: Finite Automata

            5.1 A Finite-State Machine

            5.2 Deterministic Finite Automata

            5.3 State Diagrams and Examples

            5.4 Nondeterministic Finite Automata

            5.5  Lambda-Transitions

            5.6 Removing Nondeterminism

            5.7 DFA Minimization

Chapter 6: Properties of Regular Languages

            6.1 Finite-State Acceptance and Regular Languages

            6.2 Expression Graphs

            6.3 Regular Grammars and Finite Automata

            6.4 Closure Properties of Regular Languages

            6.5 A Nonregular Language

            6.6 The Pumping Lemma for Regular Languages

            6.7 The Myhill-Nerode Theorem

Chapter 7: Pushdown Automata and Context-Free Languages

            7.1 Pushdown Automata

            7.2 Variations on the PDA Theme

            7.3 Pushdown Automata and Context-Free Languages

            7.4 The Pumping Lemma for Context-Free Languages

            7.5 Closure Properties of Context-Free Languages


Part III Computability

Chapter 8: Turing Machines

            8.1 The Standard Turing Machine

            8.2 Turing Machines as Language Acceptors

            8.3 Alternative Acceptance Criteria

            8.4 Multitrack Machines

            8.5 Two-Way Tape Machines

            8.6 Multitape Machines

            8.7 Nondeterministic Turing Machines

            8.8 Turing Machines as Language Enumerators

Chapter 9: Turing Computable Functions

            9.1 Computation of Functions

            9.2 Numeric Computation

            9.3 Sequential Operation of Turing Machines

            9.4 Composition of Functions

            9.5 Uncomputable Functions

            9.6 Toward a Programming Language

Chapter 10: The Chomsky Hierarchy

            10.1 Unrestricted Grammars

            10.2 Context-Sensitive Grammars

            10.3 Linear-Bounded Automata

            10.4 The Chomsky Hierarchy

Chapter 11: Decision Problems and the Church-Turing Thesis

            1.1 Representation of Decision Problems

            11.2 Decision Problems and Recursive Languages

            11.3 Problem Reduction

            11.4 The Church-Turing Thesis

            11.5 A Universal Turing Machine

Chapter 12: Undecidability

            12.1 The Halting Problem for Turing Machines

            12.2 Problem Reduction and Undecidability

            12.3 Additional Halting Problem Reductions

            12.4 Rice’s Theorem

            12.5 An Unsolvable Word Problem

            12.6 The Post Correspondence Problem

            12.7 Undecidable Problems in Context-Free Grammars

Chapter 13: Mu-Recursive Functions

            13.1 Primitive Recursive Functions

            13.2 Some Primitive Recursive Functions

            13.3 Bounded Operators

            13.4 Division Functions

            13.5 Gödel Numbering and Course-of-Values Recursion

            13.6 Computable Partial Functions

            13.7 Turing Computability and Mu-Recursive Functions

            13.8 The Church-Turing Thesis Revisited


Part IV: Computational Complexity

Chapter 14: Time Complexity

            14.1 Measurement of Complexity

            14.2 Rates of Growth

            14.3 Time Complexity of a Turing Machine

            14.4 Complexity and Turing Machine Variations

            14.5 Linear Speedup

            14.6 Properties of Time Complexity of Languages

            14.7 Simulation of Computer Computations

Chapter 15: Tractability and Decision Problems

            15.1 Time Complexity of Nondeterministic Turing Machines

            15.2 The Classes P and NP

            15.3 Complexity of Decision Problems

            15.4 Decision Problems and Complexity Classes

            15.5 The Hamiltonian Circuit Problem

            15.6 Polynomial-Time Reduction

            15.7 P = NP?

            15.8 The Satisfiability Problem

            15.9 Complexity Class Relations

Chapter 16: NP-Complete Problems

            16.1 Reduction and NP-Complete Problems

            16.2 The 3-Satisfiability Problem         

            16.3 Reductions from 3-Satisfiability

            16.4 Reduction and Subproblems

            16.5 Optimization Problems

            16.6 Approximation Algorithms

            16.7 Approximation Schemes

Chapter 17: Additional Complexity Classes

            17.1 Derivative Complexity Classes

            17.2 Space Complexity

            17.3 Relations between Time and Space Complexity

            17.3 P-Space, NP-Space, and Savitch’s Theorem

            17.4 P-Space Completeness

            17.5 An Intractable Problem


Part VI: Deterministic Parsing

Chapter 18: Parsing: An Introduction

            18.1 The Graph of a Grammar

            18.2 A Top-Down Parser

            18.3 Reductions and Bottom-Up Parsing

            18.4 A Bottom-Up Parser

            18.5 Parsing and Compiling

Chapter 19: LL(k) Grammars

            19.1 Lookahead in Context-Free Grammars

            19.2 FIRST, FOLLOW, and Lookahead Sets

            19.3 Strong LL(k) Grammars

            19.4 Construction of FIRSTk Sets

            19.5 Construction of FOLLOWk Sets

            19.6 A Strong LL(l) Grammar

            19.7 A Strong LL(k) Parser

            19.8 LL(k) Grammars

Chapter 20: LR(k) Grammars

            20.1 LR(0) Contexts

            20.2 An LR(0) Parser

            20.3 The LR(0) Machine

            20.4 Acceptance by the LR(0) Machine

            20.5 LR(1) Grammars


Appendix I: Index of Notation

Appendix II: The Greek Alphabet

Appendix III: The ASCII Character Set

Appendix IV: Backus-Naur Form Definition of Java