Top Books on Natural Language Processing

 

Natural Language Processing, or NLP for short, is the study of computational methods for working with speech and text data.

The field is dominated by the statistical paradigm and machine learning methods are used for developing predictive models.

In this post, you will discover the top books that you can read to get started with natural language processing.

After reading this post, you will know:

• The top books for practical natural language processing.
• The top textbooks for the theoretical foundations of natural language processing.
• The NLP books I have on my shelf.

Kick-start your project with my new book Deep Learning for Natural Language Processing, including step-by-step tutorials and the Python source code files for all examples.

Let’s get started.

Top Practical Books on Natural Language Processing

As practitioners, we do not always have to grab for a textbook when getting started on a new topic.

Code examples in the book are in the Python programming language.

Although there are fewer practical books on NLP than textbooks, I have tried to pick the top 3 books that will help you get started and bring NLP method to your machine learning project.

1. Natural Language Processing with Python

Written by Steven Bird, Ewan Klein and Edward Loper.

Natural Language Processing with Python

This book provides an introduction to NLP using the Python stack for practitioners.

The book focuses on using the NLTK Python library, which is very popular for common NLP tasks.

Contents include:

1. Language Processing and Python
2. Accessing Text Corpora and Lexical Resources
3. Processing Raw Text
4. Writing Structured Programs
5. Categorizing and Tagging Words
6. Learning to Classify Text
7. Extracting Information from Text
8. Analyzing Sentence Structure
9. Building Feature-Based GRammars
10. Analyzing the Meaning of Sentences
11. Managing Linguistic Data

This book is perfect if you are looking at getting into classical NLP using the go-to NLTK platform.

Resources

• Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit
• Natural Language Processing with Python (free version)

2. Taming Text

This book provides an introduction to a suite of different NLP tools and problems, such as Apache Solr, Apache OpenNLP, and Apache Mahout.

Taming Text

Code examples are in Java.

It may be more suited to developers getting started with larger enterprise-grade NLP tools on work projects.

Written by Grant Ingersoll, Thomas Morton and Drew Farris.

Notably, Grant Ingersoll is a cofounder of the Apache Mahout project.

Contents include:

1. Getting Started Taming Text
2. Foundations of Taming Text
3. Searching
4. Fuzzy String Matching
5. Identifying People, Places and Things
6. Clustering Text
7. Classification, Categorization and Tagging
8. Building an Example Question Answering System
9. Untaming Text: Exploring the Next Frontier

Resources

• Taming Text: How to Find, Organize, and Manipulate It
• Book homepage
• Book GitHub Repository (code and data)

3. Text Mining with R

Written by Julia Silge and David Robinson.

Text Mining with R

This book demonstrates statistical natural language processing methods on a range of modern applications.

Code examples are in R.

Code focuses on the “tidy” principles by Hadley Wickham (paper) and the tidytext package by the authors.

Of the three books, this is the most recently published and has a more practical and modern feel to the demonstrations.

Contents include:

1. The Tidy Text Format
2. Sentiment Analysis with Tidy Data
3. Analyzing word and Document Frequency: tf-idf
4. Relationships Between Words: N-grams and Correlations
5. Converting to and from Nontidy Formats
6. Topic Modeling
7. Case Study: Comparing Twitter Archives
8. Case Study: Mining NASA Metadata
9. Case Study: Analyzing Usenet Text

Resources

• Text Mining with R: A Tidy Approach
• Book Homepage (and book for free)
• Book GitHub Repository (code and data)

Do you know of other great practical books on natural language processing?
Let me know in the comments.

Top Textbooks on Natural Language Processing

There are a ton of textbooks on natural language processing and on specific sub-topics.

In this section, I have tried to focus on what I (and consensus) seems to see as the best books on the topic for beginners, e.g. undergraduate or graduate students and practitioners looking to step deeper into the theory.

I have tried to pick a mix of general NLP books as well as books on highly studied topics like translation and speech.

The first two books in this section are essentially cannon for NLP students.

1. Foundations of Statistical Natural Language Processing

Written by Christopher Manning and Hinrich Schütze.

Foundations of Statistical Natural Language Processing

Notably, Christopher Manning teaches NLP at Stanford and is behind the CS224n: Natural Language Processing with Deep Learning course.

This book provides an introduction to statistical methods for natural language processing covering both the required linguistics and the newer (at the time, circa 1999) statistical methods.

This book provides a strong foundation to better grasp the newer methods and encodings.

Contents include:

1. Introduction
2. Mathematical Foundations
3. Linguistic Essentials
4. Corpus-Based Work
5. Collocations
6. Statistical Inference: n-gram Models over Sparse Data
7. Word Sense Disambiguation
8. Lexical Acquisition
9. Markov Models
10. Part-of-Speech Tagging
11. Probabilistic Context Free Grammars
12. Probabilistic Parsing
13. Statistical Alignment and Machine Translation
14. Clustering
15. Topics in Information Retrieval
16. Text Categorization

Resources

• Foundations of Statistical Natural Language Processing
• Book Homepage

2. Speech and Language Processing

Written by Daniel Jurafsky and James Martin.

Speech and Language Processing

This book provides coverage of NLP from both speech and text perspectives with a strong focus on applications (one in each chapter).

Coverage of the topic feels exhaustive.

Contents include:

1. Introduction
2. Regular Expressions and Automata
3. Words and Transducers
4. N-grams
5. Part-of-Speech Tagging
6. Hidden Markov and Maximum Entropy Models
7. Phonetics
8. Speech Synthesis
9. Automatic Speech Recognition
10. Speech Recognition: Advanced Topics
11. Computational Phonology
12. Formal Grammars of English
13. Syntactic Parsing
14. Statistical Parsing
15. Features and Unification
16. Language and Complexity
17. The Representation of Meaning
18. Computational Semantics
19. Lexical Semantics
20. Computational Lexical Semantics
21. Computational Discourse
22. Information Extraction
23. Question Answering and Summarization
24. Dialog and Conversational Agents
25. Machine Translation

Resources

• Speech and Language Processing
• Book Homepage

4. Statistical Machine Translation

Written by Philipp Koehn.

Statistical Machine Translation

This book provides an introduction to the topic of statistical machine translation, a s subfield of NLP.

Contents include:

1. Introduction
2. Words, Sentences, Corpa
3. Probability Theory
4. Word-Based Models
5. Phrase-Based Models
6. Decoding
7. Language Models
8. Evaluation
9. Discriminative Training
10. Integrating Linguistic Information
11. Tree-Based Methods

Resources

• Statistical Machine Translation
• Book Homepage

5. Statistical Methods for Speech Recognition

Written by Frederick Jelinek.

Statistical Methods for Speech Recognition

This book provides an introduction to the topic of statistical speech recognition, another subfield of NLP that saw an overhaul in the 1990s with statistical approaches.

Contents Include

1. The Speech Recognition Problem
2. Hidden Markov Models
3. The Acoustic Model
4. Basic Language Modeling
5. The Viterbi Search
6. Hypothesis Search on a Tree and the Fast Match
7. Elements of Information Theory
8. The Complexity of Tasks – The Quality of Language Models
9. The Expectation-Maximization Algorithm and Its Consequences
10. Decision Trees and Tree Language Models
11. Phonetics from Orthography: Spelling-to-Base Form Mappings
12. Triphones and Allophones
13. Maximum Entropy Probability Estimation and Language Models
14. Tree Applications of Maximum Entropy Estimation to Language Modeling
15. Estimation of Probabilities from Counts and the Back-Off Method

Resources

• Statistical Methods for Speech Recognition

NLP Books that I Own

I like to have a mixture of practical and reference texts on my shelf.

The hard part of NLP (for me) is simply the large number of sub-problems and the specialized terminology and theory used.

For this reason I have the following 3 NLP textbooks on my shelf:

• Natural Language Processing with Python
• Foundations of Statistical Natural Language Processing
• Neural Network Methods in Natural Language Processing

I also really like the look of:

• The Oxford Handbook of Computational Linguistics

I recommend choosing the NLP books that are right for you and your needs or project.

Let me know which books you chose or own.
Leave a comment below.

Further Reading

This section provides more resources on the topic if you are looking go deeper.

Top NLP Books

• Best Sellers in Natural Language Processing on Amazon
• Popular Natural Language Processing Books on GoodReads

Quora

• What are some books that people interested in NLP must read?
• What are the best books on NLP?
• What is the best Natural Language Processing textbook(s)?
• Best NLP books

Summary

In this post, you discovered the top books on natural language processing.

Specifically, you learned:

• The top books for practical natural language processing.
• The top textbooks for the theoretical foundations of natural language processing
• The NLP books I have on my shelf.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.