Practical Natural Language Processing: A must-read for anyone who wants to become seriously involved in NLP

So you like our media brand Neural? You should join our Neural event track at TNW2020, where you’ll hear how artificial intelligence is transforming industries and businesses. 

By many accounts, linguistics is one of the most complicated functions of the human mind. Likewise, natural language processing (NLP) is one of the most complicated subfields of artificial intelligence. Most books on AI, including educational books on machine learning, provide an introduction to natural language processing. But the field of NLP is so vast that covering all its aspects would require several separate books.

When I picked up Practical Natural Language Processing: A Comprehensive Guide to Building Real-World NLP Systems, what I expected was a book that covered Python machine learning for NLP in depth. Though the book didn’t exactly turn out to be what I had in mind, it provided the exact kind of coverage that the field misses in the craze and hype that surrounds deep learning today.

The best way to describe Practical Natural Language Processing is a zoomed-out view of the NLP landscape, a close-up of the NLP process, and plenty of practical tips and guidelines to avoid making mistakes in one of the most important fields of AI.

Two types of audience

What you take away from Practical Natural Language Processing depends on two things: Your previous background in mathematics and Python machine learning, and your involvement in the field. I recommend this book to two types of readers:

• Seasoned Python machine learning experts: People who already have experience coding with machine learning and deep learning libraries in Python will find plenty of new directions to follow in this book.
• Decision-makers who want to learn more about the mechanics of NLP systems: Executives and product managers who are building solutions that use conversational interfaces and other language features (search, sentiment analysis, etc.) must understand the NLP landscape and pipeline. The insights Practical Natural Language Processing provides will help guide technology decisions when building new products and making upgrades to old solutions.

Ironically, these two audiences are at two ends of the NLP spectrum. On the one hand, you have hardcore Python machine learning coders while on the other, you have people whose daily routine doesn’t involve coding.

But the authors of Practical Natural Language Processing, who have extensive experience implementing NLP in different fields, have managed to write a book that will provide value to both audiences. Veteran coders can go through the entire book, the accompanying Jupyter Notebooks, and the many references the authors provide. Executives, on the other hand, can skip over the code and read the high-level overview that each chapter provides before digging into the technical Python coding.

I would not recommend this book to novice Python machine learning coders. If you don’t have experience with numpy, pandas, matplotlib, scikit-learn, tensorflow, and keras libraries, then this book probably isn’t for you. I suggest you go through a book or course on data science and another on Python machine learning before picking up Practical Natural Language Processing.

The NLP pipeline

Anyone who has done machine learning knows that the development cycle of ML applications is different from the classic, rule-based software development lifecycle. But many people mistakenly think that the NLP development pipeline is identical to the data gathering, modeling, testing cycle of any machine learning application. There are some similarities, but there are also many nuances that are specific to NLP.

Some of the most valuable parts of Practical Natural Language Processing are the overview of the NLP development pipeline for different applications. The book brilliantly gives a high-level view of natural language processing that is detached from machine learning and deep learning.

You’ll get to know a lot of the challenges involved in gathering, cleaning, and preparing data for NLP applications. You’ll also learn about important NLP disciplines such as information extraction, name-entity recognition, temporal information extraction, and more.

One of the key challenges of NLP is that data tends to be very application-specific. Recent advances in deep learning have enabled the development of very large language models that can adapt to different tasks without further tuning. But in a lot of applications and settings, the use of expensive deep learning language models is still not feasible.

Practical Natural Language Processing shows how you can start with small and simple NLP applications and gradually scale them and transition to more complex and automated AI models as you gather more data on your problem domain.

As you go deeper into the book, you’ll get to know the specific development cycle for different NLP applications such as chatbots, search engines, text summarization, recommender systems, machine translation, ecommerce, and more.

At the end of the book, you’ll get a review of the end-to-end NLP process and some of the key questions that can determine which path and technology to choose when starting a new application. There are also important guidelines on storing and deploying models and problems you’ll need to solve in real-world NLP applications, such as reproducing results, which is a great challenge in machine learning in general.

These high-level discussions make Practical Natural Language Processing a valuable read to both developers, team leaders, and executives at tech companies.

But the book also contains a lot of coding examples, which I’ll get to next.

You don’t need machine learning for everything

Although a large part of Practical Natural Language Processing is about using Python machine learning libraries for language tasks, the book has much more to offer. Interestingly, the most important takeaway is that you don’t need machine learning for every single task. In fact, large deep learning­–based language models should be your last resort in most cases. This is a recurring theme across the book and—in my opinion—a very important one, given all the excitement surrounding the use of larger and larger neural networks for NLP tasks.

As the authors will show you, in many cases, simple solutions such as regular expressions will provide better results. At the very least, the simpler rule-based systems will provide you with an interim solution to gather the data required for the more complex AI models.

With that out of the way, the book does go deep on Python machine learning and deep learning for natural language processing. Practical Natural Language Processing provides in-depth coverage of many critical concepts such as word embeddings, bag of words, ngrams, and TF-IDF.

Aside from the popular machine learning and NLP libraries, the book also introduces and explore Python NLP libraries that basic machine learning books don’t cover, such as spacy and genism. There’s also a good deal on using other Python tools to better assess the performance of language models, such as t-SNE for visualizing embeddings and LIME for dealing with AI explainability issues.

As you go into the details of each technique and its associated libraries, the authors continue to provide some key tips, such as how to decide between general-purpose embeddings and hand-crafted features.

The book also introduces you to Google’s BERT (but you have to bring your own pytorch skills).

I had mixed feelings about the code samples and how they’re presented in the book. On the one hand, there are plenty of valuable material in the Jupyter Notebooks. However, the code tends to get a bit buggy due to bad file addresses in some of the notebooks. (You also need to spend a great deal of time downloading embedding and models from other sources, which is inevitable anyway.)

In the book, snippets are presented very scantly with only the very important parts highlighted. But in many cases, those parts have been selected from the middle of a sequence, which means you won’t make sense of it unless you also open the original code file and read what has come before (to their credit, the authors have taken great care to comment the code comprehensively.)

There’s also a lot of “we leave it as an exercise for the reader” holes, which give some great directions for personal research but would not be appealing to less experienced developers.

Don’t reinvent the wheel

Another great thing about Practical Natural Language Processing is the introduction of a roster of tools and application programming interfaces (API) that let you get started with language tasks without much coding.

The point is, you don’t need to reinvent the wheel, and there’s a likely chance that there’s already a tool out there that can boost your initial efforts to integrate NLP into your applications.

Throughout the book, you’ll get to use tools such as DialogFlow, Elasticsearch, and Semantic3 for different NLP applications. You’ll also see how APIs such as Bing can abstract language tasks (if you have the financial means to rent them).

You’ll also get an idea of how these different pieces can be integrated into other NLP applications or gradually transitioned to your own custom-made language models.

Familiarity with these tools will also be very useful for product managers who must decide on which direction to take with the development of their applications given their time, budget, and talent constraints.

Final verdict

Practical Natural Language Processing is a must-read for anyone who wants to become seriously involved in NLP. Whether you’re a c-level executive or a hands-on coder, this book is for you.

However, don’t pick up this book if you just want to learn the basics of NLP. There are plenty of good Python machine learning books and courses that will introduce you to the basics. Once you feel comfortable with Python machine learning and the basics of natural language processing, this will be your go-to book for taking the next step.

This article was originally published by Ben Dickson on TechTalks, a publication that examines trends in technology, how they affect the way we live and do business, and the problems they solve. But we also discuss the evil side of technology, the darker implications of new tech and what we need to look out for. You can read the original article here.