If you were part of the machine learning Twitter, last Thursday, it was impossible to miss OpenAI’s press release of their new GPT-2 model and all the heated Twitter conversation about that. Many people tried to summarize it in their own ways. A lot of these posts are of “He said, She said” flavor. While […]
Archive | Natural Language Processing
The Real Problems with Neural Machine Translation
TLDR: No! Your Machine Translation Model is not “prophesying”, but let’s look at the six major issues with neural machine translation (NMT). So I saw a Twitter thread today with the editor-in-chief of Motherboard tweeting, “Google Translate is popping out bizarre religious texts and no one is sure why“. This post in response to that. I […]
Design Patterns for Production NLP Systems
This post is an excerpt from the final chapter of our upcoming book on Deep Learning and NLP with PyTorch. The book is still a draft under review so your comments on this section are appreciated! Production NLP systems can be complex. When building an NLP system, it is important to remember that the system […]
Differentiable Dynamic Programs and SparseMAP Inference
Two exciting NLP papers at ICML 2018! ICML 2018 accepts are out, and I am excited about two papers that I will briefly outline here. I think both papers are phenomenally good and will bring back structured prediction in NLP to modern deep learning architectures. Differentiable Dynamic Programming for Structured Prediction and Attention Arthur Mensch […]
When (not) to use Deep Learning for NLP
We are preparing for the second edition of our PyTorch-based Deep Learning for NLP training. It’s a two-day affair, crammed with a lot of learning and hands-on model building where we get to play the intricate dance of introducing the topics from the ground up while still making sure folks are not far from the […]
The Two Tribes of Language Researchers
TL;DR not-a-rant rant When I talk to friends who work on human language (#nlproc), I notice two tribes of people. These are folks who do Natural Language Processing and folks who do Computational Linguistics. This distinction is not mine and is blurry, but I think it explains some of the differences in values different researchers […]
A Billion Words and The Limits of Language Modeling
In this post, I will talk about Language Models, when (and when not) to use LSTMs for language modeling, and some state of the art results. While I mostly discuss the “Exploring Limits” paper, I’m adding a few things elementary (for some) here for completeness sake. The Exploring Limits paper is not new, but I think it’s a good illustration […]