TLDR: No! Your Machine Translation Model is not “prophesying”, but let’s look at the six major issues with neural machine translation (NMT). So I saw a Twitter thread today with the editor-in-chief of Motherboard tweeting, “Google Translate is popping out bizarre religious texts and no one is sure why“. This post in response to that. I […]
Archive | Natural Language Processing
Design Patterns for Production NLP Systems
This post is an excerpt from the final chapter of our upcoming book on Deep Learning and NLP with PyTorch. The book is still a draft under review so your comments on this section are appreciated! Production NLP systems can be complex. When building an NLP system, it is important to remember that the system […]
Differentiable Dynamic Programs and SparseMAP Inference
Two exciting NLP papers at ICML 2018! ICML 2018 accepts are out, and I am excited about two papers that I will briefly outline here. I think both papers are phenomenally good and will bring back structured prediction in NLP to modern deep learning architectures. Differentiable Dynamic Programming for Structured Prediction and Attention Arthur Mensch […]
When (not) to use Deep Learning for NLP
We are preparing for the second edition of our PyTorch-based Deep Learning for NLP training. It’s a two-day affair, crammed with a lot of learning and hands-on model building where we get to play the intricate dance of introducing the topics from the ground up while still making sure folks are not far from the […]
The Two Tribes of Language Researchers
TL;DR not-a-rant rant When I talk to friends who work on human language (#nlproc), I notice two tribes of people. These are folks who do Natural Language Processing and folks who do Computational Linguistics. This distinction is not mine and is blurry, but I think it explains some of the differences in values different researchers […]
A Billion Words and The Limits of Language Modeling
In this post, I will talk about Language Models, when (and when not) to use LSTMs for language modeling, and some state of the art results. While I mostly discuss the “Exploring Limits” paper, I’m adding a few things elementary (for some) here for completeness sake. The Exploring Limits paper is not new, but I think it’s a good illustration […]