TLDR: This is a weird post; I am making an entry from my personal journal public. So, it’s likely useless and worth skipping, esp. if you are not interested in writing. This is also not advice on how to write (I’m in no place to give that), but an exploration of writing as a noun […]
Unstoppable AI Flywheels and the Making of the New Goliaths
TL;DR: AI creates engines for relentless optimization at all levels. Read the article to figure out how and its consequences to what you’re doing. Some time ago I wrote about how everything is a model when reviewing a paper from Kraska et al (2017) where they show how traditional CS data structures like B-Tree indexes, […]
The Twelve Truths of Machine Learning for the Real World
TLDR: Scroll and read the headings in bold Last month I gave an informal talk to an intimate gathering of friends with this title that I am putting down in words. This post is mainly for people who are using machine learning to build something as opposed to people who are working on machine learning […]
Fragility of Everything
The day before heading to Neurips, I packed my luggage as usual at the last minute before going to bed (early am flight). As I was packing, I received a call from India. It was my mom. My dad had developed “breathing issues”, which we later found out was a combination of congestive heart failure, […]
The Doers and The Clarion Callers
There is an unnecessary drama unfolding on Twitter on “the war” between connectionists and symbolists. This drama is absurd and perpetuated by people whose relevance exists only in the discussion of the “difference” and the “war.” Most of us who are too busy learning or doing, happily occupy the liminal space between symbolism and connectionism. […]
Joining AI Foundation
TL;DR: I am joining AI Foundation to help build uncanny content generation and detection systems A few friends and family knew this, but today, I am excited to announce widely that I joined the AI Foundation as their VP of Research. I am overseeing their deep learning/machine learning research roadmaps in speech, language, and vision for […]
When OpenAI tried to build more than a Language Model
If you were part of the machine learning Twitter, last Thursday, it was impossible to miss OpenAI’s press release of their new GPT-2 model and all the heated Twitter conversation about that. Many people tried to summarize it in their own ways. A lot of these posts are of “He said, She said” flavor. While […]
The Real Problems with Neural Machine Translation
TLDR: No! Your Machine Translation Model is not “prophesying”, but let’s look at the six major issues with neural machine translation (NMT). So I saw a Twitter thread today with the editor-in-chief of Motherboard tweeting, “Google Translate is popping out bizarre religious texts and no one is sure why“. This post in response to that. I […]
Design Patterns for Production NLP Systems
This post is an excerpt from the final chapter of our upcoming book on Deep Learning and NLP with PyTorch. The book is still a draft under review so your comments on this section are appreciated! Production NLP systems can be complex. When building an NLP system, it is important to remember that the system […]
Differentiable Dynamic Programs and SparseMAP Inference
Two exciting NLP papers at ICML 2018! ICML 2018 accepts are out, and I am excited about two papers that I will briefly outline here. I think both papers are phenomenally good and will bring back structured prediction in NLP to modern deep learning architectures. Differentiable Dynamic Programming for Structured Prediction and Attention Arthur Mensch […]