On Reading Papers
One of the most important skill I learnt in grad school was to break down any complex paper into bite sized parts, digest them, and ask critical questions about them. On the face of it, it sounds like an obvious skill, right? Just like knowing how to talk doesn't make you a good public speaker, knowing how to read doesn't automatically make you prepared to read (and absorb) papers. You have to train it, like training a muscle. Typical machine learning papers have a fair bit of math in them (unless you're reading deep learning papers :-p). Reading a math paper, or a math-heavy paper, will require additional skills. For example, reading in a linear fashion, and focusing on the nitty gritties at first is the worst thing to do when you confront a math-heavy paper. You might want to ask:
- What are the major claims/results advertised?
- What are the assumptions around those claims?
- What are the (hidden) constants?
- If there is an equation, how does it behave when you fiddle with the exponents and the other parameters.
- Can you do a quick dimensional analysis to see if this makes sense? (very useful for linear algebra heavy papers)
But none of the existing resources capture the essence of what I try to achieve in my reading and what I want my students to be able to do. It’s a kind of Thurstonian ideal of understanding: where the proof falls away and what you’re left with is a deep conceptual appreciation of how the parts of a proof (or a theorem, or a set of definitions) fit together, and what they really mean.
In today's fast-moving data science world, knowing how to read and digest technical literature is vital survival skill. Reading papers critically will allow you to come up with better implementations than some random appropriated code from github. I'm truly excited about Suresh's new course. Follow his blog!