How to Read a Research Paper: The Three-Pass Method
Reading a paper from the first line to the last is slow and easy to lose the point in. S. Keshav's three-pass method reads it three times with a different goal each time: a 5–10 minute bird's-eye view, up to an hour on the content, and a deep third pass only for the papers that matter. What to do in each pass, the five Cs, how to use the passes for a literature survey, and a 3-minute video that shows them on a tablet.
Most of us read a research paper the way we read a book: from the first line to the last. It is slow, and it is the easiest way to lose the main point. The method that works is to read the paper three times, each pass with a different goal. The first pass takes five to ten minutes and tells you whether to go on; the second takes up to an hour and gives you the content; the third takes hours and is only for the papers you have to know deeply. It comes from a two-page note by S. Keshav, "How to Read a Paper" (2007), and it is the single most useful thing a new PhD student can learn about reading.
Here it is in three minutes, shown on a tablet. The text below has the detail.
The three passes in one table
| What you read | How long | What you have at the end | |
|---|---|---|---|
| Pass 1 | Title, abstract, introduction; the section headings; the conclusions; a glance at the references | 5–10 minutes | The five Cs, and a decision: stop here, or read on |
| Pass 2 | The whole paper with care, skipping details such as proofs; the figures closely; notes in the margins; the references worth reading | Up to an hour | The paper's main thrust, with its supporting evidence, well enough to explain to someone else |
| Pass 3 | Everything, while re-creating the work from the authors' own assumptions | 4–5 hours for a beginner, about an hour with experience | The structure from memory, the strong and weak points, and ideas for your own work |
Pass 1: the bird's-eye view
The first pass is a quick scan to get a sense of the paper, and it should take five to ten minutes. Keshav's steps:
- Read the title, the abstract and the introduction carefully.
- Read the section and sub-section headings, and ignore everything else.
- Read the conclusions.
- Glance over the references, ticking off the ones you have already read.
At the end of the first pass you should be able to answer the five Cs:
- Category. What type of paper is this? A measurement paper, an analysis of an existing system, a description of a research prototype?
- Context. Which other papers is it related to? Which theoretical bases were used to analyze the problem?
- Correctness. Do the assumptions appear to be valid?
- Contributions. What are the paper's main contributions?
- Clarity. Is the paper well written?
Then decide. You may choose not to read further, because the paper does not interest you, or you do not know enough about the area to understand it, or the authors make invalid assumptions. Most of the papers you pick up should end here; that is the point of the pass, not a failure of it. Keshav adds a lesson for writers: most reviewers and readers will make exactly one pass, so choose section titles that make sense on their own and write an abstract that stands alone.
Pass 2: the content
In the second pass, read the paper with greater care, but ignore details such as proofs. Jot down the key points, or make comments in the margins, as you read. Two things get specific attention:
- The figures, diagrams and other illustrations. Pay special attention to graphs. Are the axes properly labeled? Are results shown with error bars, so that the conclusions are statistically significant? A paper's evidence is mostly in its figures, and they are rarely next to the text that discusses them, so this is where the time goes.
- The references. Mark the relevant unread references for further reading. This is how you learn more about the background of the paper, and it feeds the literature survey below.
This pass takes up to an hour for an experienced reader. At the end of it you should be able to grasp the content of the paper and summarize its main thrust, with supporting evidence, to someone else. That is the right level of understanding for a paper you are interested in but that is not in your own research specialty.
Sometimes you will not understand a paper at the end of the second pass. The subject may be new to you, the authors may use a proof or an experimental technique you do not know, or the paper may be badly written. Keshav gives three ways out: set the paper aside, hoping you will not need it; return to it later, after reading the background material; or persevere and go on to the third pass.
Pass 3: virtually re-implement the paper
The third pass is for a paper you have to understand fully, such as one closest to your thesis or one you are reviewing. The key is to attempt to virtually re-implement it: make the same assumptions as the authors and re-create the work. By comparing your re-creation with the actual paper you find not only its innovations but its hidden failings and assumptions. The pass asks for great attention to detail: challenge every assumption in every statement, and think about how you yourself would present each idea. That comparison is also where ideas for future work come from, so jot them down as you go.
It takes a beginner four or five hours, and an experienced reader about an hour. At the end you should be able to reconstruct the entire structure of the paper from memory, identify its strong and weak points, and point to implicit assumptions, missing citations and potential problems with its techniques.
Using the three passes for a literature survey
Paper-reading skills are put to the test in a literature survey, where you read tens of papers, often in a field that is new to you. Keshav uses the passes to choose them:
- Use an academic search engine and a few well-chosen keywords to find three to five recent papers in the area. Do one pass on each, then read their related-work sections. If you are lucky, one of them is a survey, and you are done; if not, go on.
- Find the citations those papers share and the author names that repeat. They are the key papers and the key researchers of the area. Download the key papers and set them aside, then look at the key researchers' recent work on their websites.
- Find where the area publishes: go to the websites of the top conferences or journals, look at the recent proceedings, and pick out the high-quality related work.
Those papers, together with the ones you set aside, are the first version of your survey. Make two passes through them, and when one points to a key paper you did not find, get it and iterate.
Where a reader built for papers helps
The method works with a printout and a pencil. On a tablet, a few things make it faster, and they are what the video shows:
- Pass 1 is a scan of headings and the abstract, which is slow on a two-column page you have to pan across. Reading Mode reflows the paper into one column, so the headings line up and the abstract reads at a glance.
- Pass 2 lives in the figures and the margins. Pin the figure beside the text and keep reading; tap a citation to see the reference without losing your place; and your highlights and notes go back to your Zotero or Mendeley library rather than staying on one device.
- Pass 3 is writing: circle the step you doubt and write the question next to it, by hand. When you have written your own summary, compare it with a paper card, which lists the research question, method, data, findings and stated limitations with a page on every sentence, and where you disagree, open the page.
Frequently asked questions
How long does each pass take?
Five to ten minutes for the first pass, up to an hour for the second, and four or five hours for the third pass when you are new to the method, falling to about an hour with experience. The numbers are Keshav's.
Do I have to read every paper three times?
No. The point of the first pass is to decide, and for most papers the decision is to stop. A second pass is for the papers you are interested in; a third pass only for the ones you have to know deeply, such as those closest to your own thesis or a paper you are reviewing.
What are the five Cs?
The five questions you should be able to answer after the first pass: Category (what type of paper is this?), Context (which papers and theoretical bases is it built on?), Correctness (do the assumptions appear valid?), Contributions (what are its main contributions?) and Clarity (is it well written?).
Is the three-pass method only for computer science?
Keshav wrote it for computer science and networking, where it has been handed to new graduate students for almost twenty years, but nothing in it is specific to the field. The structure of a research paper is the same elsewhere: abstract, introduction, method, results, discussion, references. Only the question of what counts as "re-implementing" changes.
What should I write down in the second pass?
The key points, as they come, and comments in the margins: a claim you doubt, a figure whose axes or error bars do not support the headline result, a reference you want to read. Keep the notes with the paper rather than in a separate file, so that they are there in the third pass and in your write-up.
What if I still do not understand the paper after the second pass?
Keshav gives three choices: set it aside and hope you will not need it; come back to it after reading the background material it rests on; or persevere into the third pass. Which one is right depends on how close the paper is to your own work.
Where is the original?
S. Keshav, "How to Read a Paper," ACM SIGCOMM Computer Communication Review, vol. 37, no. 3, July 2007, pp. 83–84, doi:10.1145/1273445.1273458. A free copy is on web.stanford.edu, as a course handout. It is two pages, and worth the ten minutes.