Lies, damn lies and open data -- Reinhart and Rogoff challenged

This is one of several cool XKCD cartoons on
statistics. See http://bit.ly/XGLLju for others.

In 2010, economists Carmen Reinhart and Kenneth Rogoff published "Growth in a Time of Debt." They analyzed national income statistics from 44 nations over a span of as much as 200 years. Their dataset incorporated over 3,700 annual observations.

Their main conclusions were:

• Above 90 percent debt/GDP, median growth rate falls by one percent, and average growth falls considerably more.
• In emerging markets, "When external debt reaches 60 percent of GDP, annual growth declines by about two percent; for higher levels, growth rates are roughly cut in half."
• For the advanced nations as a group, there is no apparent contemporaneous link between inflation and public debt levels, but in emerging markets inflation rises sharply as debt increases.

The study was influential. It has 450 Google Scholar citations and was widely quoted by Paul Ryan and other politicians arguing for austerity.

But, a new paper by University of Massachuets economists Thomas Herndon, Michael Ash and Robert Pollin analyzes the same data and comes to different conclusions.

This does not mean Reinhart and Rogoff are charlatans or Ryan and other politicians are liars. Ryan (hapilly) accepted Reinhart and Rogoff's conclusions as fact and Reinhart and Rogoff built a number of questionable assumptions into the spreadsheet they used to analyze their data. They were also careless, evidently not catching a spreadsheet coding error.

But, there are many ways to analyze any multivariate dataset. Data analysis is like the proverbial blind men describing an elephant while touching different parts. A study like this does not give The Answer, it gives an answer.

The best way I can think of to cope with such complexity is to let many people analyze the same data. As the open source software aphorism states "given enough eyeballs, all bugs are shallow."

While I may not agree with the assumptions they made in their analysis, I applaud Reinhart and Rogoff for publishing their data and spreadsheet. You can download them and do your own analysis.

-----

Updated, 4/21/2013

Paul Krugman has weighed in on the Reinhart and Rogoff affair, commenting on its practical political and economic influence in a column on "The Excel Depression" (http://nyti.ms/14IrRd5). I typically agree with what Krugman writes, including this column, but its title is a bit over the top.

-----

Updated 5/31/2013

Reinhart and Rogoff have come in for more criticism -- does debt cause slow growth or vice versa? See details of the data analysis in this paper by University of Massachusetts economist Arindrajit Dube.

One might also compare recent performance of the Japanese and pro-austerity European economies.