Lies, damn lies and open data -- Reinhart and Rogoff challenged

This is one of several cool XKCD cartoons on
statistics. See http://bit.ly/XGLLju for others.

In 2010, economists Carmen Reinhart and Kenneth Rogoff published "Growth in a Time of Debt." They analyzed national income statistics from 44 nations over a span of as much as 200 years. Their dataset incorporated over 3,700 annual observations.

Their main conclusions were:

• Above 90 percent debt/GDP, median growth rate falls by one percent, and average growth falls considerably more.
• In emerging markets, "When external debt reaches 60 percent of GDP, annual growth declines by about two percent; for higher levels, growth rates are roughly cut in half."
• For the advanced nations as a group, there is no apparent contemporaneous link between inflation and public debt levels, but in emerging markets inflation rises sharply as debt increases.

The study was influential. It has 450 Google Scholar citations and was widely quoted by Paul Ryan and other politicians arguing for austerity.

But, a new paper by University of Massachuets economists Thomas Herndon, Michael Ash and Robert Pollin analyzes the same data and comes to different conclusions.

This does not mean Reinhart and Rogoff are charlatans or Ryan and other politicians are liars. Ryan (hapilly) accepted Reinhart and Rogoff's conclusions as fact and Reinhart and Rogoff built a number of questionable assumptions into the spreadsheet they used to analyze their data. They were also careless, evidently not catching a spreadsheet coding error.

But, there are many ways to analyze any multivariate dataset. Data analysis is like the proverbial blind men describing an elephant while touching different parts. A study like this does not give The Answer, it gives an answer.

The best way I can think of to cope with such complexity is to let many people analyze the same data. As the open source software aphorism states "given enough eyeballs, all bugs are shallow."

While I may not agree with the assumptions they made in their analysis, I applaud Reinhart and Rogoff for publishing their data and spreadsheet. You can download them and do your own analysis.

-----

Updated, 4/21/2013

Paul Krugman has weighed in on the Reinhart and Rogoff affair, commenting on its practical political and economic influence in a column on "The Excel Depression" (http://nyti.ms/14IrRd5). I typically agree with what Krugman writes, including this column, but its title is a bit over the top.

-----

Updated 5/31/2013

Reinhart and Rogoff have come in for more criticism -- does debt cause slow growth or vice versa? See details of the data analysis in this paper by University of Massachusetts economist Arindrajit Dube.

One might also compare recent performance of the Japanese and pro-austerity European economies.

Chrome benchmark: math is slow, but will improve -- bad news for Microsoft

Several people ran benchmarks when Google Chrome came out, and concluded that it was faster than Internet Explorer and Firefox. But others, for example this article on 10 things to love (and hate) about Google Chrome, found Chrome to be no faster. Relative speed depends upon the benchmark.

I created a Google Spreadsheet that computes 200 sines, cosines, squares and square roots. Recalculation was noticeably slower with Chrome (23 seconds) than Firefox (2 seconds) or Internet Explorer (10 seconds).

The bad news for Google is that they will have to improve the efficiency of their math libraries. The good news is that Firefox has demonstrated that it can be done. More good news -- the user will not have to install new software when it is ready. It will automatically download when the application is used.

The bad news for Microsoft is that they too will have to improve the speed of math calculations in Internet Explorer. That will not be difficult for them to do, but doing so will have a bad-news side effect -- it will narrow the speed gap between Excel and Google Spreadsheet.

This is just one example. Microsoft will be forced to improve Internet Explorer to keep up with Google, Firefox and others. But every time they improve Internet Explorer, they chip away at the advantage their dedicated Office applications have over the network applications of Google and others. The performance gap between desktop and network-based programs narrows.

Can you think of other new technologies that painted an old technology into a corner? If you were running Microsoft, how would you deal with this situation?