Bailout

Why official statistics are like corporate earnings

By Felix Salmon

Mark Gimein uncovers some statistical legerdemain at the Harlem Village Academies, a pair of middle schools in New York. Let’s see how good you are at middle school math, by seeing whether you can answer this question correctly.

A middle school runs from 5th grade to 8th grade. In the 2006-7 year, it had 66 students in its fifth-grade class. In the 2009-10 year, 100% of its eighth-graders were proficient in math. If all the eighth-graders who passed their math test were members of the fifth grade in 2006-7, how many eighth-graders passed their math test this year?

If you said 66, you get an F. If you said “I don’t know, probably a few of those fifth-graders moved away, maybe 60?” you get a D. If you said 19, you get an A.

Of course, the public announcements from the Academies are all about that 100% math-proficiency rate, and make no mention of the fact that clearly you’re only allowed to even enter eighth grade in the first place if you’re going to pass that test.

Gimein puts his finger on the broader syndrome:

Steve Koss, a former math teacher at Manhattan?s Lab School, paints the city?s relationship with stats as an example of Campbell?s Law at work. Named for sociologist Donald T. Campbell, the precept holds, essentially, that the more that numbers are used for political purposes, the more they will be manipulated?and distort the decisions they were supposed to inform.

So whenever you see a politician citing some datapoint you never much heard about before, be very suspicious. And the same goes double when money’s on the line: if inflation-linked or GDP-linked bonds ever become a large part of a government’s liabilities, expect shenanigans surrounding the official inflation and GDP figures. Statistics can be useful, but only if nobody much cares about them. In that sense, they’re a bit like headline corporate earnings.

Full post as published by Macroprudential on August 30, 2010 (boomark / email).