Academic

How not to design survey questions

By Tung Yin

For some reason, my wife got an official looking piece of mail the other day, titled "2010 Congressional District Survey -- OR 3rd" . . . from the Republican National Committee, of which she is not a member.  The survey started off with some reasonably plausible questions designed to highlight differences between the Republicans and Democrats.  Here are a couple; all responses are Yes/No/Undecided:

Do you support the Obama Administration's efforts to eliminate further testing and deployment of an intercontinental missile defense system?

Okay, it's a plausible difference, but as I understand it, missile defense systems are far beyond our engineering ability at the present, so the question might be a little deceptive.  Still, I suppose you could say that we'll never succeed if we don't try, so at least it's a fair difference between the parties.

Next question:

Should Republicans fight congressional Democrats' efforts to grant full unconditional amnesty to illegal immigrants?

Apart from whether it really is "full unconditional amnesty," this also seems to point out a legitimate policy difference.

However, we soon verge from seemingly plausible questions into a bizarro land of wildly distorted and hyperbolic questions.  Here are a few examples:

Do you support the Democrats' efforts to create a massive new federal government bureaucracy that would be run by unionized government employees and would have complete control of your healthcare costs and choices?

Do you believe Barack Obama and Democrats in Congress have the best interests of you, you family and your community in mind?

Do you support Barack Obama's plan to get Medicare to fund a new bloated entitlement program?

Do you agree that President Obama and Democrats in Congress seem more concerned about passing their liberal pet-programs than creating jobs and getting the economy going?

Well, who's in favor of bloated entitlements and massive bureaucracies?  But of course, the real issue is whether these characterizations are caricatures. . . .  This survey may generate responses favorable for the Republican agenda, but that's not hard to do if your questions assume the very worst view of the opposing side's stances.

Full post as published by The Yin Blog on April 23, 2010 (boomark / email).