Home -> Law Blog Directory -> Legal Commentary Blogs -> Agoraphilia

Legal Commentary

: Agoraphilia

Musing about Correlations

By Tom W. Bell and Glen Whitman

In the course I'm currently teaching, the exams usually consist of two sections: multiple choice and short answer. I just finished recording the midterm grades, and for curiosity's sake I calculated the correlation between the two sections. For one class, the correlation between multiple choice and short answer was 0.59; for the other class, it was 0.70.

The question is, what correlation should I hope to get if the test has been optimally designed? Two things occur to me:

1. I don't want a correlation of zero, because I expect smarter and better prepared students to well on both parts (and dimmer and less prepared students to do poorly on both parts). A correlation of zero would raise the worry that my test doesn't really measure ability and preparation.

2. I don't want a correlation of one, because that means the sections are effectively redundant. Multiple choice is much easier to grade, so why bother grading the short answers if the multiple choice section tells me everything I need to know? If the two sections measure different things, correlation should be less than perfect.

So the correlation should be in between zero and one. But that's a pretty wide range. How much correlation should I aim for? Is there even a right answer to the question?

Full post as published by Agoraphilia on March 27, 2008 (boomark / email).