Law Technology

Rankings for the Rest - Measuring Teaching Impact

By JohnPMayer

Professor Leiter is conducting a study of law faculty scholarly productivity (methodology here). The goal (or one goal) is...

"...what is the most effective and efficient way to measure the scholarly impact of a law faculty..."

What if you replace the word "scholarly" with "teaching"?

If you have seen this video...

...then you can imagine how hard it is to measure teaching impact in the same way as Professor Leiter is measuring scholarly impact but perhaps there are some objective numbers that can be discovered and measured.

• number of classes taught
• number of students in those classes
• number of credit hours those classes are worth

These can be culled from the course catalog and a the class sizes by survey.

• casebooks published (first time)
• casebooks updated
• supplemental materials published

Theoretically, we could find this information out from the commercial publisher websites and matching up authors and co-authors with the publication dates to get an average per year.

Harder to get would be...

• adoption rates for casebooks
• sales figures for casebooks and supplemental materials
• whether supplemental materials are required, suggested or recommended.

Some of this could be culled from the course syllabi - many of which are online. There is a certain amount of "everyone knows" regarding what are the most popular casebooks in the major subject areas, but the long tail is harder to measure.

Having a casebook in the marketplace will generate a certain amount of traffic for the authors who must "teach the teachers" as well. This illustrates the point that teachers teach students and teachers teach teachers. Should we measure both? It would seem that teaching teachers has more downstream impact than teaching students. The places that teachers teach teachers include presentations given relating to teaching at ...

• AALS
• Conferences
• Workshops

What about non-traditional supplemental materials?

• CALI lessons written
• Websites related to educational activities
• Blogs and podcasts with educational intent like Classcaster

I can certainly measure the first item and there are over 100 law professors who have written a CALI lesson at some time in the past, but the others would be much harder to track. A lot of web-based educational material is behind a password or inside a learning management system.

None of this speaks to quality, however and so we look back to Leiter's study. He measures quality by citation. If someone else (either a court or another faculty member) has made a citation to your work, it is an indicator of impact and therefor some measure of the works value.

The nearest thing to citation in teaching is casebook/material adoption, but this would not capture the whole picture.

It would seem that potential law students would be very interested in this type of information, especially in the 150 law schools that are not going to end up in Professor Leiter's Top 30 ranking of scholarly impact.

Full post as published by CALIopolis on June 04, 2007 (boomark / email).