Tuesday, December 30, 2008

Algorithms for Designing Exams

The discussion about designing grading schemes (1/4th the points for "I Don't Know", choice of answering X out of Y questions, etc) got me thinking: is there published work on algorithms for designing online exams (such as in ETS) where each question is dynamically determined online depending on the performance thus far, under a suitable model and optimization criteria? Are there interesting game-theoretic issues in how students can manipulate the outcome by deliberately not answering a question correctly?

Labels:

3 Comments:

Anonymous D. Eppstein said...

You might find section 2.3 "Assessment Procedures" of http://arxiv.org/abs/0803.4030 relevant. (Although it's my paper, that part of the paper is not my work: it describes the work of some mathematical psychologists in using combinatorial models describing which sets of concepts from some domain make sense as states of knowledge of learners, in order to guide a Bayesian procedure for quickly narrowing down which concepts an individual knows.)

6:09 PM  
Anonymous Anonymous said...

Hi David,
Thanks for the ref. There are $'s involved in automating and scaling the process of exams, one would think there will be a lot of prior work (modulo the challenges in making up an Algorithms exam).
-- metoo

4:17 AM  
Blogger Vineet Pandey said...

I might be making a foolish statement here, but isn't that pattern followed in GRE. The questions become difficult progressively.
(I'm from India, so can be excused :) )

3:56 AM  

Post a Comment

<< Home