UWEE Tech Report Series

How to Analyze Paired Comparison Data


UWEETR-2011-0004

Author(s):
Kristi Tsukida and Maya R. Gupta

Keywords:
Thurstone's Law, paired comparisons, Bradley-Terry model

Abstract

Thurstone's Law of Comparative Judgment provides a method to convert subjective paired comparisons into one-dimensional quality scores. Applications include judging quality of different image reconstructions, or different products, or different web search results, etc. This tutorial covers the popular Thurstone-Mosteller Case V model and the Bradley-Terry logistic variant. We describe three approaches to model- fitting: standard least-squares, maximum likelihood, and Bayesian approaches. Proofs of key results are given in the appendix. This tutorial assumes basic knowledge of random variables and probability distributions.

Download the PDF version

Download the Gzipped Postscript version