Sample size requirements for agreement studies

Abstract

This thesis examines requirements of subject sample size while planning a medical experiment, describes some known types of measures of inter-rater agreement and discovers some new useful results in this study area. The main goal is to determine how many patients to include in clinical study. There is a particular focus on studying the Gwet’s AC1 agreement coefficient by comparing it to the well known Cohen’s kappa coefficient and some others, which are popular used in the inter-rater agreement studies. As is shown most agreement coefficients have paradoxical behaviour, which gives unreasonably low estimates of agreement in some situations. Gwet’s AC1 has been claimed to be ”paradox-free”, however, we find its own paradox. That is why the focus holds on studying some useful properties of this measure and analyzing the formula for variance which has the main role in finding the required sample size.