Correspondence

The challenge of assessing reader performance in mammography

Breast screening programmes enhance the probability of early breast cancer detection in many countries worldwide; however, the success of these efforts is highly dependent on the ability of breast screen readers to detect abnormalities in the screened population, which has low prevalence. Therefore, this task can be challenging. Clinical audit is a key quality assurance measure that aims to keep the screen reading performance within acceptable standards. Auditing, nonetheless, is a lengthy process, and its accuracy is dependent on available clinical data, which often can be limited. Mammographic standardised test sets are a different screen reading evaluation approach that provides participants with instant feedback based on a simulated environment. Although a test set provides unique evaluative qualities, its ability to represent clinical performance is debated. This article describes the distinctive roles of clinical audit and test sets in measuring and improving the quality of breast screening and highlights the relationship between test sets and clinical performance.

The purpose of this article is to review the limitations associated with current methods of assessing reader accuracy in mammography screening programmes. Clinical audit is commonly used as a quality-assurance tool to monitor the performance of screen readers; however, a number of the metrics employed, such as recall rate as a surrogate for specificity, do not always accurately measure the intended clinical feature. Alternatively, standardized screening test sets, which benefit from ease of application, immediacy of results, and quicker assessment of quality improvement plans, suffer from experimental confounders, thus questioning the relevance of these laboratory-type screening test sets to clinical performance. Four key factors that impact on the external validity of screening test sets were identified: the nature and extent of scrutiny of one's action, the artificiality of the environment, the over-simplification of responses, and prevalence of abnormality. The impact of these factors on radiological and other contexts is discussed, and although it is important to acknowledge the benefit of standardized screening test sets, issues relating to the relevance of test sets to clinical activities remain. The degree of correlation between performance based on real-life clinical audit and performances at screen read test sets must be better understood and specific causal agents for any lack of correlation identified.