Method comparison studies in biomedical science: an introduction

All analytical tests we perform are expected to be sufficiently reliable and accurate so that any clinical decision based on results can be taken with confidence. For a test to achieve this it is then labelled as fit for purpose. No matter which discipline we work in, most of the ‘things’ we are trying to measure are well out of our reach to measure directly. As much as measuring tree rings is used as a surrogate for past temperatures, we are limited to indirect measurements of physiological processes. Right from the start we realise that what we measure may be far removed from actually what we want to know.

This may cause issues when we want to add an assay to our repertoire, or update an existing one. We are limited to what we are able to compare it against. Comparability is the fact or quality of being similar and able to be compared. If it exists, we compare ourselves to another, equally indirect, measure of the thing we want to know. If one does not exist, we need a way of being sure the method works. Review of the literature in the field suggests this is a process that is poorly done. At its simplest, we need to know whether multiple methods of measuring the same thing do so in a way that we can call equivalent. We will try to address that in the coming series of articles.

Recently, we have been required to implement new assays to help with the diagnosis of SARS-CoV-2 and COVID-19 disease, from serology testing for antibodies to detecting the presence of viral RNA. How do we compare new methods to what we have… or even do not have? We compare ourselves to gold standards when available. Does the new method provide the same (or acceptably similar) results on the same samples?