Researchers often need to evaluate the agreement between two measurements of continuous data (as opposed to categorical data); for example, blood pressure, tumour diameter, forced expiratory volume in one second. This could be measurement of a single feature by two people, or two measurements of the same feature by the same person. Martin Bland and Doug Altman give good advise on how to address this problem. They recommend 1) a graphical method and 2) an arithmetic method, the latter consistent with the British Standards Institution guide for reproducibility of a standard test method. (See: British Standards Institution. Precision of Test Methods 1:
Guide for theDetermination and Reproducibility for a Standard
Test Method (BS 597, Part 1). BSI: London, 1975).
Bland and Altman’s graphical method for assessing agreement is clearly and elegantly described in their 1986 paper ‘ Statistical methods for assessing agreement between two methods of clinical measurement’ Statistical methods for assessing agreement between two methods of clinical measurement (https://www.ncbi.nlm.nih.gov/pubmed/2868172). The paper guides the reader through the process of constructing what is now known as a ‘Bland-Altman Plot’, a plot of the mean of each pair of measurements against the difference of each pair of measurements, which visually identifies the agreement between the pairs of measurements; those outside the ‘limits of agreement’, conventionally within +/- two standard deviations of the mean difference, are instantly apparent. The plot also depicts any bias (for example, does one method of measurement consistently measure higher or lower than the other), and any change in agreement with the magnitude of the measurement, for example, measurements might have good agreement at low values and poorer or more biased agreement at higher values. For further excellent discussion see Watson and Petries’ paper “Method agreement analysis: A review of correct methodology” http://www.theriojournal.com/article/S0093-691X(10)00023-3/abstract and Bland and Altman’s more recent 2003 paper “Applying the right statistics: analysis of measurement studies (https://www.ncbi.nlm.nih.gov/pubmed/12858311).