 |
|
|
 |
Variability of Precision in Scoring Radiographic Abnormalities in Rheumatoid Arthritis by Experienced Readers
JOHN T. SHARP, FREDERICK WOLFE, MARISSA LASSERE, MAARTEN BOERS, DÉSIRÉE van der HEIJDE, ARVI LARSEN, HAROLD PAULUS, ROLF RAU and VIBEKE STRAND
ABSTRACT.
Methods. Radiographic scores from 6 datasets in which 2 or more readers had scored film sets were analyzed. Datasets included scores by 11 different readers, 6 of whom scored films by both the Larsen (global) and Sharp (composite) methods. Scores of each possible combination of 2 readers were compared in calculating the smallest detectable difference (SDD) on raw scores and on scores normalized for each individual reader (nSDD). Intraclass correlation (ICC), Pearson's r, and the correlation between differences in score and their mean scores were determined. Agreement on progression of radiographic damage scores was also examined. Results. Variability among readers was greater than previous studies suggested. Agreement was better for intra- than interreader comparisons; average intrareader SDD was 24.4 for the composite method and 9.0 for the global. The larger SDD for the composite method reflect their greater range of possible scores. When normalized scores were used to adjust for the range difference, there was minimal difference in the SDD; nSDD was 10.1 for the composite method, 8.0 for the global. Interreader variability was larger: SDD of 53.7 for the composite method and 23.3 for the global; nSDD 12.9 and 14.4, respectively. ICC varied between 0.465 and 0.999, with all but one value below 0.925 occurring in composite scores with a range below 100. Differences in repeated scores were frequently associated with the mean of those scores and this was greater for inter- than for intrareader comparisons. Agreement between progression scores showed a similar pattern. The SDD was better for intrareader comparisons and smaller for global scores: compare 13.7 (composite, intrareader) and 5.4 (global, intrareader) to 18.1 (composite, interreader) and 8.7 (global, interreader). The ICC was lower for progression scores than for raw scores, averaging between 0.661 and 0.885. Conclusion. The variability in scoring radiographic abnormalities is considerable among this group of 11 expert readers. This has important implications for power calculations in comparison studies such as therapeutic trials and for cross-trial comparisons. The correlation between the difference in repeated scores and their means indicates systematic error (bias), which, if corrected, may improve the detection of treatment effects when using a responder-type analysis. These and other design and analysis issues are discussed. (J Rheumatol 2004;31:1062-72) Key Indexing Terms:
RHEUMATOID ARTHRITIS
From the University of Washington School of Medicine, Seattle, Washington, USA; National Data Bank for Rheumatic Disease, Wichita, Kansas, USA; University Medical Center, Amsterdam, The Netherlands; University Hospital Maastricht, Maastricht, The Netherlands; Kongsvinger Hospital, Kongsvinger, Norway; St. George Hospital, University of New South Wales, Sydney, Australia; H. Paulus, University of California at Los Angeles, Los Angeles, California, USA; Evangelisches Fachkrankenhaus, Ratingen, Germany; and Stanford University, Palo Alto, California, USA. J.T. Sharp, MD, University of Washington School of Medicine; F. Wolfe, MD, National Data Bank for Rheumatic Disease; M. Boers, MD, University Medical Center Amsterdam; D. van der Heijde, MD, University Hospital Maastricht; A. Larsen, MD, Kongsvinger Hospital; M. Lassere, MD, St. George Hospital, University of New South Wales; H. Paulus, MD, University of California at Los Angeles; R. Rau, MD, Evangelisches Fachkrankenhaus Ratingen; V. Strand, MD, Stanford University. Address reprint requests to Dr. J.T. Sharp, 8387 NE Sumanee Place, Bainbridge Island, WA 98110, USA. E-mail: johntsharp@worldnett.att.net Submitted February 19, 2003; revision accepted November 24, 2003.
|