How Meaningful Is Our Evaluation of Meaningful Change in Osteoarthritis?

GILLIAN A. HAWKER, MD, MSc, FRCPC,
Associate Professor,
Division of Rheumatology,

Download PDF View Table of Contents

Department of Medicine,
Director, Clinical Epidemiology and
Health Care Research Program,
Department of Health Policy, Management and Evaluation,
Faculty of Medicine, University of Toronto,
Toronto, Ontario, Canada;
MONIQUE A.M. GIGNAC, PhD,
Senior Scientist, Division of Outcomes and
Population Health, University Health Network,
Associate Professor, Department of Public Health Sciences,
University of Toronto.
 
Dr. Hawker receives salary support as the F.M. Hill Chair in Academic Women's Medicine.

Address reprint requests to Dr. G.A. Hawker, Sunnybrook/Women's College Health Science Centre, 8th Floor East, Room 815, 76 Grenville Street, Toronto, Ontario M5S 1B2. E-mail: gillian.hawker@sw.ca

Osteoarthritis (OA) has a major impact on the functioning and independence of the population, ranking among the top 10 causes of disability worldwide¹. The last 5 years have seen a resurgence of interest in this common condition, with increased attention to the development not only of new interventions to control OA symptoms and reduce OA disability, but also to novel therapies aimed at slowing the structural damage that occurs with OA. An essential element in the evaluation of OA, whether in clinical practice or research, or in clinical trials of interventions, is the need for reliable, valid, and sensitive measurement tools.

A number of instruments or tools have been developed over the years aimed largely at assessing the pain and disability associated with OA. Overall, these measures have worked — they have been useful in evaluating the influence of pain therapies and surgical interventions on OA pain and functioning. However, the report by Pollard, et al in this issue of The Journal² suggests that there is a need for rethinking not only how we evaluate patients with OA but also what our goals are in this evaluation.

What has changed? Why are we taking a new look at OA measures? A number of factors are likely driving this renewed interest. For example, a wealth of data has been amassed that has characterized OA not only at one point in time but across several years^3-6. Making sense of OA changes, trying to determine meaningful change, and grappling with standards for care to improve the quality of life of people with OA are the focus of much emerging OA research. Pollard, et al highlight some of these issues, and touch upon 3 important points. The first is that reliance on conceptual models can help arthritis researchers in a number of important ways. Second, global health assessments traditionally used by arthritis researchers may mask our understanding of the processes by which changes related to treatments are actually occurring. Finally, by using conceptual models in our research we may be better able to identify and measure constructs that are important and meaningful in the lives of people with arthritis.

To make sense of arthritis research overall, as well as clinical and outcomes data related specifically to OA, Pollard and others are increasingly relying on conceptual frameworks to guide their thinking. The use of theoretical and conceptual frameworks in the social and behavioral sciences underpins most research. However, for clinical epidemiologists their use is relatively new and provides a means to help identify gaps in research and clinical care, as well as generate hypotheses for future research. The framework used most often to date in the field of arthritis is the World Health Organization's International Classification of Functioning, Disability and Health, known as the ICF model⁷. When applied to OA, this framework represents conventional wisdom whereby OA can lead to impairment (e.g., joint space narrowing, pain), which gives rise to limitations in activities, such as walking and self-care. These, in turn, may restrict participation in broad roles and societal activities (e.g., employment, education, social involvement, personal relationships, and leisure). Explicit in the model is the need to clearly evaluate OA at all 3 levels, impairment, activity limitation, and participation. Yet, as Pollard and colleagues find, this is rarely the case. Of the 13 instruments they assessed, 12 were found to measure a mixture of constructs with few or no conceptual distinctions made among them.

What is especially interesting and apparent in the approach taken by Pollard and colleagues is a concerted effort to more closely examine assessments of health status and quality of life that have underpinned much of arthritis research over the past 2 decades. While problems with health measures, particularly those assessing global health, have frequently been discussed^8,9, the fact remains that clinicians and researchers have embraced global outcome measures. The latter have been lauded as broadening the scope of outcomes research, providing a standard by which to compare treatments that incorporates the patient's perceptions.

However, as research using the measures has evolved, concerns have been expressed that global or composite measures may be obscuring more than illuminating the processes that contribute to changing outcomes. In other words, the use of instruments measuring a combination of domains or constructs may mask true treatment effects, or effects that are specific to some domains/constructs, but not others. Alternatively, they may mislead researchers into believing that treatment effects have occurred where none exist. For example, statistically significant improvements in an outcome measure may lead researchers to believe in the efficacy of a particular treatment/intervention. More detailed examination of subscales or individual constructs using an approach similar to that proposed by Pollard and colleagues, however, may reveal that reductions in pain exist along with few changes in activity limitations and increases in participation restrictions. While this may satisfy researchers and clinicians who had primarily aimed to reduce pain with a treatment/intervention, the "truth" may be more complicated. Research finds that people with arthritis often give up or limit important roles or activities^10-13. Hence, a reduction in symptoms may not necessarily signify that a treatment is effective, but may come about because people have limited the time spent on participation in activities and roles like socializing, leisure, or employment. If these factors are the explanation for observed improvements in pain over time, then this suggests that self-reported changes in impairment (pain) or activity limitation (walking) must be interpreted in the context of concomitant changes in participation in broader social roles. A focus solely on impairment or activity limitation or the reliance on a total score of a composite measure will miss such shifts. Of equal if not greater concern is that individuals who experience minimal pain or disability as a result of restricting their participation in roles and activities may perceive themselves as doing well, and may not seek out appropriate healthcare, thinking that they do not require it. Clinicians, therefore, may receive the wrong message and not offer appropriate medical or surgical interventions to patients who may potentially benefit from them. Here again, the use of conceptual models and a clearer distinction among concepts being assessed in measurement tools can guide our interpretation of changes over time.

The example above raises another important issue. That is, are we measuring what matters to people with arthritis? Pollard, et al discuss the concept of participation in broader roles and life activities as an important dimension of the ICF model. Although not the primary focus of their study, their work indicates that there is significant underrepresentation of one of the 3 ICF model constructs in available OA measures, and that is participation. This is a cause for concern, given the link between valued life roles and psychological well-being found in arthritis research^12,14. Yet, with some exceptions, there has been relatively little attention to the construct of participation and its earlier incarnation, handicap, in either arthritis research or clinical care^15-18. There are signs, however, that this may be changing. A significant focus of several recent arthritis meetings, such as the CARE III conference, an international gathering of arthritis health professionals¹⁹, and the Summit on Standards for Arthritis Prevention and Care held in Ottawa, Canada²⁰, both in 2005, was the need for conceptually clear measures of participation to be included as core outcomes in evaluation of arthritis treatments and interventions. Perhaps of greatest relevance, however, was the clear message that was given by people with arthritis to clinicians and researchers attending these meetings: It isn't enough to measure participation as an outcome; greater attention also needs to be paid to designing interventions and treatments that have as their primary aim increasing people's levels of participation in valued roles like employment, education, social involvement, personal relationships, and leisure pursuits. People with arthritis tell us that it is maintaining or resuming such roles that is most important to them — this is where our real challenge as arthritis healthcare providers and researchers lies. It is high time we accepted the challenge.

REFERENCES

Search PubMed for:

1. Murray CJL, Lopez AD, editors. The Global Burden of Disease: A comprehensive assessment of the mortality and disability from diseases, injuries and risk factors in 1990 and projected to 2020. Boston: Harvard School of Public Health on behalf of the World Health Organisation, and the World Bank; 1996.

2. Pollard B, Johnston M, Dieppe P. What do osteoarthritis health outcome instruments measure? Impairment, activity limitation, or participation restriction? J Rheumatol 2006;33:757-63.

3. Leffondre K, Abrahamowicz M, Regeasse A, et al. Statistical measures were proposed for identifying longitudinal patterns of change in quantitative health indicators. J Clin Epidemiol 2004;57:1049-62. [MEDLINE]

4. Dieppe P, Cushnaghan J, Tucker M, Browning S, Shepstone L. The Bristol "OA500 study": Progression and impact of the disease after 8 years. Osteoarthritis Cartilage 2000;8:63-8. [MEDLINE]

5. Dougados M, Gueguen A, Nguyen M, et al. Radiological progression of hip osteoarthritis: definition, risk factors and correlations with clinical status. Ann Rheum Dis 1996;55:356-62. [MEDLINE]

6. Peters TJ, Sanders C, Dieppe P, Donovan J. Factors associated with change in pain and disability over time: a community-based prospective observational study of hip and knee osteoarthritis. Br J Gen Pract 2005;55:205-11. [MEDLINE]

7. World Health Organization. International Classification of Functioning, Disability and Health. Geneva: World Health Organization; 2001.

8. McDowell I, Newell C. Measuring health: A guide to rating scales and questionnaires. 2nd ed. New York: Oxford University Press; 1996.

9. Bergner M. Quality of life, health status, and clinical research. Med Care 1989;27 Suppl:S148-S156.

10. Beaton DE, Tarasuk V, Katz JN, Wright JG, Bombardier C. "Are you better?" A qualitative study of the meaning of recovery. Arthritis Rheum 2001;45:270-9. [MEDLINE]

11. Katz PP, Yelin EH. Life activities of persons with rheumatoid arthritis with and without depressive symptoms. Arthritis Care Res 1994;7:69-77. [MEDLINE]

12. Gignac MAM, Cott C, Badley EM. Adaptation to chronic illness and disability and its relationship to perceptions of independence and dependence. J Gerontol: B Psychol Sci Soc Sci 2000; 55:P362-P372. [MEDLINE]

13. Gignac MAM. Arthritis and employment: An examination of behavioral coping efforts to manage workplace activity limitations. Arthritis Care Res 2005;53:328-36. [MEDLINE]

14. Katz PP, Yelin EH. Activity loss and the onset of depressive symptoms: Do some activities matter more than others? Arthritis Rheum 2001;44:1194-202. [MEDLINE]

15. Badley EM. The ICIDH: format, application in different settings, and distinction between disability and handicap. A critique of papers on the application of the International Classification of Impairments, Disabilities, and Handicaps. Int Disabil Stud 1987;9:122-5. [MEDLINE]

16. Cardol M, de Jong BA, van den Bos GA, Beelem A, de Groot IJ, de Haan RJ. Beyond disability: perceived participation in people with a chronic disabling condition. Clin Rehabil 2002;16:27-35. [MEDLINE]

17. Weigl M, Cieza A, Harder M, et al. Linking osteoarthritis-specific health-status measures to the International Classification of Functioning, Disability, and Health (ICF). Osteoarthritis Cartilage 2003;11:519-23. [MEDLINE]

18. Wilkie R, Peat G, Thomas E, Croft PR. Measuring the consequences of osteoarthritis and joint pain in population-based studies: can existing health measurement instruments capture levels of participation? Arthritis Rheum 2004;51:755-62. [MEDLINE]

19. Li LC, Backman C, Bombardier C, et al. Focusing on care research: a challenge and an opportunity. Arthritis Rheum 2004;51:874-6. [MEDLINE]

20. Website for the Alliance for the Canadian Arthritis Program. Available at: www.arthritisalliance.ca. Accessed January 30, 2006.