Why Questionnaire Scores Are Not Measures: A Question-Raising Article.
Journal
American journal of physical medicine & rehabilitation
ISSN: 1537-7385
Titre abrégé: Am J Phys Med Rehabil
Pays: United States
ID NLM: 8803677
Informations de publication
Date de publication:
01 01 2023
01 01 2023
Historique:
pubmed:
15
6
2022
medline:
17
12
2022
entrez:
14
6
2022
Statut:
ppublish
Résumé
Any person is provided by characteristics that can be neither located in body parts nor directly observed (so-called latent variables): these may be behaviors, attitudes, perceptions, motor and cognitive skills, knowledge, emotions, and the like. Physical and rehabilitation medicine frequently faces variables of this kind, the target of many interventions. Latent variables can only be observed through representative behaviors (e.g., walking for independence, moaning for pain, social isolation for depression, etc.). To measure them, behaviors are often listed and summated as items in cumulative questionnaires ("scales"). Questionnaires ultimately provide observations ("raw scores") with the aspect of numbers. Unfortunately, they are only a rough and often misleading approximation to true measures for various reasons. Measures should satisfy the same measurement axioms of physical sciences. In the article, the flaws hidden in questionnaires' scores are summarized, and their consequences in outcome assessment are highlighted. The report should inspire a critical attitude in the readers and foster the interest in modern item response theory, with reference to Rasch analysis.
Identifiants
pubmed: 35700126
doi: 10.1097/PHM.0000000000002028
pii: 00002060-202301000-00010
pmc: PMC9770109
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
75-82Informations de copyright
Copyright © 2022 The Author(s). Published by Wolters Kluwer Health, Inc.
Déclaration de conflit d'intérêts
Financial disclosure statements have been obtained, and no conflicts of interest have been reported by the authors or by any individuals in control of the content of this article.
Références
Hambleton RK, Cook LL: Latent trait models and their use in the analysis of educational test data. J Educ Meas 1977;14:75–96
Steyer R, Schmitt M, Michael E: Latent state-trait theory and research in personality and individual differences. Eur J Pers 1999;13:389–408
Nunnally JC, Bernstein IH: Psychometric Theory , 3rd ed. McGraw-Hill, Inc, 1994
Luce RD, Tukey JW: Simultaneous conjoint measurement: a new type of fundamental measurement. J Math Psychol 1964;1:1–27
Carlson JE, von Davier M: Item response theory, in Bennett R, von Davier M (eds): Item Response Theory. Advancing Human Assessment. Methodology of Educational Measurement and Assessment . Springer, 2017:133–78. doi:10.4324/9781410605269
Rasch G: Probabilistic Models for Some Intelligence and Attainment Tests . University of Chicago Press, 1980
Wright BD, Linacre JM: Observations are always ordinal; measurements, however, must be interval. Arch Phys Med Rehabil 1989;70:857–60
Andrich D: Rasch Models for Measurement . Sage Publications, 1988
Tesio L, Caronni A, Kumbhare D, et al.: Interpreting results from Rasch analysis 1. The “most likely” measures coming from the model. Disabil Rehabil 2022. (submitted)
Tesio L, Caronni A, Simone A, et al.: Interpreting results from Rasch analysis 2. Advanced model applications and the data-model fit assessment. Disabil Rehabil 2022. (submitted)
Wright B: Measuring and counting. Rasch Meas Trans 1994;8:371–1
Becker KA: History of the Stanford-Binet Intelligence Scales: content and psychometrics, in: Stanford-Binet Intelligence Scales , 5th ed. Riverside Publishing, 2003
Tesio L: Measuring behaviours and perceptions: Rasch analysis as a tool for rehabilitation research. J Rehabil Med 2003;35:105–15
Tesio L, Alpini D, Cesarani A, et al.: Short form of the Dizziness Handicap Inventory: construction and validation through Rasch analysis. Am J Phys Med Rehabil 1999;78:233–41
Yamaguchi J: Positive vs. negative wording. Rasch Meas Trans 1997;11:567–7
Collin C, Wade D: Assessing motor impairment after stroke: a pilot reliability study. J Neurol Neurosurg Psychiatry 1990;53:576–9
Franchignoni F, Tesio L, Benevolo E, et al.: Psychometric properties of the Rivermead Mobility Index in Italian stroke rehabilitation inpatients. Clin Rehabil 2003;17:273–82
Wright B, Linacre MJ: Fundamental measurement for outcome evaluation. MESA Memorandum 66 . 1997. Available at: https://www.rasch.org/memo66.htm . Accessed April 30, 2022
Hassan S, Kumbhare D: Validity and diagnosis in physical and rehabilitation medicine: critical view and future perspectives. Am J Phys Med Rehabil 2022;101:262–9
Schafer JL, Graham JW: Missing data: our view of the state of the art. Psychol Methods 2002;7:147–77
Chatfield C, Little RJA, Rubin DB: Statistical analysis with missing data. J R Stat Soc Series A 1988;151:375–6
Little RJA, Rubin DB: Statistical Analysis with Missing Data . John Wiley & Sons, Inc, 2002. doi:10.1002/9781119013563
Franchignoni F, Mandrioli J, Giordano A, et al.; ERRALS Group: A further Rasch study confirms that ALSFRS-R does not conform to fundamental measurement requirements. Amyotroph Lateral Scler Frontotemporal Degener 2015;16(5–6):331–7
Tennant A, Penta M, Tesio L, et al.: Assessing and adjusting for cross-cultural validity of impairment and activity limitation scales through differential item functioning within the framework of the Rasch model: the PRO-ESOR project. Med Care 2004;42:I37–48
Arnould C, Vandervelde L, Batcho CS, et al.: Can manual ability be measured with a generic ABILHAND scale? A cross-sectional study conducted on six diagnostic groups. BMJ Open 2012;2:e001807
Simone A, Rota V, Tesio L, et al.: Generic ABILHAND questionnaire can measure manual ability across a variety of motor impairments. Int J Rehabil Res 2011;34:131–40
Miller GA: The magical number seven plus or minus two: some limits on our capacity for processing information. Psychol Rev 1956;63:81–97
Ma WJ, Husain M, Bays PM: Changing concepts of working memory. Nat Neurosci 2014;17:347–56
Franchignoni F, Salaffi F, Tesio L: How should we use the Visual Analogue Scale (VAS) in rehabilitation outcomes? I: how much of what? The seductive VAS numbers are not true measures. J Rehabil Med 2012;44:798–9
Kelley TL: Interpretation of Educational Measurements . Yonkers-on-Hudson, NY, World Book Company, 1927
Guilford JP: New standards for test evaluation. Educ Psychol Meas 1946;6:427–38
Borsboom D, Mellenbergh GJ, Van Heerden J: The concept of validity. Psychol Rev 2004;111:1061–71
Agazzi E: Scientific Objectivity and Its Contexts . Heidelberg, Springer International Publishing, 2014. doi:10.1007/978-3-319-04660-0
Buzzoni M, Tesio L, Stuart MT: Holism and Reductionism in the Illness/Disease Debate, in Wuppuluri S, Stewart I (eds): From Electrons to Elephants and Elections. The Frontiers Collection . Cham, Springer, 2022:743–78
Tesio L: Measurement in clinical vs. biological medicine: the Rasch model as a bridge on a widening gap. J Appl Meas 2004;5:362–6