Abstract
Occupational therapy practitioners who provide low vision rehabilitation (LVR) require assessment tools that focus on vision-dependent activities of daily living (ADLs) to efficiently and accurately identify their clients’ limitations and to provide effective interventions. The Self-Report Assessment of Functional Visual Performance (SRAFVP; Velozo, 1996) has been widely adopted by occupational therapy practitioners to identify ADL performance limitations in adults with low vision (Gilbert & Baker, 2011).
The SRAFVP consists of 38 items divided into nine categories addressing personal care (4 items), clothing care (2 items), meal preparation (7 items), financial management (4 items), telephone use (2 items), leisure (3 items), reading (9 items), writing (3 items), and mobility (4 items). The assessment is administered in an interview format, with clients being asked to rate their ability to complete each of the 38 tasks. Therapists used a 3-point rating scale with the following verbal qualifiers: 1 = unable, 2 = some difficulty, and 3 = independent (Velozo et al., 2013).
The 38-item SRAFVP showed unidimensionality and satisfactory model fit, and it demonstrated acceptable levels of internal construct validity (Velozo, 1996). Evidence of construct validity for the SRAFVP was supported through the known-groups method in older adults with homonymous hemianopia resulting from stroke (Mennem et al., 2012). In addition, consistent with the item-difficulty hierarchy established via the Rasch model, tasks clustered to functional reading (which required near vision) were more difficult for participants with homonymous hemianopia to complete than eye–hand coordination tasks, which, in turn, were significantly more difficult than mobility tasks (which required distance vision; Mennem et al., 2012; Velozo et al., 2013). Overall, the SRAFVP demonstrated acceptable reliability and validity to evaluate performance of vision-dependent ADLs in older adults with central field impairment and homonymous hemianopia (Mennem et al., 2012; Velozo et al., 2013).
However, it is clear that the 20-yr-old assessment has become dated. Digital technologies, such as email, social media, iPods, iPads, e-readers, tablets, and smartphones, introduced after the assessment was developed, have been incorporated into older adults’ ADL and daily routines. Conversely, older adults no longer complete some of the tasks in the SRAFVP, such as consulting a newspaper TV guide. The purpose of this study was to describe the process used to update and establish content validity for the revised SRAFVP (R–SRAFVP). The study was the first of a two-phase project to update the SRAFVP for clinical utility. The second phase will establish the psychometric properties of the R–SRAFVP.
Method
In this study, we used a cross-sectional survey research design to establish the content validity of the R–SRAFVP for older adults with low vision. Content validation was completed with a three-stage process: (1) item modification and development, (2) item evaluation, and (3) item comprehensibility evaluation. The institutional review board of the University of Alabama at Birmingham approved the study.
Stage 1: Item Modification and Development
Participants.
For the first two stages of this study, occupational therapy experts in LVR were recruited via a database of occupational therapy low vision experts willing to participate in research studies. Inclusion criteria included having provided LVR services to adults with age-related eye disease for a minimum of 16 hr/wk for at least 2 yr and having routinely administered the SRAFVP to clients with low vision. Database members received an email describing the first stage of the study and an invitation to participate. An affirmative response to the email served as consent to participate in the study; the first three occupational therapists to respond were selected for the panel. The mean (M) ± standard deviation (SD) of the experts’ work experience in LVR was 9.5 ± 1.3 yr (range = 8–10.5 yr). They provided outpatient LVR services in Georgia, Maryland, and Mississippi.
Procedure.
The original 38-item SRAFVP in Word (Microsoft Corp., Redmond, WA) format was sent to the panel experts via email with a set of questions addressing needed modifications to the instrument. The experts were asked to identify items and item descriptions that should be revised, added, deleted, or moved to another category; items that should be divided to create more discrete items; and categories that should be revised, added, deleted, or combined to create a broader category. Each expert independently reviewed the items using the provided questions and identified those items that should be included, deleted, or modified on the basis of her experience with the ADLs commonly performed by clients with age-related eye disease. The experts also commented on the number of response options and verbal qualifiers for the rating scale. After the feedback was returned, the first author (Cheryl L. Zemina) conducted a telephone interview with each expert to affirm and clarify her response to each item, item description, and the rating scale. The interviews were audiotaped and transcribed verbatim, and the expert’s feedback (written and interviews) was compiled, organized, and returned for fact checking and comment.
Results.
Suggestions and comments from the three experts were reconciled to produce the first draft of the R–SRAFVP with 59 items. Twenty-five new items were added to almost every category except clothing care and leisure, and 4 items in the category of reading were deleted or combined for a net addition of 21 items (Table 1). The original item addressing medication routine was divided and expanded to 4 items related to health management; the original item on grooming was divided and expanded to 4 items related to personal hygiene. Four new items were proposed for the meal preparation category, including 2 items in a new subcategory of kitchen management. Two new items were added to the categories of financial management, telephone usage, and writing, respectively, and 3 items were added to the reading category.
Changes to the Original 38-Item Self-Report Assessment of Functional Visual Performance in Stage 1 Item Review and Stage 2 Item Evaluation
Note. Item numbers in bold indicate items in the original 38-item version of the SRAFVP. CVI = content validity index; deleted = item was deleted after Stage 1 item modification; excluded = item was eliminated after Stage 2 item evaluation; min reword = minor rewording; mod reword = moderate rewording.
Item description included digital technology.
The original item addressing community/social participation was expanded to 4 items. The original item on shopping located in the financial management category was expanded to 2 items, which were moved to the subcategory of community/social participation under the functional mobility category. Two additional new items were included in the functional mobility category. Finally, 4 items in the categories of personal care (1 item), telephone usage (1 item), and leisure (2 items) were significantly revised. The three experts recommended that the number of rating points be expanded from the original 3 to at least 4 with appropriate verbal qualifiers such as great difficulty, moderate difficulty, minimal difficulty, and independent.
Stage 2: Item Evaluation
Participants.
Using the same recruitment procedure as in Stage 1, a new panel of eight qualified licensed occupational therapists with expertise in LVR was recruited to participate in item evaluation of the first draft of the R–SRAFVP. The M ± SD of the panel’s work experience in LVR was 10.0 ± 2.3 yr (range = 8–15 yr). The eight panel members provided LVR services in various geographic regions: Southeast (4), Northeast (2), Midwest (1), and Northwest (1).
Procedure.
The purpose of this stage was to assess the relevance of each of the 59 items on the newly created draft to determine which items were essential for inclusion in the R–SRAFVP. The 59-item R-SRAFVP draft with instructions was emailed to each panel member. The cover letter informed the panel members that the new draft was created by adding, deleting, and amending items on the original 38-item SRAFVP. To assist the reviewers, we color coded the draft using green for new items, red for deleted items, and blue for modified items.
The panel members independently rated the relevance of each of the 59 items using a 4-point rating scale: 1 = not relevant, 2 = unable to assess relevance without major revision, 3 = relevant but needs minor alteration, and 4 = very relevant and succinct. The reviewers were instructed to assign the rating on the basis of two criteria: (1) whether ability to complete the item was important to a client’s ability to take care of him- or herself, live independently and safely within his or her home, and participate independently and safely in the community and (2) whether the panel member frequently queried clients about performance of this item during the initial ADL assessment and addressed it in intervention because it was important in their daily lives.
Panel members were encouraged to highlight words or portions of the item descriptions that were unclear and use the comment column to suggest alternate phrasing and improvements in the layout and format of the instrument. Finally, panel members were asked to appraise the change in the R–SRAFVP rating scale from a 3-point scale (1 = unable, 2 = some difficulty, and 3 = independent) to a 4-point scale (1 = great difficulty, 2 = moderate difficulty, 3 = minimal difficulty, and 4 = independent).
Results.
We analyzed the data for item relevance by determining the percentage of agreement among the ratings of the eight experts using the content validity index (CVI; Lynn, 1986). The formula to compute the CVI for item relevance was the number of experts giving a rating of 3 or 4 divided by the number of experts on the panel. The criterion to retain an item was set at 75%, which required six of the eight experts to provide the item a rating of 3 or 4 (i.e., item CVI ≥ .75). The item CVI cutoff criterion was slightly more lenient than the .88 proposed by Lynn for eight experts. Our rationale for doing so was the importance of retaining a relatively larger pool of items for conducting the item and factor analyses in the construct validation phase of the project to avoid prematurely excluding an item.
Using the CVI ≥ .75 criterion, we reduced the second draft of the R–SRAFVP to 50 items with nine categories (personal care [10 items], clothing care [2 items], meal preparation [9 items], financial management [5 items], telephone usage [2 items], leisure [3 items], reading [8 items], writing [5 items], and functional mobility [6 items]; see Table 1). The scale CVI of the second draft of the R–SRAFVP, which is the average of the item CVI of the 50 items (Polit et al., 2007), was .91; 26 items achieved unanimous agreement (CVI = 1), 15 items had a CVI of .88, and 9 items had a CVI of .75. After reviewing the panel’s comments and recommendations, we determined that the 4-point rating scale was appropriate because it provided a more discrete measurement of the client’s limitations and enabled the client to more accurately rate his or her ability to perform the items. The panel members also provided extensive suggestions on the wording of the items and item descriptions that were incorporated into the R–SRAFVP for pilot testing in Stage 3.
Stage 3: Item Comprehensibility Evaluation
Participants.
We recruited 5 community-dwelling older adults (3 women and 2 men) with limitations in completing ADLs because of low vision to be assessed using the 50-item R–SRAFVP. Eligibility criteria included >50 yr of age, self-reported eye disease or condition causing low vision, limitations in completing ADLs because of vision loss, absence of memory problems, and absence of considerable hearing loss. Potential participants were identified from the caseload or acquaintances of the first author. The M ± SD age of the participants was 65 ± 13.4 yr (range = 52–79 yr). Their reported diagnoses included macular degeneration, trauma to the face and eye, severe corneal scarring due to bilateral eye infection, and visual field loss from stroke (this participant met all other inclusion criteria).
Procedure.
The purpose of this stage of item evaluation was to determine whether adults with vision impairment experienced difficulty comprehending and interpreting the items, item descriptions, and the rating scale response options and verbal qualifiers of the 50-item R–SRAFVP. The cognitive interviewing method (Drennan, 2003) was used to evaluate the participants’ comprehension and accurate interpretation of the instrument items and rating. Cognitive interviewing in the form of “think alouds” has been used in self-report questionnaires to understand how older adults formulate responses and to reveal their understanding of item content to avoid responses that may misrepresent the intended meaning of a survey question (Jobe & Mingay, 1990).
After obtaining written informed consent and explaining the R–SRAFVP and the rating scale to the participants, the first author read aloud each of the 50 items and item descriptions to the participant. The participant was asked to provide the rating response that best described his or her ability to complete each described task from the following rating scale: 1 = great difficulty (may perform some aspect of the task but requires assistance for the majority of the task or cannot perform the task in a safe and efficient manner), 2 = moderate difficulty (performs the task with difficulty even under optimal conditions; i.e., difficulty performing the task in a timely manner, questionable safety and efficiency, makes errors, or assistance is required for a substantial part of the task), 3 = minimal difficulty (performs the task with some difficulty; i.e., can perform the task only under optimal conditions or may require assistance for a small part of the task), 4 = independent (experiences no difficulty performing the task safely, accurately, and efficiently), or not applicable (does not perform or is no longer performing the task for various reasons).
After the participant provided the rating response to a particular item, the first author assessed the participant’s understanding of the item’s intent by asking the participant to provide his or her interpretation of the item (e.g., what it meant to him or her) and to explain why he or she chose a particular rating response and whether he or she had any suggestions to make the item clearer. If the participant demonstrated difficulty understanding an item description, the first author slowly repeated the description verbatim phrase by phrase or paraphrased the description. Two participant interviews were completed by phone, and three were completed in person at the participants’ homes. The interviews were audiotaped and transcribed verbatim for analysis. The participants’ wording suggestions were considered and incorporated into the final version of the R–SRAFVP to improve the clarity of item descriptions, the response options, and the verbal qualifiers on the rating scale.
Results.
The participants’ responses showed that they used the full range of the 4-point rating scale. Of the 250 responses provided by the participants, 5.2% of the items were rated as not applicable, 14.4% were rated as great difficulty, 20.0% were rated as moderate difficulty, 24.8% were rated as minimal difficulty, and 35.6% were rated as independent. This pattern of responses was similar to that obtained on the original SRAFVP (Velozo et al., 2013), with a higher percentage of responses scored as independent and a lower percentage rated as great difficulty. In addition, the distribution of responses rating the degree of difficulty of performing the task items mostly mirrored the findings of the previous study (Velozo et al., 2013). For example, participants rated mending and functional reading as the most difficult items to perform, and they rated oral care, dressing (locate, identify, and match clothing), and meal preparation (locate/organize items in kitchen) as the least difficult items to perform. The 5 participants reported no difficulty understanding the description of 43 items and suggested alternate words to simplify phrasing for 7 items (e.g., use walk instead of ambulate). Overall, the participants verified the clarity of all items and the rating scale and suggested only minimal changes to the form.
Discussion
In this study, we used a rigorous three-stage process to establish the content validity of the R–SRAFVP (Lynn, 1986). The process used sequential expert review to establish the authenticity of the content. Three occupational therapy experts expanded the original 38-item SRAFVP to 59 items in Stage 1, and eight experts condensed the instrument to 50 items and retained the original nine categories in Stage 2. The rigor of this process was supported by the fact that 8 of the 26 new items introduced by the first panel of occupational therapy experts achieved an item CVI of less than .75 from the second panel of occupational therapy experts and were eliminated (see Table 1). The eliminated items were in the categories of meal preparation (2 items), telephone usage (2 items), and functional mobility (4 items).
Moreover, the original item addressing community/social participation was also eliminated because of a low item CVI even after it was modified by the first panel. To reflect the digital technology incorporated into the current ADLs of older adults, the Stage 1 panel developed 6 new items addressing ability to proficiently use digital technology, including reading the screen on a personal computer, e-readers, and tablets; using computers for managing finances; and smartphones. The panel modified an additional 5 items to include reading financial statements on computers, playing computer games, and using iPods, apps on smartphones, and the program guide on TV. The Stage 2 panel subsequently eliminated the 2 new items in the telephone usage category addressing retrieval of phone numbers from the cell phone directory and computer because they were considered to be duplicates.
Pilot testing of the R–SRAFVP with the 5 adults with low vision in Stage 3 provided evidence that the distribution of rating responses was consistent with the difficulty level in performing task items reported by older adults with low vision on the original SRAFVP form (Velozo et al., 2013). Cognitive interviewing confirmed that adults with low vision were able to understand the items and verified the clarity of all items on the R–SRAFVP.
With determination of the content validity of the R–SRAFVP completed, determining the construct validity of the assessment is the next step in the process of updating the form. This step will be completed by evaluating the R–SRAFVP on a sample of older adults with low vision receiving occupational therapy services.
Study Limitations and Recommendations
One limitation of the study was that the process of updating items in the item modification and development stage depended largely on the experts’ beliefs about which items were relevant. This process is subject to recall bias and may not produce a comprehensive representation of the typical ADLs that older adults complete in their daily lives. To ensure that the content of the SRAFVP represents the types of ADLs (especially those incorporating digital technology) that older adults with low vision commonly report difficulty performing would require abstracting such information from the experts’ occupational therapy evaluation records of this population in the past few years.
Implications for Occupational Therapy Practice
The findings of this study have the following implications for occupational therapy practice:
The 50 R–SRAFVP items reflect the ADLs that older adults with low vision commonly perform and that have high relevance to them; moreover, the experts believe older adults with low vision are most likely to experience difficulty completing these ADLs.
Preliminary evidence supporting the content validity of the R–SRAFVP confirms that the content is suitable to evaluate older adults with vision impairment.
Conclusion
The R–SRAFVP is a standardized self-care assessment that measures the vision-dependent ADLs that older adults with age-related eye disease most often report as being difficult to perform. This study used expert and consumer input and review to establish the content validity of the revised version of the assessment. The R–SRAFVP includes activities that older adults now complete using digital technologies not available when the original form was created in 1995. The revised assessment will be tested on a population of older adults with low vision from age-related eye disease to establish the construct validity and create the final version of the form.
Footnotes
Acknowledgments
We thank the 5 adults with low vision who participated in this study and the 11 occupational therapists (Deann Bayerl, Michelle Bianchi, Robin Deacy, Joan Gillard, Donna Inkster, Karen Kendrick, Kristen Lindeman, Cheri Nipp, Serena Speaker, Lind Stevens, and Nilima Tanna) who assisted in updating the SRAFVP.