Abstract
Keywords
Approximately one-third of people age 65 yr or older in the United States experience a fall in a given year, and in half of such cases, the falls are recurrent (Moylan & Binder, 2007). Falls among older adults are a common cause of hospitalization, elevated fear of falling, and decreased physical activity and quality of life (Boulgarides, McGinty, Willet, & Barnes, 2003) as well as reduced function, disability, and even early death (Mann, Locher, Justiss, Wu, & Tomita, 2005). In addition, fall-related injury is one of the most expensive medical conditions among community-dwelling older adults (Carroll, Slattum, & Cox, 2005). Therefore, falls in older adults are a public health concern.
The World Health Organization (2012) listed home assessment and environmental modification for people with known fall risk factors or a history of falling as one of five important strategies to prevent falls. The Centers for Disease Control and Prevention (CDC) has published a tool kit, Check for Safety: A Home Fall Prevention Checklist for Older Adults, to help health care providers prevent falls in older adults (CDC, 2005). The CDC’s key strategy is home safety, and the agency's checklist includes 17 items older adults can use to make their homes safer. The checklist addresses whether a problem exists and offers potential solutions. Online and hard-copy versions are available free of charge. The evidence of its usability and effectiveness has not been established.
Other recently published comprehensive tools include The SAFE AT HOME: A Quick Home Safety Assessment (Robnett, Hopkins, & Kimball, 2003), the Falls Risk Assessment (Flemming, 2006), and The Safe Living Guide: A Guide to Home Safety for Seniors (Public Health Agency of Canada, 2011). The SAFE AT HOME is a screening tool to determine whether a person can recognize common home safety issues and fix the problems independently by following a few specific directions. Its use is limited to licensed occupational therapists or certified occupational therapy assistants under the supervision of an occupational therapist (Robnett, 2006). The Falls Risk Assessment is a comprehensive fall risk assessment instrument listing many fall risk factors, but home fall risk is limited to one item, “environmental hazards.” The Safe Living Guide offers checklists in nine areas (outside, inside, stairs, fire and hazardous materials, bathroom, kitchen, bedroom, garage/basement/workroom, and childproofing) and is aimed at all populations, but it does not offer solutions for home risk factors. The Safe Living Guide is not targeted for use by older adults.
Many older adults need to improve their home environment to prevent falls, and they may be able to do so if support is available. The Home Safety Self-Assessment Tool (HSSAT) has been developed especially for older adults and their informal caregivers to help assess fall risks in their homes and guide them in improving their home environment. The HSSAT has been used by Area Agencies on Aging (AAA), home care providers (Horowitz, Nochajski, & Schweitzer, 2013), assisted living facilities, and university researchers in 17 states and in Brazil, Canada, Hong Kong (China), and England. The purpose of this study was to establish reliability and validity of the risk factor section (HSSAT) and responsiveness to change using the HSSAT.
Home Safety Self-Assessment Tool
The HSSAT was developed for community-dwelling older adults and their informal caregivers, such as families and friends, as a guide to prevent home falls. The risk factor section is comprehensive, covering 64 risk items in nine areas of the home (front and back entrances, hallway or foyer, living room, kitchen, bedroom, bathroom, staircases, and laundry room/basement) and offering solutions and advice for each item on the list. In addition, the HSSAT includes information regarding useful products to prevent falls currently available on the market, examples of how to improve the home environment, tips for fall prevention, lists of regional home improvement service providers and vendors for home safety and assistive devices, a list of organizations that provide free services related to home safety, Americans With Disabilities Act of 1990 (Pub. L. 101–336) installation guidelines, and an action log to record the history of home improvements. The uniqueness of the HSSAT is characterized by pictures drawn for nine home areas that depict problems as well as solutions, numbered to coincide with the item number in the fall risk factor section and for each solution. Risks can be summed for each area and a grand total generated for all areas within the home.
The HSSAT serves as educational material for users and an instrument to assess fall risk factors. In the HSSAT, risk factors in each home area are listed to raise awareness for current and possible future falls. For example, some people may not be aware that clutter is a fall risk until they see it on the list. By reviewing each risk item, users may be able to match the risks listed in the HSSAT with identified risks in their own home environment.
The booklet includes sections to help users modify their own fall risks. For example, after finding grab bars a necessity in the bathroom, older adults may not know what kind of grab bars are available and whom they should call for installation. They may be contemplating home improvement, but they cannot do it until they have identified the product, the source for procuring it, and the viable options for its installation. Relevant information is included in the HSSAT to reduce this barrier.
After the initial development of the draft of the HSSAT booklet, four small focus groups were conducted with a total of 24 community-dwelling older adults (age 65 yr or older), who evaluated the comprehensiveness, relevance, clarity, ease of understanding risk items, and general contents (including wordings, pictures, and spacing) and expressed preferences for the structures, length, color, and pictures. The completed HSSAT was published in a booklet and in three online versions (downloadable, interactive, and accessible), which are available free of charge at http://agingresearch.buffalo.edu/hssat/index.htm.
Method
We tested psychometrics for the HSSAT for content validity, test–retest reliability, interrater reliability, construct validity, and responsiveness to change. Procedures and results are described under each type of psychometric test. Approval was obtained from the Health Sciences Institutional Review Board at the University at Buffalo.
Study 1: Content Validity
Content validity has been defined as the “degree to which an instrument has an appropriate sample of items for the construct being measured” (Polit & Beck, 2004, p. 423). Lynn (1986) developed criteria that if five or fewer raters are involved, the item content validity index (I–CVI) should be 1.0 to account for chance agreement. We aimed for the I–CVI on the HSSAT to be close to 1.0.
Procedures.
Content validity was tested by four occupational therapists who are experts in geriatric and home safety. First, I–CVI was calculated for 64 fall risk items. Each expert categorized the items using a 4-point scale (1 = not essential, 2 = somewhat essential, 3 = quite essential, and 4 = highly essential). We then identified the ratio of the number of agreements on each risk factor to the total number of reviews. The raters also identified additional risk factors as essential. Items with a low agreement were discussed and discarded or revised. This process was repeated 3 times until the results were close to 1.0.
Results.
The mean I–CVI index of four raters was initially .91. Seven items had 50% agreement, 9 items had 75% agreement, and 51 items had 100% agreement. The items that had a low agreement owing to a clarity issue were reworded. Finally, we included “other” in an open-ended question format to capture fall risks unique to individual homes. For this item, for example, if two risks were listed, the item was counted as two. The final I–CVI was .98 (Table 1).
Item Content Validity Index
Note. I–CVI = item content validity index.
We concluded that the 64 fall risk items have satisfactory content validity. Two items that had 50% agreement were outside grab bar at the door and unmarked or raised threshold. These items remained in the fall risk list as a result of a positive remark by a focus group participant who was poststroke and had installed a grab bar and a comment by another participant who had a problem with a high threshold. The subsequent psychometric tests used the validated fall-risk items.
Study 2: Test–Retest Reliability
Test–retest reliability is obtained by administering the same test on multiple occasions to a sample from the population of interest. High reliability indicates that the questions are formed in such a way that people respond to them in the same manner, when asked in the same environment and under the same conditions. In other words, it is a measure of the precision of the measurement (Lavrakas, 2008). Our hypothesis was that the HSSAT has very good test–retest reliability (>.80).
Procedures.
We used a one-sample, two-trial design with an interval of 1 wk. Inclusion criteria were participants who were age 65 or older, lived in their home, and could read English. Exclusion criteria were Alzheimer’s disease, mild to severe cognitive limitation, or residence in a long-term care or an assisted living facility. After we posted flyers in senior centers, 28 older adults responded. One respondent was not eligible, and data for another contained missing values; therefore, data from 26 participants were analyzed.
Participants were requested to use the HSSAT by themselves and identify fall risks that existed in their own home twice, 2 wk apart. The intraclass correlation coefficient, two-way random model (ICC2,1), was used for the total score. Because ICC concerns relative consistency, which is the consistency of the position or rank of individuals in the group relative to others, absolute consistency was also measured using the standard error of measurement (SEM) to identify consistency of scores of individuals. The SEM was calculated as follows:
where SD is the standard deviation of the scores from all participants (Weir, 2005). The SEM was used to identify the SEM %, which is the percentage of SEM of the mean of all measurement from both sessions.
Results.
Demographic characteristics for 26 participants are summarized in Table 2. The mean total fall risk factors were 4.5 (SD = 3.3) for the first trial and 4.8 (SD = 3.3) for the second trial. The ICC2,1 was high, .972 (p < .001; Zegers et al., 2010). Between the two trials, the difference in identified risk factors (.31) was not statistically significant, t (25)= 1.990, p = .58. The SEM was 0.39, which represents 8.2% of the mean of all cases.
Demographic Characteristic of Participants in Various Tests
The number of fall risks the 26 participants identified in each area was the same in two trials for hallway (8), living room (19), kitchen (23), bathroom (24), and stairs (6). Risk factors identified for front door and yard and entrance to back or side door increased by 1 at the second trial (8 to 9), those for bathroom increased by 1 (13 to 14), and those for basement increased by 2 (9 to 11). The minor changes show the assessment's stability in the HSSAT. These results suggest that the HSSAT has high test–retest reliability as measured in community-dwelling older adults 2 wk apart.
Study 3: Interrater Reliability
Interrater reliability concerns the degree to which different raters give consistent estimates of the same phenomenon (Web Center for Social Research Methods, 2006). Initially, we planned to conduct an interrater reliability test between older adults assessing the same home. Instead, because we encountered difficulty finding a person whose home could be evaluated by another older person, we tested interrater reliability between two occupational therapists who were trained for home safety assessment using older adults’ homes. The hypothesis was that interrater reliability between two occupational therapists who have experience in home assessment would be good.
Procedures.
Two occupational therapists conducted the interrater reliability test by assessing the same home. Interrater reliability between the two occupational therapists was tested in five older adults’ homes. The occupational therapists, who have similar experience in home assessments over the past 3 yr, visited homes within 1 wk at the homeowner’s convenience. Before visiting homes, they were provided with the HSSAT booklet and discussed the procedure of assessment with researchers. In each home an occupational therapist (Rater 1) first interacted with and observed an older adult for about 20 min, then using the HSSAT, the occupational therapist assessed risk factors in each area, taking another 20 min. A second occupational therapist (Rater 2) used the same method. The results were not shared between the two occupational therapists until the analysis was completed. A two-way random method for the ICC2,1 was used. Agreement in all risk factors was calculated using the κ statistic.
Results.
For Rater 1, the mean number of risk factors identified was 4.4 (SD = 1.9); for Rater 2, the mean was 4.8 (SD = 4.6). Rater 2 tended to identify more risk factors than Rater 1, and the total risk factors between the two raters did not completely agree for 4 of the 5 participants, when the total risk factors were summed for each home environment. However, the ICC2,1 was good, .893 (Zegers et al., 2010). As a general guideline, Portney and Watkins (2009) suggested that ICC values >.75 are indicative of good reliability, although this cutoff point should not be considered an absolute standard.
One difference between the two raters occurred in the kitchen. Rater 2 considered a dog coming into the kitchen when the homeowner was cooking a fall hazard, whereas Rater 1 viewed the dog not as a fall risk because the dog was big, old, and quite immovable under the kitchen table. Another difference involved a bathroom. Rater 2 thought not having a grab bar near the toilet was a fall risk, whereas Rater 1 thought there would be no room for the homeowner to move if a grab bar were installed. The third difference concerned a hallway. Rater 2 thought the way the door to the basement was installed was a fall risk, not leaving much room to maneuver, whereas Rater 1 did not notice a problem. The last difference was for electric cords in a bedroom. In this case, Rater 1 considered electric cords a fall hazard because there were three separate cords on the floor in the corner, whereas Rater 2 did not think they were a hazard because the person rarely used the corner.
Although no perfect agreements were obtained for the total risk factors for 4 participants, of 336 fall risk factors assessed in 5 participants, including one other hazard, the two raters agreed on 324 risk factors. Removing a chance agreement, the κ statistic was .91, which is considered high agreement. We conclude that the HSSAT has good interrater reliability among occupational therapists.
Study 4: Construct, Convergent, and Discriminant Validity
Construct validity refers to the validity of inferences that observations or measurement tools actually represent or measure the construct being investigated (Polit & Beck, 2004). Because constructs have a theoretical basis, an instrument’s validity can also be assessed by using it to test a specific hypothesis that supports theory (Portney & Watkins, 2009). The HSSAT provides a list to measure fall risk factors at home; therefore, if older adults use the HSSAT booklet, they should be able to identify fall risks more than without using the HSSAT booklet.
We also tested convergent and discriminant validity. Convergent validity indicates that two measures believed to reflect the same underlying phenomenon will yield similar results or will correlate highly. Discriminant validity indicates that different results or low correlation is expected from measures that are believed to assess different phenomena. For convergent validity, fall risks identified by the HSSAT booklet and the CDC’s checklist were correlated. For discriminant validity, fall risks at home and fear of falling were correlated. Fall risks at home are existing home environmental hazards that may cause a fall, whereas fear of falling is one’s perception of behaviors and states that may cause a fall. The hypotheses were, first, that risk factors identified using the HSSAT and CDC’s checklist have a high correlation and, second, that risk factors measured by the HSSAT and fear of falling have a low correlation.
Participants and procedures.
Participants were 34 older adults who had a history of a fall within the past year and reported that they had visited a primary care physician or an emergency department after their unintentional fall. They were recruited by a convenience sampling method from hospitals, senior centers, and religious facilities by means of meetings, flyers, and newsletters. Participant demographic characteristics are described in Table 2.
To test construct validity, the 34 older adults assessed their own homes first with the CDC Home Safety Checklist. They were asked to check items and write additional fall risks that they identified. Within 2 wk, they were asked to assess their homes again, using the HSSAT booklet. We analyzed their answers using a paired t test.
For convergent validity, the data were analyzed using Pearson correlation coefficients. For discriminant validity, at the second assessment when participants used the HSSAT booklet, the Fall Efficacy Scale (Tinetti, Richman, & Powell, 1990) was administered to assess fear of falling. These results were correlated using a Pearson product–moment correlation.
Results.
Using the HSSAT, participants identified a mean of 11.2 fall risks (SD = 10.6); without the HSSAT, they identified a mean of 7.2 risk factors (SD = 6.1). The difference in mean identified risk factors between the two methods was 4.1, which was statistically significant, t(33) = 2.764, p = .005, using a one-tailed test. For convergent validity, the correlation between the two methods was .650 (p < .001), which is high. The correlation between the fall risk factors and fear of falling was .103 (p = .506). These results indicate that construct validity for the HSSAT is established.
Study 5: Responsiveness to Change
Responsiveness to change is the ability of a measure to change in home risk factors over a predetermined time frame (Husted, Cook, Farewell, & Gladman, 2000). The HSSAT booklet was created not only to identify fall risk factors in the home environment but also to encourage changes in users’ behaviors to make their home safer. The results of the behavior changes should be captured using the HSSAT. If participants recognize fewer risk factors after their improvement in home safety, we can conclude that the HSSAT is sensitive enough to identify the changes that took place. We hypothesized that the risk factors identified by older adults would be reduced at the postassessment because of their behavior in making their home safer (as determined according to their records of what had been done).
Participants and procedures.
This test used a one-sample pretest–posttest design with the interval of 6 wk. A total of 107 community-dwelling older adults (age ≥65 yr) were recruited through the same convenience sampling method described for the construct validity test. The participants were different from those in previously mentioned tests. Three participants withdrew during the study period, and data from 104 older adults were analyzed. Their demographic characteristics are summarized in Table 2. At pretest, an occupational therapist provided and explained how to use the HSSAT. Then, participants assessed their own homes using the HSSAT. Four weeks later, they were asked to reassess their homes using the HSSAT. Statistical analyses included the calculation of the standardized response mean (SRM), which was calculated as the individual change in score divided by the standard deviation of change (Crosby, Kolotkin, & Williams, 2003). Cohen’s interrelation of effect sizes was then used to interpret it (Cohen, 1977).
Results.
At baseline, a mean of 8.4 risk factors (SD = 7.1) was identified, and 4 wk later, a mean of 5.5 risk factors (SD = 5.5) were recognized (Figure 1). The difference (2.9) was divided by the standard deviation of the change score (5.15). The SRM was 0.57. The effect size for this is considered moderate. We concluded that the responsiveness of the HSSAT is moderate as measured in older adults after 4 wk.
Changes in the means of identified risk factors using the Home Safety Self-Assessment Tool.
Discussion
In this study, we tested the psychometric properties of the recently developed HSSAT and concluded that these properties have been established. Because the HSSAT focuses on older adults, we tested its psychometrics on older adult participants, except for content validity and interrater reliability.
We found that the HSSAT has satisfactory content validity (I–CIV = 0.98), as determined with the four experts who participated in this study. We also found that the HSSAT has high test–retest reliability (ICC = .97, SEM = 0.39) and interrater reliability (ICC = .89, κ = .91) for 336 items. The HSSAT has construct validity: The number of identified risk factors was greater among older adults using the HSSAT than among those not using it (p = .005). We found a high correlation (r = .65) for the number of risk factors identified by older adults using the HSSAT and the CDC brochure and a very low correlation (r = .103) between the number of fall risk factors and fear of falling. Finally, responsiveness to change measured in older adults was moderate (SRM = 0.57) over 4 wk.
The HSSAT was more effective in identifying risk factors than the CDC’s checklist, possibly because the HSSAT lists recurring risk factors in each area of the home, whereas the CDC checklist only has broad categories such as floors, stairs and steps, kitchen, and bedroom. Older adults may not remember general risk items when they go to different areas in the home to assess risk factors.
The HSSAT is an instrument to identify fall risk factors inside the home. By exposing these risk factors, it appears that older adults raise their awareness for fall risk factors in general. One of the users reported,
After I reviewed fall risks in the [HSSAT] book, I am more aware. We went to a restaurant the other day, and some ladies put their purses under their chairs. I noticed that one of their shoulder straps was in the walking path and I thought someone might trip on it, so I asked her to put it under the chair. You see, I am more mindful now than before.
Moreover, use of the HSSAT can bring about actual changes in fall risk reduction. In this study, responsiveness to change was very good, as measured by the reduction in identified fall risk factors in a home environment. The reduction was 2.9 on average over 6 wk, but this change will be greater with a longer intervention period. If home modification service providers were to be involved (e.g., contractors installing railings and grab bars), it could take longer than 1 mo until the job is completed. Another study revealed that although more than half of home safety improvement was made within about 1 mo, approximately 78% of home improvements were made when observations were extended to 3 mo (Tomita & Rajendran, 2010).
The tests we conducted showed rather large standard deviations in relation to the mean scores. A standard deviation that is close to the mean score is considered large. Some participants recognized many risk factors, whereas others did not notice as many, indicating that older adults in this study are not a homogeneous group and some variables influenced the way they responded to our questions about fall risks. These variables could include direct experience with falls, vicarious experience with falls, income status, presence or absence of caregivers, motivations, and educational status. Using homogeneous samples would result in statistical significance. Identifying these variables helps us discern those who can benefit from using the HSSAT and develop strategies for fall prevention education for specific populations.
One of the study’s limitations is that although the HSSAT has also been created for older adults’ informal caregivers, caregivers were not used to establish psychometrics. Educating caregivers about fall risk factors may be more effective than educating older adults alone.
Another limitation is found in two sections of the HSSAT booklet that were developed to facilitate behaviors to make the home safer. This booklet was originally developed for residents in Erie County, NY, and it included information about home improvement service providers and organizations that provide free services; however, this information is not helpful for HSSAT users who live outside of Erie County. Therefore, responsiveness to change without these informational supports may not be as good when tested with older adults who cannot access them. We assume that the number of actual home safety improvements may be lower for people who are not given referrals to service providers. Thus, the difference between the pretest and posttest may not be significant. If AAAs provided this type of information, older adults and caregivers would likely benefit.
A participant who fell in her garage, hit her head, and was hospitalized for 3 days suggested that a garage area should be added to the HSSAT. None of the current home safety instruments include a garage, to our knowledge. The cause of the participant’s fall appeared to be clutter in the garage. Inclusion of garage hazards should be examined in future studies.
The final study limitation is the lack of interrater reliability of home fall risk assessments by older adults. Instead, we used occupational therapists who assessed only five homes. We found it difficult to find older adults who were willing to offer their homes for these tests. In one county, an AAA created rooms that resemble drawings in the HSSAT to educate home safety assessors and residents. This educational tool may offer a setting to test interrater reliability. However, the lack of consensus described in this article clearly indicates that reasonable people can disagree on what constitutes a home hazard. Also, even though people recognize risk factors, if there are no clear solutions, they may not mitigate a given risk factor.
The HSSAT does not provide detailed considerations and solutions for unique situations. Therefore, we strongly believe that occupational therapists’ insight into the interaction between an individual and the environment can achieve higher levels of solutions and recommendations for fall risks in a home environment. Occupational therapists also usually have professional contacts and networks to discuss cases, which can enhance occupational therapy interventions. In addition, the interrater reliability between older adults and caregivers should be tested in the future.
Community-dwelling older adults cannot access occupational therapists unless they need rehabilitation. Therefore, to translate occupational therapists’ knowledge about home safety to older adults, the HSSAT has been developed and can be used as an educational and assessment tool with established psychometric properties.
Conclusion
Content validity, test–retest reliability, interrater reliability, construct validity, and responsiveness to change have been established for the HSSAT. The HSSAT booklet can be used to identify fall risks and facilitate fall hazard reduction in a home environment for community-dwelling older adults.
Implications for Occupational Therapy Practice
The results of this study have the following implications for occupational therapy practice:
Occupational therapists can use the HSSAT with confidence to assess home hazards to prevent falls.
Use of the HSSAT raises older adults’ awareness of safety hazards in their home environment by returning locus of control to residents when identifying environmental hazards and plans for correction within their home and providing an opportunity to analyze the activity and environment interface for safe engagement in home-based occupations
The HSSAT can be used with older adult populations where the occupational therapist is a consultant to a medical, nonmedical, or social service agency in a community setting to prevent falls.
Footnotes
Acknowledgments
This study was supported by grants from the Health Foundation for Western and Central New York. We are especially thankful for Christine Klotz’s and Amanda Norton’s guidance. The Health Foundation for Western and Central New York is an independent, private foundation whose mission is to improve the health and health care of the people and communities of Western and Central New York. Our appreciation also extends to the Mark Diamond Research Fund for financial contribution and Meals on Wheels for Western New York and the Visiting Nursing Association of Western New York for data collection.