Medical students’ intention to integrate digital health into their medical practice: A pre-peri COVID-19 survey study in Canada

Theoretical model

To address the above-mentioned research questions, we first developed a theoretical model. One of the main theoretical foundations of this study is Triandis’ theory of interpersonal behavior ³⁸ which posits that individuals’ behavioral intention is influenced by their beliefs toward the behavior. Triandis defines beliefs as assessments of what an individual thinks about the object of interest There is no implied goodness or badness in beliefs, but only an assessment of what one thinks exists or does not exist For instance, an individual may hold a belief that computers in general contribute to the improvement of society. In this study, medical students’ beliefs were operationalized regarding AI-related technologies specifically rather than dHealth technologies generally, as AI-based tools have the most potential to fundamentally alter and significantly improve the practice of medicine.^23,39,40 We posit that the stronger the medical students’ beliefs that AI can positively impact the medical profession as well as their own medical practice, the greater their intention to integrate dHealth technologies (including AI tools) into their future practice.

Second, we postulate that perceived facilitating conditions are another critical determinant of medical students’ behavioral intention related to dHealth adoption. Originating in the technology acceptance model (TAM), a theory that models how users come to adopt a new technology, ⁴¹ facilitating conditions are external factors that influence an individual's perceptions of the difficulty with which a task (eg, use of dHealth technologies) may be performed.^42–46 In the present study, facilitating conditions are operationalized as students’ level of exposure to or experimentation with dHealth technologies during their medical education. We posit that the more medical students experiment with dHealth technologies during their degree, the higher their intention to integrate dHealth in their medical practice.

Third, our theoretical model includes perceived usefulness, another TAM variable, which is defined as the degree to which a person believes that using a particular technology or information system would enhance their job performance. ⁴¹ Adapted to the present study context, perceived usefulness refers to medical students’ perceptions of the importance and relevance of integrating dHealth into the medical curriculum. We posit that the more one thinks that all students should receive training in dHealth as part of their medical degree, the more their intention to integrate dHealth in their own medical practice.

Last, following prior research on dHealth training [eg,^18,32,47] as well as various studies testing the TAM [eg,^42–44] another construct, called individual background, was added to our research model. This construct was operationalized as a composite of three individual characteristics: gender, age, and academic level. Due to the exploratory nature of this study, we simply assert that students’ individual background is likely to influence their views and perceptions on dHealth technologies.

The theoretical model to be empirically tested in this study is shown in Figure 1. Given the relationships postulated by the theoretical lens mobilized above as well as the empirical relationships identified in the extant literature, seven hypotheses were formulated as summarized in Table 1. The first two hypotheses emanate from the TAM wherein medical students’ individual background is deemed to influence their beliefs about the role of dHealth technologies (more particularly AI) in medicine and their perceived dHealth education needs. For their part, hypotheses 3, 4 and 5 also emanate from the TAM wherein external facilitating conditions are deemed to influence beliefs toward a behavior, perceived usefulness, and behavioral intention. Hypothesis 6 is derived from the theory of behavior proposed by Triandis and states that medical students’ beliefs are deemed to positively influence their behavioral intention. The final hypothesis also originates from the TAM and posits a positive relationship between perceived usefulness and behavioral intention.

OPEN IN VIEWER

Figure 1.

Theoretical model.

OPEN IN VIEWER

Table 1.

Research hypotheses.

#	Hypotheses
1	Medical students’ individual background will likely influence their beliefs about the role of AI in the future of medicine
2	Medical students’ individual background will likely influence their perceptions relating to the importance of integrating dHealth into medical education
3	The greater the students’ experimentation with dHealth during their medical education, the stronger and more positive their beliefs about the role of AI in the future of medicine
4	The greater the students’ experimentation with dHealth technologies during their medical degree, the greater their perceived dHealth education needs
5	The greater the students’ experimentation with dHealth technologies during their medical education, the greater their intention to integrate dHealth into their medical practice
6	The stronger the medical students’ beliefs about the positive impact of AI on medicine, the greater their intention to integrate dHealth into their medical practice
7	The stronger the medical students’ perceptions related to the inclusion of dHealth into the medical curriculum, the greater their intention to integrate dHealth into their medical practice

Setting and data collection

The present study was conducted at the University of Montreal's (UM) medical school in Canada. During the 5-year long undergraduate medical curriculum, no formal dHealth education or training is provided to students. However, students have access to the EDUlib online training platform which offers educational content on a variety of subjects including health and information technologies, as well as to symposia and conferences on different aspects of dHealth. Also, workshops on mobile health and EMR systems are mandatory for students in their clerkship, a stage at which students are called upon to experiment with different dHealth technologies. Finally, issues related to the use of social media, email, and mobile applications by healthcare professionals are also covered in elective seminars.

The study population consisted of 1367 UM medical students. The survey questionnaire was administered in two phases, that is, an initial survey (t₀) in February 2020, before the Covid-19 pandemic, and a replication survey (t₁) in January 2021, during the pandemic. It was distributed to all students via the medical school's mailing list The study was also promoted via social media groups that are only accessible by UM medical students and the local medical student association sponsored the survey via its newsletter. There was no incentive for students to fill out the online questionnaire and there were no negative consequences if students did not participate.

The invitation contained a hyperlink directing the participants to the questionnaire through a secure Web site. The online questionnaire was developed on the Qualtrics survey platform. Qualtrics complies with applicable data privacy laws in its role as a data processor of customer data, as indicated on the company's website.

Questionnaire development

We were unable to locate any pre-existing questionnaire that assessed the variables included in our research model; hence, we decided to develop our own instrument. The survey design underwent several rounds of iteration, and final validation was performed with a group of 10 UM medical students who were excluded from the sampling population. The final survey instrument consisted of 70 5-point Likert questions and 8 yes/no questions. The four tables located in Appendix I provide essential information on the questionnaire's content and on measurement items and scales.

Statistical analysis

The data were first analyzed through descriptive statistics (mean, standard deviation, percentage) and further examined through analyses of variance and principal component analyses, using the IBM SPSS software v28. Component-based structural equation modeling (SEM) was then used to test the research hypotheses (cf. Figure 1). As implemented in the SmartPLS software, the partial least squares (PLS) technique was chosen for its robustness with regard to the distribution of residuals and its greater affinity for exploratory rather than confirmatory research purposes when compared to covariance-based SEM techniques such as AMOS and EQS. ⁴⁸

Ethics approvals

The survey questionnaire was approved by the UM's ethics committee on October 29, 2019 (#CERSES-19-108-D). Informed consent was obtained from all participants. All methods were carried out in accordance with relevant guidelines and regulations.

Demographics

A total of 184 students responded to the initial survey at t₀ (13%), whereas 138 responded to the replication survey at t₁ (10%). As shown in Table 2, most participants were female (65% at t₀ and 70% at t₁). The mean age was 23 years, which is comparable to the average age of medical students at UM. While our sample was evenly distributed between the first and fifth year of medical studies at t₀, there were fewer fourth- and fifth-year medical students at t₁.

OPEN IN VIEWER

Table 2.

Profile of the respondents.

Medical students’ background		Pre-COVID-19		Peri-COVID-19
		t0 (n = 184)		t1 (n = 138)
		N	%	N	%
Academic level	Preparatory year	40	22%	28	20%
	First year preclinical	36	20%	32	23%
	Second year preclinical	43	23%	55	40%
	First year clerkship	33	19%	8	6%
	Second year clerkship	32	17%	14	10%
Gender	Female	119	65%	92	70%
	Male	65	35%	40	30%
Age	Mean	22.9		22.6
	Standard deviation	3.5		2.7
	Minimum	18		18
	Maximum	38		35

Descriptive analysis

For analytical purposes, individual dHealth technologies and applications were grouped under five technology “bundles” that were named basic IT systems (eg, electronic medical records, clinical information systems), advanced dHealth (eg, robotics, virtual reality), telehealth (eg, teleconsultation, tele-expertise), AI-related technologies (eg, artificial intelligence, machine learning) and mobile applications (eg, UpToDate, BMJBestPractice).

First, with regard to the questions relating to experimentation with dHealth technologies, a large majority of the medical students had little occasion to experiment with advanced dHealth, telehealth, and AI-related technologies in the course of their medical curriculum (see Table A1 in Appendix I). A minority of the participants reported having been somewhat or very exposed to basic IT systems (i.e. EMR, CIS, and iEHR systems) and two specific mobile apps, namely, UpToDate and the Quebec Health Technology Assessment (HTA) Institute's mobile app. ³

Second, a vast majority of respondents agreed that all students should receive formal dHealth education as part of their medical degree. As shown in Table A2 in Appendix I, medical students believe that the most important dHealth education needs concern, by order of importance, basic IT systems, telehealth, AI-related technologies, and robotics. A majority of respondents, however, perceive lesser need concerning other advanced dHealth technologies including blockchain, internet of things, virtual reality, and augmented reality.

Third, a large majority of the sampled students at both t₀ and t₁ are observed to have strong beliefs toward AI-related technologies, be it in terms of these technologies’ impact on the medical profession, in general, as well as on many medical specialties such as radiology and anatomopathology (see Table A3 in Appendix I). Moreover, most students indicate that they expect to use AI-based tools in support of one or more of their future medical activities including image and data analysis, diagnosis and, to a lesser extent, prognosis.

Fourth, a large majority of students declared an intention to integrate dHealth into their medical practice. As shown in Table A4 in Appendix I, this intention is most important with regard to disease prevention, diagnosis, and treatment, followed by patient communication and consultation, and patient monitoring and follow-up activities.

Last, when comparing the variable means between the two samples (t₀ and t₁), as presented in Table 3, t-test analyses confirm statistically significant differences (P < 0.001) on a few research variables. For one thing, medical students in the replication study (t₁), on average, experimented more with telehealth technologies and, conversely, experimented less with basic IT systems and mobile applications than those in the initial survey (t₀). This last result may be due to the fewer proportion of fourth- and fifth-year students who participated at t₁ in comparison to t₀. Further, medical students surveyed at t₁ expressed a greater need for education on telehealth technologies than those surveyed at t₀. This specific need was ranked first by students in the replication study, whereas it was ranked second by participants in the initial study. Here, one may tentatively explain these differences by the advent of the COVID-19 pandemic, which has brought telemedicine to the forefront of medical training and medical practice. ⁴⁹

OPEN IN VIEWER

Table 3.

Comparison of medical students’ views and intention between t₀ and t₁.

Research construct	Pre-COVID-19	Peri-COVID-19	T-test
Research variable	(n = 184)	(n = 138)	(two-tailed)
	mean	mean
Individual Background
Age	22.9	22.6	1
Gender	0.65	0.7	-0.9
Academic level	2.9	2.6	1.9
Experimentation with dHealth technologies
Basic IT systems	1.8	1.4	4.2***
Advanced dHealth	1.2	1.1	2.4**
Telehealth	1.2	1.5	−4.6***
AI-related technologies	1.3	1.2	2
Mobile applications	1.5	1.3	3.6***
Importance of dHealth in medical curriculum
Basic IT systems	4.1	4.1	0.3
Advanced dHealth	3.4	3.4	-0.1
Telehealth	3.7	4.1	−5.0***
AI-related technologies	3.5	3.5	0.3
Beliefs about impact of AI-related technologies
On the medical profession	3.6	3.5	1.2
On various medical specialties	3.4	3.3	0.8
On one's own medical practice	3.9	3.6	0.8
Intent to integrate dHealth in medical practice
Patient communication and consultation	3.4	3.3	0.6
Patient monitoring and follow-up	3.3	3.1	1.5
Disease prevention, diagnosis and treatment	3.6	3.3	2.1**

** P < 0.05; *** P < 0.001.

Measurement model

Component-based SEM was used to empirically test our set of research hypotheses. The PLS technique was selected because it is suited to measurement models such as ours that include both exogenous (reflective) and endogenous (formative) constructs. ⁵⁰ Precisely, whereas the research construct “individual background” is modeled as being “formative” given its composite and multidimensional nature, the four other constructs are modeled as being “reflective”. ⁵¹

The initial step consisted of estimating the measurement model at t₀ and t₁ using the SmartPLS software. We started by assessing the internal consistency of the measures (manifest variables) as well as the unidimensionality, reliability, predictive validity, and discriminant validity of the four reflective research constructs (latent variables). As shown in Table 4, all the Cronbach α values (except one at t₁) were above the .70 threshold, thus confirming the internal consistency of the research variables. The unidimensionality of a reflective construct is assessed by looking at each of its indicators’ loading (λ), the threshold being 0.40 for newly developed scales. ⁵²

OPEN IN VIEWER

Table 4.

Psychometric properties of the research variables and constructs.

	Pre-COVID-19 (t0)				Peri-COVID-19 (t1)
Research construct		CR b	AVE c			CR b	AVE c
Research variable	α a			VIF d	α a			VIF d
Individual background		-	-			-	-
Age (yrs.)	-			1.11	-			1.23
Gender (0: male, 1: female)	-			1.02	-			1.08
Academic level (1 to 5)	-			1.12	-			1.2
Experimentation with dHealth technologies		0,81	0.47			0,81	0.48
With basic IT systems	0.86			-	0.87			-
With advanced dHealth	0.88			-	0.81			-
With telehealth	0.78			-	0.69			-
With AI-related technologies	0.84			-	0.84			-
With mobile applications	0.88			-	0.86			-
Importance of dHealth in the curriculum		0.90	0.70			0.93	0.76
On basic IT systems	0.93			-	0.97			-
On advanced dHealth technologies	0.93			-	0.93			-
On telehealth	0.86			-	0.87			-
On AI-related technologies	0.84			-	0.84			-
Role of AI in the future of medicine		0.91	0.76			0.92	0.80
For the medical profession	0.82			-	0.78	-		-
For the medical specialties	0.82			-	0.79			-
For their medical practice	-			-	-			-
Intention to integrate dHealth in future practice		0.98	0.95			0.99	0.98
For patient communication and consultation	0.94			-	0.97			-
For patient monitoring and follow-up	0.91			-	0.95			-
For disease prevention, diagnosis and treatment	0.95			-	0.97			-

^a

Cronbach alpha coefficient of reliability.

^b

CR: composite reliability. [CR = (Σλ_i)²/((Σλ_i)² + Σ(1-λ_i²))] .

^c

AVE: average variance extracted by a construct from its associated variables. [AVE = ∑λ_i² / n] .

^d

VIF: variance inflation factor. VIF = 1 / (1 – R_i²), where R_i² is the unadjusted R² obtained when variable i is regressed against all other variables forming a construct.

Moreover, the reflective constructs’ composite reliability (CR) must be above the 0.7 threshold, which is the case for all four reflective constructs. There is also evidence of the convergent validity of these constructs because their average variance extracted (AVE) is above the 0.50 threshold, except for the “Experimentation with dHealth” construct which is just below this threshold (0.47). Despite this, it was decided not to remove any of this last construct's indicators from its measure, given that it demonstrated adequate reliability and unidimensionality. Finally, there is evidence of discriminant validity, that is, of the extent to which a research construct differs from other constructs. In the case of the four reflective constructs, the shared variance between each construct and the other constructs was found to be less than the AVE from its indicators. ⁵² In the case of the sole formative construct, individual background, the fact that it shared less than 70% variance with the other construct in the measurement model, and thus correlated less than perfectly with these constructs, was again indicative of strong discriminant validity. ⁵¹

Next, as the usual validity criteria for reflective constructs are inapplicable to a formative construct, one must instead verify that there is no multicollinearity among the formative construct's indicators. One uses the variance inflation factor (VIF) statistic to do so, a common rule being that a variable's VIF value be less than 3.3, or in other words, that less than 70% of the variance in the variable be jointly explained by the other variables. ⁵³ As shown in Table 4, the outer VIF values estimated by PLS for the three formative indicators of the individual background construct varied from 1.02 to 1.12 at t₀ and from 1.08 to 1.23 at t₁, well below the 3.3 threshold, thus indicating the absence of multicollinearity.

Measuring the research variables through a self-administered questionnaire with a single respondent poses a risk of common method bias (CMB). ⁵⁴ As precautionary measures, we chose different question formats and scale types. Further, we examined the correlation matrix of the five research constructs to determine if any two constructs correlated above 0.90, as this could signal the presence of CMB in the data. ⁵⁵ In our case, all construct correlations (not shown here) were well-below this threshold. We also employed Harman's single-factor test to check for CMB, examining the unrotated factor solution for all variables in the measurement model. As multiple factors emerged from the factor analysis and as no single factor accounted for 50% or more of the covariance among the variables, this further suggests the absence of CMB. ⁵⁶

Causal analysis at t₀

Next, the causal paths inferred from the conceptual framework were tested by assessing the path coefficients (β) estimated by the SEM procedure as executed by the SmartPLS software. As shown in Figure 2, the performance of the theoretical model at t₀ that interrelates the five research constructs is indicated by the strength and significance of the βs and the proportion of explained variance (R² = 0.53), as befits PLS's focus on prediction and concern with generalization. ⁵⁷

Hypothesis 1 (partly confirmed). Given the results of the initial causal analysis (t₀) provided by the SEM procedure, an initial finding lies in the negative and highly significant path coefficient that links the medical students’ characteristics to their beliefs about the role of AI in the future of medicine (β = −0.26, P < 0.05). As shown in Figure 2, the primary explanatory characteristic is the academic level, as students who are in their preparatory and preclinical years have stronger beliefs about the positive impact of AI on the medical profession, various medical specialities, and their own medical practice. Age and gender were not significantly associated with students’ beliefs about AI technologies.

OPEN IN VIEWER

Figure 2.

Causal analysis results at t₀.

Causal analysis at t₁

As depicted in Figure 3, the causal paths inferred from the research model were tested anew at t₁ by assessing the path coefficients (β) estimated by the SEM procedure. As indicated by the strength and significance of the βs and the proportion of explained variance, the performance of the theoretical model was found in the replication study to be superior to that of the initial study (R²_t0 = 0.53 vs. R²_t1 = 0.69). These results provide further confirmation of the validity and predictive ability of our theoretical model. Moreover, the advent of the COVID-19 pandemic appears to have had no impact on the theorized relationships between the research constructs, as replicating the causal analysis produced results similar to those observed at t₀, that is, in terms of the strength and significance of these relationships. The reader may thus essentially refer to the above presentation of the causal analysis at t₀ for further analysis of the t₁ results.

OPEN IN VIEWER

Figure 3.

Causal analysis results at t₁.

Principal findings

The present study provides a clear and concise explanation of medical students’ intention to integrate dHealth technologies into their practice. We found the sampled students, especially those who are in their preparatory or preclinical years, to have had little occasion to experiment with dHealth technologies during their medical studies. A plausible explanation might be that during their preclinical years, students do not have direct access to EMRs and other clinical information systems available in hospital settings. The lower scores at t₁ might be related to the pandemic where students were even less exposed to hospital practice (and hospital IT systems), with many parts of the curriculum being offered virtually.

This lack of exposure may explain why a vast majority of respondents felt that all medical students should receive dHealth training as part of their formal education. Further, most had strong beliefs about the positive impact of AI-related technologies in the future of medicine, in general, and the progress of various medical disciplines. Importantly, a majority of students declared an intention to integrate dHealth technologies, including AI-based tools, into their medical practice.

The main differences observed between t₀ and t₁ brings telehealth, and teleconsultation in particular, to the forefront of medical education. Indeed, the pandemic's advent appears to have shifted medical students’ view of telemedicine, now considered to be an important aspect of their dHealth education and future medical practice. Moreover, this shift is made at the expense of another component, basic IT systems (eg, EMRs), that could be now considered as a ‘given’, ⁵⁹ that is, as being fully and seamlessly integrated within medical education and medical practice.

Last, as predicted by our theoretical model, the combination of three factors (experimentation with dHealth, perceived importance of integrating dHealth into medical curriculum, and beliefs about AI in medicine) explains, in both phases of the study, medical students’ intention to integrate dHealth into their practice.

Study contributions and implications

The factors included in the theoretical model constitute important descriptive, predictive, and explanatory keys upon which to reflect on the issue of dHealth education in medical schools. The proposed model may thus constitute an initial conceptual framework for researchers and practitioners concerned with informing, motivating, and preparing medical students to make effective use of dHealth technologies, in general, and of AI tools, in particular. In this regard, it appears that the emphasis on basic IT systems such as EMRs still remains, whereas telehealth appears to have taken on added importance with the advent of the COVID-19 pandemic. Furthermore, as the students’ future use of dHealth technologies is determined above all by their beliefs about the positive impact of AI on modern medicine, it ensues that the introduction of AI training in medical curricula should be further considered and investigated. ⁶⁰

This study has several practical implications. For one thing, our study provides medical schools with a conceptual template with which to make a strategic assessment of their dHealth situation and thus obtain actionable insights as to the technologies to be included in the medical curriculum. Our findings also allow us to make a few recommendations to medical schools in Canada and elsewhere wishing to better serve their students through the mindful integration of dHealth within the curriculum: (1) integrate dHealth training as part of the doctorate in medicine; (2) create one or several specialized diplomas in dHealth (eg, telehealth, AI in health); (3) set-up a working group within each faculty of medicine, comprised of professors and students of all levels, to periodically review dHealth training needs in light of existing and emerging technologies; (4) periodically assess students’ satisfaction with their dHealth training, and make the necessary adjustments; and (5) foster collaboration and exchange between medical students and those from other faculties (eg, computer science, software engineering) interested in dHealth technologies through the organization of “hacking health” type of events.

Importantly, our results have implications for key institutional actors in the development and deployment of dHealth technologies in Canada, as they seek to guide and support medical schools in their integration of these technologies within the medical curriculum. For instance, the Association of Faculties of Medicine of Canada (AFMC) and the Canadian Medical Association (CMA) could collaborate on developing and implementing an evidence standards framework for dHealth technologies, like the one developed in the United Kingdom by the National Institute for Health and Care Excellence. ⁶¹ This framework would enable a more agile evaluation of these technologies in medical school settings, as well as render them more meaningful to students and add value to dHealth education.

Study limitations and suggestions for future research

The results of this study must be interpreted with caution due to its inherent limitations. Given the nature of the sample, its representativeness in relation to all Canadian medical students limits the scope of these results. Future research could investigate the nature and scope of dHealth education offered in medical schools located elsewhere in Canada as well as in other countries and compare students’ level of proficiency in dHealth as well as their intention to integrate dHealth technologies in their medical practice. It would also be important to conduct a follow-up survey once the COVID-19 pandemic is behind us. Moreover, the rather low response rate may have induced a non-response bias that is hard to evaluate. ⁶² Notwithstanding our aim for parsimony, the theoretical model could also be extended in future research by including other variables such as social influence and effort expectancy to be more comparable to the previously cited behavioral studies. Further, one could include, in addition to dHealth technologies and applications, the IT-enabled medical knowledge management capabilities such as e-healthcare intelligence and e-collaboration that students must develop to practice modern medicine and, hence, be both innovative and productive.^63,64