Consistency of Patient Preferences About a Secure Internet-Based Patient Communications Portal

Abstract

Internet-based secure communication portals (portal) have the potential to enhance patient care via improved patient–provider communications. This study examines differences among primary care patients’ perceptions when contemplating using, enrolling to use, and using a portal for health care purposes. A total of 3 groups of patients from 1 Midwestern academic medical center were surveyed at different points in time: (1) Waiting Room survey asking about hypothetical interest in using a portal to communicate with their physicians; (2) patient portal Enrollment survey; and (3) Follow-up postenrollment experience survey. Those who enroll and use a patient portal have different demographic characteristics and interest levels in selected portal functions (eg, e-mailing providers, viewing medical records online, making appointments) and initially perceive only limited improvements in care because of the portal. These differences have potential market implications and provide insight into selecting and maintaining portal functions of greater interest to patients who use the portal.

Keywords

patient portal secure Internet communications quality of care asynchronous communications

Effective patient–provider communication is key to ensuring high-quality and safe patient care. Secure Internet-based communication portals (portals) are being used increasingly by health care systems and providers to enhance asynchronous communications with patients outside of face-to-face interactions.^1-7 Because of its asynchronous nature, portal-based communication can play an important role in improving patient care quality and safety because it allows for 2-way communication and access to potentially important information outside of office visits or hospitalizations. Examples include more timely communication between patient and provider regarding rapidly deteriorating conditions such as heart failure; more timely medication adjustments, especially for newly prescribed drugs; quicker follow-up postdischarge communications and interventions; previsit communication of questions and concerns; and the potential for e-visits to substitute for office visits.

Although portals are intuitively appealing to enhance communication between patients and their providers, little is known about differences in patients’ preferences when they are asked to think about the relative importance of different functions of a “hypothetical” portal compared with preferences of patients who actually enroll for and use a portal. Understanding these differences is important to identify which patient groups may be more likely to take advantage of the asynchronous communication potential of patient portals as a tool to improve patient care quality and value. These differences are also important when decisions are being made about portal design, workflows potentially affected by the introduction of a portal, and ongoing portal operation.

The literature suggests a wide range of potential portal features, including secure electronic messaging with physicians,^1-15 nonurgent consultations,^16,17 access to lab and diagnostic test results,^{6,7,11,12,14,17,18} and requesting appointments,^12,18 referrals,^6,7,12 and prescription refills.^6,7,12 Patient expectations for disease-specific portal applications may include easy access to providers,^17-19 reliable information resources,^11,19,20,21 disease-specific education,^19,21,22 symptom monitoring,^11,19,21 reminders,^20,21 and personalized care.¹⁸ Patients also may have relatively high expectations for timely responses to their e-mails from providers^14,16,23 and timely reporting of diagnostic test results,^14,16,24 both of which may require significant changes in physician and office staff workflows. Finally, a common patient concern is maintaining the security and confidentiality of their health information.^11,13,17,18 Categories of potential portal functions can be summarized as follows.

patient–provider communications (e-mails, send and receive data);

making orders and requests (prescription refills, appointments, referrals, and forms);

patient and surrogate viewing of electronic health record (EHR), test results, and medication lists;

updates and inputs from patient (change in status/ clinical data from patient, previsit updates, online clinical history, and health risk appraisals/ screenings);

read/learn about disease-specific condition(s) and general health/wellness resources; and

patient reminders and alerts.

As more features are added, implementation and maintenance costs increase as well as unanticipated consequences such as the need for redesigned workflow processes, increased and/or uncompensated provider workload, increased patient expectations of provider responsiveness, and potential for breaches in patient data confidentiality.

Little is known about how closely the perceived desirability of portal functions that might be offered matches the desirability of portal functions for patients who enroll in and use portals. Using a pilot implementation of a portal at 1 Midwestern academic medical center, the authors explored differences in hypothetical interest in potential portal functions among primary care patients versus the interests and experiences of patients who actually enrolled in and used the portal.

Methods

The study methods were approved by the local institutional review board prior to data collection. A total of 3 patient surveys were conducted (hereafter referred to as Waiting Room, Enrollment, and Follow-up surveys) as part of the academic medical center’s pilot implementation of a patient portal. All participants were adult patients from family and community medicine or general internal medicine clinics. Survey questionnaires were developed using the following process: (1) survey items were constructed based on literature review, functions of the portal system being considered for implementation, and the research team’s expertise; (2) response options were developed to assess frequency of use and relative importance of items using Likert-type scales (eg, 5-point scale: not at all important to very important); and (3) surveys were pilot tested followed by revision. Survey instruments are available on request from the authors. Figure 1 displays a schematic of the 3 survey groups and the specific data (as denoted by the shaded boxes) analyzed in this article.

Figure 1.

Study samples

The Waiting Room survey was a paper instrument administered in the waiting rooms of 5 primary care clinics between February and March 2008, prior to implementing the patient portal. It asked about current use of the Internet in general and about health care–related purposes specifically and respondents’ level of interest in different portal functions that might be included in a future portal. To obtain a representative sample, patients were recruited from 5 primary care clinics on different days of the week and during different clinic sessions (morning, afternoon, and evening). Of the 713 patients asked to participate, 638 (89%) completed surveys. No patient identifiers were collected. Because patient portals rely on use of the Internet, the authors analyzed only the responses of current “Internet users” (N = 499; 78%), identified as those who reported use of the Internet for any purpose in the previous month (eg, sending or receiving e-mails, shopping, looking up information).

The Enrollment survey was conducted between late November 2008 and April 2009; 3 clinics served as pilot sites for portal implementation. The Enrollment survey asked patients about their current Internet use for health-related purposes and their level of interest in using the portal’s various functions. Up to March of 2009, enrolled patients received a portal-generated message asking them to confirm that they could receive messages. Those who responded to this request (n = 208) were sent a regular (nonportal) e-mail request to participate in the Enrollment survey. Subsequently, 25 additional enrollees (after the end of portal-generated messages) and 62 individuals who had enrolled in 2009 but had not responded to the original portal message also were invited to participate. In all, of the 369 enrolled patients, 74 were not sent surveys either because of incorrect e-mail addresses or failure to confirm system access early in the rollout process. Patients were contacted through their personal e-mail addresses rather than through the portal’s secure e-mail to enable use of an online survey tool (Survey Monkey) because the portal did not allow embedded hyperlinks. The net result was that 163 (55%) of the 295 patients asked to participate completed the Enrollment survey.

The Follow-up survey occurred approximately 3 months after enrollment (ie, March and June of 2009) and also was administered using Survey Monkey. A total of 124 (35%) of the 355 enrollees with correct e-mail addresses completed the survey. The authors were able to match the responses of 91 patients completing both the Enrollment and Follow-up surveys; 79 of these patients (48% of the Enrollment survey completers) reported in the Follow-up survey that they also had logged in to use the portal.

Responses from the 3 groups of patients indicated by the shaded blocks in Figure 1 were analyzed using descriptive statistics and χ² statistics to test for differences in proportions. The Mantel-Haenszel χ² was used to compare ordinal variables such as education. To examine the relationship between the intensity of respondents’ current Internet use for health purposes and their interest in potential portal functions, an Internet use for health purposes index was calculated as the mean response over 9 of the 11 current Internet use questions (never = 1, less than monthly = 2, monthly = 3, weekly = 4, and daily = 5) included in the Waiting Room survey. Items were included related to current Internet use except for sending e-mail and ordering goods or services because nearly everyone reported current use of both, and neither was related to health care. Spearman rank order correlations and ordinal logistic regression were used to evaluate the relationship between the index and respondents’ level of interest in potential portal functions.

Results

Table 1 shows respondent demographics for the 3 surveys. Compared with Waiting Room survey respondents, Enrollment survey respondents were older, had higher household income, were more likely to have a college degree, and reported having a chronic illness. Respondents who had completed both the Enrollment and Follow-up surveys and had logged into the portal exhibited similar differences relative to Waiting Room survey respondents, except that education did not differ significantly between groups.

Table 1.

Respondent Demographics for Waiting Room, Enrollment, and Follow-up Surveys^a

Characteristic	Waiting Room Survey: Internet Users (N = 499)	Enrollment Survey (n = 163)	Follow-up Survey (n = 79)^b
Female	70.6%	62.2%	71.4%
Age (years)^c,d
>60	16.6%	25.6%	29.9%
46-60	28.4%	43.6%	44.2%
18-45	55.0%	30.8%	26.0%
Annual household income^c,d
<$40 000	37.6%	21.5%	19.4%
$40 000-$59 000	17.9%	25.7%	30.6%
≥$60 000	44.5%	52.8%	50.0%
Education^c
High school/GED or less	16.2%	9.8%	11.7%
Trade/Some college	24.9%	17.0%	16.9%
College graduate	30.7%	37.3%	41.6%
Postgraduate degree	28.1%	35.9%	29.9%
Chronic illness^c,d	39.1%	57.7%	64.9%
Self-reported health status
Very good/Excellent	46.6%	52.9%	55.9%
Good	37.0%	31.8%	29.9%
Fair/Poor	16.4%	15.3%	14.3%

Abbreviation: GED, general equivalency diploma.

Reported as n (%).

Follow-up survey includes those logging in and also completing the enrollment survey.

Difference between Waiting Room and Enrollment survey respondents, P < .05.

Difference between Waiting Room and Follow-up (portal users) survey respondents, P < .05.

Waiting Room Survey

Table 2 summarizes Waiting Room respondents’ reported Internet use for both non-health-related purposes (eg, e-mail, order goods and services) and health-related purposes (eg, read about health conditions, e-mail physician). “Weekly” and “daily” were combined to identify the most intensive users and, thus, those potentially most likely to use a portal. As might be expected, e-mail was the most frequent use (92% weekly or daily), and nearly 50% of respondents reported using the Internet to order goods and services at least monthly. Health-related use was much less frequent. About 25% reported e-mailing their physician (using regular e-mail); only 6% reported e-mailing their physicians at least monthly. Finally, when asked where they accessed the Internet, 84% indicated at home, 49% at work, 15% at a friend’s, and about 10% at libraries.

Table 2.

Frequency of Internet Use for Waiting Room Survey Respondents (N = 499 Internet Users)

	Frequency of Use (%)
Internet Use To	Never	> Monthly	Monthly	Weekly	Percentage Daily	Weekly or Daily
Send or receive e-mail	2.4%	2.0%	3.4%	9.9%	82.2%	92.1%
Read about general health issues^a	10.1%	37.3%	28.4%	16.7%	7.6%	24.3%
Order goods or services	15.7%	35.9%	32.3%	10.9%	5.2%	16.1%
Read about medications	21.7%	45.8%	19.4%	10.0%	4.0%	14.0%
Read about specific health condition(s)^a	21.3%	48.1%	19.5%	8.1%	3.0%	11.1%
Track diet or exercise^a	52.2%	30.5%	7.7%	5.9%	3.7%	9.6%
Track health condition(s)^a	47.0%	38.5%	8.5%	3.8%	2.2%	6.0%
Find physician services^a	42.1%	43.9%	9.6%	3.2%	1.2%	4.4%
Order prescription medications^a
E-mail doctor^a	74.8%	19.2%	3.4%	1.4%	1.2%	2.6%
Look at medical record^a	89.7%	7.7%	1.8%	0.8%	0.6%	1.4%

Included in the calculation of the Internet use for health purposes index.

The Internet use for health purposes index—the mean response for 9 of the 11 items in the Waiting Room Survey (Table 2)—had a range of 1 to 5 (median = 1.8; mean = 1.8; standard deviation = 0.64). All Spearman rank order correlations between Internet use and interest in potential portal functions were moderate, positive, and statistically significant (all P < .001), with the highest for requesting medication refills (r = 0.45), viewing medications (r = 0.44), sending clinical data to their physician (r = 0.41), and scheduling appointments (r = 0.40). The lowest correlations were for sharing information with family members (r = 0.31), viewing lab test results (r = 0.34), and tracking health conditions (r = 0.35). Odds ratios from ordinal logistic regression ranged from 3.0 to 5.3, indicating that a 1-point increase in Internet use (eg, moving from monthly to weekly use) is associated with a 3- to 5-fold increase in interest in using various health-related portal functions (all P < .001).

When respondents in the Waiting Room survey were asked about their level of interest in different portal functions, more than 40% reported being very interested in scheduling appointments, viewing lab results, e-mailing their physician, viewing prescribed medications, and looking at their medical record (Table 2). Conversely, fewer than 30% of respondents reported being very interested in using a portal to read about specific health conditions, to track their health conditions, share information or clinical data with their physicians, track diet and exercise, share information with family members, or participate in support groups. About 37% were very interested in medication refills.

Enrollment Survey

Of the 163 Enrollment Survey respondents, approximately half (50.3%) reported having e-mailed their physicians in the past year, with 37% reporting having sent 4 or more e-mails. Most respondents (79.8%) had never used a computer to order prescription medications (refills), and only 13.9% reported having previously used an online personal health record.

Substantial differences were found in the percentages of Waiting Room versus Enrollment respondents who reported being very interested in e-mailing their doctors (48% vs 73%), viewing lab and test results (54% vs 75%), and requesting medication refills (37% vs 52%). Relatively little difference was found between Waiting Room and Enrollment survey respondents who reported being very interested in making appointments (54% vs 57%), viewing medication lists (44% vs 52%), and sending information to the doctor (22% vs 28%).

Follow-up Survey

Responses of patients completing both the Enrollment and Follow-up surveys and who also reported using the portal indicated, at best, modest use during the first 3 months. For example, although more than 70% of enrollees reported e-mailing their doctors at least once, about 25% reported sending 4 or more e-mails. Although 40% had viewed their medication list, 32% reported viewing the medication list only once. Only 20% reported requesting a medication refill, and 18% reported requesting an appointment.

Enrollees’ perspectives about their experience with using the portal to e-mail their physician and view prescribed medications indicated significant room for improvement. Fewer than 33% reported strong agreement that the messaging function was easy to navigate or that they received a timely response to their e-mails. Only 29% of portal users strongly agreed that the medication list was accurate, 26% that the medication list was easy to access, and 19% that they felt that their medication lists were more accurate.

Figure 2 presents the perceptions of Follow-up survey respondents regarding how using the portal affected their participation in the care process and communication with their doctors. The most common response to each item was neutral/no opinion. Among the remaining respondents, there was generally more positive than negative perception of the portal’s effect. In sum, these data suggest that users reported no overwhelming effect of the portal on their health care.

Figure 2.

Perceptions of how much using the secure communications patient portal is affecting patients’ participation in the care process and communications with their providers (n = 79)

Discussion

If patient portals are to have the intended impact on patient care quality and safety, it is essential to understand which patients will actually use them, how the portals are being used, and the portal’s usability from the patients’ perspectives. This study provides several insights into differences in patients’ perceptions when contemplating hypothetical portal features versus perceptions when enrolling and using a portal. First, our data clearly identified the clinics’ primary care patients as generally having a high degree of Internet use for non-health-related purposes; they reported only limited use for health care purposes. From the health care organization’s standpoint, having portal users who are experienced Internet users is an advantage because they already have the necessary equipment to access and skills to use a portal. Conversely, experienced Internet users represent a potential disadvantage in terms of higher expectations that the portal will be easy to access and use and contain features and functions that patients actually desire. Meeting these expectations may be a challenge depending on the particular portal selected.

From portal planning and implementation perspectives, our data suggest that there are substantial differences in patients’ responses when being asked a hypothetical question (Waiting Room survey) versus responses of patients who actually sign up for and use the portal (Enrollment and Follow-up surveys). There were significant demographic differences between patients who actually enrolled to use the portal and our patient population of Internet users. When compared with a general primary care patient population (Waiting Room survey), patients who enrolled and used the portal tended to be older and female, have higher levels of income and education, or be more likely to have a chronic disease. Understanding the characteristics of patients who actually enroll to use a portal is important from planning and implementation perspectives as well as the broader organizational service area and market share perspectives.

Because Internet use is a prerequisite to portal use, it is useful to consider patients’ relative level of sophistication in using the Internet. For example, analysis of the Waiting Room survey responses found that greater Internet use for health-related purposes was positively associated with higher levels of interest in using various potential portal features (eg, e-mail, viewing lab/test results). Similarly, although respondents reported being active Internet users primarily for non-health-related purposes, about 25% of Waiting Room survey respondents, compared to 60% of the Enrollment survey respondents, reported having e-mailed their physician using traditional e-mail. This suggests that compared with our general primary care population, patients who enrolled to use the portal already were using the Internet to communicate with their physician. In addition, about 72% of respondents who had logged on to use the portal and completed both the Enrollment and Follow-up surveys reported that they had e-mailed their physicians at least once using the secure communications portal. Combined, these findings demonstrate the importance of understanding how a health care organization’s patient subgroups, defined in terms of demographic and Internet use characteristics, may differentially value and use a portal.

It is assumed that patients are generally very interested in communicating with their physician, knowing test results, making appointments, and refilling medications. The Waiting Room and Enrollment survey findings suggest that there are differential levels of interest in using a portal when comparing hypothetical use with interest levels of those enrolling and ultimately using the portal. For example, although being able to e-mail one’s physician and look up one’s lab and other test results online was seen as important for all respondent groups, there was an absolute increase of more than 20% in those viewing these as being very important between the Waiting Room and Enrollment surveys. What patients desire in a communication portal, in contrast to the realities of what a system has to offer (in terms of functions, organizational policies, and clinic work flow), has the potential to leave patients wanting. Patients generally will not understand the complexity of linking the portal to the EHR and may feel that the portal does not have all the bells and whistles they typically find with other online interfaces. An implication of particular importance is the need to pay special attention to what portal functions and features are offered to patients as well as to ensure that these features and functions perform as expected. Thus, from both planning and marketing perspectives, our data suggest that there may be important differences between those who are being asked hypothetically about their preferences compared with those who ultimately use the portal.

Our data also identified differences in patients’ interest in potential portal functions. For example, in both the Waiting Room and Enrollment surveys, fewer than 30% of respondents were very interested in “sending information to my doctor.” In contrast, nearly 75% of patients in both surveys were very interested in viewing lab and test results as well as e-mailing their physicians. Clearly, these 3 functions would have different implications for the health care organization or provider. In the case of patients viewing their EHRs, decisions governing which portions could be viewed may raise concerns about whether patients would become confused about what they are reading, which in turn might generate more telephone calls to their providers. In contrast, patients’ ability to send their clinical data (eg, blood pressures, blood glucose levels, and weight) directly to their providers requires that policies and workflows be developed to ensure that these data are examined in a timely manner and potentially entered into the patients’ EHRs. Although potentially feasible and of interest to some patients, our data suggest that current interest levels might make this a lower priority for initial implementation.

For portals to increase patient care quality and safety through improved communication and access to needed information, it is essential that patients find the portal and its various functions easy to access and use. Similar to Web site design in general, portals that are not seen as being user-friendly, organized, and efficient to use will be used less frequently. As demonstrated in this study, asking patients a few questions about their experiences with using the portal is essential to identify potential impediments to its use.

When implementing a portal, there may be an underlying assumption or explicit expectation that the portal will enable patients to become more engaged in their care. It is interesting to note that respondents to the Follow-up survey tended to disagree or be neutral about statements related to whether the portal was helping them take a more active role in managing their health. This is a somewhat surprising finding that may have been partly related to several factors: (1) the follow-up period of 3 months may have been too short; (2) the portal’s limitations in terms of not allowing patients to view significant portions of their EHR or diagnostic test results may have made the portal less valuable to the patient; (3) patients’ use of a portal may be more intermittent and reflect the episodic nature of changes in disease states and health conditions; and (4) although a portal may facilitate communication with one’s physician, this does not automatically affect patient–provider communication.

A final example of patients’ potentially low interest in something that providers might value is the use of portals to participate in support groups. Barriers to in-person attendance at support groups may not be the only factors that keep patients from participating. Availability of online support is not likely to overcome low motivation or interest. However, an alternative explanation for the low interest in Web-based support groups may be that patients cannot envision this possibility as they have not yet experienced it. Additionally, they may have privacy concerns about communicating online about their health with other people whom they do not know or whom they may meet in person.

It is perhaps instructive to consider patient portal acceptance and use by patients in the context of what has been learned about consumer acceptance and use of other non-health-related online information and transaction services. For example, online banking and health care services share some common attributes, in that both have great potential for information sharing (eg, information pulled by the client vs information pushed by the organization), significant potential for client–organization transactions (ie, paying bills, scheduling appointments, requests for specific actions), significant security and confidentiality concerns, and increased convenience offered by the potential for 24/7 access and asynchronous communications. Although online banking probably is more widespread than patient portals, a recent estimate is that only about 40% of US banks use it versus more than 70% of Northern European banks.²⁵

The characteristics of online banking adopters are also similar to some of the demographic characteristics associated with patient portal adoption (ie, higher level of education, higher income). This raises the potential for application of what has been learned from online banking to patients’ adoption and use of health portals. For example, a Finnish study of consumer resistance to Internet banking found systematic relationships between different groups of “resistors” (ie, postponers, opponents, rejecters) and their perceptions of the ease of use, values, and risks associated with online banking.²⁶ Other studies have found greater consumer adoption of online banking to be related to the perceived usefulness, ease of use, trust,²⁵ relative advantage, compatibility, complexity,²⁷ convenience, and extensive customer support.²⁶ These characteristics associated with the use of online banking are all directly applicable to patient portals.

A piece of advice from the banking literature that is applicable in health care is that “banks will be better able to manage consumer experiences with moving to Internet banking if they understand that such experiences involve a process of adjustment and learning over time, and not merely the adoption of a new technology.”^26(p450) Unlike online banking, the process of adjustment and learning include not only the customer (or patient) but also the patient’s providers, who will need to change work processes to optimize the use of patient portals for both the patients’ and their own benefit.

Limitations

This study has several limitations. First, it is limited to primary care patients within 1 academic medical center, which limits our ability to generalize our findings to other health care organizations and unique patient populations. Second, the actual portal implementation was limited to 3 clinics as part of a pilot study, with only 3 months of follow-up. Because the pilot clinics were not selected at random, we are unable to assess the extent to which their selection may have introduced bias into the study. Third, we were not able to match respondents in the Waiting Room survey with respondents in the Enrollment survey. Finally, the Follow-up survey responses may have been affected in an unknown manner by the specific features and usability of the patient portal.

Conclusion

Significant effort is being expended to develop and design patient portals to enable secure communication between patients and their health care providers. The promise of patient portals to potentially improve the timeliness, quality, safety, and even the cost of health care, although great, cannot be realized if the portals do not become routinely used as part of patient care processes. Our data suggest that although many patients may indicate a hypothetical interest in using such a portal, those who actually enroll and use portals may represent a unique subgroup of a practice’s or health system’s population. Greater attention paid to understanding differences between the preferences of hypothetical and actual users is needed to optimize portal design and implementation, patient acceptance and use, and ultimately patients’ engagement in their clinical care.

Footnotes

Authors’ Note

Material related to this manuscript has been presented at the following meetings: HIMSS (Healthcare Information and Management Systems Society) 2011, February 2011, in Orlando, FL. 2011 ACHE (American College of Healthcare Executives) Congress, March 2011, in Chicago, IL. Department of Veterans Affairs VIReC Clinical Informatics Seminar, April 2011, webinar.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by grant number R18HS017035 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.

References

Zhou

Garrido

Chin

Wiesenthal

Liang

LL.

Patient access to an electronic health record with secure messaging: impact on primary care utilization. Am J Manag Care. 2007;13:418-424.

McGeady

Kujala

Ilvonen

The impact of patient–physician web messaging on healthcare service provision. Int J Med Inform. 2008;77:17-23.

Virji

Yarnall

Karuse

. Use of email in a family practice setting: opportunities and challenges in patient- and physician-initiated communication. BMC Med. 2006;4:18.

Klein

An empirical examination of patient-physician portal acceptance. Eur J Inf Syst. 2007;16:751-760.

Silvestre

Sue

Allen

JY.

If you build it, will they come? The Kaiser Permante model of online health care. Health Aff (Millwood). 2009;28:334-344.

Emont

. Measuring the impact of patient portals: what the literature tells us. http://www.chcf.org/publications/2011/05/measuring-impact-patient-portals. Accessed August 10, 2011.

Molyneaux

O’Donnell

Oakley

. Patient Portal Technologies. Fredericton, NB, Canada: National Research Council; 2009.

Wallwiener

Kansy

Seeger

Rajab

TK.

Impact of electronic messaging on the patient-physician interaction. J Telemed Telecare. 2009;15:243-250.

Bryne

Elliott

Firek

Initial experience with patient–clinician secure messaging at a VA medical center. J Am Med Inform Assoc. 2009;16:267-270.

10.

Lin

Wittevrongel

Moore

Beaty

Ross

SE.

An Internet-based patient–provider communication system: randomized controlled trial. J Med Internet Res. 2005;7:e47.

11.

Hess

Bryce

Paone

. Exploring challenges and potentials of personal health records in diabetes self-management: implementation and initial assessment. Telemed J E Health. 2007;13:509-517.

12.

Kleiner

Akers

Burke

Werner

EJ.

Parent and physician attitudes regarding electronic communications in pediatric practices. Pediatrics. 2002;109:740-744.

13.

Denton

IC.

Will patients use electronic personal health records? Responses from a real-life experience. J Healthc Inf Manage. 2001;15:251-259.

14.

Couchman

Forjuoh

Rascoe

Reis

Koehler

Walsum

KL.

E-mail communications in primary care: what are patients’ expectations for specific test results?

Int J Med Inform. 2005;74:21-30.

15.

Sittig

King

Hazlehurst

BL.

A survey of patient–provider e-mail communication: what do patients think?

Int J Med Inform. 2001;61:71-80.

16.

Couchman

Forjuoh

Rascoe

TG.

E-mail communication in family practice: what do patients expect?

J Fam Pract. 2001;50:414-418.

17.

Katz

Moyer

Cox

Stern

DT.

Effect of a triage-based e-mail system on clinic resource use and patient and physician satisfaction in primary care. J Gen Intern Med. 2003;18:736-744.

18.

Bergman

Brown

Wilson

Teen use of a patient portal: a qualitative study of parent and teen attitudes. Perspect Health Inf Manag. 2008;5:13.

19.

Atreja

Mehta

Miller

. One size does not fit all: using qualitative methods to inform the development of an Internet portal for multiple sclerosis patients. AMIA Annu Symp Proc. 2005:16-20.

20.

Bryce

Zickmund

Hess

. Value versus user fees: perspectives of patients before and after using a web-based portal of management of diabetes. Telemed J E Health. 2008;14:1035-1043.

21.

Earnest

Ross

Wittevrongel

Lin

Use of a patient-accessible electronic medical record in a practice for congestive heart failure: patient and physician experiences. J Am Med Inf Assoc. 2004;11:410-417.

22.

Winkelman

Leonard

Rossos

PG.

Patient-perceived usefulness of online electronic medical records: employing grounded theory in the development of information and communication technologies for use by patients living with chronic illness. J Am Med Inf Assoc. 2005;12:306-314.

23.

Shaw

Howard

Chan

. Access to Web-based personalized antenatal health records for pregnant women: a randomized controlled trial. J Obstet Gynaecol Can. 2008;30:38-43.

24.

Matheny

Gandhi

Orav

. Impact of an automated test results management system on patients’ satisfaction about test result communication. Arch Intern Med. 2007;167:2233-2239.

25.

Aldas-Manzano

Lassala-Navarre

Ruiz-Mafe

Sans-Blas

Key drivers of Internet banking services use. Online Inf Rev. 2009;33:672-695.

26.

Laukkanen

Sinkkonen

Laukkanen

Consumer resistance to Internet banking: postponers, opponents, and rejectors. Int J Bank Manag. 2008;26:440-455.

27.

Lichtenstein

Williamson

Understanding consumer adoption of Internet banking: an interpretative study in the Australian banking context. J Electron Commerce Res. 2006;7(2):50-66.