Electronic Health Record and Problem Lists in Leeds,United Kingdom: Variability of general practitioners’ views

Abstract

Data sharing of Electronic Health Records from general practices to secondary care in Leeds occurs through the so-called Leeds Care Records, which collects a specific set of codes from primary care, known as ‘Active Problems’, and presents it to the user. Variability on its content is a known issue. To explore general practitioners’ views on their use of ‘Active Problems’ and on sharing data, so lessons could be learnt on how to homogenise and improve shared data. Assessing Leeds general practitioners’ views through two parallel processes (60 online surveys and 17 interviews). General practitioners feel they do not have the time nor the training required for keeping a shared approach to concise and current Problem Lists in electronic patient records. Action is needed to reduce current variability, and to improve the quality of shared information. Some types of codes currently present in Problem Lists have very little support among general practitioners who consider the focus should be on long-term conditions and probably adding current acute diagnoses and life expectancy items and not omitting sensitive information. There is a perceived need of training and time to update Problem Lists if their quality is to improve.

Keywords

attitude of health personnel Electronic Health Record primary care Problem-Oriented Medical Record software/utilisation

Introduction

Patients expect professionals looking after them to communicate information in a timely manner and in ways that guarantee continuity of care across different organisations. There is a considerable amount of information stored in each clinical record, and not everything is relevant where sharing is considered. For that purpose, information flow, understanding what content should be shared and how it should be selected is paramount.

The way the records are catalogued determines what information is shared. If transfer of information among clinicians is to improve, it is vital to understand how the process of selection occurs. In Leeds, for example, when a shared record between primary and secondary care was created (Leeds Care Record (LCR)), it was decided to collect general practice information present at ‘Active Problem Lists’. It is a concept created by Weed¹ and available in all software products used by primary physicians in England. The ‘Problem List’ is the equivalent to a ‘table of contents’,¹ containing Active Problems and issues that are currently relevant, while issues that are no longer relevant for the care of the patient are put aside, as part of the past history. It needs a clinician’s continuous review to be pertinent: some problems need to be removed from the active list (like a pneumonia already resolved), and some could be in need of getting back (like a recurrence of depression). The content of the list is not static, but fluent with the patient journey.

There are two different clinical software packages used in general practice in Leeds. The main difference between software products is the fact that one (EMIS Web) is a truly Problem-Oriented Medical Record (POMR), while the other (SystmOne) although can be used as POMR also allows a parallel ‘Summary List’, which is the equivalent of the concept of past medical history, significant events grouped in a section of the record without indication of whether they are currently affecting the patient or whether there are past episodes. The concern is that a general practitioner (GP) could create a record without problems, simply using a Summary List for recap, and the consequence would be that the LCR would be empty of GP information, even though many diseases could be populating the Summary List.

Doctors tend to be sceptic about changing their way of work,² so when they get used to a routine, it is not easy to implement new engagement rules. With the advent of LCR, different organisations, new sharing rules have been implemented; a POMR rule could be said. Is there a smooth flow of information behind, is it reducing the risk of clinical error? If the attitudes and points of view among clinicians entering data are too variable, it could be that it is not the case.

Background

Clinical record keeping is a basic professional responsibility and with the progression from paper to electronic records, it requires additional tasks to keep the records informative.³ It is a concern that many clinicians see EHR as ‘Electronic Paper Records’⁴ resulting in poor coding, with no benefit deriving from the advanced features that health informatics provide. In the past, perhaps it was not noted as limited interoperability reduced access to entries by others,⁵ but now, when records are shared, there is concern that inaccuracies on the data like Problem Lists could affect decisions based on them.⁶ When comparing the prescribing for specific chronic disease to Problem Lists reflecting these conditions, completeness has been generally poor⁷ and dependent on the thoroughness of individual clinicians.⁸

It is known that clinicians’ attitude towards EHR depends on many factors including computer literacy,⁹ cost and interoperability,⁵ public opinion,¹⁰ privacy concerns,^5,10,11,12 reliability,¹³ and the will to share among the clinical teams¹² and with the patients,¹⁴ but there is no guideline or policy on what should be shared and what type of code should populate the Problem List.

Implementation and use of EHR is affected positively by demonstration of the EHR¹⁵ and access to simulations.¹⁶ Perhaps more important is the fact that, even if there are initial barriers to EHRs, they are used successfully by all clinicians¹⁷ despite concerns about the interference with the process of the consultation^18,19 or its effect on decision-making.²⁰

Training is required as well as regular monitoring to maintain the quality of the records and their ‘Problem Lists’,^11,13,21 with a 40 per cent of variance on quality as a result of the software package used,²² as primary physicians use different EHR packages in varying ways,²³ and as described earlier, the two software used in Leeds offer different functionality adding more complexity to shared records at city level.

When data flow among organisations using different software packages with clinicians with differing attitudes to EHR and ‘Problem Lists’ use, knowing there are difficulties maintaining and sharing the large volume of information present in EHR,¹² there are a few questions to be considered: is it helping managing patients or is it causing risk by misinformation? Are shared electronic records improving patient care? Unfortunately, current use of electronic records has not been necessarily linked to better outcomes.²⁴

Methods

Study design

This is part of an academic project, a final year MSc assignment, for which ethical approval was obtained, and where P.M.M. was the student and L.S. the supervisor. The aim of the project was to assess how sharing of information could improve healthcare at local level, assessing clinicians’ views and analysing patient records, and determining what could be done to enhance the benefits EHR could offer.

Study method

The assessment of GPs’ views was planned to have two streams: online questionnaires and interview with a working GP. General practice in United Kingdom is currently in crisis with an excessive workload²⁵ and not many clinicians would agree to a face-to-face interview, so the option of a short online questionnaire was conceptualised. The aim was to recruit 15–20 GPs for interviews as well as achieving 60–100 online survey responses.

The questionnaire was divided into three themes: (a) the use of EHRs in general, (b) the use of Summaries and Problems Lists in particular, and (c) presenting groups of possible record entries and asking whether they should or should not be part of the shared Problem List. The rationale behind the questionnaire was that it was not only about understanding how GPs interact with their shared records, but if usability of the records were to improve, a discussion on the content was needed, and to facilitate the answering and analysis, some form of cataloguing the types of entries was required. As there were a large number of different possible codes that could be entered on Problem Lists, they were grouped under 11 different headings, based on codes seen already in Problem Lists (see Table 1 for headings). Supplemental Appendix A, slide 2 for details).

Table 1.

Types of codes considered.

Code headings
Major long-term conditions
Minor long-term conditions
Acute condition
Grouped conditions
Health factors
Symptoms and examination findings
Tests
Administrative codes
No clear meaning codes
Sensitive information
Life expectancy codes

The piloting of the online survey was conducted with the support of two GPs, and resulted in some minor changes in the questions so that they were clearer to understand (see in Supplemental Appendix B the final online questionnaire).

It was considered quite important to be able to explore in depth how and why the EHR are used in such varying ways and ‘qualitative interview techniques offer tools for bringing these meanings to the surface’²⁶ (p. 91). The GP-researcher (P.M.M.) would lead with focused guiding questions (see Supplemental Appendix A) that allowed discussion to follow in a flexible way but without losing sight of the purpose of the study, following Witzel’s concepts of problem-centred interviews.²⁷ The interviews were designed to be semi-structured, but within this frame and depending on the timing of the interview, they could be used in two different ways, following postpositivism and constructivism:

- On one hand, to triangulate findings, as postpositivism research would encourage: Looking into what has been commented in previous interviews and building up from acquired knowledge. In this regards, ‘if we keep trying, we will eventually discover whether or not the beliefs we have accepted are defective, for the quest of knowledge is to a considerable extent self-corrective’²⁸ (p. 3).

- And, on the other hand, to look for ways forwards, to determine the appetite and the willingness to change practice and to follow guidelines that could be created following these discussions, to modify the way EHR are used and in particular Problem Lists. In that respect constructivism was the other layer embedded in the methodological framework used in the interview.

The invitations to the study were done through the Leeds Clinical Commissioning Group’s (CCG) electronic weekly communiqué that is sent to all GPs in Leeds. Ethics committee approval included the requisite of not contacting GPs directly, which limited the ability to engage participants. In addition, brief presentations were made during two meetings organised by the CCG for other matters. GPs were invited to participate through an online questionnaire and/or interview.

The invitation was aiming to reach all GPs in Leeds, around 400, but it was not possible to estimate how many of them read the invitation, and furthermore, it was likely in many occasions the invitation was missed as it was in a menu of over 20 items to explore in the communiqué. A consequence has been the poor uptake, less than 15 per cent of potential GPs.

Results

Data analysis

A total of 60 online questionnaires and 17 interviews were collected and analysed. There was a reasonable distribution of GP characteristics (Table 2). The online questionnaires were collected between November 2017 and February 2018, interviews were carried out by P.M.M. between November 2017 and April 2018. During interviews, some GPs declared they had done the online questionnaire, others could not remember, and other did not. There was an overlap, undetermined, reducing the number of participants, not the value of their contribution.

Table 2.

Participant characteristics.

GP characteristics	Online survey		Interview
Gender
Female	31	52%	7	41%
Male	29	48%	10	59%
Current role
Principal	37	61%	12	70%
Salaried	6	10%	2	12%
Locum	7	12%	3	18%
Registrar (in training)	10	17%	0	0%
GP qualification
Before 2000	28	46%	8	47%
Between 2000 and 2007	12	20%	2	12%
Between 2008 and 2015	13	22%	4	23%
After 2015	7	12%	3	18%
Clinical software familiarity
EMIS Web	12	20%	0	0%
SystmOne	31	52%	10	59%
Both	15	25%	7	41%
None	2	3%	0	0%
Number of responses	60		17

GP: general practitioner.

The interview durations varied considerably in length, between 13 and 61 min, as some GPs were straight to the point and others expanded considerably. The interviews were transcribed, and using the software Nvivo, they were coded so the different emerging themes were determined and compared. As Ryan and Bernard²⁹ pointed out, themes do not come only from the replies to questions, but ‘from the investigator’s prior theoretical understanding of the phenomenon under study (an a priori approach)’ (p. 88). Theoretical sampling finally followed, allowing the development of a substantive theory that should be the base of a guideline on a common approach to the use of Problem Lists and Summaries.

GPs’ views on EHRs

The online survey questions and corresponding responses concerning this matter are presented in Figure 1. Of note, GPs strongly considered that the purpose of electronic records was not to simply upgrade the existing Lloyd George notes, the name given to the traditional paper records used in the NHS, and not just a way to allow activity payments, but there was a way to improve the quality of care that could be provided. The understanding of the coding was variable, and the use of specifically embedded templates to facilitate coded entries did not seem popular. There was general lack of understanding of how clinical codes work.

During interviews, the issue of how entries were made was explored. It should be clear that ‘The design and development of EHRs requires the capture of narrative information to reflect patient/clinician communication and computable data for administration and research purposes’.³⁰ Some clinicians used mainly simple text (also called free text, like traditional paper records), while others used more coded text (text associated with a clinical code). Although the importance of coding was recognised, variations were detected:

I only code things I think are important. (GP09)

I am an obsessive coder. I do, personally, I do as much as possible using codes, and I try to associate text with codes. (GP16)

There was recognition of the need to change but

How do you start doing things halfway through when everything is a mess already? How do we do this without tidying up the mess? (GP13)

And more considering,

It takes time and it takes effort. And time . . . effort we got, time we lack. (GP14)

Understanding Problems Lists and Summary Lists use

The online survey on Problem Lists suggested a larger proportion of GPs use them rather than not (62% of respondents strongly disagreed or disagreed to the statement that Problems are not used), and that they are commonly checked before consultation. It was relevant to see that the length of the Problem Lists did not have an impact on their use. Other aspects assessed on the online survey included enclosing consultations under problems (variable), level of regularly tidying them up (not common, as 57% of respondents strongly disagreed or disagreed to the statement they regularly tidy up the Problem List), and trust on accuracy of the Problem Lists (mainly disagreeing somehow). Data are presented in Figure 2.

From the interviews, it was clear the distinction between Problem Lists and Summaries was somehow blurred and not fully understood. Equally, there were concerns when presented with Problem Lists, with their variable length and accuracy, so GPs would explore further the records to confirm entries.

It was relevant the sense presented that EHR were not owned by clinicians:

Actually what they don’t see is the tool that they are using for their profession. They need to own that. You wouldn’t have a shoemaker that would not look after his tools. You wouldn’t have an engineer that would not look after his tools. So, why we don’t? It is a major tool for how we work. (GP11)

Although tidying up of Problem List entries was not a common activity according to the online questionnaire, it was reported by many of the GPs interviewed, as well as its importance.

There was even the question as to why the term ‘Problem’:

A Problem suggests that is something that is wrong, Something that is abnormal . . . whether the health issue suggests it actually is . . . it is something is there . . . it is no value judgement based on whether it is good bad or indifferent. Pregnancy is one of those. (GP11)

Understanding summaries

The online survey on Summary Lists showed a similar pattern regarding their use, with a considerable number of GPs summarising all diagnoses. Not all GPs responded to this section of the questionnaire. In any case, there were less GPs checking these lists before consultations but, as with Problems Lists, their use was not significantly affected by their length. Tidying up was again uncommon (60% of respondents strongly disagreed or disagreed to the statement they regularly tidy up the summaries; see Figure 3).

During interviews, the extensive use of Summaries was highlighted as a risk:

If you could get rid of old summaries or information that you don’t need, that will also help a lot, because most practices are still running old Summary Lists. (GP03)

A common approach towards Problem Lists

Questions on training specific to the uses of these lists were included in the online survey, and the general feeling was that training was insufficient or absent: 43 per cent respondents felt strongly about the lack of training received (see Figure 4).

When assessing training during interviews, the general feeling was that of ‘no formal training’, ‘basic’, or being ‘too accidental’ or people simply ‘learnt on the job’. There was the occasional exception, but the general view was that more training was needed.

Would GPs adhere to guidelines? The survey found just 51 per cent of respondents would strongly agree or agree to follow guidelines (see Figure 4).

What information should be then part of the Problem List? Online responses are present in Figure 5. There were three groups of items:

- Likely to be included in the Problem List by the majority of clinicians:

Major long-term conditions (91.5% of respondents likely to add it);

Minor long-term conditions (81.4%).

- Not so clear whether they should be included:

Acute conditions (69.5% of respondents likely to add it);

Sensitive information (69.5%);

Life expectancy codes (59.3%);

Grouped conditions (34%).

- Unlikely to be included:

‘No clear meaning’ (96.6% of respondents unlikely to add it)

Administrative codes (86.2%);

‘Symptoms/examination’ (81.4%);

Test related codes (76.3%);

Health factors (72.9%).

During interviews, GPs showed an interest on some guidelines to follow, although some flexibility would be expected.

A different matter was the actual content of the Problem List:

- Regarding long-term conditions like cardiovascular disease, hypertension, diabetes, or cancer, there was agreement on their need to be present.

- Regarding grouped conditions, there was some polarity, some GPs considering it bad practice, and others considering their use as a way of folders you look inside, which would not be good if the content of that folder was not available.

- Regarding acute conditions, there was a general agreement on their need.

- Regarding health factors, they were considered irrelevant for the Problem List, and information that should be presented elsewhere.

- Regarding symptoms and examination, they were also considered not ideal.

- Regarding tests, only in particular circumstances, as data are present in other sections of the record:

Probably not. Not unless that person has got a history of breast cancer and needs yearly mammogram. That would help. (GP03)

- Regarding administrative codes, there was some polarity also, as some clinicians considered their presence unfortunate while others favoured seeing all referrals made, for example.

- Regarding ‘no clear meaning’ codes, if anything there was a perceived need to clean them out.

- Regarding sensitive information, the concern was more about who could access the information rather than there is a need for it, and if anything the understanding all data can be considered sensitive.

- And finally regarding ‘life expectancy’ codes, it was also generally agreed the need for this data to be present in the Problem List.

A summary of findings is presented in Table 3.

Table 3.

Highlights.

Highlights of GPs’ views
Training is needed. Quality cannot be high without adequate initial and also regular training on the use of the records.
A Problem List should basically be a list of chronic diseases, although acute conditions, sensitive information and life expectancy items were generally accepted.
Items like administrative codes, health factors, no clear meaning, symptoms, and tests should not be part of the Problem List.
The general feeling is that there is not enough time to keep Problem Lists up to date.
Some Problem Lists seem to be too vast, too large, to be informative.

GPs: general practitioners.

Discussion

‘Modern healthcare is complicated, inefficient, expensive and messy’³¹ as there are many different views, organisations, processes interacting for the care of a single patient, mixed with distinct software companies providing packages where the influence of the user is limited: ‘Knowledge of how to develop, implement, and continuously improve EHRs for patient safety is currently limited and not accessible to most health care organisations’.³² Leeds sample presented is not an exception: GPs acknowledge the difficulties, and work positively understanding the limitations of their EHR. The Leeds GPs who participated in this research felt generally confident navigating through the electronic medical records looking for information and adding data on them, even though considerable variation was noted, for example, on the use of coded text versus free text. Far from the notion that EHRs have little quality³³ and are unusable and a burden to clinical care,³⁴ Leeds GPs had a strong feeling that EHR improved quality of care, although there was not necessarily a sense of ownership.

‘Problem List’ usage by clinicians has remained highly variable and far from ideal.³⁵ Furthermore, current available codes are ‘insufficient to express the complete medical thoughts required for the task’³⁶ and although there is overall agreement ‘Different physicians can view the same clinical situation in different ways, leading to different Problem Lists’,³⁷ and what Leeds GPs would add to the lists corroborate this. There is no common way of using the Problem List.

Problem Lists are not properly maintained^36,38,39 although Problems Lists done in primary care were more accurate than those done by specialists.⁴⁰ The Problem List should facilitate transfer of care among organisations,⁴¹ the fact is that its completeness varies considerably,⁴² preventing it from fully happening, as it happens in Leeds.

It has also been conveyed that correct and complete Problem Lists resulted in better care.⁴³ A reason already reported in the literature is the need to update the list manually⁴⁴ and lack of time to do so was the main obstacle reported among respondents, so they are not duly kept up to date,⁴⁵ deteriorating data quality, considered to be poor.⁷

Discussions on the way data are presented on EHR would result in differences from current models but more pressing than this there is a need to fix current issues with EHR, to produce clear records that can be shared and understood. This leads to the need for training. Knowing the use of codes is necessary,⁴⁶ as without proper understanding data quality would remain poor. In this respect, training increased the level of morbidity coding from 64 to 82 per cent.²¹ Still, health informatics is rarely provided in UK medical schools,⁴⁷ and it was confirmed by the sample in this study that training is an unmet need.

The study is limited by the poor participation response but the message from those who replied is clear and strong, as summarised in Table 3. Bias should be considered among respondents who are more interested in the subject, but some GPs with little experience on the topic participated to learn about EHR and Problem Lists. Hopefully, it reduced the final potential imbalance.

Conclusion

There are gaps in the way Leeds GP clinical records reflect current patient health issues, and there is considerable variability among practices hindering the understanding of the Problem List when shared. The GPs still felt strongly that their records improve patient care despite the issues existent with them.

Problem Lists according to Leeds GPs should basically be a list of chronic diseases, although acute conditions, sensitive information, and life expectancy items could be entered, but items like administrative codes, health factors, those with no clear meaning, symptoms, and tests should not be included.

Supplemental Material

Appendix_A_Interview_script – Supplemental material for Electronic Health Record and Problem Lists in Leeds, United Kingdom: Variability of general practitioners’ views

Supplemental material, Appendix_A_Interview_script for Electronic Health Record and Problem Lists in Leeds, United Kingdom: Variability of general practitioners’ views by Pablo Millares Martin and Laura Sbaffi in Health Informatics Journal

Supplemental Material

Appendix_B_Online_Questionnaire-jhi – Supplemental material for Electronic Health Record and Problem Lists in Leeds, United Kingdom: Variability of general practitioners’ views

Supplemental material, Appendix_B_Online_Questionnaire-jhi for Electronic Health Record and Problem Lists in Leeds, United Kingdom: Variability of general practitioners’ views by Pablo Millares Martin and Laura Sbaffi in Health Informatics Journal

Footnotes

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Although P.M.M. works in a practice using systmOne software, he is familiar with both and has no commercial association with any of the two companies providing these packages.

Ethical approval

The study received approval by NHS Health Research Authority (CAG reference: 17/CAG/0133, IRAS project ID: 224706, REC reference: 17/IEC08/0040).

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Pablo Millares Martin

Supplemental material

Supplemental material for this article is available online.

References

Weed

LL.

Medical records that guide and teach. N Engl J Med 1968; 278(11): 593–600.

Audet

Doty

Shamasdin

, et al. Measure, learn, and improve: physicians’ involvement in quality improvement. Health Aff 2005; 24(3): 843–853.

Adane

Muluye

Abebe

Processing medical data: a systematic review. Arch Public Health 2013; 71(1): 27.

Price

Singer

Kim

Adopting electronic medical records: are they just electronic paper records?

Can Fam Physician 2013; 59(7): e322–e329.

Bates

DW.

Physicians and ambulatory electronic health records. Health Aff 2005; 24(5): 1180–1189.

Widdifield

Young

Bombardier

, et al. FRI0194 Identifying patients with rheumatoid arthritis in primary care electronic medical records. Ann Rheum Dis 2014; 73(Suppl. 2): 452–453.

Singer

Kroeker

Yakubovich

, et al. Data quality in electronic medical records in Manitoba: do problem lists reflect chronic disease as defined by prescriptions. Can Fam Physician 2017; 63(5): 382–389.

Singer

Yakubovich

Kroeker

, et al. Data quality of electronic medical records in Manitoba: do problem lists accurately reflect chronic disease billing diagnoses. J Am Med Inform Assoc 2016; 23(6): 1107–1112.

Al Alawi

Al Dhaheri

Al Baloushi

, et al. Physician user satisfaction with an electronic medical records system in primary healthcare centres in Al Ain: a qualitative study. BMJ Open 2014; 4(11): e005569.

10.

Steininger

Stiglbauer

EHR acceptance among Austrian resident doctors. Health Policy Techn 2015; 4(2): 121–130.

11.

Rooney

Hornby

From troglodytes to information managers: information management and technology needs to achieve the primary care NHS modernization agenda – the views of three GPs. Health Libr Rev 2000; 17(3): 148–156.

12.

Perera

Holbrook

Thabane

, et al. Views on health information sharing and privacy from primary care practices using electronic medical records. Int J Med Inform 2011; 80(2): 94–101.

13.

Pillai

Thomas

Garg

The electronic immediate discharge document: experience from the South West of Scotland. Inform Prim Care 2004; 12(2): 67–73.

14.

Walker

Leveille

Ngo

, et al. Inviting patients to read their doctors notes: patients and doctors look ahead. Ann Intern Med 2011; 155(12): 811–819.

15.

Beiter

Sorscher

Henderson

, et al. Do electronic medical record (EMR) demonstrations change attitudes, knowledge, skills or needs? J Innov Health Inf 2008; 16(3): 221–227.

16.

Reis

Sagi

Eisenberg

, et al. The impact of residents’ training in electronic medical record (EMR) use on their competence: report of a pragmatic trial. Patient Educ Couns 2013; 93(3): 515–521.

17.

Bishop

Ryan

McCullough

, et al. Do provider attitudes about electronic health records predict future electronic health record use? Healthcare 2015; 3(1): 5–11.

18.

Christensen

Grimsmo

Instant availability of patient records, but diminished availability of patient information: a multi-method study of GPs use of electronic patient records. BMC Med Inform Decis Mak 2008; 8(1): 12.

19.

Makam

Lanham

Batchelor

, et al. The good, the bad and the early adopters: providers attitudes about a common, commercial EHR. J Eval Clin Pract 2014; 20(1): 36–42.

20.

Hor

O’Donnell

Murphy

, et al. General practitioners attitudes and preparedness towards Clinical Decision Support in e-Prescribing (CDS-eP) adoption in the West of Ireland: a cross sectional study. BMC Med Inform Decis Mak 2010; 10(1): 2.

21.

Porcheret

Hughes

Evans

, et al. Data quality of general practice electronic health records: the impact of a program of assessments, feedback, and training. J Am Med Inform Assoc 2004; 11(1): 78–86.

22.

Van Der Bij

Khan

Ten Veen

, et al. Improving the quality of EHR recording in primary care: a data quality feedback tool. J Am Med Inform Assoc 2017; 24(1): 81–87.

23.

Raymond

Paré

Marchand

Extended use of electronic health records by primary care physicians: does the electronic health record artefact matter?

Health Informatics J 2017; 25: 71–82.

24.

Lau

Price

Boyd

, et al. Impact of electronic medical record on physician practice in office settings: a systematic review. BMC Med Inform Decis Mak 2012; 12(1): 10.

25.

Dale

Potter

Owen

, et al. The general practitioner workforce crisis in England: a qualitative study of how appraisal and revalidation are contributing to intentions to leave practice. BMC Fam Pract 2016; 17(1): 84.

26.

Hatch

JA.

Doing qualitative research in education settings. Albany, NY: State University of New York Press, 2002.

27.

Hayek

Nieva

Corrigan

, et al. End-of-life care planning: improving documentation of advance directives in the outpatient clinic using electronic medical records. J Palliat Med 2014; 17(12): 1348–1352.

28.

Phillips

Burbules

NC.

Postpositivism and educational research. Lanham, MD: Rowman & Littlefield, 2000.

29.

Ryan

Bernard

HR.

Techniques to identify themes. Field Method 2003; 15(1): 85–109.

30.

Morrison

Fernando

Kalra

, et al. National evaluation of the benefits and risks of greater structuring and coding of the electronic health record: exploratory qualitative investigation. J Am Med Inform Assoc 2014; 21(3): 492–500.

31.

McMurray

Grindrod

Burns

How appropriate is all this data sharing? building consensus around what we need to know about shared electronic health records in extended circles of care. Healthc Q 2017; 19(4): 28–36.

32.

Walker

Carayon

Leveson

, et al. EHR safety: the way forward to safe and effective systems. J Am Med Inform Assoc 2008; 15(3): 272–277.

33.

Edwards

Neri

Volk

, et al. Association of note quality and quality of care: a cross-sectional study. BMJ Qual Saf 2013; 23(5): 406–413.

34.

Declerck

Aime

Reasons (not) to spend a few billions more on EHRs: how human factors research can help. Yearb Med Inform 2014; 9(1): 90–96.

35.

Wright

Maloney

Feblowitz

JC.

Clinician attitudes toward and use of electronic problem lists: a thematic analysis. BMC Med Inform Decis Mak 2011; 11(1): 36.

36.

Van Vleck

Wilcox

Stetson

, et al. Content and structure of clinical problem lists: a corpus analysis. AMIA Annu Symp Proc 2008; 2008: 753–757.

37.

Rothschild

Lehmann

Hripcsak

Inter-rater agreement in physician-coded problem lists. AMIA Annu Symp Proc 2005; 2005: 644–648.

38.

Kaplan

DM.

Clear writing, clear thinking and the disappearing art of the problem list. J Hosp Med 2007; 2(4): 199–202.

39.

Ancker

Kern

Edwards

, et al. How is the electronic health record being used? Use of EHR data to assess physician-level variability in technology use. J Am Med Inform Assoc 2014; 21(6): 1001–1008.

40.

Luna

Franco

Plaza

, et al. Accuracy of an electronic problem list from primary care providers and specialists. Stud Health Technol Inform 2013; 192: 417–421.

41.

Krauss

Boonstra

Vantsevich

, et al. Is the problem list in the eye of the beholder? An exploration of consistency across physicians. J Am Med Inform Assoc 2016; 23(5): 859–865.

42.

Wright

McCoy

Hickman

, et al. Problem list completeness in electronic health records: a multi-site study and assessment of success factors. Int J Med Inform 2015; 84(10): 784–790.

43.

Hartung

Hunt

Siemienczuk

, et al. Clinical implications of an accurate problem list on heart failure treatment. J Gen Intern Med 2005; 20(2): 143–147.

44.

Buchanan

Accelerating the benefits of the problem oriented medical record. Appl Clin Inform 2017; 8(1): 180–190.

45.

Bevilacqua

Manzotti

Padilla

, et al. Caracterización de la lista de problemas de la historia clínica ambulatoria informatizada en el Hospital Alemán. In: VII Congreso Argentino de Informática en Salud (CAIS 2016)-JAIIO 45 (Tres de Febrero, 2016), http://sedici.unlp.edu.ar/bitstream/handle/10915/57795/Documento_completo.pdf?sequence=1 (accessed 15 July 2018).

46.

Thiru

de Lusignan

Hague

Have the completeness and accuracy of computer medical records in general practice improved in the last five years? The report of a two-practice pilot study. Health Inform J 1999; 5(4): 224–232.

47.

Walpole

Taylor

Banerjee

Health informatics in UK medical education: an online survey of current practice. JRSM Open 2017; 8(1): 2054270416682674.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.13 MB

0.18 MB