A tool to measure whether business management capacity in general practice impacts on the quality of chronic illness care

Abstract

Our aim was to develop a tool to identify specific features of the business and financial management of practices that facilitate better quality care for chronic illness in primary care.

Domains of management were identified, resulting in the development of a structured interview tool that was administered in 97 primary care practices in Australia. Interview items were screened and subjected to factor analysis, subscales identified and the overall model fit determined. The instrument's validity was assessed against another measure of quality of care.

Analysis provided a four-factor solution containing 21 items, which explained 42.5% of the variance in the total scores. The factors related to administrative processes, human resources, marketing analysis and business development. All scores increased significantly with practice size. The business development subscale and total score were higher for rural practices. There was a significant correlation between the business development subscale and quality of care.

The indicators of business and financial management in the final tool appear to be useful predictors of the quality of care. The instrument may help inform policy regarding the structure of general practice and implementation of a systems approach to chronic illness care. It can provide information to practices about areas for further development.

Introduction

The management of chronic diseases is a major priority for all health-care systems, with chronic conditions anticipated to account for 60% of the global burden of disease by 2020.¹ In Australia, the 2007–2008 National Health Survey found that 77% of respondents reported at least one long-term condition.² Apart from the impact on the quality of life and mortality of people with these illnesses, the costs to governments and society of management and treatment are significant.

The majority of patients with chronic conditions are managed in the primary health-care sector by general practitioners (GPs). Research has shown that the quality of management is related to a number of key variables.^3,4 It suggests that a multidimensional systems approach to the assessment and management of patients is required, which integrates all areas of practice activity and involves all levels of staff. While financial incentives are used to encourage GPs to improve their management of chronic diseases,⁵ there has been less focus on the role played by specific aspects of the management of practices as organizations.

The Australian government provided our research team with funding to identify the organizational and structural characteristics of general practice that were critical for successful chronic disease management. Through an iterative process of focus groups and interviews with key stakeholders,⁶ we determined that there were four major factors that should be investigated. These were multidisciplinary teamwork, information management, clinical linkages, and the business and financial management capabilities of the practices.

The hypothesis was that a practice with a more sophisticated or mature level of understanding and application of good management structures, systems, principles and processes would be best able to systematically deliver quality care to patients with chronic illnesses. As there was an absence of tools capable of easily identifying and measuring the features of management that contribute to quality care, we created a new tool. This paper describes the development and validation of the Business and Finance Practice Profiling Interview (PPI). It also provides an assessment of whether there is a link between this aspect of management and the provision of evidence-based, high-quality patient care.

Methods

Item selection

In Australia, general practices are mainly small-to-medium businesses that provide a personal professional service so we began by determining common themes through searching literature relating to the principles of successful small business management.⁷ We considered the relevance of concepts such as the balanced scorecard,⁸ the plan, do, study, act cycle (PDSA)⁹ and the use of key performance indicators. Journals relating specifically to family practice and practice management were then searched.¹⁰ Further information about practice management and standards for accreditation was gained from government websites and publications of Australian medical organizations.^11–13 Performance expectations for practice management outlined in the General Medical Services Contract in the UK¹⁴ were reviewed.

A draft list of items was created from a synthesis of these sources, within six domains:

The existence of clinical systems to ensure a thorough, systematic management of patients and recording of treatment and results;

Financial recording and financial management processes which enhance the economic viability of the practice;

Application of principles of human resource management;

Appropriate provision and maintenance of the facilities and equipment required for patient care and efficient operation of the practice;

Provision and marketing of a range of patient-centred services; and

Evidence of long-term practice development, including strategic planning, practice accreditation, continuous evaluation and benchmarking.

Development and piloting of the interview tool

Sample questions based on the six domains were developed and circulated to all members of the research team for comment. After further modification, the items were distributed to three key individuals in the Australian general practice environment for their feedback. The final version of the interview contained 27 questions, but some questions had subsections, bringing the total to 63 items. The interview was piloted in 11 practices, leading to minor modifications in wording to ensure that questions were understandable, appropriate and able to discriminate between practices.

Administration of the interview tool

The tool was administered in 97 practices during 2004 as part of our wider investigation into the organizational capacity of Australian general practices for chronic disease care. The practices were recruited across five states and one territory of Australia, and represented a cross-section geographically (metropolitan versus rural) and in terms of size (from solo GP to large and/or corporate practices).

Ethics approval for the entire study was gained from the Human Research Ethics Committees of The University of Adelaide and The University of New South Wales.

Nine researchers, who had undertaken standardized training, administered the PPIs. Each interview using the Business and Finance PPI was conducted face to face with the principal GP of the practice and, where applicable, the practice manager. The tool took about 15 minutes to complete and responses were entered directly into an electronic form on a laptop computer. The scoring applied by the different interviewers was reviewed by three of the authors (CH, JP and JG) for consistency and inter-rater reliability.

Statistical methods

Item screening

Frequencies of item responses were inspected for signs of restriction in range. Items were excluded if one response accounted for more than 90% of the sample. Interitem correlations were inspected, with only one item from a pair being included where correlations were above 0.9.

Cronbach alpha coefficients were calculated as a baseline measure of internal consistency. Only items with moderate reliability (alpha of at least 0.6–0.7) were included in the next stage. A coefficient of greater than 0.9 was used to indicate high level of item redundancy.¹⁵

Factor analysis procedure

The number of respondents for effective factor analysis is ideally five times the number of items on the questionnaire.¹⁶ As we had 97 practices participating, and the original tool consisted of 63 variables, data items were screened to reduce the item pool prior to factor analysis using SPSS v13.0.

Correlation matrices were then examined to identify moderately correlated items. Principal axis factoring was used as the method of estimation, with Oblimin rotation (with Kaiser normalization). The amount of variance explained by each factor (initial eigenvalues greater than 1) and inspection of the Scree plot were used as criteria for retention of factors.

Identifying and calculating the subscales

Simple approximation methods were used to calculate subscale scores from the original data-set.¹⁷ Items with a factor loading coefficient equal to or approaching 0.4 were included, with the highest loading option being used where items loaded on more than one factor.^17,18 Cronbach's alpha coefficient was calculated to establish the internal consistency of the items within each subscale.

Overall model fit and utility of scores

The overall model fit was assessed via a structural equation model using analysis of moment structures (AMOS).¹⁹ Criteria used to assess goodness of fit were: discrepancy ratio of less than 2; goodness of fit index (GFI), adjusted goodness of fit index (AGFI), Tucker–Lewis index (TLI) and comparative fit index (CFI) with values around 0.9, and a root mean square error of approximation (RMSEA) around 0.05.

The frequency distributions of the four subscales and total scores were examined for normality (skewness less than 1). Utility was examined by testing for significant differences between the area of practice (metropolitan versus rural) and size of practice (one GP, 2–3 GPs, 4 or more GPs) using standard parametric methods.

Analysis of association with quality of care

To investigate the relationship between the Business and Finance PPI and the quality of care being provided to patients, the scores were analysed against those obtained from another new tool, the general practice clinical care interview (CCI). This tool was designed to assess the level of GP adherence to evidence-based best practice guidelines for the care of patients with asthma, diabetes and cardiovascular disease. Full details of the development and evaluation of the CCI are provided elsewhere.²⁰

Associations between the four subscales of the PPI and the five subscales and total score from the CCI were analysed using multiple linear regression, controlling for area and size of practice. Each CCI subscale was analysed separately. All analyses were performed at the practice level.

Results

Screening of items

There were no missing values in responses to the 63 items of the Business and Finance PPI, but 10 items with restriction in range were unsuitable for further inclusion. Interitem correlations revealed another 18 items relating to paired variables, where the leading question was omitted. Hence, 35 items were suitable for inclusion in the first factor analysis, giving an initial respondent to item ratio of 3:1. The Cronbach alpha coefficient for the 35 items was 0.902.

Factor analysis

Initial factor analysis suggested a solution with four factors. Twelve items with loading coefficients less than 0.4 were omitted. A second factor analysis resulted in the deletion of a further two items loading at borderline levels.

The final analysis converged in 12 iterations and identified a four-factor solution with 42.5% of the variance explained. The final structure of the Business Finance Model contained 21 items (Table 1). The four factors were labelled as:

Organizational and administrative processes (7 items);

Human resources/staff development (5 items);

Marketing analysis (3 items);

Business development (6 items).

Table 1

Pattern matrix (final factor analysis)

Factor description	Item particulars	Scoring	Factor
Factor description	Item particulars	Scoring	1	2	3	4
Organizational and administrative processes	Has your practice registered for the Chronic Disease Initiatives (financial incentives from the govt for completing certain work)?	1 if yes	0.769
	Has the practice made any claims for the Chronic Disease Initiatives?	1 if yes	0.749
	Describe or give examples of protocols for administrative procedures	1 for good example	0.581
	What internal communication system(s) does the practice use to share information within the practice?	½ for each acceptable system, max of 2	0.528
	Are there written job descriptions for practice staff members?	1 if all staff	0.509
	Is the practice accredited?	2 if yes1 if in process	0.425
	How would you describe the procedure used for recalling patients for follow-up?	1 if describe systematic approach	0.407
Human resources/staff development	Are formal staff performance appraisals carried out?	1 if yes		0.874
	How often are formal staff performance appraisals carried out?	1 if at least annually		0.778
	Does the practice have a patient through-put target?	1 if yes		0.415
	How often is the written strategic/business plan reviewed and updated?	1 if at least five yearly		0.408
	How often are written job descriptions for practice staff members reviewed?	1 if annually		0.405
Marketing analysis	When did your practice last perform a SWOT analysis (identifying strengths, weaknesses, opportunities and threats), or similar process of analysing its market position?	1 if done in last two years			0.831
	Were the results of the SWOT analysis used to introduce any lasting changes in the practice?	1 if yes			0.817
	What strategies were developed as a result of the practice examining, in the last 12 months, its capacity to increase its gross revenue?	1 if positive strategies reported			0.403
Business development	Describe or give examples of formal evaluation of the financial viability of introducing system changes, new services etc.	1 for good example				0.529
	Describe or give examples of risk management strategies	1 for good example				0.524
	Describe or give examples of your stock control system	1 for good example				0.512
	What opportunities are offered to staff for regular upskilling?	1 if active strategies				0.484
	Describe or give examples of financial planning	1 for good example				0.435
	Does the practice have regular practice meetings?	1 if yes				0.434

Extraction method: Principal axis factoring

Rotation method: Oblimin with Kaiser normalization. Rotation converged in 12 iterations

Descriptive statistics and reliability coefficients for the four subscales and total score are detailed in Table 2, with alpha coefficients indicating a high level of reliability. A confirmatory factor analysis using AMOS showed that the overall model was an acceptable fit (discrepancy of 1.33 with GFI = 0.826, AGFI = 0.770, TLI = 0.895, CFI = 0.912 and RMSEA = 0.06).

Table 2

Descriptive statistics and reliability coefficients for the subscales and total for business and finance

	Organizational and admin processes (max score 9)	HR/staff development (max score 5)	Marketing analysis (max score 3)	Business development (max score 6)	Total score (max score 23)
Mean	5.77	1.53	1.23	3.99	12.49
Std deviation	2.27	1.61	1.14	1.69	4.85
Range	0–9	0–5	0–3	0–6	1–21
Cronbach alpha	0.78	0.75	0.74	0.71	0.84

Association with quality of care

Simple linear regression, with no adjustment, showed that Factor 4 (Business development) was consistently correlated with the individual CCI scores and the total CCI score (P = 0.016). Multilevel regressions were then conducted with area and size of practice entered first, followed by Factors 1–3, and Factor 4 in the final step. The change in R ² was computed at each step (Table 3).

Table 3

Changes in the multiple correlations for Business and Finance subscales and elements of the clinical care interview

Section of clinical care interview	Change in R			Total R ²
Section of clinical care interview	1. Area and size of practice	2. Factors 1, 2 and 3 combined	3. Factor 4	Total R ²
Diabetes assessment	0.012	0.057	0.025	0.094
Asthma assessment	0.033	0.020	0.037	0.090
Assessment of risk factors	0.036	0.027	0.060*	0.122
Use of registers and recall system	0.043	0.168**	0.082*	0.293*
Care planning	0.003	0.010	0.027	0.039
Clinical care interview – Total	0.039	0.056	0.091*	0.186

*Statistically significant (<0.05); **Only Factor 1 was statistically significant (P = 0.010)

In the final model, Factor 4 increased the R ² value significantly (ΔR ² = 0.091, F _1,89 = 9.92, P = 0.002) and was a significant predictor of CCI total score (B = 1.52, P = 0.002). It was also found to be statistically significant in predicting the use of registers and recall systems for chronic care (P = 0.002), and the assessment of risk factors (P = 0.016). Factor 1 also contributed to register and recall systems (P = 0.010). In addition, the total Business and Finance score was associated with the CCI score for the use of registers and recall systems (P < 0.001).

Utility

Table 4 shows the means and standard deviations for the subscales and total scores, according to the size of practice. All increased significantly as practice size increased. Post hoc pairwise tests indicated that significant differences were recorded between all three groups, with significance increasing as the number of GPs in the practice increased.

Table 4

Mean subscale and total scores for the business and finance model by practice size

Factor	Size of practice	Mean	Standard deviation	Valid N
Organizational and administrative processes (max score 9)	Solo	3.96	2.41	N = 25
	2–3 GPs	5.78	2.19	N = 32
	4 or more GPs	6.89	1.38	N = 40
Human resources/staff development (max score 5)	Solo	1.04	1.59	N = 25
	2–3 GPs	1.16	1.32	N = 32
	4 or more GPs	2.15	1.66	N = 39
Marketing analysis (max score 3)	Solo	0.56	0.87	N = 25
	2–3 GPs	1.09	1.17	N = 32
	4 or more GPs	1.75	1.03	N = 40
Business development (max score 6)	Solo	3.20	1.98	N = 25
	2–3 GPs	3.66	1.62	N = 32
	4 or more GPs	4.75	1.21	N = 40
Business and Finance – Total (max score 23)	Solo	8.76	5.34	N = 25
	2–3 GPs	11.69	3.66	N = 32
	4 or more GPs	15.55	3.25	N = 39

GP = general practitioner

The analysis of the four subscales and total scores showed significantly higher scores for practices in rural areas compared with metropolitan areas for the subscale of Business Development (t = 3.0, df = 92, P = 0.004) and Business and Finance total scores (t = 2.4, df = 94, P = 0.018) (Table 5). The other three subscales were not significant.

Table 5

Mean subscale and total scores of the business and finance model by area

Factor	Area	Mean	Standard deviation	Valid N
Organizational and administrative processes (max score 9)	Metropolitan	5.44	2.33	N = 63
Organizational and administrative processes (max score 9)	Rural/remote	6.38	2.06	N = 34
Human resources/staff development (max score 5)	Metropolitan	1.40	1.61	N = 62
Human resources/staff development (max score 5)	Rural/remote	1.76	1.60	N = 34
Marketing analysis (max score 3)	Metropolitan	1.17	1.14	N = 63
Marketing analysis (max score 3)	Rural/remote	1.32	1.15	N = 34
Business development (max score 6)	Metropolitan	3.67	1.84	N = 63
Business development (max score 6)	Rural/remote	4.59	1.18	N = 34
Business and Finance – Total (max score 23)	Metropolitan	11.64	5.21	N = 62
Business and Finance – Total (max score 23)	Rural/remote	14.06	3.69	N = 34

Multivariate analysis

Multiple regression was used to investigate the impact of practice managers on practices' quality of business and financial systems. The mean total score for practices with no practice manager was 9.9 (range 1–17) and for those with one or more practice managers 14.2 (range 2–20). After adjusting for both size and location of practice, there was a significant difference in the total score between practices that did or did not have a practice manager (P = 0.007). When practice management was a specific, independent role, the total scores were also significantly higher (P = 0.021).

Discussion

Development and validation of the tool

A new tool was developed to identify the aspects of business and financial management that may increase the ability of general practice to provide quality care for chronic disease. After the item analysis and screening, the final interview tool comprised 21 items. Factor analysis revealed four underlying factors.

The four factors point to the elements that contribute to effective patient management. A review of the items retained in the factor analysis suggests that most have face validity as they match the findings of our literature review and the consultation carried out with stakeholders. While the link between individual items and the quality of chronic disease care may not appear to be direct, collectively they reflect critical success factors in both financial and non-financial areas of performance. While it can be argued that these are important in management of all patients, they are especially vital for the coordinated and continuous management required for complex chronic illnesses.

Items in Factor 1 include accessing of government incentives relating to chronic disease management and having clear, efficient administrative procedures and processes, including those for recalling patients for follow-up. Such policies and systems are an essential requirement for successful practice accreditation.

The items loading in Factor 2 reflect the practice's application of principles of human resource management. Preparing and reviewing job descriptions and carrying out regular staff appraisals are indicators that the practice recognizes the value of having qualified staff in appropriate, clearly defined roles, which is central to providing a multidisciplinary team care approach to chronic disease management.

Items in Factor 3 include those relating to use of techniques of planning and marketing, and the implementation of positive strategies to advance the practice. This would include analysis and understanding of their client base and recognizing the business case for offering chronic disease care services that will provide income as well as good health outcomes.

The items constituting Factor 4 reflect an objective and sophisticated approach to running the business with a view to long-term survival and growth. They relate to protection and enhancement of the practice's resources (physical, human and financial) and the application of business development principles (planning and evaluation of activities) as a strategy for coping with changing demands.

The procedure employed to identify the categories of questions was similar to that used to develop a framework of quality indicators applicable to practices in Europe (although not specifically for chronic disease management).²¹ Expert panels considered five domains of practice management – infrastructure, staff, information, finance, and quality and safety – which covered the same scope as the factors emerging from our analysis. Further testing of the European model revealed some differences from our variables, such as the omission of recall of patients and less importance on written protocols and marketing of the business.²²

There are also similarities with the methodological and statistical approaches employed in the development of a tool to measure organizational attributes of primary care practices in USA.²³ The broader context for that study was the ability of practices to adapt their care and management practices in the face of ongoing pressure for change. Many items relating to effective communication and staff involvement in decision-making were comparable with ours. The importance of resources such as information management and management infrastructure was also noted, but not explored further.

Correlation with quality of care

The analysis of the association between the business and finance factors and the elements of the CCI provide encouraging results but do not prove cause and effect. It could be argued, however, that having a systematic approach to practice management creates the right environment for carrying out recommended elements of best-practice care, such as having disease registers and recalling patients regularly. The validity, strength and direction of the associations between quality of business management, practice capacity and quality of care need further investigation.

Effect of practice size and location

We found a clear relationship between practice size and business management scores (subscales and total). One explanation is that larger practices are more likely to be able to employ a specialist practice manager who has the skills and time to take responsibility for many of the tasks described, including the management of nursing and other staff. This would appear to be supported by the finding that having a practice manager was a significant factor in explaining the total score, independent of size of practice.

With regard to the higher total scores (and scores for the subscale Business Development) attained by rural practices compared with metropolitan, the explanation for this is less obvious. It could be that rural practices are more consciously aware of their important role in the community, and of their relative disadvantage in access to health resources and services, and so take more care to maximize their efficiency, be self-sufficient and ensure their survival.

Limitations

The relatively small sample size has implications for the validity of the factor analysis. While there was a good spread of type and location of practices, they may not be entirely representative as they were self-selected. The use of nine interviewers raises the possibility of inconsistency and subjectiveness in recording answers, even though procedures were implemented to counter this. The results may not be generalizable to family practices in other countries, due to differences in organizational cultures and health-care systems.

Conclusions

Having completed this initial development and validation of the Business and Finance PPI, further testing of its utility and reliability should be conducted. A full validation study would be required to investigate whether the factor structure was replicated in another sample in Australia, followed by further field testing on a larger Australian sample and overseas, to determine its universality.

The interview tool has the potential to be used to inform government policy decisions regarding the structure and funding of general practice, support for improved practice management education and the implementation of a systems approach to care. By providing feedback to individual practices about their scores, it also constitutes a relatively quick, targeted method of assessing indicators of good management practice which can then be used as the basis for quality improvement by practices.

Footnotes

Acknowledgements

The investigators sincerely thank the participating general practices and their staff for their assistance, and the other members of the PracCap research team for their valuable contribution (Edward Swan, Cheryl Amoroso, Christopher Barton and Gawaine Powell Davies). JP is grateful to the Australian National Health and Medical Research Council (Programme Grant 510135) for salary support. The study was funded by the Australian Department of Health and Ageing. The Department was not involved in the study design, data collection, analysis or interpretation, and had no influence on the writing and submission of this article.

References

World Health Organisation. Innovative Care for Chronic Conditions: Building Blocks for Action. Global Report. Geneva: World Health Organisation, 2002

Australian Bureau of Statistics. National Health Survey: Summary of Results 2007–08 (ABS Publication 4364.0) Canberra: ABS, 2009

Bodenheimer

, Wagner

, Grumbach

. Improving primary care for patients with chronic illness. JAMA 2002;288:1775–9

Campbell

, Hann

, Hacker

, Identifying predictors of high quality care in English general practice: observational study. BMJ 2001;323:1–6

Beilby

, Furler

. General practitioner services in Australia. In: Pegram

, ed. General Practice in Australia: 2004, Canberra: Australian Government Department of Health and Ageing, 2005:129–213

Proudfoot

, Infante

, Holton

, Organisational capacity and chronic disease care: an Australian general practice perspective. Aust Fam Physician, 2007;36:286–8

English

. How to Organise and Operate a Small Business in Australia. 8th edn. Crows Nest, NSW: Allen & Unwin, 2001

Kaplin

, Norton

. The Balanced Scorecard: Translating Strategy into Action. Boston: Harvard Business School Press, 1996

Langley

, Nolan

, Normal

, Provost

. The Improvement Guide: A Practical Approach to Improving Organizational Performance. San Francisco: Jossey-Bass Publishers, 1996

10.

Edsall

, Backer

, Bush

, White

, Maresh

, Hocker

. The family practice management practice self-Test. Fam Pract Manag 2001;8:41–8

11.

The Royal Australian College of General Practitioners. Standards for General Practices. 2nd edn. Melbourne: RACGP, 2000

12.

Australian General Practice Accreditation Limited/Quality in Practice Pty. Ltd. QbAY Library of Resources. See https://www.qip.com.au/library.asp

13.

Australian Government Department of Health and Ageing. Making Your Practice Work Better: a Resource Kit for General Practitioners (4 Modules). Canberra: Australian Government Department of Health and Ageing, 2001

14.

National Health Service. Investing in General Practice: the New General Medical Services Contract. Annex C: Competency Framework for Practice Management. London, 2003

15.

Nunally

. Psychometric Theory. 2nd edn. New York: McGraw Hill, 1978

16.

Streiner

. Figuring out factors: the use and misuse of factor analysis. Can J Psychiatry 1994;39:135–40

17.

Winstanley

. Clinical supervision: development of an evaluation instrument. Unpublished PhD thesis (Category A), University of Manchester, Faculty of Medicine, Dentistry and Nursing, 2000

18.

Gorsuch

. Factor Analysis. London: Saunders Company, 1974

19.

Arbuckle

. AMOS 4.0 Users Guide. Chicago: Small Waters Corporation, 1999

20.

Proudfoot

, Jayasinghe

, Infante

, Quality of chronic disease care in general practice: the development and validation of a provider interview tool. BMC Fam Pract 2007;8:21–7

21.

Engels

, Campbell

, Dautzenberg

, Developing a framework of, and quality indicators for, general practice management in Europe. Fam Pract 2005;22:215–22

22.

Engels

, Dautzenberg

, Campbell

, Testing a European set of indicators for the evaluation of the management of primary care practices. Fam Pract 2006;23:137–47

23.

Ohman-Strickland

, Orzano

, Nutting

, Measuring organizational attributes of primary care practices: development of a new instrument. Health Serv Res 2007;42:1257–73