Validity Evidence of Brazilian Portuguese Version of the Professional Fulfillment Index

Abstract

Purpose

The questionnaire “Professional Fulfillment Index” (PFI) was validated to assess emotional exhaustion, interpersonal disengagement, and professional achievement among physicians. This study presents the process of cross-cultural adaptation of the PFI to the Brazilian context.

Design

Analytical cross-sectional study carried out between July and October 2020. Settings: For conceptual equivalence, a panel of experts was constituted who conducted analyzes on the translation and back-translation processes from English to Brazilian Portuguese.

Subjects

For the operational equivalence, 432 physicians in the field of Occupational Medicine were invited.

Measures

They answered the PFI in an online platform. Analysis: Both exploratory and confirmatory factor analysis were performed to seek evidence of the hypothesized structure of PFI item responses. Reliability was assessed using three indicators.

Results

The Brazilian version can be considered similar to the original. Most of participants were female (57.6%) and mean age 49.6 years—SD 14.0. The mean time since graduation was 18.5 years—SD 9.67. Most of them had a full-time job (88.0%) and were trained through post-graduation courses 83.8%. The most frequent job task was to perform worker´s clinical evaluations (84.9%). Exploratory and confirmatory analyzes showed the adequacy of the items in measuring the instrument’s construct, with stability for use in other populations and samples.

Conclusion

The PFI was developed to evaluate positive and negative aspects of physicians’ role and performance. The process described in this article was the first cross-cultural adaptation of the questionnaire. The PFI can help in mapping potential risk situations for negative impacts on physicians’ occupational well-being and performance at work, in order to propose interventions that reduce exhaustion and expand professional fulfillment.

Keywords

burnout professional surveys and questionnaires psychometrics occupational health

Purpose

Occupational burnout is conceptualized as an exhaustion condition due to prolonged exposure to work-related stressors,¹ such as job strain with high demands and low control, effort-reward imbalance at work, and job insecurity.² It has been recognized as an important problem that impairs workers' health, in physical, psychological, and occupational dimensions. A systematic review reported cardiovascular diseases, musculoskeletal pain, depressive symptoms, job dissatisfaction, and absenteeism related to burnout.³

Low-quality services provided by healthcare professionals and medical errors are related to workers’ exhaustion.^4,5 In the United States, the average annual cost of physician turnover and reduction of hours worked attributable to burnout has been estimated to be $4.6 billion.⁶

The questionnaire most used in Brazil for measuring burnout is the Maslach Burnout Inventory™ (MBI), which is proprietary of Mind Garden (http://https://www.mindgarden.com/), including the version for evaluating this outcome among healthcare professionals, the Maslach Burnout Inventory—Human Services Survey (MBI-HSS).⁷ The limitations of this tool for measuring the various aspects of burnout have been discussed⁸ and it has been questioned whether its restricted focus on professional exhaustion truly provides a comprehensive view of worker’s well-being.^9,10

In 2018, the questionnaire “Professional Fulfillment Index” (PFI) was validated to assess emotional exhaustion, interpersonal disengagement, and professional achievement among physicians.¹⁰ The emotional exhaustion follows an approach similar that of other tools for assessing burnout, such as MBI-HSS. The construct of interpersonal disengagement focuses on empathy and connection with other people. One of the main characteristics of the PFI is that it expands the assessment of the perception of personal fulfillment through evaluation of components intrinsic to aspects of physicians’ work, like happiness, meaning, self-esteem and satisfaction, among others.¹⁰

The PFI questionnaire has been considered appropriate for assessing physicians’ well-being in intervention studies on professional exhaustion.¹⁰ A national survey of breast surgeons in the United States using the PFI found that the prevalence of burnout in this group was 41.3%, while 42.5% had high professional fulfillment.¹¹ Among these physicians, the longer their professional activity had been, the less frequent exhaustion was and the higher their professional fulfillment was. Higher workload was associated with burnout, but professional fulfillment increased if there was clinical practice.¹¹

Considering that the PFI can help in understanding the phenomenon of burnout among Brazilian physicians and can broaden the discussion to include professional fulfillment, this study presents the process of cross-cultural adaptation of the Professional Fulfillment Index (PFI) questionnaire in the Brazilian context.

Methods

Design

The original Professional Fulfillment Index (PFI) questionnaire is intellectual property of the Stanford Medicine WellMD Center (United States). The stages of the cross-cultural adaptation of the instrument followed the methodology indicated by researchers both internationally^12-14 and in Brazil,¹⁵ regarding conceptual, semantic, and operational equivalence.

For conceptual equivalence, a review of the literature on this subject was carried out and a research project that discussed its associated factors was constructed. The objective was to evaluate the relevance of the main theme of the questionnaire within the experience of Brazilian physicians. After the translation, the equivalence of items was discussed in meetings of the committee of experts to evaluate the adequacy and clarity of the statements, as well as the response options. In the process of reaching semantic equivalence (Figure 1), we sought to transfer meanings from the concepts present in the original questionnaire to the adapted version, to ensure equivalent face validity among respondents within both cultures. This was achieved through a back-translation and validation process conducted by one of the authors of the original questionnaire. The resulting version of the instrument was then subjected to pretesting.

Figure 1.

Flowchart of the semantic equivalence in cross-cultural adaptation process.

Sample

To analyze its operational equivalence, physicians who performed activities in the field of occupational medicine in any part of the national territory were invited to participate. The invitations were sent out electronically means by scientific associations that were partnering with the project and were also disseminated on the internet through the researchers' social networks.

Measure

The PFI is formatted in 3 dimensions (professional fulfillment-PF, professional exhaustion-PE, and interpersonal disengagement-ID). Each statement in the index has response options on a five-point Likert scale. Data on Brazilian version were collected between October and December 2020 via electronic means accessed through links in the worldwide computer network. All 432 participants gave their consent to participate by agreeing to an online informed consent statement.

This study was approved by the Research Ethics Committee of the of the São Camilo University Center—opinion report nº. 4.077.484/2020), with registration on the Brazil Platform (CAAE 31400620.0.0000.0062).

Psychometric Analysis

The sample size in psychometric studies is calculated by considering the number of items in the instrument under analysis.^16,17 It is recommended that there should be minimum proportions of 20 participants for each item¹⁸ (20:1) and/or at least 300 to 400 participants.¹⁹ In the present study, with 432 participations and 16 items, the proportions were 27:1.

A factor analysis was performed to seek evidence of the hypothesized structure of PFI item responses grouped by the three published sub-scales. This investigated the dimensional structure of the Brazilian Portuguese version of the instrument, using contemporary methods.^19-21

The following exploratory factor analysis (EFA) assessments were made:

1. Robust parallel analysis (RPA)^19,22,23 through optimal implementation of parallel analysis (PA), with minimum rank factor analysis (MRFA) that minimizes the common variance of residuals²⁴;

2. Schwarz’s Bayesian information criterion (BIC) dimensionality test²⁵;

3. The Hull method with Comparative Fit Index (CFI) and Root Mean Square Error of Approximation (RMSEA)²⁶;

4. Closeness to one-dimensionality assessment²⁷: one-dimensional congruence (UNICO) > .95; explained common variance (ECV) > .80; and mean of item residual absolute loadings (MIREAL) < .30.

These techniques were implemented from a polychoric matrix using Bayes modal estimation.²⁸ Factors were extracted using robust unweighted least squares (RULS), and Promax rotation was adopted, since the original instrument has more than one dimension. The robustness of the method was determined through associating a bootstrap with sample extrapolation to 5000.

The indices of replicability, quality, and effectiveness of the solution were used as a way to increase the security of the solution of the proposed model.^27,29 For construct replicability, we used the generalized G-H index with a minimum index of .80³⁰. For factor quality and effectiveness estimation, we used the factor determinacy index (FDI >.90), EAP marginal reliability (EAP >.80), sensitivity ratio (SR > 2) and expected percentage of true differences (EPTD >90%).^31,32

Reliability was assessed using three indicators: Cronbach’s alpha, McDonald’s omega, and ORION (greatest lower bound) with a cutoff value of .80. To assess the adequacy of the items and the model, the following criteria were considered: explained variance of the model from 60 to 70%; factor loading values greater than .50¹⁸; commonalities greater than .40; and problems of collinearity and multicollinearity if factor loadings were above .80 and .85, respectively.^18,33

Confirmatory factor analysis was performed using RULS and the following were used as goodness-of-fit (GOF) indices: robust mean-scaled chi square (x²/df); Tucker-Lewis index (TLI); CFI; goodness-of-fit index (GFI); and adjusted goodness-of-fit index (AGFI). For these indicators, the cutoff value adopted was .90, with RMSEA <.07 and root mean of residuals (RMR) < .08. The path diagrams were presented with factor loadings, predictive values for the items, and correlations between factors.

The analyses were performed using the Factor 10.03.01 and AMOS 23.0 software.

Results

Two independent translators who were native to Portuguese and fluent in English were hired to construct two versions of the questionnaire in the Portuguese spoken in Brazil (T1 and T2). These translations were discussed by a committee of experts, which was composed of four physicians with experience in cross-cultural adaptation of instruments and practices within occupational medicine, health promotion and epidemiology. The clarity of the translations, the use of colloquial language and the semantic equivalence of the items were objects of discussion for elaboration of the first combined version of the translation (V1).

The first topic discussed at the consensus meeting was the title. The original in English placed value only on the dimension “professional fulfillment,” but given that in the Brazilian context “burnout” is already part of the popular vocabulary and is evaluated by the questionnaire, we chose to name the Brazilian version the “Professional Fulfillment and Burnout Index.” Some phrases had been translated differently by the two translators and it was necessary to reach a consensus regarding words or terms.

The first pretest was conducted with five invited medical professionals to verify whether words and expressions used in the questionnaire were appropriate to the Brazilian context. Among the suggestions, there was an indication for a change in the opening sentence of the burnout dimension, to make it more appropriate by asking about frequency since this the responses were now in this format. Then, the expert committee produced a second version of the translation (V2). This was sent for back-translation, done by two translators who were native to English and fluent in the Portuguese spoken in Brazil (B1 and B2).

Following this, the committee met again in order to define a consolidated version of the back-translation into English (B3). Version B3 was evaluated by one of the authors of the original questionnaire, who was asked to issue an evaluation ranging from 0 to 5 points for each of the 31 items. For items that scored below 4.5, this evaluator was asked to provide a justification. The average score was 4.125 and the comments made led to modifications after another meeting of the committee of experts.

The modifications supported formatting of the third version in Portuguese (V3), from which the back-translation into English was version B4. The evaluator considered that B4 was appropriate. The V3 pretest was applied to 20 other medical professionals. As there was no difficulty in understanding the content or filling out the questionnaire, the committee of experts considered that V3 was the final version of the cross-cultural adaptation for the Portuguese spoken in Brazil, for the Professional Fulfillment Index questionnaire (Table 1).

Table 1.

Items of the Original English Language Questionnaire and Final Version of the Adaptation to the Portuguese Spoken in Brazil of the Professional Fulfillment and Burnout Index.

Code	Original	Brazilian Version
TITLE	Professional Fulfillment Index (PFI)	Índice de Burnout e Realização Profissional
DIMENSION 1	Professional fulfillment	Realização profissional
OPENING 1	How true do you feel the following statements are about you at work during the past two weeks?	O quanto você acha que as afirmações abaixo são verdadeiras, sobre você no seu trabalho nas duas últimas semanas?
LIKERT 1	not at all true; somewhat true; moderately true; very true; completely true	nada verdadeiro; um pouco verdadeiro; moderadamente verdadeiro; muito verdadeiro; totalmente verdadeiro
PF1	I feel happy at work	Eu me sinto feliz no trabalho
PF2	I feel worthwhile at work	Eu me sinto importante no trabalho
PF3	My work is satisfying to me	Meu trabalho me satisfaz
PF4	I feel in control when dealing with difficult problems at work	Eu me sinto no controle ao lidar com dificuldades no trabalho
PF5	My work is meaningful to me	Meu trabalho é muito significativo para mim
PF6	I’m contributing professionally (e.g. patient care, teaching, research, and leadership) in the ways I value most	A minha participação no trabalho (p. ex. na assistência ao paciente, no ensino, na pesquisa e na liderança) ocorre da forma que eu mais valorizo
DIMENSION 2	Burnout (work exhaustion and interpersonal disengagement)	Burnout (exaustão profissional e desengajamento interpessoal)
OPENING 2	To what degree have you experienced the following?	Em qual medida você já vivenciou as situações abaixo
LIKERT 2	not at all; very little; moderately; a lot; extremely	nada; pouco; moderadamente; muito; muitíssimo
OPENING PE	During the past two weeks I have felt…	Durante as duas últimas semanas tenho sentido…
PE1	A sense of dread when I think about work I have to do	Uma sensação de pavor quando penso no trabalho que tenho a fazer
PE2	Physically exhausted at work	Exaustão física no trabalho
PE3	Lacking in enthusiasm at work	Falta de entusiasmo no trabalho
PE4	Emotionally exhausted at work	Exaustão emocional no trabalho
OPENING ID	During the past two weeks my job has contributed to me feeling…	Durante as duas últimas semanas, meu trabalho contribuiu para me sentir…
ID1	Less empathetic with my patients	Menos empático com os pacientes
ID2	Less empathetic with my colleagues	Menos empático com meus colegas
ID3	Less sensitive to others’ feelings/emotions	Menos sensível aos sentimentos/emoções dos outros
ID4	Less interested in talking with my patients	Menos interessado em conversar com meus pacientes
ID5	Less connected with my patients	Menos conectado com meus pacientes
ID6	Less connected with my colleagues	Menos conectado com meus colegas

Exploratory Factor Analysis

The database consisting of the data from the 432 participants (female 57.6%; mean age 49.6 years–sd 14). The mean time since medical graduation was 18.5 years (sd 9.67). Most of them was trained in Occupational Medicine through post-graduation courses (83.8%) and had a full-time job (88.0%). About job tasks, they performed worker´s clinical evaluations (84.9%) and/or services management (69.0%).

A priori, data did not present any constitution that might have negatively affected the results from the analyses. The results indicated that the data presented satisfactory conditions for use of factoring according to the Kaiser–Meyer–Olkin index (KMO = .92), Bartlett’s sphericity test (X² = 4891.1; degrees of freedom = 120; P < .001) and the determinant matrix (P < .001).

A parallel analysis on dimensionality indicated the existence of one dimension through the criterion of the mean and 2 dimensions when the 95th percentile was applied. Closeness analysis indicated that diversity of dimensionality existed: regarding the general indexes, the UNICO indicated one-dimensionality (.96), the ECV indicated multidimensionality (.83) and the MIREAL was the limit for one-dimensionality (.30). Thus, the confidence intervals could be both one-dimensional and multidimensional.

Possible divergences in the configuration of dimensionality were also found in the BIC test (445.01), which indicated a model with better fit in 3 dimensions, and the Hull method, because CFI (30.31) and RMSEA (2.50) indicated a unidimensional model. Because the theoretical model for the instrument did not recommend unidimensionality, the divergence between the techniques justified testing of the three options. The variances explained by these options could be considered adequate: the unidimensional model explained 61.24%; the two-dimensional model, 72.33%; and the three-dimensional model, 79.86%.

In the three-dimensional model, analysis on the polychoric matrix indicated moderate to strong correlations. The items of “professional fulfillment” (PF) had positive correlations with each other, ranging from .418 to .772. The items of “professional exhaustion” (PE) and “interpersonal disengagement” (ID) had positive correlations between each other, ranging from .410 to .939. The PF had a negative correlation with the items that encompassed the theoretical dimensions conceived for the instrument and ranged from −.269 to −.758. This result indicated that there was a difference in behavior regarding response trends between the PF items and the items of the 2 burnout subdimensions (PE and ID).

Table 2 presents the factor loading and commonality values of the three models. In the unidimensional model, the factor loadings ranged from −.828 to .758, with commonalities from .259 to .717. In this model, the factor loadings were adequate and pointed to signing that differed from that of the items that made up the PF subdimension, in relation to the other items. In the two-dimensional model, the loads ranged from −.827 to .837, with commonalities from .410 to .835. This model presented a problem with regard to satisfactory saturation in the first dimension, but 11 of the 16 items were also saturated in the second dimension, which made it impossible to interpret.

Table 2.

Factors loads and commonalities of the models.

	1 Factor Model		2 Factors Model			3 Factors Model
Item	F1	h²	F1	F2	h²	F1	F2	F3	h²
PF1	−.828	.685	.837	.363	.832	−.198	−.014	.756	.823
PF2	−.707	.500	.716	.377	.655	.125	−.071	.888	.726
PF3	−.795	.633	.806	.389	.802	−.012	−.053	.870	.830
PF4	−.583	.340	.587	.319	.446	−.242	.112	.553	.443
PF5	−.509	.259	.517	.426	.449	.121	.101	.828	.495
PF6	−.676	.457	.682	.340	.581	.064	−.068	.788	.621
PE1	.728	.530	−.722	−.037	.523	.679	.062	−.103	.631
PE2	.645	.417	−.640	.062	.414	.927	0.017	.181	.689
PE3	.847	.717	−.842	−.127	.725	.591	.075	−.313	.783
PE4	.785	.617	−.779	−.006	.606	.974	−.007	.036	.895
ID1	.716	.513	−.727	.398	.686	−.087	.926	.021	.738
ID2	.729	.531	−.734	.317	.639	.173	.696	.041	.631
ID3	.740	.547	−.751	.402	.725	.054	.862	.073	.733
ID4	.807	.652	−.817	.372	.806	−.021	.916	−.028	.843
ID5	.815	.664	−.827	.389	.835	−.076	.976	−.035	.897
ID6	.769	.592	−.769	.229	.644	.213	.602	−.056	.637

h²-communalities.

In the three-dimensional model, the factor loadings ranged from −.553 to .976 and the commonalities from .443 to .897. This model presented better alignment of the items in the respective theoretical factors and had the best factor loadings and commonalities in relation to the other models. There was only one double saturation (PE3), but it was in a borderline situation. One aspect that will require future follow-up was the fact that 8 of the 16 items in this model had factor loadings above .80, which may indicate problems of collinearity/multilinearity of the items (Table 2). From a quantitative point of view, these could generate distortions in the proposed model and from a qualitative perspective they could indicate redundancy of the content of the items. These loads at these levels could also indicate a two-factor model.

A bifactor model and a hierarchical model were tested. The bifactor model did not present an adequate fit, while the hierarchical model fit better but does not bring better contributions to the interpretation of the model. The loadings could also have been due to the sample characteristics, which it will be possible to test in other databases.

Reliability was measured using three indicators with satisfactory results: Cronbach’s alpha (.95), McDonald’s omega (.95), and ORION (.98). These results may have been due to the possible collinearities that were found in the factor loadings, given that it is not recommended that reliability indexes should be above .90. The latent G-H indices of replicability of the three dimensions were shown to be adequate and stable, such that the latent variables could be defined: PF (.93–.88), PE (.93–.87), and ID (.95–.88). The indicators of quality and effectiveness of the model also have adequate indices: factor determinacy index (FDI) with PF (.98), PE (.97) and ID (.97); EAP marginal reliability with PF (.96), PE (.94) and ID (.94); sensitivity ratio (SR) for PF (4.89), PE (3.95), and ID (4.06); and expected percentage of true differences (EPTD) for PF (96%), PE (94.7%), and ID (94.8%) (Table 3).

Table 3.

Synthesis of models.

	Index	Technique	Correlational Method
Exploratory	Adequacy of correlation matrix	Determinant of the matrix	<0 .00001
		Bartlett	4891.1 (df = 120); P < .001
		KMO (Kaiser-Meyer-Olkin)	.918
	Explained variance (Kaiser criterion)		.76
	Explained variance (AP)		79.78
	Polychoric correlation (r_p =)		−.76 to .93
Confirmatory	Robust mean-scaled chi square (X²/df)		3.498
	Non-Normed fit index (NNFI)		0,93
	Comparative fit index (CFI)		0,95
	Goodness of fit index (GFI)		0,90
	Adjusted goodness of fit index (AGFI)		0,87
	Root mean square error of approximation (RMSEA)		0,08
	Root mean square of residuals (RMSR)		0,07
Reliability and replicability	Standardized Cronbach’s Alpha		.94
	McDonald’s Omega		.94
	Construct reliability—index G-H (latent and observed)		PF(.93–.88); PE (.93–.87); DI (.95–.88)
Clossness	Unidimensional congruence (UNICO)		0,97
	Explained common variance (ECV)		0,84
	Mean of item residual absolute loading (MIREAL)		0,30
Efetivity and quality	Factor Determinacy index (FDI)		PF(.98); PE (.97); ID (.97)
	EAP marginal reliability		PF(.96); PE (.94); ID (.94)
	Sensitivity ratio (SR)		PF(4.89); PE (3.95); ID (4.06)
	Expected percentage of true differences (EPTD)		PF(96%); PE (94.7%); ID (94.8%)

PF-Factor 1; PE-Factor 2; ID–Factor 3.

Confirmatory Factor Analysis (CFA)

The CFA was tested in the correlational model and in the hierarchical model. The fit indexes of the model did not present recommendable results, and there was a need to control for covariance errors. Moreover, in the hierarchical model, it was necessary to leave the PF dimension without parameters, because the correlation with the other factors of the three-dimensional model was negative. The covariance error controls made between items ID1-ID2 and ID1-ID6 improved the fit indexes of the hierarchical model: robust mean and variance-adjusted chi square x²/df (100) = 350.49; P < .001; NNFI = .931; CFI = .950; GFI = .904; AGFI = .869. The residual indicators were at good levels (RMSEA = .08; RMSR = .07).

In the path diagram of the confirmatory factor analysis, the factor loadings could be ascertained: these were between .62 and .96. The predictive values of the items could also be determined, and these ranged from .38 to .88 (Figure 2). Factor loadings above .80 were found, as in the exploratory analysis. It was not possible to identify any problems generated in the model at this preliminary stage, due to the high loadings.

Figure 2.

Confirmatory factor analysis path diagram.

Discussion

The process described in this article was the first cross-cultural adaptation of the Professional Fulfillment Index questionnaire. This limits comparisons with other attempts to verify the applicability of this instrument outside the context of its origin, which can be characterized as an Anglo-Saxon developed country. Adaptation of scales for use in different languages and contexts can help in discussing similar aspects that have theoretical importance and practical application. Use of the same instrument allows comparison of results and promotes exchanges between researchers at the international level.

The version for the Portuguese language spoken in Brazil can be considered similar to the original. Adapting an instrument to another language is a complex process involving technical, linguistic, and semantic aspects that need to be taken into account.¹⁴ Comparison between the instrument in English and the back-translation of the third version in the Portuguese spoken in Brazil showed that the meanings of the title, opening phrases, statements and Likert scale were satisfactory, in the view of one of the authors of the original questionnaire. There were small changes between the literally translated version and the final version, which can be considered to represent cross-cultural adaptation that maintained the original context of the tool and added characteristics peculiar to the Brazilian reality.

As an acculturation strategy, the recommended process of obtaining two translations and back-translations was followed in order to expand the options for terms that could be ambiguous in one of the languages.¹⁴ This was observed by the committee of experts, in making choices for producing consolidated versions of the translation and back-translation. Nonetheless, there was a need to review the work until it achieved better evaluation by an author of the original version. The knowledge of the English language among all the members of the committee and the professional field of the questionnaire were components that placed value on the result from this study.¹⁴

The validation study on the original tool was conducted in the USA among a sample of 250 physicians at a single academic center.¹⁰ The present study was conducted among a larger group, with more women, with a higher mean age and with lower variability of medical practice specialty. Although Reichenheim and Moraes¹⁵ indicated that differences in the characteristics of populations surveyed with the same instrument might cause variability in the parameters, the psychometric study on this instrument demonstrated that its validity and reliability were acceptable. This shows that this instrument presents evidence of a consistent and reliable internal structure for measuring the desired construct.

Regarding the factor validity of the scale, the analysis indicated that the version composed of 16 statements presented the possibility for studying two or three dimensions. Like in the original, there was a positive correlation between the burnout dimensions (PE and ID) and a negative correlation between these and the dimension of professional fulfillment.¹⁰ Because PE and ID, as negative aspects for the well-being of physicians, show a correlation with PF as a positive aspect, a two-dimensional model may be ideal. Hence this has already been used by researchers.³⁴ However, it is possible to categorize burnout into its 2 categories, which makes it possible to go more deeply into discussion about the negative aspects of exhaustion and disengagement.

Exploratory and confirmatory analyses showed the adequacy of the items for measuring the construct of the instrument, with adequate factor loading levels and commonalities; variance explained by the AFE and the factor loadings in the CFA; and good levels for regression values and fit index models. The high level of replicability of the scale demonstrated that the instrument was stable for use in other populations and samples. In short, the model would maintain its psychometric properties under other conditions, according to these preliminary data.³⁰

Regarding internal consistency, the analysis indicated that the format and method of the instrument had the capacity to reliably evaluate the constructs. Cronbach’s alpha values were higher than those found in the analysis on the original instrument, perhaps due to the greater homogeneity of the population (occupational physicians vs a variety of medical professionals). Other tests that were applied to corroborate this result regarding the adaptation were not used in the validation of the original.

The contemporary psychometric approach used in this study, which gave rise to indicators that adequately and satisfactorily explained the construct measured, including all the indicators of exploratory and confirmatory factor solutions, as well as those of reliability, quality, and efficacy of the model, demonstrated evidence of instrument validity.

The PFI was developed to evaluate positive aspects of physicians’ role and performance, i.e. professional fulfillment; and also to assess negative aspects, that is, burnout,¹⁰ which can influence their well-being, quality of life and productivity. Use of this questionnaire can help to understand the physicians’ professional experiences, which have an impact on the quality of service delivery, with social and economic repercussions. It can also provide information to those involved in the development of practices for promoting well-being among those healthcare professionals.¹⁰ This can assist in analyzing the relationship between workloads and workers’ capacity, and in mapping out critical situations in order to propose micro-level individual and collective interventions that reduce exhaustion and expand professional fulfillment, and macro-level policies in order to mitigate stressful occupational contexts.

Individual and collective interventions can be designed or specified based on PFI assessment results. For assessment of change over time, we recommend tracking change in total scores (or average item scores), rather than tracking of percentages of individuals scoring above cut-points. However, use of cut-points for categorization can assist with cost effective intervention targeting, with more intensive intervention going to healthcare workers with higher levels of distress.

Regular assessment using the PFI can facilitate a population health approach³⁵ to iterative improvement of healthcare workers occupational well-being, using previously published cut-points.¹⁰ Individuals who do not score high on professional fulfillment and do not score high on overall burnout may be considered at higher risk for burnout than those who score below the burnout cut-point and do score high on professional fulfillment. Selective primary prevention interventions for this group are appropriate.

More intensive indicated interventions for those with high burnout (high occupational distress) are appropriate and may serve as indicated primary prevention to mitigate risk for more generalized distress that manifests as clinical syndromes such as clinical depression and anxiety disorders. It is important to note that the PFI assesses occupational well-being (professional fulfillment) including occupational distress (burnout) and is not intended as a clinical screen for generalized or clinical distress. Other established instruments designed to assess clinical problems such as depression and anxiety are needed and also have value in an overall population health approach to occupational wellness.

Limitations and Strengths

This study involved a sample that differed slightly from the group studied by Trockel and his colleagues in validating the original instrument,¹⁰ which compromises the direct comparability of the data. Although the participants of the present study were a national sample, they were recruited according to convenience. Thus, the study group was composed of physicians who were providing occupational health services and was not representative of all physicians in Brazil. In addition, some characteristics may be specific to these professionals, including deeper technical knowledge regarding risk factors at work and outcomes related to the constructs analyzed by the instrument.

One strong point of this study was its contemporary statistical techniques, which demonstrated that the model works well in the Brazilian context. Therefore, it is recommended that use of this instrument should be replicated in populations with characteristics differing from those of the present study, in order to confirm (or not) that the psychometric properties described here are maintained.

Studying exhaustion and professional fulfillment among occupational physicians can help in mapping out situations of potential risk that have impacts on the health and quality of life of these professionals. The PFI is a simple and low-cost instrument that may be useful for organizations to evaluate their medical staff, in order to develop actions to improve well-being and engagement among active physicians. This study presented the process of cross-cultural adaptation of this instrument, to produce a version in the Portuguese spoken in Brazil, which was done in accordance with the equivalence steps that have been recommended in the scientific literature.

Use of tools during the data collection process that have been validated through scientifically recognized methods may minimize measurement bias. Future scientific evidence demonstrated through research using this instrument may help in formatting programs, strategies, and policies for health promotion and disease prevention, thereby minimizing individual and collective damage, given that physicians’ occupational well-being affects their performance at work, which affects the teams they work with and the patients they care for.

So what?

What is already known on this topic?

The questionnaire “Professional Fulfillment Index” (PFI) was developed to evaluate positive aspects of physicians’ role and performance, that is, professional fulfillment; and also to assess negative aspects, that is, burnout and interpersonal disengagement, which can influence their well-being, quality of life, and productivity.

What does this article add?

This study presents the process of cross-cultural adaptation of the PFI to the Brazilian context, and the version can be considered similar to the original. Psychometrics analyses showed the adequacy of the items for measuring the construct of the instrument among physicians of Brazil.

What are the implications for health promotion practice or research?

The PFI can help in mapping potential risk situations for negative impacts on physicians' occupational well-being and performance at work in Brazil, in order to propose individual and collective interventions that reduce exhaustion and expand professional fulfillment, such as stress management strategies and improvement of the physical and psychosocial work environment.

Footnotes

Acknowledgements

The authors would like to acknowledge the Care Global Partners Brasil for financial support. The Brazilian National Association of Occupational Medicine (Associação Nacional de Medicina do Trabalho–ANAMT) and the Occupational Medicine Association of São Paulo State (Associação Paulista de Medicina do Trabalho–APMT) provided support for contacting participants.

Author Contributions

JSSJ made a substantial contribution to the concept and design of the work, acquisition, analysis and interpretation of data; and drafted the article. AJNO, BCS and RBD made a substantial contribution to the concept and design of the work; and revised it critically for important intellectual content. AAC made a substantial contribution to the acquisition, analysis and interpretation of data; and revised it critically for important intellectual content. MT made a substantial contribution to the acquisition, analysis and interpretation of data; and revised it critically for important intellectual content. All authors approved the version to be published and take public responsibility for appropriate portions of the content.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Care Global Partners Brasil

Ethical approval

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (Research Ethics Committee of the São Camilo University Center - opinion report no. 4.077.484/2020), with registration on the Brazil Platform (CAAE 31400620.0.0000.0062). Informed consent was obtained from all patients for being included in the study.

ORCID iDs

João Silvestre Silva-Junior

Alberto José Niituma Ogata

Bruno Correia Scarpellini

Arthur Arantes da Cunha

Mickey Trockel

Rodrigo Bornhausen Demarch

References

Guseva Canu

Marca

Dell’Oro

, et al. Harmonized definition of occupational burnout: A systematic review, semantic analysis, and Delphi consensus in 29 countries. Scand J Work Environ Heal. 2021;47(2):95-107.

Aronsson

Theorell

Grape

, et al. A systematic review including meta-analysis of work environment and burnout symptoms. BMC Public Health. 2017;17:264.

Salvagioni

DAJ

Melanda

Mesas

González

Gabani

de Andrade

. Physical, psychological and occupational consequences of job burnout: A systematic review of prospective studies. PLoS One. 2017;12(10):e0185781.

Tawfik

Scheid

Profit

, et al. Evidence relating health care provider Burnout and quality of care a systematic review and meta-analysis. Ann Intern Med. 2019;171(8):555-567.

Tawfik

Profit

Morgenthaler

, et al. Physician Burnout, Well-being, and Work Unit Safety Grades in Relationship to Reported Medical Errors. Mayo Clin Proc. 2018;93(11):1571-1580.

Han

Shanafelt

Sinsky

, et al. Estimating the attributable cost of physician Burnout in the United States. Ann Intern Med. 2019;170(11):784-790.

Maslach

Jackson

. The measurement of experienced Burnout. J Organ Behav. 1981;2(2):99-113.

Rotenstein

Torre

Ramos

, et al.. Prevalence of Burnout among physicians a systematic review. JAMA 2018;320(11):1131-1150.

Kristensen

Borritz

Villadsen

Christensen

. The Copenhagen Burnout Inventory: A new tool for the assessment of Burnout. Work & Stress. 2005;19(3):192-207.

10.

Trockel

Bohman

Lesure

, et al. A brief instrument to assess both burnout and professional fulfillment in physicians: Reliability and validity, including correlation with self-reported medical errors, in a sample of resident and practicing physicians. Acad Psychiatry. 2018;42(1):11-24.

11.

Zhang

Riba

Magrini

, et al. Assessing burnout and professional fulfillment in breast surgery: Results from a national survey of the american society of breast surgeons. Ann Surg Oncol. 2019;26(10):3089-3098.

12.

Herdman

Fox-Rushby

Badia

. “Equivalence” and the translation and adaptation of health-related quality of life questionnaires. Qual Life Res. 1997;6(3):237-247.

13.

Herdman

Fox-Rushby

Badia

. A model of equivalence in the cultural adaptation of HRQoL instruments: the universalist approach. Qual Life Res. 1998;7(4):323-335.

14.

Guillemin

Bombardier

Beaton

. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46(12):1417-1432.

15.

Reichenheim

Moraes

. Operationalization of cross-cultural adaptation of measurement instruments used in epidemiology. Rev Saude Publica. 2007;41(4):665-673.

16.

MacCallum

. 2001 Presidential Address: Working with imperfect models. Multivariate Behavioral Research. 2003;38(1):113-139.

17.

Costello

Osborne

. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Pract Assessment, Res Eval 2005;10(1):7.

18.

Hair

Black

Babin

Anderson

. Multivariate data analysis. 8th ed. Boston: Cengage; 2019.

19.

Auerswald

Moshagen

. How to determine the number of factors to retain in exploratory factor analysis: A comparison of extraction methods under realistic conditions. Psychol Methods. 2019;24(4):468-491.

20.

Goretzko

Pham

TTH

Bühner

. Exploratory factor analysis: Current use, methodological developments and recommendations for good practice. Curr Psychol. 2021;40(7):3510-3521.

21.

Bandalos

. Measurement theory and applications for the social sciences. New York, NY: Guilford Publications; 2018.

22.

Finch

. Using fit statistic differences to determine the optimal number of factors to retain in an exploratory factor analysis. Educ Psychol Meas. 2019;80(2):217-241.

23.

Dobriban

Owen

. Deterministic parallel analysis: an improved method for selecting factors and principal components. J R Stat Soc Series B Stat Methodol. 2019;81(1):163-183.

24.

Timmerman

Lorenzo-Seva

. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychol Methods. 2011;16(2):209-220.

25.

Schwarz

. Estimating the Dimension of a Model. Ann Stat. 1978;6(2):461-464.

26.

Lorenzo-Seva

Timmerman

Kiers

HAL

. The hull method for selecting the number of common factors. Multivariate Behav Res. 2011;46(2):340-364.

27.

Ferrando

Lorenzo-Seva

. Assessing the quality and appropriateness of factor solutions and factor score estimates in exploratory item factor analysis. Educ Psychol Meas. 2017;78(5):762-780.

28.

Choi

Kim

Chen

Dannels

. A comparison of maximum likelihood and Bayesian estimation for polychoric correlation using Monte Carlo simulation. J Educ Behav Stat. 2011;36(4):523-549.

29.

Hancock

Mueller

. Rethinking construct reliability within latent variable systems. In: Cudeck

du Toit

Soèrbom

, eds. Structural equation modeling: Present and future A Festschrift in honor of Karl Jöreskog. Chicago: ScienticSoftware International; 2001.

30.

Rodriguez

Reise

Haviland

. Evaluating bifactor models: Calculating and interpreting statistical indices. Psychol Methods. 2016;21(2):137-150.

31.

Grice

. Computing and evaluating factor scores. Psychol Methods. 2001;6(4):430-450.

32.

Rodriguez

Reise

Haviland

. Applying Bifactor Statistical Indices in the Evaluation of Psychological Measures. J Pers Assess. 2016;98(3):223-237.

33.

Tabachnick

Fidell

. Using multivariate statistics. Boston: Pearson; 2018.

34.

Shanafelt

Makowski

Wang

, et al. Association of burnout, professional fulfillment, and self-care practices of physician leaders with their independently rated leadership effectiveness. JAMA Netw Open. 2020;3(6):e207961.

35.

Trockel

Corcoran

Minor

Shanafelt

. Advancing physician well-being: a population health framework. Mayo Clinic Proceedings. 2020;95(11):2350-2355.