The Bifocal Lens Model and Equation

Abstract

Background. Brunswik’s Lens Model and lens model equation (LME) have been applied extensively in medical decision making. Clinicians often face the dual challenge of formulating a judgment of patient risk for some adverse outcome and making a yes or no decision regarding a particular risk-reducing treatment option. Objective. In this article, I examine the correlation between clinical risk judgments and treatment-related decisions, referring to this linkage as “cohesion”. A novel form of the LME is developed to decompose cohesion. The approach is “bifocal” in that it focuses on 2 sets of linked responses from the same individual. Methods. Data from 2 studies were analyzed to illustrate how individual differences in cohesion could be explained by differences in the parameters of the bifocal lens model equation (BiLME). Results. Cohesion varied because of differences in cognitive control, similarities in the judgment and decision policies, and a possible reliance on a subjective threshold value applied to the judgments to make decisions. The parameters of the BiLME accounted for individual differences in cohesion; however, their relative influences differed between the two studies. Conclusion. The BiLME links the results from two regression models—one linear and one logistic—based on the same set of cases. In its current form, the equation holds promise for understanding cognitive individual differences that could underlie practice variation. With minor modifications, it becomes possible to apply the equation to traditional, dual-system judgment analysis studies, where continuous judgments are compared with an ecology composed of dichotomous outcomes, or vice versa. In this regard, the BiLME is quite flexible and adds to the set of tools available to judgment analysts.

Keywords

Brunswik’s Lens Model clinical judgment lens model equation

In many patient encounters, clinicians face the dual challenge of formulating a judgment of patient risk for some adverse outcome and making a yes or no decision regarding a particular risk-reducing treatment option; for example, judging a hospital patient’s risk for acute deterioration and the decision to alert the hospital’s rapid response team. When asked to review the same set of several patient profiles, clinicians may show variation in the degree to which their quantitative judgments of patient risk are correlated with their yes or no decisions to take a specific risk-reducing action. The term “cohesion” is introduced to describe how well these two related cognitive responses (one quantitative judgment, and one dichotomous decision) correlate or hang together within an individual. In this sense, cohesion implies that, for a given clinician, higher likelihoods of taking risk-reducing actions will co-occur with higher judgments of patient risk, whereas lower likelihoods of taking risk-reducing actions will co-occur with lower judgments of risk. Individual differences in cohesion may contribute to practice variation among clinicians, or at least provide insight into the cognitive origins of such variation. In this article, I develop and apply a novel form of Brunswik’s Lens Model and the lens model equation (LME) to decompose these correlations to gain an understanding of such individual differences in the cohesion of clinical risk judgments and treatment-related decisions.

Over the years, the LME has evolved as a useful tool to help researchers understand judgments in various contexts. It has proven particularly useful in studying expert judgment and decision making in health care.^1-13

The remainder of this article is laid out as follows: First, Brunswik’s Lens Model and some key developments in the LME over several years are reviewed. Second, the bifocal lens model and equation, a modification that targets the linkage between judgments of risk and related decisions, is introduced. Third, the bifocal lens model equation is applied to data from 2 studies to illustrate how it can provide insight into the cognitive sources underlying individual differences in the cohesion of clinical risk judgments and treatment-related decisions. Finally, how this new approach can be used to reveal performance variation in judgment tasks and the generality of the equation are discussed.

The Lens Model

Brunswik’s Lens Model is a general approach to the analysis of an individual’s judgments made under conditions of uncertainty where outcomes are probabilistically related to perceptible cues in the environment.¹⁴ This model is useful in describing a clinician’s use of clinical variables (cues) in forming judgments and the importance of these cues to their judgment, and facilitates comparison of the relationship of cues to criterion with that of cues to clinical judgment.

Referring to Figure 1, X_k denotes the clinical variables (cues) that vary in the degree to which they relate to an actual clinical outcome (criterion). Y refers either to the criterion of interest (on the left side) or to judgments (on the right side). The subscripts e and s designate respectively the ecology (“real world”) and the subject (i.e., the study participant). The hats (^) denote predicted values of Y (either criterion, or judgments) and are usually obtained from linear regression models of the respective “sides” of the lens. Cues are related to criterion values via some weight (w_ek) and correspondingly to the subjective judgments (w_sk). These weights are typically standardized regression coefficients; thus, the model allows for the cues to be inter-correlated. Clinical prediction or diagnostic rules, derived in epidemiology using multiple regression techniques, correspond to the ecological side of the lens model.

Figure 1

Brunswik’s Lens Model. See text for details.

The term r_a is the correlation between the criterion values (Y_e) and the individual clinician’s judgments (Y_s) and is referred to as “achievement” or accuracy. For example, the risk for coronary heart disease for each of a sample of patients based on the Framingham risk equation (criterion) and the clinician’s judgments of each patient’s risk for coronary heart disease, could be quantified with r_a. The correlation between the predicted values $({\hat{Y}}_{e} and {\hat{Y}}_{s})$ obtained from the two linear regression equations (G), referred to as “policy matching”, represents the amount of knowledge that the individual clinician has about the relationships of the cues to the criterion. As the similarity in the individual’s cue-weighting strategy (set of w_sk) and the weights from the ecology (set of w_ek) increases, G also increases. The multiple correlations from the two regression equations convey the predictability of the ecology (R_e) and the amount of “cognitive control” or how systematically the individual applies his or her judgment policy (R_s). The correlation between the residuals $(Z_{e} and Z_{s})$ from the two regression equations (C) reflects the extent to which unmodeled knowledge in the individual’s judgments matches unmodeled aspects of the ecology (i.e., common reliance on variables not included in either model), or chance agreement between model errors.

The Lens Model Equation

The accuracy of judgments can be decomposed using the LME that combines two linear regression models: one of the relationships between the ecological criterion and the cues, and another between the individual’s judgments and the cues, both fit to the same set of cases.^15,16

The LME defines judgment accuracy, or achievement, as the correlation between judgments and criterion values, r_a, and then descriptively decomposes achievement into the 4 components: G, R_e, R_s, and C, all of which are themselves, correlations. The components are related in the classic equation:

r_{a} = G R_{e} R_{s} + C \sqrt{(1 - R_{e}^{2})} \sqrt{(1 - R_{s}^{2})}

wherein each term represents a correlation: $G = r ({\hat{Y}}_{e}, {\hat{Y}}_{s})$ , $R_{e} = r (Y_{e}, {\hat{Y}}_{e})$ , $R_{s} = r (Y_{s}, {\hat{Y}}_{s})$ and $C = r (Z_{e}, Z_{s})$ , and where $Z_{e} = (Y_{e} - {\hat{Y}}_{e})$ and $Z_{s} = (Y_{s} - {\hat{Y}}_{s})$ .¹⁶

Cooksey¹⁷ and Stewart^18,19 extended the LME to examine the relationships between dichotomous decisions and a dichotomous criterion applying logistic, rather than linear, regression. Unlike linear regression, in which the correlation between predicted values and residual values resulting from a given model is zero, this is not the case with logistic regression, and so the logistic LME requires three C terms to represent the relationship between the two regression models. Stewart^{18, 19} showed that R_e needs to be replaced by the ratio of the computed standard deviations of predicted values (probabilities) to the standard deviation of actual (dichotomous) values $(σ_{{\hat{Y}}_{e}} / σ_{Y_{e}})$ and that $\sqrt{1 - R_{e}^{2}}$ be replaced by the ratio of the standard deviation of the residuals to that of the actual values $(σ_{Z_{e}} / σ_{Y_{e}})$ in the ecology and similarly for R_s measuring cognitive control. Thus, using their notation, the logistic form of the LME is:

r_{a} = G \frac{σ_{{\hat{Y}}_{e}}}{σ_{Y_{e}}} \frac{σ_{{\hat{Y}}_{s}}}{σ_{Y_{s}}} + C_{1} \frac{σ_{Z_{e}}}{σ_{Y_{e}}} \frac{σ_{Z_{s}}}{σ_{Y_{s}}} + C_{2} \frac{σ_{{\hat{Y}}_{e}}}{σ_{Y_{e}}} \frac{σ_{Z_{s}}}{σ_{Y_{s}}} + C_{3} \frac{σ_{Z_{e}}}{σ_{Y_{e}}} \frac{σ_{{\hat{Y}}_{s}}}{σ_{Y_{s}}}

where C₁ is the correlation between the residuals of the 2 models, $r (Z_{e}, Z_{s})$ , and C₂ and C₃ are cross-correlations between the predictions of one model and the residuals of the other, $r ({\hat{Y}}_{e}, Z_{s})$ , and $r (Z_{e}, {\hat{Y}}_{s})$ . While all three C terms are needed to fully reconstruct r_a in the logistic LME, C₁ is sufficient for representing the extent to which unmodeled knowledge in the individual’s judgments matches unmodeled aspects of the ecology.

In the context of studying interpersonal conflict, Naylor and Schenck²⁰ and Brehmer and Hammond²¹ adapted the linear LME (Equation 1) to examine the amount of agreement between two individuals judging the same set of cases by replacing the ecology regression equation with a regression equation relating the second individual’s judgments and cues:

r_{a} = G R_{1} R_{2} + C \sqrt{1 - R_{1}^{2}} \sqrt{1 - R_{2}^{2}}

In this form, r_a now refers to agreement between two individuals; R₁ and R₂ represent the cognitive control of the first and second individual, respectively; G may be interpreted as the amount of similarity between the judgment policies of the two individuals, and C is the similarity in unmodeled aspects of their policies that contribute to their agreement. Beckstead et al.⁸ used this form to examine agreement among physicians judging the benefits of cardiac rehabilitation for the same set of patients.

The Bifocal Lens Model and Bifocal Lens Model Equation

When an individual provides two related responses (one quantitative judgment of risk and one dichotomous decision to take a risk-reducing action) to the same set of cases, it is possible to examine the correlation, or cohesion, between these two sets of responses. This type of dual-response task can be represented with the Bifocal Lens Model shown in Figure 2.

Figure 2

The Bifocal Lens Model in clinical decision-making context. See text for details.

Consistent with the classic Lens Model, the ecology is represented on the left-hand side and the subjective responses from the clinician are on the right-hand side. Towards the center are the cues, in this case a set of patient characteristics. The lines originating from the cues converge to the ecological criterion (Risk) and to the subjective judgment of risk, in much the same way that light passing through a lens can be focused at a point lying on either side. These lines represent the influence of the cues and are typically standardized coefficients estimated from multiple regression equations. Achievement (r_a) or the accuracy of the individual’s judgments compared with the ecological criterion is represented by the arc in the lower left-hand side of the figure and is the correlation coefficient decomposed in the classic LME.

In the Bifocal Lens Model, the metaphor is extended to encompass two subjective focal points on the right-hand side, one proximal and one distal. As the figure illustrates, both the proximal judgment of risk and the distal decision to act are modeled on the same set of cues via linear and logistic regression, respectively. Cohesion (r_jd), or the correlation between the two sets of responses from an individual, is represented by the arc connecting judgments of risk and decisions to act in the lower right-hand side.

Using the interpersonal conflict form of the LME (Equation 3) that links the judgments from two individuals as a point of departure, replacing the quantitative judgments from individual number 2 with a second set of dichotomous responses (i.e., decisions) from individual number 1, and incorporating modifications for a logistic regression (Equation 2) to handle this second set of dichotomous responses, we have the Bifocal Lens Model Equation (BiLME):

r_{j d} = G_{j d} R_{j} \frac{σ_{\hat{d}}}{σ_{d}} + C_{1} \sqrt{1 - R_{j}^{2}} \frac{σ_{Z_{d}}}{σ_{d}} + C_{2} R_{j} \frac{σ_{Z_{d}}}{σ_{d}} + C_{3} \sqrt{1 - R_{j}^{2}} \frac{σ_{\hat{d}}}{σ_{d}}

In this form, cohesion, r_jd—the correlation between a clinician’s judgments (j) of patient risk and decisions (d) to take a risk-reducing action—is a function of internal policy matching (G_jd) or the extent to which the clinician used the same cue-weighting strategy when formulating these two responses, the amount of cognitive control in the clinician’s judgment policy (R_j), and decision policy, $R_{d} = σ_{\hat{d}} / σ_{d}$ , and three C terms. As in Equation 2, C₁ is the correlation between the residuals of the two models, $r (Z_{j}, Z_{d})$ , in this case, the judgment model and the decision model in this case, and is sufficient for representing the degree to which the two sets of residuals are systematic and related. C₂ and C₃ are cross correlations, as in Equation 2, $r ({\hat{Y}}_{j}, Z_{d})$ and $r (Z_{j}, {\hat{Y}}_{d})$ .

In the traditional linear forms of the LME, (Equations 1 and 3), the correlation between the two sets of residuals, is simply C. In the logistic form of the LME (Equation 2), C₁ is defined as the correlation between the two sets of residuals. Whether discussing C or C₁, this term may reflect common reliance on variables not included in either regression model.

Unique to the BiLME, however, the correlation between the two sets of residuals can also represent important psychological aspects of the relationship between the individual’s judgments and decisions. Much like a partial correlation coefficient, C₁ represents the relationship remaining between an individual’s judgments and decisions when the influences of the cues have been partialed out (removed) from both via regression. That is, in the BiLME, C₁ is a component of cohesion that is unrelated to systematic cue utilization. For instance, a clinician may have a tendency to decide “take action” following risk judgments that are above a certain subjective value and to decide “take no action” following judgments that fall below that same value. In the BiLME, one interpretation of C₁ is that it reflects the extent to which, having formed a judgment based on the cues, the clinician simply compares them to a subjective threshold value in order to make a decision. It may also be the case that C₁ is unrelated to risk judgment accuracy (i.e., achievement, r_a).

Also, because the two regression equations forming the BiLME (one modeling risk judgments, and one modeling decisions to act) are based on responses from the same clinician, it is expected that C₁ will be larger than in traditional LME applications, where the two regression equations are based on data originating from two independent sources (e.g., ecology and judge in the standard Lens Model, or, judge 1 and judge 2 in the interpersonal conflict form). Historically, in traditional LME studies, the correlation between residuals tends to be near zero (0.04; 95% CI, 0.02 to 0.06).²²

Applications

Having laid out the development and rationale for the BiLME, data from two studies are presented in which the cohesion between judgments of patient risk and decisions to take risk-reducing actions is examined to illustrate its usefulness. The first study involved registered nurses (RNs) who judged the risk of patients experiencing critical events during hospital stays and decisions to alert hospital rapid response teams. The second study examined nurse practitioners (NPs) who made judgments of patient risk for developing coronary heart disease and decisions to refer patients to cardiologists. In both studies, individual differences in cohesion (r_jd) were examined in relation to BiLME parameters (R_j, R_d, G_jd and C₁) and in relation to judgment accuracy (r_a).

Both studies used representative designs, with the intent to sample the substance of the ecology (i.e., patient profiles) so that it retains its realistic content and feel from the participant’s point of view. Hammond discusses the difficulty of doing so and suggests that one approach is to retain the distributions and intercorrelations among cues that the participant would ordinarily encounter.²³ The advantage here comes in the form of greater generalizability. In both studies, following Wigton’s suggestions, published distributions and correlations of the cues and criterion (described in the Method sections below) were used to construct a population of patient cases.²⁴ After constructing such a population, cases were randomly selected for presentation. The means, standard deviations, and correlations among cues and criterion in the sample of cases were checked for consistency with those reported in the referenced articles before presentation to participants.

Study 1: Judging Risk for Acute Deterioration and Decision to Alert Rapid Response Teams

Method

Participants

Twenty-five RNs participated in this online study.

Materials

Participants read brief descriptions, or profiles, of 100 hypothetical (and representative) patients admitted to the emergency room complaining of chest pain. Each description presented 7 pieces of information (cues): heart rate, respiratory rate, oxygen saturation, respiration support, level of consciousness, urine output in the past 4 h, and systolic blood pressure. These cues have been shown to be important predictors of the clinical condition of hospital patients and have been incorporated into the modified early warning score (MEWS), a risk index for acute deterioration and critical events, such as cardiopulmonary arrest, unplanned admission to the ICU, and death.²⁵ The means, standard deviations, and correlations among cues and criterion in the sample of profiles were then checked for consistency with those reported by Subbe et al.²⁵ Patient values on the MEWS served as the ecological standard for risk (as a proxy for actual dichotomous outcome) and permitted calculation of r_a.

Procedure

Each patient profile was presented on a separate screen. For each of the 100 profiles, participants provided a subjective judgment of how likely (0% chance to 100% chance) they believed the patient was to deteriorate or to experience an adverse critical event over the next 12 h and indicated whether or not they would alert the hospital’s rapid response team to monitor the patient. Completion of the task took 33 min on average (range, 25 to 52 min).

Data Analysis

Data were analyzed for each participant separately using multiple linear and multiple logistic regressions. BiLME parameters and judgment accuracy (r_a) were calculated for each participant. Fisher’s r-to-Z transformation was applied to all correlation coefficients before aggregation, except where noted.

Results

Data from 2 participants were discarded; 1 because the participant only completed 74 of the 100 profiles, and the other because the participant gave only 4 alert decisions, all based on the same value of a single cue, which prevented computation of the parameter covariance matrix in the logistic regression model. This left n = 23 for analysis. Of these, 20 were female. The average age of participants was 36.9 y (SD, 11.5 y) and participants had been in practice as a nurse, on average, 10.5 y (SD, 9.9 y).

The various BiLME parameters (r_jd, G_jd, R_j, R_d, and C₁) as well as judgment accuracy (r_a) are shown for each participant in Table 1. As the BiLME focuses on a new aspect of the Lens Model, that is, the relationship between judgments and decisions based on the same cases (r_jd), I felt it important to first comment on the accuracy (r_a) of the participants’ judgments in order to establish comparability with previous research involving the traditional LME before discussing decomposition of r_jd. In a meta-analysis of 86 studies that employed the LME between 1954 and 2007, Karelaia and Hogarth²² reported that average judgment accuracy was 0.56 (95% CI, 0.53 to 0.59). In a second meta-analysis of 35 studies employing the LME conducted by Kaufmann and Athanasou²⁶, judgment accuracy was reported for 9 studies in medicine (mean r_a = 0.40; 95% CI, 0.31 to 0.59). Three additional studies (not included in either meta-analysis) involving nurses or NPs found average judgment accuracy values of 0.42⁹, 0.553¹⁰, and 0.675.¹² In the current study, judgment accuracy (r_a) ranged from 0.474 to 0.851, with an average of 0.682. Thus, judgment accuracy seems comparable to that found in previously published studies.

Table 1

Results of Study 1; MEWS Task

Participant	r_jd	G_jd	R_j	R_d	C ₁	r_a
1	0.784	0.888	0.756	0.813	0.425	0.644
2	0.815	0.847	0.788	0.803	0.358	0.752
3	0.748	0.807	0.839	0.796	0.457	0.742
4	0.875	0.926	0.795	0.693	0.724	0.756
5	0.748	0.859	0.710	0.756	0.518	0.474
6	0.774	0.930	0.789	0.724	0.499	0.716
7	0.735	0.870	0.816	0.834	0.325	0.614
8	0.659	0.838	0.774	0.741	0.310	0.750
9	0.583	0.679	0.813	0.670	0.318	0.750
10	0.823	0.830	0.852	0.905	0.195	0.576
11	0.646	0.901	0.786	0.727	0.345	0.707
12	0.773	0.869	0.748	0.765	0.465	0.708
13	0.851	0.894	0.929	0.868	0.516	0.851
14	0.669	0.883	0.724	0.813	0.399	0.599
15	0.777	0.622	0.806	0.883	0.429	0.683
16	0.663	0.679	0.753	0.771	0.374	0.705
17	0.783	0.852	0.730	0.687	0.617	0.560
18	0.925	0.897	0.793	0.739	0.818	0.696
19	0.729	0.801	0.795	0.714	0.459	0.717
20	0.675	0.655	0.663	0.664	0.539	0.524
21	0.787	0.881	0.824	0.804	0.518	0.808
22	0.794	0.760	0.606	0.695	0.564	0.520
23	0.685	0.861	0.711	0.724	0.466	0.579
Min	0.583	0.622	0.606	0.664	0.195	0.474
Max	0.925	0.930	0.929	0.905	0.818	0.851
Mean	0.765	0.844	0.783	0.774	0.477	0.682
SD	0.210	0.253	0.184	0.185	0.202	0.182
Spearman’s rho with r_jd		0.331	0.280	0.256	0.503	0.105
Pearson’s r with r_jd (r-to-Z)		0.442	0.309	0.229	0.705	0.197

MEWS, modified early warning score; r_jd, cohesion (the correlation between judgments and decisions); G_jd, internal policy matching; R_j is cognitive control over judgments; R_d, cognitive control over decisions; C₁, correlation between the residuals from the two regression models; and r_a, judgment accuracy (the correlation between judgments and the ecological standard).

It is evident from inspection of Table 1 that there was notable variation in all the model parameters across the participants. The average cohesion (r_jd) was 0.765 (SD, 0.210). The C₁ coefficients were large, as expected, because the two sets of responses upon which they were based came from the same source (mean, 0.477). They also showed considerable variation (SD, 0.202) across participants. The average cognitive control parameters were reasonable (R_j = 0.783; R_d = 0.774) although slightly lower than those reported by Karelaia and Hogarth (mean, 0.80; 95% CI 0.79 to 0.82) and by Kaufmann and Athanasou (mean, 0.81; 95% CI, 0.76 to 0.86). The average G_jd = 0.844 fell slightly above the upper bound reported in the studies reviewed by Karelaia and Hogarth (mean, 0.80; 95% CI, 0.76 to 0.83) and Kaufmann and Athanasou (mean, 0.70; 95% CI, 0.64 to 0.77). Note, however, that G_jd in the current study refers to the correlation of predicted values from two regression equations originating from the same individual (i.e., internal policy matching), and so one might expect it to be larger than that found in traditional LME studies.

To examine individual differences in cohesion as functions of BiLME parameters (G_jd, R_j, R_d, C₁) and judgment accuracy (r_a), two approaches—each defensible but neither perfect when correlation coefficients are treated as data points—were used. Spearman’s correlation, rho, was calculated among the raw values and also Pearson’s correlation, r, with Fisher’s r-to-Z transform. Both results are shown in Table 1. The two approaches concur that individual differences in cohesion were most strongly related to differences in C₁, somewhat less related to internal policy matching G_jd, and only modestly related to cognitive control (R_j, and R_d). This pattern of results suggests that variation in cohesion, across individuals, was due to factors other than differences in cue utilization. Interestingly, individual differences in cohesion showed the lowest correlations with differences in accuracy (r_a). In other words, across individuals, cohesion was relatively independent of accuracy.

Discussion

This study is the first to employ the BiLME to examine individual differences in the cohesion between clinical judgments of patient risk and decisions to take risk-reducing action. While differences in cohesion were correlated with all of these parameters to some degree, the strongest relationship was found to be with individual differences in C₁, the correlation between the residuals of the modeled judgments and the modeled decisions, rather than between the two policies themselves (G_jd). Taken at face value, this suggests that decisions were linked to judgments for reasons orthogonal to cue utilization. As this finding seemed somewhat counter-intuitive, one might question whether it may be unique to the particular judgment or decision task employed (i.e., risk for acute deterioration and decision to alert rapid response teams) and limited to the population of participants sampled (i.e., RNs). To address these concerns, data from a second study that involved a different population of participants (NPs) and a different judgment/decision task are presented.

Study 2: Judging Risk of Developing Coronary Heart Disease and Referrals to Cardiologists

Method

Participants

Thirty NPs participated in this pencil and paper survey.

Materials

One of the most widely circulated prediction rules for coronary heart disease (CHD) is based on the ongoing Framingham study. Eight patient characteristics—gender, age, smoking status, total cholesterol level, high-density lipid level, systolic blood pressure, and whether or not the patient has been diagnosed with diabetes or left ventricular hypertrophy—represent an optimal set of risk factors. These were selected from a larger set of potential risk factors using stepwise selection procedures (see Anderson et al.²⁷ for details) and served as cues. Scores on this risk index served as the ecological standard. A representative sample of 70 patient profiles was constructed for presentation using the same approach as in Study 1. The means, standard deviations, and correlations among cues and criterion in the sample of profiles were then checked for consistency with those reported by Anderson et al.²⁷ before presentation.

Procedure

The 70 patient profiles were presented, one per page, in booklets. The participants provided a subjective estimate of how likely (0% chance to 100% chance) they believed each patient was to develop CHD in the next 10 y, and were then asked if they would refer the patient to a cardiologist, (yes or no). Completion of the task took an average of 32 min (range, 20 to 47 min).

Data Analysis

The methods of data analysis were identical to those described in Study 1.

Results

Data from 6 participants were excluded from analysis: 2 participants failed to complete all 70 profiles, and 4 made no referrals. This left n = 24 for analysis. The average age of participants was 48.2 y (SD, 60.8 y), 22 were female, and participants had been in practice as an NP, on average, 8.9 y (SD, 6.3 y).

The BiLME parameters (r_jd, G_jd, R_j, R_d, and C₁) as well as judgment accuracy (r_a) are shown for each participant in Table 2. There was considerable variation in all of the parameters across the participants. The average accuracy (r_a) of participants was 0.732, which is comparable to previously published LME studies. Cohesion (r_jd) averaged 0.721. The two measures of cognitive control were high, on average (R_j = 0.873, R_d = 0.875). The average value of G_jd was also high, 0.828. Although C₁ values were high (mean, 0.260) compared with traditional LME studies, they were significantly lower than those in Study 1 (t = 40.807; df = 45; P < 0.001; η² = 0.339). [η2 is an index of effect size. It is the proportion of variance in the dependent variable explained by the independent variable. Cohen²⁸ provides guidelines for interpretation: small (0.01) medium (0.06), and large (>0.14)].

Table 2

Results of Study 2; CHD Task

Participant	r_jd	G_jd	R_j	R_d	C ₁	r_a
1	0.564	0.652	0.895	0.937	0.012	0.763
2	0.730	0.850	0.913	0.885	0.132	0.825
3	0.740	0.807	0.907	0.922	0.214	0.716
4	0.759	0.842	0.921	0.925	0.183	0.785
5	0.583	0.835	0.821	0.637	0.356	0.747
6	0.673	0.842	0.836	0.822	0.237	0.687
7	0.747	0.836	0.722	0.863	0.333	0.498
8	0.653	0.754	0.881	0.895	–0.042	0.815
9	0.755	0.783	0.801	0.800	0.488	0.693
10	0.691	0.820	0.773	0.860	0.407	0.644
11	0.703	0.823	0.925	0.864	0.171	0.797
12	0.719	0.901	0.774	0.647	0.530	0.730
13	0.765	0.888	0.950	0.871	0.132	0.787
14	0.796	0.834	0.915	0.939	0.221	0.777
15	0.743	0.824	0.888	0.926	0.277	0.705
16	0.730	0.856	0.725	0.891	0.323	0.566
17	0.678	0.823	0.880	0.857	0.174	0.662
18	0.684	0.741	0.791	0.873	0.364	0.663
19	0.804	0.857	0.935	0.925	0.252	0.777
20	0.731	0.830	0.899	0.849	0.319	0.800
21	0.693	0.782	0.917	0.902	0.125	0.785
22	0.784	0.895	0.871	0.793	0.434	0.690
23	0.698	0.822	0.875	0.817	0.321	0.781
24	0.768	0.833	0.836	0.939	0.143	0.681
Min	0.564	0.652	0.722	0.637	–0.042	0.498
Max	0.804	0.901	0.950	0.939	0.530	0.825
Mean	0.721	0.828	0.873	0.875	0.260	0.732
SD	0.117	0.151	0.247	0.263	0.153	0.157
Spearman’s rho with r_jd		0.521	0.294	0.281	0.130	0.032
Pearson’s r with r_jd (r-to-Z)		0.569	0.245	0.293	0.222	–0.050

CHD, coronary heart disease; r_jd, cohesion (the correlation between judgments and decisions); G_jd is internal policy matching; R_j is cognitive control over judgments; R_d is cognitive control over decisions; C₁ is the correlation between the residuals from the two regression models; and r_a is judgment accuracy, the correlation between judgments and the ecological standard.

In contrast with the results from Study 1, the individual differences in cohesion (r_jd) seen here were most strongly related to differences in internal policy matching (G_jd) and weakly correlated with differences in C₁, the correlated residuals from the two regression models (see Table 2). Again, individual differences in cohesion were nearly independent of differences in judgment accuracy (r_a).

Statistical comparisons of all BiLME parameters with those from Study 1 are shown in Table 3. Neither the accuracy of judgments (r_a) nor the cohesion (r_jd) between judgments and decisions differed significantly between the 2 studies; although, the effect sizes were notable (η²s were 0.081 and 0.080, respectively). The average internal policy matching component (G_jd) was nearly identical across the two studies. Both measures of cognitive control (R_j and R_d) were, on average, significantly higher than in Study 1.

Table 3

Comparison of Average BiLME Parameters between Studies

Parameter	Study 1	Study 2	t	p	η²
r_jd	0.765	0.721	1.983	0.053	0.080
G_jd	0.844	0.828	0.862	0.393	0.016
R_j	0.783	0.873	4.528	0.001	0.313
R_d	0.774	0.875	4.763	0.001	0.335
C ₁	0.447	0.260	4.807	0.001	0.339
r_a	0.682	0.732	1.990	0.053	0.081

Study 1 is the modified early warning score (MEWS) task; Study 2 is the coronary heart disease (CHD) task; r_jd, cohesion (the correlation between judgments and decisions); G_jd, internal policy matching; R_j, cognitive control over judgments; R_d, cognitive control over decisions; C₁, the correlation between the residuals from the two regression models; and r_a, judgment accuracy (the correlation between judgments and the ecological standard). Degrees of freedom (df) for each t test was 45. As all data were correlation coefficients, Fisher’s r-to-Z transformation was used before calculating t tests. Analyzed values were then transformed back (Z-to-r) to correlations for presentation.

Discussion

In this study, the BiLME was again used to examine individual differences in cohesion. Differences in cohesion were correlated with all of the BiLME parameters to some degree. The most notable difference in the results of this study, relative to those from Study 1, was the reversal in the magnitudes of the two correlations relating individual differences in cohesion to individual differences in internal policy matching (G_jd) and to individual differences in C₁, the correlated residuals from the two regression models. Here, individual differences in cohesion showed a weaker correlation with differences in C₁. Further, using r-to-Z transforms, the Pearson correlation of cohesion scores with C₁ scores was significantly smaller in this study than in Study 1 (0.222 v. 0.705; t = 20.184; df = 45; P = 0.034), but the correlation of cohesion scores with G_jd scores did not differ significantly (0.569 v. 0.442; t = 0.575; df = 45; P = 0.568) between studies.

One explanation for these differences between Study 1 and Study 2 may be the mode of administration (online in Study 1, pencil-and-paper in Study 2). Another possibility may be a qualitative distinction in the type of judgment-to-decision connection examined in each study. While both studies elicited clinical judgments of patient risk for an adverse outcome, the decision in Study 1 (to alert, or not) exhausted all the options open to the decision maker. In the current study, however, the decision whether or not to refer the patient to a cardiologist represents only one of various options that are not mutually exclusive (others could include prescribing blood pressure medications, or cholesterol lowering agents, or both).

General Discussion

In this article, the cohesion between clinical risk judgments and treatment-related decisions was examined. Novel forms of Brunswik’s Lens Model and the LME were developed to decompose cohesion. The approach is considered “bifocal” in the sense that it focuses on two sets of linked responses from the same individual. Data from two studies were used to illustrate how individual differences in cohesion could be related to differences in the parameters of the BiLME (Equation 4). Cohesion (r_jd) varied due to individual differences in cognitive control (R_j and R_d), similarity in the judgment and decision policies (G_jd), and a possible reliance on a subjective threshold value applied to the judgments in order to make decisions (C₁). Whether C₁ is indicative of a subjective risk threshold for action decisions as suggested here, or simply a response-set bias, is not directly discernable from the current data; it does, however, pose an interesting direction for future research.

The parameters of the BiLME were associated with individual differences in cohesion in both studies; however, their relative influences differed between the two studies. In Study 1, where RNs judged hospital patients’ risk for critical events and decided to alert (or not) the rapid response team, differences in cohesion were most strongly associated with differences in C₁, the correlation between the residuals from the regression models of judgments and decisions. This finding suggests that individual differences in cohesion may have been due to factors unrelated to cue utilization because C₁ is the correlation between an individual’s judgments and decisions with the influence of the cues partialed out. In Study 2, where NPs judged patients’ risk for CHD and decided to refer to a cardiologist (or not), individual differences in cohesion were most strongly related to differences in internal policy matching (G_jd), suggesting that variation in cohesion was due to individual differences in the similarity of the two cue-weighting strategies used to form judgments and to make decisions.

The BiLME permits the testing of two hypotheses relevant to the following question: Why are some individuals (performing the same judgment-decision task) more cohesive than others? H₁: Individuals differ in the extent to which they use the same cue-weighting strategy to form their judgments and to make their decisions (i.e., individual differences in G_jd will correlate with r_jd values). H₂: Having formed their judgments based on the cues, individuals differ in the extent to which they simply apply a subjective threshold value to their judgments in order to make their decisions (i.e., individual differences in C₁ will correlate with r_jd values).

The BiLME (Equation 4) links the results from two regression models—one linear and one logistic—based on the same set of cases. Substituting r_a for r_jd makes it possible to apply the equation to traditional, dual-system judgment analysis studies, where continuous judgments are compared with an ecology composed of dichotomous outcomes, or where dichotomous judgments are compared with a continuous outcome in the ecology. In this regard, Equation 4 is quite flexible and adds to the set of tools available to judgment analysts. Hamm and Yang²⁹ have recently compared a similar “hybrid” method for deriving lens model parameters from the linear and logistic forms of the LME.

Across individuals (within each study), cohesion showed little to no relationship with individual differences in judgment accuracy (r_a). That is, the cohesiveness among an individual’s judgments of patient risk and decisions to take risk-reducing actions was relatively independent of how accurately the individual judged patient risk relative to the ecological standard. What makes this finding interesting is that some individuals can show strong connections between their decisions and their judgments regardless of whether their judgments of risk are accurate. And conversely, individuals who can accurately judge risk may not always rely on their own judgments when making decisions.

References

Wigton

. Applications of judgment analysis and cognitive feedback to medicine. In: Brehmer

Joyce

CRB

, eds. Human Judgment: The SJT View. Amsterdam: Elsevier Science Publishers; 1988. p 227–45.

Wigton

. What do the theories of Egon Brunswik have to say to medical education? Adv Health Sci Educ Theory Pract. 2008;13:109–21.

Tape

Wigton

. Medical students’ and residents’ estimates of cardiac risk. Med Decis Making. 1989;9:170–5.

Skaner

Strender

Bring

. How do GPs use clinical information in their judgments of heart failure? Scand J Prim Health Care. 1998;16L:95–100.

Skaner

Bring

Ullman

Strender

. The use of clinical information in diagnosing chronic heart failure: a comparison between general practitioners, cardiologists, and students. J Clin Epidemiol. 2000;53:108–8.

Dhami

Harries

. Fast and frugal versus regression models of human judgment. Think Reason. 2001;7:5–27.

Harries

Evans

JSBT

Dennis

Dean

. Clinical judgment analysis of prescribing decisions in general practice. Le Travail Humain. 1996;59:87–109.

Beckstead

Pezzo

Beckie

Shahraki

Kentner

Grace

. Physicians’ tacit and stated policies for determining patient benefit and referral to cardiac rehabilitation. Med Decis Making. 2014;34:63–74.

Thompson

Bucknall

Estabrookes

. Nurses’ critical event risk assessments: A judgment analysis. J Clin Nurs. 2007;18:601–12.

10.

Yang

Thompson

Hamm

Bland

Foster

. The effect of improving task representativeness on capturing nurses’ risk assessment judgments: A comparison of written case simulations and physical simulations. BMC Medical Inform Decis Mak. 2013;13:62–72.

11.

Adderley

Thompson

. Community nurses’ judgment for the management of venous leg ulceration: A judgment analysis. Int J Nurs Stud. 2015;52:345–54.

12.

Beckstead

Stamp

. Understanding how nurse practitioners estimate patients’ risk for coronary heart disease: A judgment analysis. J Adv Nurs. 2007;60:436–46.

13.

Stamp

. How nurse practitioners make decisions regarding coronary heart disease risk: a social judgment analysis. Int J Nurs Knowl. 2012;23:29–40.

14.

Brunswik

. The Conceptual Framework of Psychology. Chicago: University of Chicago Press. 1952.

15.

Stewart

. The lens model equation. In: Hammond

Stewart

eds. The Essential Brunswik: Beginnings Explications, Applications. New York: Oxford University Press; 2001. p 357–62.

16.

Tucker

. A suggested alternative formulation in the developments by Hursch, Hammond, and Hursch, and by Hammond, Hursch, and Todd. Psych Rev. 1964;71:528–30.

17.

Cooksey

. Judgment Analysis: Theory Methods, and Applications. San Francisco: Academic Press;1996.

18.

Stewart

. Components of correlation and extensions of the lens model equation. Psychometrika. 1976;41:101–20.

19.

Stewart

. Notes on a form of the lens model equation for logistic regression analysis. Paper presented at the 20^th Annual International Meeting of the Brunswik Society, Minneapolis; 2004.

20.

Naylor

Schenck

. ρ_m as an “error-free” index of rater agreement. Educ Psychol Meas. 1966;26:815–24.

21.

Brehmer

Hammond

. Cognitive sources of interpersonal conflict: Analysis of interactions between linear and nonlinear cognitive systems. Organ Behav Hum Perform. 1973;10:290–313.

22.

Karelaia

Hogarth

. Determinants of linear judgment: A meta-analysis of Lens Model studies. Psychol Bull. 2008;134:404–26.

23.

Hammond

. Probabilistic functionalism: Egon Brunswik’s integration, of the history theory, and method of psychology. In: Hammond

ed. The Psychology of Egon Brunswik New York:Holt Rinehart and Winston; 1966. p 15–80.

24.

Wigton

. Use of linear models to analyze physicians’ decisions. Med Decis Making. 1988;8:241–52.

25.

Subbe

Kruger

Rutherford

Gemmel

. Validation of a modified early warning score in medical admissions. QJM. 2001;94:521–6.

26.

Kaufmann

Athanasou

. A meta-analysis of judgment achievement as defined by the lens model equation. Swiss J Psychol. 2009;68:99–112.

27.

Anderson

Wilson

PWF

Odell

Kannel

. An updated coronary risk profile: A statement for health professionals. Circulation. 1991;83:356–62.

28.

Cohen

. Statistical power analysis for the behavioral sciences, 2nd ed.Hillsdale: Erlbaum; 1998.

29.

Hamm

Yang

. Alternative lens model equations for dichotomous judgments about dichotomous criteria. J Behav Decis Mak. 2016. DOI:100.1002/bdm0.169.