An approach for quantifying small effects in regression models

2.1 General ideas

We define vectors to be column vectors and set Σ XZ ≡ Cov ( X , Z ) for random vectors X and Z . Sample means and covariance matrices are labelled x ¯ , S xx and S xz .

Assume that the distribution of Y given Z = z i satisfies the model Y i = β 0 + z i ' β 1 + e i for i = 1 , 2 , … , n where E ( e i ) = 0 and Var ( e i ) = σ 2 . The predictor Z has no effect on the conditional mean of the responses when the regression function is constant on z 1 , … , z n . This suggests that a measure of the regression effect can be based on variation in the regression function, for example

δ 2 = Var ( β 0 + Z ' β 1 ) σ 2 = β 1 ' Σ ZZ β 1 σ 2 ,

(1)

The distribution on Z can be viewed multiple ways. One possibility is a sampling distribution for z 1 , … , z n in a joint model for ( Y , Z ) . Another view is that this distribution is solely an averaging distribution or device for quantifying the deviation of the regression function from a constant. In the first setting, E ( Y | Z ) is the minimum mean squared error predictor of Y. The accuracy of this predictor is measured by the correlation ratio Var { E ( Y | Z ) } / Var ( Y ) , which is η 2 = β 1 ' Σ ZZ β 1 / ( σ 2 + β 1 ' Σ ZZ β 1 ) when E ( Y | Z ) = β 0 + Z ' β 1 and Var ( Y | Z ) = σ 2 . Also, η 2 is the percentage of Var ( Y ) that is explained by E ( Y | Z ) . This interpretation is central to our calibration of η 2 even though we derived this summary differently.

Regardless of how the distribution of Z is viewed, inferences on δ 2 and η 2 will be based on the likelihood from the regression model (i.e. conditional on observed covariates). This requires the distribution of Z to be completely specified and Σ ZZ to be given. The empirical distribution for Z , which assigns probability 1 / n to each z i , is a logical averaging distribution because this option only assumes that the model holds for z 1 , … , z n . This choice is appealing when covariate levels are fixed by design and gives η 2 = β 1 ' S zz β 1 / ( σ 2 + β 1 ' S zz β 1 ) , which is the population R² for the model. ⁵ An alternative option with continuous predictors is to specify that a sampling or averaging density for Z has Σ ZZ = S zz . This also gives η 2 = R 2 so different perspectives on the role of Z lead to the same R² effect. The regression effect η 2 is different from R² when Σ ZZ ¬ = S zz .

To extend our idea to a more general setting, suppose that the distribution of Y given ( X , Z ) = ( x i , z i ) satisfies the linear model

Y i = β 0 + z i ' β 1 + x i ' θ + e i for i = 1 , 2 , … , n ,

(2)

δ p 2 = 1 σ 2 E { Var ( β 0 + Z ' β 1 + X ' θ | Z ) } = 1 σ 2 θ ' Σ * θ ,

(3)

An alternative derivation of η p 2 = R p 2 follows from reparametrizing equation (2) as Y i = β 0 * + z i ' β 1 * + t i ' θ + e i , where t i = ( x i - x ¯ ) - S xz S zz - 1 ( z i - z ¯ ) for i = 1 , … , n . The t i s result from orthogonalizing x i relative to the other effects in equation (2). As a result, t i and x i are equivalent for predicting Y after accounting for z i . Details are provided in Appendix 1. Treating the reparametrized model as a specification for the distribution of Y given ( T , Z ) = ( t i , z i ) , the effect of T is δ p 2 = E { Var ( β 0 * + Z ' β 1 * + T ' θ | Z ) } / σ 2 . If we assume that the marginal distribution of T is the empirical distribution of the t i s and that T and Z are independent, which is reasonable given that t i and z i are uncorrelated, then δ p 2 = θ ' S tt θ / σ 2 , where S tt = S xx - S xz S zz - 1 S zx is the covariance matrix of t 1 , … , t n . Thus this simpler averaging distribution also gives η p 2 = R p 2 .

To focus the discussion, Sections 2.2–2.5 consider η p 2 = R p 2 as a summary, with an understanding that R p 2 = R 2 when z is absent from the regression model. Subsequent sections will further examine the specification of the distribution of ( X , Z ) and suggest averaging distributions that produce alternatives to R p 2 as regression effects.

2.2 Bayesian inference

Let τ = ( β 0 , β 1 , θ , σ 2 ) be the parameter vector for equation (2) assuming the e_i are independent and normally distributed. Bayesian inference for R p 2 = R p 2 ( τ ) is straightforward using Monte Carlo Markov Chain (MCMC) or related simulation methods to generate a sample from the posterior distribution of τ . To calibrate R p 2 , note that R p 2 ≤ 0 . 01 is often viewed as a small effect. ⁶ However, the issue is more nuanced because it may be sensible to set this threshold as a function of the reduced model R 2 . As a slightly more liberal rule of thumb, we consider the effect of x to be negligible if the posterior 50th and 95th percentiles of R p 2 are at most 0.02 and 0.04, respectively, provided Z is not strongly associated with Y (i.e. posterior median for reduced model R 2 ≤ 0 . 25 ).

2.3 A Bayes test to establish a negligible effect

Our primary focus is the estimation of regression effects. However, we will briefly examine hypothesis testing, as this is a main interest in the literature. A Bayes test to establish a negligible effect of x contrasts H 0 : R p 2 ≥ Δ and H a : R p 2 < Δ , where Δ is a user-defined and problem specific tolerance. For the standard 0-1 loss function, the H₀ is rejected when Pr ( H a | y ) > 1 / 2 , or equivalently, when the posterior median of R p 2 < Δ . Equivalent hypotheses are H 0 : δ p 2 ≥ Δ * and H a : δ p 2 < Δ * where δ p 2 = R p 2 / ( 1 - R p 2 ) and Δ * = Δ / ( 1 - Δ ) . We often use Δ = 0 . 02 and Δ * = 0 . 02 / 0 . 98 = . 0 . 02 .

Wellek’s ¹ ANOVA F-tests to establish a negligible effect specify H_a as the negligible effect. This is consistent with a frequentist’s goal to establish that the effect, even if present, is small enough to be ignored. We follow this convention but stress that the Bayes test does not depend on which hypothesis is labelled H a . To contrast a Bayes and frequentist approach in equation (2), note that the F-statistic for testing θ = 0 has a F ( n δ p 2 / 2 ) distribution, ⁷ which reduces a central F-distribution when θ = 0 . Appendix 1 gives details. This result can be used to construct a size α test ¹ for δ p 2 , leading to the same hypotheses considered by the Bayes test. In particular, H 0 : δ p 2 ≥ Δ * is rejected in favor of H a : δ p 2 < Δ * when F < F α ( n Δ * / 2 ) , where F α ( n Δ * / 2 ) is the α th percentile of the reference non-central F-distribution. In essence, large F-values support a conclusion that a regression effect exists whereas small values support a conclusion that the effect is negligible. The two F-tests use the same statistic but with different reference distributions to address different questions. Both conclusions, or one or neither, are possible. This issue may also arise in Bayes tests.

2.4 CHDS analysis

Selvin ⁴ considers the model BW i = β 0 + z i ' β 1 + x i ' θ + e i to predict the birth weight (in pounds) of 680 live-born white male infants, where x i is the vector of paternal features (PA = age; PE = years of education; PS = cigarettes smoked/day; PH = height), and z i is the vector with gestation length (GL) and four maternal features (MA = age; MS = cigarettes smoked/day; MH = height; MW = pre-pregnancy weight). Table 1 gives posterior means and standard deviations (SDs) for ( β 0 , β 1 , θ ) based on a reference prior ⁸ p ( β 0 , β 1 , θ , σ ) ∝ 1 / σ 2 . The predictors were standardized so that the regression coefficients are in one pound units. The estimates are based on 10,000 posterior samples and match a least squares (LS) analysis given the large sample size. The posterior mean and SD of σ are 0.95 and 0.03, respectively. The posteriors for several regression parameters (GL, MS, MH, MW and PH) are concentrated away from zero but only gestation length has much impact on mean birth weight.

OPEN IN VIEWER

Table 1.

Posterior means and SDs for regression coefficients in CHDS analysis.

Effect	Mean	SD
Constant	7.517	0.036
GL	0.438	0.037
MA	−0.007	0.064
MS	−0.172	0.038
MH	0.099	0.044
MW	0.151	0.042
PA	0.000	0.064
PE	0.009	0.039
PS	0.026	0.039
PH	0.103	0.039

As mentioned in Section 1, one might hypothesize that the paternal predictors have limited information after accounting for GL and maternal features. In Selvin’s ⁴ frequentist analysis, adding x to the model increases the estimated R² from 0.252 to 0.262. Although the four paternal characteristics are significant at the 10% level (p = 0.09), the tiny estimated partial R p 2 = 0 . 012 suggests that these features have minimal impact on prediction. This is reinforced by the F-test to establish that the effect of x is negligible, which has a p-value of 0.09 when Δ * = 0 . 02 .

A Bayesian analysis leads to a similar conclusion. If one considers the question as an issue of model selection, then the difference between the Deviance Information Criterion ⁸ (DIC) for the full model and the reduced model without the paternal features ( Δ DIC = 0 . 08 ) does not suggest a clear preference. Least squares estimates were used to compute the effective number of parameters in DIC but an identical conclusion was obtained using other estimators. Although one might chose the simpler model based on parsimony, this decision is bolstered by noting that the posterior distribution of R p 2 is concentrated near zero; see Figure 1. In particular, the posterior median of R p 2 is 0.016 and Pr ( R p 2 < 0 . 04 | y ) = 0 . 98 , which suggest that the effect of paternal features on prediction is negligible using our benchmarks. OPEN IN VIEWER

2.5 Discussion of regression effects

The specification of a regression effect should reflect the purpose of the summary, which can range from reporting an effect size to addressing explanatory power. The units of measurement also play a role when deciding whether the summary should be standardized. Wilkinson ⁹ recommends regression coefficients as effect sizes when the units of measurement have practical interpretation, as in the CHDS analysis. Otherwise they suggest a standardized measure such as R p 2 , which we consider given our focus on prediction.

The uncertainty of an estimated effect impacts whether an effect is negligible. Small non-null effects may be estimated well enough to support a conclusion that they are negligible but poorly estimated null effects may not. Furthermore, conflicting conclusions can be reached from different effect measures. The CHDS analysis highlights some of these points. In particular, there is a 0.99 posterior probability that the paternal height (PH) regression coefficient exceeds zero and DIC increases by 5.30 when PH is omitted from the model, both of which support a conclusion that PH improves the fit of the model. However, Pr ( R p 2 < 0 . 03 | y ) = 0 . 95 from adding PH last to the model, which suggests that PH has a negligible effect on prediction.

2.6 Alternatives to R p 2

It is important to emphasize that R p 2 is a measure of predictive ability that is specific to the observed ( x i , z i ) pairs and depends on the distribution of these pairs through S tt . These pairs may have been fixed by design, simply observed as in the CHDS analysis, or resulted from sampling. We follow standard practice and treat the ( x i , z i ) s as fixed regardless of the process for choosing these pairs. Weisberg ¹⁰ argues that R p 2 is most meaningful when covariate values are randomly sampled, in part, because the value of R p 2 can be dramatically decreased simply by restricting the range of the observed ( x i , z i ) pairs.

There are two clear ways to alter the dependence of δ p 2 and η p 2 on the distribution of the observed predictor values. One approach that is implicit in Section 2.1 and illustrated in Sections 3 and 4 is to include unobserved ( x , z ) pairs in the definition of the effect. This requires an assumption that the model extends beyond the observed covariate pairs. A second approach that has broad application in ANOVA models is to modify the weight assigned to each unique ( x i , z i ) pair. This can often be done by using the constraints implied by the reduced model with θ = 0 to define effects. For example, consider a one-way ANOVA model Y ij = μ i + e ij for observations j = 1 , 2 , … , n i in groups i = 1 , 2 , … , K . The F-test ¹ to establish negligible differences in means has δ p 2 = ∑ i w i ( μ i - μ ¯ W ) 2 / σ 2 , where w i = n i / n and μ ¯ W = ∑ i = 1 K w i μ i . For this model, δ p 2 and R p 2 have the undesirable property of depending on group weights. Let μ ¯ = ∑ i μ i / K and note that the reduced model of equal means satisfies the constraints μ i - μ ¯ = 0 for i = 1 , 2 , … , K . A simple measure of a non-null effect that is independent of the group sample sizes is ∑ i ( μ i - μ ¯ ) 2 / ( K σ 2 ) . The same issue arises in a test for negligible interaction in two-way ANOVA, ³ with an analogous remedy.

Bayesian inference on effect measures is straightforward, so deciding that an alternative to R p 2 is more appropriate poses no real challenges. In contrast, linear model inferences based on F-statistics are restricted to R p 2 as a summary measure. Bootstrap methods may provide a general frequentist approach to inference on regression effects, but this has not been extensively explored in the literature.

3.1 Development

We defined linear models effects based on variation in the mean function. This idea applies to non-linear models and generalizes to other measures of dispersion. Bayesian inferences are straightforward. For example, suppose Y i = μ ( z i ; γ ) + e i with e i ∼ N ( 0 , σ 2 ) , where μ ( z i ; γ ) depends on predictors z and parameters γ . We define a general measure of regression effect over a set Ω as

D p ( s ) = ∫ Ω d p ( μ ( z ; γ ) - μ ¯ ( γ ) ; s ) d P ( z ) for p > 0 ,

(4)

To relate equation (4) to earlier summaries, note D 2 ( σ ) = Var { μ ( Z ; γ ) } / σ 2 when Z ∼ P . This reduces to equation (1) in the linear model and ∑ j ( μ j - μ ¯ ) 2 / ( K σ 2 ) in a one-way ANOVA where μ ( z j ; γ ) = μ j and Pr ( Z = z j ) = 1 / K for group labels z j = 1 , … , K . More generally, D p ( s ) = E | μ ( Z ; γ ) - μ ¯ ( γ ) | p / s p . The summary D 1 ( s ) is proportional to the mean absolute deviation in μ ( Z ; γ ) , which is an alternative to Var { μ ( Z ; γ ) } as a measure of dispersion in the regression function.

To give an example where equation (4) is useful, suppose γ = ( β , κ ) and μ ( z ; γ ) = w ( z ; κ ) ' β , where the elements of w ( z ; κ ) are a basis for functions that admit a finite series representation, for example a spline or polynomial. The covariate values w ( z ; κ ) depend on observed values z 1 , … , z n for a scalar feature Z, for example age, and may depend on a parameter vector κ , for example the knots in a spline. The model is non-linear when κ is unknown, so linear models effects do not apply. In general, the effect of Z is negligible if the mean function is nearly constant over the predictor space. However, with a flexible spline or polynomial model it is also meaningful to assess whether the effect is negligible over subsets of the predictor space. For example, Figure 2 plots a quadratic spline 0 . 35 + 1 . 93 z - 0 . 19 z 2 + 0 . 24 ( z - κ ) + 2 with a knot at κ = 6 and the average of the spline over Ω 0 = [ - 3 , 14 ] and Ω 1 = [ 3 . 5 , 14 ] . Here ( z - κ ) + = z - κ when z > κ and 0 otherwise. The spline varies much less over Ω 1 than Ω 0 , which we interpret as a weaker effect of z over Ω 1 than is observed overall. OPEN IN VIEWER

We can use D p ( s ) to measure variation in μ ( z ; γ ) over an interval Ω. Also, a continuous distribution for P is often sensible for splines and polynomials because it is plausible to assume that the model holds over the entire predictor space. As a result, equation (4) need not depend exclusively on the observed covariate levels. One can also consider multiple Ω . For example, the local variation in μ ( z ; γ ) can be described by setting Ω = [ z - c , z + c ] for some c > 0 and then varying z across the predictor space.

3.2 Selecting a summary

The effect D p ( s ) is intended to be flexible and easily interpreted, with problem-specific choices for s, p, and ( Ω , P ) . Standard options are s = 1 , σ , or | μ ¯ ( γ ) | and p = 1, 2 or the limiting case D ∞ ( s ) ≡ lim p → ∞ D p ( s ) 1 / p = sup z ∈ Ω | μ ( z , γ ) - μ ¯ ( γ ) | / s . Natural choices for ( Ω , P ) are the empirical distribution on z 1 , … , z n , a discrete uniform distribution on the unique z i s, a uniform distribution on the predictor space or an interval Ω , or the distribution of Z when z 1 , … , z n are sampled.

The specifications for s and p should depend on the role of the summary and the units of measurement, as noted in Section 2.5. To contrast options for s, suppose p = 2 . A measure of predictive ability results when s = σ because D 2 ( σ ) / { 1 + D 2 ( σ ) } is the proportion of var ( Y ) explained by μ ( Z ; γ ) . Alternatively, D 2 ( | μ ¯ ( γ ) | ) is the squared coefficient of variation of μ ( Z ; γ ) , which is a descriptive summary of relative variation in the regression function. These two measures are unitless whereas D 2 ( 1 ) 1 / 2 = SD { μ ( Z ; γ ) } has the same units as Y. This summary may be difficult to calibrate if the units are unnatural and ill considered if prediction is the interest. Another consideration is that D p ( s ) is less robust to extreme deviations from μ ¯ ( γ ) for larger values of p, with D ∞ ( s ) providing an assessment of whether the variation in μ ( Z , γ ) is uniformly small over Ω. We prefer p = 1 to p = 2 for a descriptive summary because D 1 ( s ) is proportional to the mean absolute deviation in μ ( Z ; γ ) which we find easier to interpret than Var { μ ( Z ; γ ) } or SD { μ ( Z ; γ ) } .

Some guidance can be given for calibrating the posterior median of D p ( σ ) for p = 1, 2 and ∞ . Given the connection with R², we propose Δ 2 * = 0 . 02 as a bound for D 2 ( σ ) . Appendix 2 suggests D ∞ ( σ ) < 0 . 48 and D 1 ( σ ) ≤ b Δ 2 * with b = 0.50 and b = 1. The posterior median bound D 1 ( σ ) ≤ Δ 2 * is liberal and follows from Jensen’s inequality. These bounds can be used in tests to establish a negligible effect.

3.3 Example: Peptide analysis

Figure 3 plots the logarithm of serum C-peptide in pmol/ml (Y) against age (z) in years for 43 participants in a study of insulin-dependent childhood diabetes. ¹¹ A least squares fit to a quadratic spline model Y i = w ( z i ; κ ) ' β + e i with a knot at the average age (8.80 years) is superimposed on the plot. Here w ( z ; κ ) ' = [ 1 , z , z 2 , ( z - κ ) + 2 ] with κ = 8 . 80 . The curve mimics the observed trend reasonably well, and varies by two across ages but only by 0.10 for 8 < z < 15 (approximately the maximum age). OPEN IN VIEWER

As an illustration, suppose we are interested in the variation in μ ( z ; γ ) over ages [ 8 , 15 ] , treating κ as fixed. We consider D 1 ( μ ¯ ( γ ) ) with P uniform on [8,15] (i.e. P ∼ U [ 8 , 15 ] ) and a reference prior p ( γ , σ ) ∝ 1 / σ 2 . If we view a 5% relative variation about μ ¯ ( γ ) to be unimportant, then Pr ( D 1 ( μ ¯ ( γ ) ) < 0 . 05 | y ) > 0 . 99 based on 20,000 posterior samples. Ignoring that this evaluation was suggested by the data, this analysis strongly suggests that the variation in the regression curve relative to the average level over [ 8 , 15 ] is unimportant. If we consider D 2 ( σ ) instead then Pr ( D 2 ( σ ) < 0 . 02 | y ) = 0 . 44 . This second analysis gives some support, but does not strongly suggest, that the predictive ability of the spline is negligible over [ 8 , 15 ] . Not surprisingly, the choice of summary has implications on the conclusion.

4.1 Motivating examples

We consider two analyses that highlight extensions of equation (4) to regression models with predictors and a factor. As background for our first analysis, Burt ¹² discusses a study of 27 identical twins raised apart, one by foster parents and one by the natural parents. An interest is whether the relationship between their IQ scores varies by the social stratum of the natural parents (low, middle, high). To answer this question, Weisberg ¹⁰ considers the model Y kj = β k 0 + β k 1 z kj + e kj where Y_kj is the IQ score for the twin raised by a foster parent for pair j in stratum k and z_kj is the IQ score for the twin raised by the natural parents. A plot of the data shows strong linear relationships with minor differences across strata. Weisberg ¹⁰ concludes based upon an F-test that the lines do not vary significantly across strata, which might suggest that the environmental effect on intelligence is unimportant. An alternative analysis is to evaluate whether differences in the regression functions are negligible. Although this may be assessed using R p 2 , Section 4.3 will consider an extension of equation (4) that eliminates dependence of the effect on the design matrix.

As another example, Figure 4 plots a time trend of the logarithm of the incidence rates (per 100,000) for melanoma of the skin for non-Hispanic white males and females. The rates are from SEER (Surveillance, Epidemiology and End Results) 13 study areas (San Francisco, Connecticut, Detroit, Hawaii, Iowa, New Mexico, Seattle, Utah, Alaska native registry and rural Georgia) for years 1992–2011, age adjusted to the 2000 US standard population. The standard errors of the estimates are about 0.02. OPEN IN VIEWER

A standard modeling approach for rates is to fit splines to the sex-specific log-incidence rates. ¹³ We consider a linear spline for males and a quadratic spline for females, each with one knot. We assume that the knot locations ( κ 1 , κ 2 ) are unknown and have independent discrete uniform distributions from years 1995–2008. Gibbs sampling was used to simulate the posterior distribution of ( γ 1 , γ 2 ) , where γ i = ( β i , κ i ) , based on large sample normal approximations to the distribution of β i given κ_i and the data, assuming independence between the male and female models.

Figure 4 plots the posterior mean of the regression functions based on 5000 samples after a 40,000 sample burn-in. The estimated trends closely follow the data. The temporal increase in rates evident in Figure 4 mirrors worldwide trends, which led to the coining of the phrase “the melanoma epidemic.” However, the sex-specific age-adjusted annual U.S. death rates for melanoma have remained relatively constant from 1990 to 2006. ¹⁴ Recent research ¹⁵ suggests that the inconsistency between the trends in incidence and mortality rates is caused to a large degree by the diagnosis of an increased number of early-stage melanomas, possibly due to more intense screening. Interestingly, the between-sex difference in log-rates has stayed relative constant, ranging from 0.37 to 0.44 over the 20 year period. We quantify this interaction in Section 4.4, taking into account that the models are different and nonlinear.

4.2 Statistical approach

We consider measures of variation and interaction among regression functions when data are available from multiple groups or strata. We assume Y kj = μ k ( z kj ; γ ) + e kj for j = 1 , 2 , … , n k and k = 1 , 2 , … , K , where e i ∼ N ( 0 , σ 2 ) and μ k ( z ; γ ) = w k ( z ; κ k ) ' β k . Here γ = ( γ 1 , … , γ K ) , where γ k = ( β k , κ k ) , and w k ( z ; κ k ) is a stratum-specific covariate that is function of z and κ k , for k = 1 , 2 , … , K . The dimension of γ k may vary by strata. This allows different models by strata, for example, different order polynomials, or splines. The parameter κ k may be absent or fixed, which is the case in standard polynomials and splines where the knot locations are assumed known.

A natural pointwise measure of differences among the mean functions is the discrepancy d p ( z ; s ) = ∑ i = 1 K | μ i ( z , γ ) - μ ∼ ( z , γ ) | p / ( Ks p ) , where s = 1 , σ or | μ ∼ ( z , γ ) | and μ ∼ ( z , γ ) = ∑ i = 1 K μ i ( z , γ ) / K . The discrepancy is zero at z ∈ Ω if and only if the regression functions are equal μ 1 ( z ; γ ) = μ 2 ( z ; γ ) = … = μ K ( z ; γ ) . Note that d 2 ( z ; | μ ∼ ( z , γ ) | ) is the squared coefficient of variation in the means at z. An overall measure of differences among the regression functions, D p O ( s ) , is obtained by averaging d p ( z ; s ) with respect to a distribution P over Ω , as in equation (4).

A characterization of no interaction is that μ i ( z ; γ ) - μ j ( z ; γ ) is independent of z ∈ Ω for all (i, j) or equivalently that μ i * ( z , γ ) ≡ μ i ( z , γ ) - μ ¯ i ( γ ) does not vary across strata, where μ ¯ i ( γ ) = ∫ Ω μ i ( z , γ ) d P ( z ) . An aggregate measure of interaction, D p I ( s ) , is obtained by averaging d p ( z ; s ) = ∑ i = 1 K | μ i * ( z , γ ) - μ ∼ * ( z , γ ) | p / ( Ks p ) as in equation (4), where s = 1 or σ and μ ∼ * ( z , γ ) = ∑ i = 1 K μ i * ( z , γ ) / K . The difference between D p I ( s ) and D p O ( s ) is that D p I ( s ) centers μ i ( z ; γ ) to have mean 0 over Ω .

4.3 Example: IQ scores

We assessed the relative differences among regression functions using D 1 O ( s ) with s = | μ ∼ ( z , γ ) | . We considered P ∼ U [ 70 , 130 ] , P ∼ N ( 100 , 15 ) and a reference prior. The second distribution approximates the IQ distribution in the general population. The posterior 50th and 95th percentiles of D 1 O ( s ) based on 20,000 samples are 0.04 and 0.07, respectively, for each of the two distributions P . Thus, the mean absolute deviation relative to the mean of the regression functions is typically about 0.04. Although a subject matter expert is needed to assess whether this small difference is important, we note that the simpler model of equal regression lines is preferred to the general model based on DIC, as Δ DIC . = 6 , with slight differences in DICs depending on the method used to estimate the effective number of parameters. An interesting historical detail is that Burt’s research on the heritability of intelligence has been extensively scrutinized, with accusations of falsification of research data. ¹⁶

4.4 Example: Incidence rates for melanoma

We summarized the interaction using the unstandardized D 1 I ( 1 ) (i.e. s = 1) and P ∼ U [ 1992 , 2011 ] . The posterior 50th and 95th percentiles of D 1 I ( 1 ) are 0.01 and 0.02, respectively. The 95th percentile is modest relative to the typical yearly difference (0.40) in the observed log-rates between sexes and suggests that differences between the male and female log-rates have not varied much over the 20 year period.