Behavioral,Information,and Monetary Interventions to Reduce Energy Consumption in Households: A Living Systematic Review and Network Meta-Analysis

The Review in Brief

This living systematic review (LSR) and network meta-analysis (NMA) provides evidence on the efficacy of information, behavior, and monetary interventions in reducing household energy consumption. Monetary interventions seem to be more effective than other interventions. Optimal policy packages where interventions are paired with other interventions can increase overall effectiveness.

What Is This Review About?

This review demonstrates how scientific evidence on the effectiveness of one set of policy interventions, namely interventions in household energy demand, can be kept up to date to deliver rigorous, solution-oriented knowledge to policymakers to meet their needs. The policy interventions studied in this review include monetary incentives that offer households a tangible financial reward for reducing energy consumption, behavioral interventions like nudging, appealing to norms, motivation techniques, and providing easily interpretable information at the point of decision-making, as well as improving skills required to perform or forego behaviors. This review assesses the extent to which such policy interventions can reduce household energy consumption.

What Is the Aim of This Review?

We sought to answer three questions: By how much can information, nudges, and cash rewards reduce household energy use on average? Which of these three approaches works best? Do combinations of them deliver even bigger savings?

What Are the Main Findings of This Review?

The evidence suggests that behavioral, monetary and information interventions can cut energy consumption in households by 4%–6%, though some interventions or combinations of interventions have larger impacts. Because study quality varies, there is a need to incorporate study quality metrics while deriving policy implications of studies in this field.

What Do the Findings of This Review Mean?

The effectiveness of interventions aimed at reducing household energy consumption is shaped by several factors, including the frequency of relevant decisions, the scalability and cost of interventions, and their broader welfare implications. Ultimately, a well-calibrated approach driven by evidence that accounts for study quality, intervention combinations, and policy feasibility is necessary to maximize the impact of household energy interventions.

The Problem, Condition, or Issue

Policymakers only have little time left to prevent the worst impacts of climate change and limit global warming to well below two degrees (IPCC 2022, 2023). Assessments by the Intergovernmental Panel on Climate Change (IPCC) have provided evidence about the extent of climate change and possible pathways for mitigation (IPCC 2023). However, rigorous solution-oriented knowledge of how to facilitate those pathways is missing (Berrang-Ford et al. 2020; Minx et al. 2017). In particular, a systematic assessment of scientific evidence on what climate policies work, to what extent, in what context, why, and for whom is not available (Berrang-Ford et al. 2020). This LSR and NMA demonstrate how scientific evidence on the effectiveness of a particular set of policy interventions, namely behavioral, information, and monetary interventions in household energy demand, can be kept up-to-date to deliver timely, rigorous, and solution-oriented knowledge to policymakers.

Finding low-energy-demand pathways is necessary to hedge against the risks involved in decarbonizing energy supply and is key to finding socially acceptable ways of meeting the Paris climate goals (Creutzig et al. 2022; Grubler et al. 2018; Van Vuuren et al. 2018). There has been a surge in interest in using demand-side policies, particularly aimed at behavioral change, for reducing energy demand and the related emissions. To reduce the energy demand of households, primary studies have experimented with using monetary incentives, behavioral interventions including nudging, appealing to norms, information incentives like providing easily interpretable and credible information at the point of decision-making, and so forth. The relevant literature is spread across various disciplinary fields: economics, psychology, power systems, and engineering studies. This LSR and NMA integrates the evidence available from all these sources to compare the effectiveness of different types of interventions (sometimes deployed simultaneously) and to understand what drives the variation in outcomes across studies.

The Intervention

We reviewed the literature on interventions in residential energy demand to reduce energy usage. These interventions include monetary incentives that offer households a tangible financial reward for reducing energy consumption, behavioral interventions like nudging, appealing to norms, information interventions that provide easily interpretable and credible information at the point of decision-making, as well as improving skills required to perform or forego behaviors. In this review, we broadly classify the interventions into three categories: monetary, behavioral, and information. The more detailed typology in Table 1 further breaks behavioral interventions into feedback, social comparison, and motivation interventions. This classification of interventions into five categories is then used for analysis in the subsequent sections. This typology is consistent with those used by previous reviews on energy consumption in households (Buckley 2020; Delmas et al. 2013; Khanna et al. 2021). This classification has been adopted across a series of reviews and meta-analyses that form an “ecosystem of reviews” that evaluates interventions in households' consumption choices in food, transport, and buildings. This series of coordinated systematic reviews explores the effectiveness of demand-side interventions to induce behavioral change in key GHG emissions sectors, including food (Lohmann et al. 2024), transport (Javaid et al. 2022), and buildings (Khanna et al. 2021).

How the Intervention Might Work

There is a rich theoretical literature on possible pathways that drive reduction in household energy consumption in response to monetary, information, or behavior-based interventions.

Information measures reduce energy consumption by reducing information gaps, decreasing the costs of acquiring information about energy-saving measures, improving skills required to perform or forego behaviors, and appealing to intrinsic motivations. These interventions may take the form of general energy campaigns or tailored advice, such as energy audits. Often, access to energy efficiency information in homes is costly, especially in rental properties, and poses barriers to energy conservation. Energy audits can address these information asymmetries by providing households with accessible and actionable information (Ramos et al. 2015). According to the theory of norm activation, energy saving tips and audits can enhance consumers' perceived control over their behavior (Fischer 2008), whereby learning about energy consumption behavior and energy saving tips can lower the perceived cost of adopting energy efficient practices(Delmas et al. 2013). Further, individuals tend to be more receptive to those they have meaningful relationships with (Cialdini 2009; Cialdini and Goldstein 2004), hence, are more likely to act on information that was delivered through strong social ties (e.g., families) (Abrahamse and Steg 2013; Christakis 2009) and through those they perceived more similarities with (Burger et al. 2004).

Feedback reduces energy consumption by influencing consumers' behavior through the provision of information on their past behaviors and energy usage patterns. Interventions can include home displays, historical usage patterns, smart meters, and redesigning utility bills. Individual feedback influences individual behavior through self-efficacy (Bandura 1997). In both types, feedback provides insights into the link between desired outcome (e.g., energy saving) and the behavior change needed to reach that outcome (e.g., turn off lights) (Kluger and DeNisi 1996). Matthies (2005) presents a heuristic model of environmentally relevant behavior that provides a framework for how feedback interrupts habitual behavior of energy consumption and initiates changes in three consecutive processes: norm activation, motivation, evaluation, and action. The process of norm activation requires three building blocks, such that consumers become conscious of their environmental problems, of the relevance of their behaviors, and their ability to influence their behaviors and outcomes (sense of control). Succeeding this is the process of weighing and evaluating personal environmental norms, social norms, and other motives to decide their course of action. Building on this framework, feedback presents consumers with a problem that requires conscious choices, offering an opportunity for self-reflection. By linking consumption feedback to a specific appliance, it raises awareness of the significance of individual actions while also providing a direct solution, thereby enhancing the sense of control. All three components are required for norm activation. Feedback is crucial in influencing energy consumption habits where consumers are unaware of their own behaviors, and do not know how to address them (Fischer 2008; Matthies 2005).

Social comparison reduces energy usage by benchmarking household consumption with comparable households. Social comparison is a provision of social norms on the standard level of usage given household characteristics and the socially acceptable level of usage, guiding consumers' behaviors, especially under uncertain circumstances (Allcott and Mullainathan 2010; Delmas et al. 2013). For reference-dependent consumers (Kahneman 2003), social norms become an important reference point to avoid disutility from socially disapproved behaviors (Fischer 2008; Schubert and Stadelmann 2015). Behavioral changes driven by social comparisons are motivated by individuals' desire to feel better off relative to a worse group (downward comparison) (Suls et al. 2002) or to see themselves in a better light relative to the better-performing group (upward comparison) (Cialdini et al. 1976). Social comparisons also correct biased beliefs of individuals' consumption (environmentalist overconfidence on their energy savings) (Andor and Fels 2018) generate competition and reinforce consumers' desire to be above average (Abrahamse et al. 2005).

Motivation interventions tackle consumers' time-inconsistent preferences by binding their future behaviors, sometimes with a reference point (O'Donoghue and Rabin 1999). Consumers with time-inconsistent preferences tend to procrastinate on and delay actions to reduce energy consumption or may never achieve them due to present bias (O'Donoghue and Rabin 2008). Motivations, like personal commitment, “appeals to a person's norm”—desire to satisfy their self-expectation (Abrahamse et al. 2005). Public commitment motivates changes in behavior through social pressure appealing to consumers' desire to meet others' expectations, be it their peers or an institution. A motivation with reference point (specific target) appeals to a person's loss aversion, where failure to meet their commitment creates more “disutility” than accomplishing it (Bandura 1986; Kahneman and Tversky 2013).

Monetary incentives reduce energy consumption through financial consequences in the form of pricing, rebates, and rewards (Dietz 2015; Mi et al. 2021; Sloot et al. 2021). Indirect incentives provide users with financial information on their energy consumption to motivate cost savings through reducing consumption. Direct incentives provide immediate financial benefits, including rebates for energy reduction and dynamic electricity pricing–varying prices through the day or during peak periods in response to demand. The economic theory of rational choice explains the effects of financial incentives in motivating energy conservation based on the assumption of rational consumers who act in their self-interest and select options where benefits outweigh costs (Hutton and McNeill 1981; Scott 2000). In the context of both direct and indirect incentives, rational consumers will reduce consumption overall or during peak times when the financial benefits (e.g., cost savings, paying lower prices, rebates) of doing so are higher than the costs (e.g., using electricity at a particular time) (Jackson 2005). However, recent psychological research suggests consumers also consider the “wider social consequences” such as environmental intrinsic values and personal norms in addition to the costs and benefits of their energy consumption behavior (Sloot et al. 2018; Stern 2000). Studies have shown that overemphasis on extrinsic and monetary values to promote pro-social behavior can crowd out “intrinsic motivation to act altruistically” and reduce desired behaviors (Frederiks et al. 2015; Frey and Jegen 2001; Menges et al. 2005).

Why It Is Important to Do This Review

There is a well-established and fast-growing literature on monetary, behavioral, and information interventions in energy consumption in households, including nudging, appealing to norms, providing easily interpretable and credible information at the point of decision-making, and so forth. The relevant literature is spread across various disciplinary fields: economics, psychology, and power system studies.

Previous meta-analyses on this topic are intra-disciplinary (e.g., Abrahamse et al. 2005) and/or focused on a subset of interventions. For instance, Faruqui and Sergici (2013) focus on pricing interventions, Karlin et al. (2015) on feedback, and Andor and Fels (2018) on social comparison, commitment devices, goal setting, and labeling. Nisa et al. (2019) consider evidence from a wider range of household behaviors that are relevant for climate change mitigation, but do not review interventions in energy consumption exhaustively. The meta-analysis by Delmas et al. (2013) was based on a narrower literature search and did not include studies published after 2012. A review with a similar scope but with studies updated till 2019 was published by Buckley (2020). Khanna et al. (2021) provide a comprehensive meta-analysis, but the literature on behavioral interventions is increasing fast, so reviews need to be constantly updated.

The latest systematic review by Khanna et al. (2021) contained 360 effect sizes from 122 studies with evidence from 25 different countries published by mid-2020. Since then, considerably more evidence has emerged in this fast-growing field of study as technological advancements in metering energy and information provision have made it easier to experiment with such interventions. The new knowledge gained from these studies may lead to changes in evidence-based recommendations for policymakers and practitioners. However, there are long lags in incorporating new evidence. To prevent such gaps in knowledge, this review will systematically and continually update the evidence using the “living systematic review” concept, hitherto mainly discussed in clinical sciences (Akl et al. 2017; Elliott et al. 2017; Simmonds et al. 2017; Thomas et al. 2017). This LSR and NMA provide up-to-date evidence, with almost double the number of effect sizes, while simultaneously preventing duplication of effort in incorporating past studies.

Methodologically, this review fills several gaps: understanding the role of machine learning (ML) in updating reviews, resolving the statistical challenges in updating meta-analyses, and setting up guidelines for updating policy recommendations based on a living review.

Criteria for Considering Studies for This Review

Search Methods for Identification of Studies

The starting point of our search is the studies identified by Khanna et al. (2021) through their comprehensive search of the relevant literature in previous systematic reviews and meta-analyses, and bibliographic databases. Khanna et al. (2021) used a prioritized screening approach (Callaghan and Müller-Hansen 2020) to identify relevant evidence from over 64,000 studies at the abstract level (through a mix of manual and ML approaches), whereby 934 studies were screened manually at the full-text level and 122 relevant studies were identified, which are twice as many as identified by any previous systematic review with this scope and includes all the studies identified by previous reviews. We update this pool of relevant studies through string-based searches of bibliographic databases and gray literature since 2020, when the databases were searched by Khanna et al. (2021). The search string we use follows the PICOS (population, intervention, comparator, outcome, and study design) logic to target empirical studies covering one or more of such interventions and household energy consumption as the outcome variable (Table 2).

Data Collection and Analysis

Assessment of Reporting Biases

We assessed publication bias using funnel plots and Egger's test. The funnel plot shows the distribution of effect sizes in conjunction with the standard error. We use statistical tests that are more formal than the funnel plot to confirm the presence of publication bias and to estimate the “true” average effect size adjusted for publication bias. We use the precision-effect test (PET) and precision-effect estimate with SE (PEESE) for estimating/testing the “true” effect in the presence of publication bias (Stanley and Doucouliagos 2014). PET and PEESE use the following regression models:

y ^ i j = β 0 + β 1 SE i + s i + m j + u i PET , ${\hat{y}}_{ij}={\beta }_0+{\beta }_1{\mathrm{SE}}_i+s_i+m_j+u_i\mathrm{PET},$

y ^ i j = γ 0 + γ 1 SE i 2 + s i + m j + u i PEESE , ${\hat{y}}_{ij}={\gamma }_0+{\gamma }_1{\mathrm{SE}}_i^2+s_i+m_j+u_i\mathrm{PEESE},$

where

y ˆ ij ${\hat{y}}_{{ij}}$

is the i-th estimate from the j-th study, and

SE i ${{SE}}_i$

is the corresponding standard error,

m j $m_j$

is the between-study effect for the j-th study, normally distributed with variance

σ m 2 ${\sigma }_m^2$

,

s i $s_i$

is the within-study effect, for the i-th effect size, distributed with a mean of zero and the variance

σ s 2 ${\sigma }_s^2$

; and

u i $u_i$

is the sampling error. The coefficient of the standard error measures publication bias and can be thought of as a test of asymmetry in the funnel plot. The intercept of the models (

β 0 ${\beta }_0$

and

γ 0 ${\gamma }_0$

) estimates the “true” effect beyond publication bias. PET is a powerful test for the presence of an authentic effect beyond publication bias (Stanley 2008), but

β 0 ${\beta }_0$

tends to underestimate the true mean effect when there is a nonzero intervention effect. If an indication is given for the presence of a genuine effect,

γ 0 ${\gamma }_0$

is the preferred estimate (Stanley and Doucouliagos 2014).

Subgroup Analysis and Investigation of Heterogeneity

Our data set includes studies that employ different types of interventions: monetary incentives, information, feedback, social comparison, and motivation, as well as combinations of them, to reduce the energy consumption of households. The diversity of interventions and their combinations used in our data set could be a source of heterogeneity in our analysis. We investigate such heterogeneity using two approaches: a multilevel meta-regression model and an NMA approach.

Heterogeneity could also arise out of context in which the studies were carried out. The previous version of this review (Khanna et al. 2021) investigates this heterogeneity by adding variables that represent both context and setting of interventions and study design characteristics as moderator variables in the meta-analysis models. Such analyses have also been conducted by previous reviews (Buckley 2020; Delmas et al. 2013) and the results from these analyses have tended to be consistent overtime. Therefore, we do not repeat the heterogeneity analysis here for the sake of the focus of this review. However, the data set that we provide with this study also contains information on these variables, and such analysis can easily be carried out by users of this data.

Meta Regression Approach for Heterogeneity Analysis

We added a dummy variable that indicated the type of intervention or the combination thereof as a moderator variable in the multilevel meta-analysis model to calculate intervention specific average intervention effects. These are presented in the form of a forest plot (Figure 1).

OPEN IN VIEWER

Figure 1

Possible comparisons in network meta-analysis.

NMA for Heterogeneity Analysis

We conducted an NMA to compare the average effects of various behavioral interventions on our primary outcome. Each effect size in our data set records the effect of interventions compared with a control group, for example, monetary incentives versus control, information versus control, and so forth. For studies where the experimental units receive multiple interventions, their interventions are recorded as a combination of interventions. For example, if participants receive both feedback and monetary incentives, then the comparison would be feedback and monetary versus control (Figure 2). The primary studies included in this network analysis were nearly always two-armed, and compared a treatment group against a control, with very few interventions compared directly (Fernández-Castilla and Van den Noortgate 2023). Since such direct evidence head-to-head comparison between interventions is rare, our network has a “star-shaped geometry.”

OPEN IN VIEWER

Figure 2

PRISMA-LSR flow diagram for updated systematic reviews which included searches of databases. created using Living PRISMA Flow: R package and ShinyApp for producing PRISMA-style flow diagrams for living systematic reviews (Version 0.0.1).

Network Geometry, Transitivity, Consistency, and Heterogeneity Assessment

An initial check while conducting NMA is that the data should satisfy the transitivity assumption, that is, included studies are sufficiently similar in effect modifiers so that it is valid to compare interventions indirectly via the common control. This allows evidence across these groups to provide a comparative ranking of the interventions (Chaimani et al. 2023; Fernández-Castilla and Van den Noortgate 2023; Wilson et al. 2016). We qualitatively assessed transitivity by reviewing study characteristics across important effect modifiers identified by Khanna et al. (2021) to ensure no major systematic differences that would violate this assumption (e.g., all studies targeted a similar outcome under comparable conditions) (Lin et al. 2021). Each intervention (or combination) was counted as a separate node in the network; in cases where the same study reported multiple comparisons (for instance, testing the same intervention in different subpopulations or timeframes), we included each comparison as if from an independent study for the purposes of the NMA (since NMA requires one effect size per study per comparison). This approach allowed us to include all relevant comparisons, but we acknowledge it treats repeated measures from a single study as independent, which we address in sensitivity considerations elsewhere.

We also evaluated whether the consistency assumption was likely to hold. Consistency refers to the agreement between direct and indirect evidence for the same comparison. Our star-shaped network lacked any loops of evidence (not many interventions were compared directly in any study), the usual methods for checking consistency could not be applied. In particular, a node-splitting analysis (which typically separates direct from indirect evidence on a particular comparison) did not produce any estimates. In other words, the network geometry provided no opportunity for inconsistency to arise—any comparison between two active interventions was informed only by indirect evidence through the control. We therefore could not calculate local inconsistency measures or obtain node-split p-values, and a design-by-treatment global inconsistency test likewise had no degrees of freedom (no between-design Q statistic could be computed, shown by the netsplit() procedure returned only NA estimates, and the blank in the table). By design, the network is structurally consistent (there is no conflict between direct and indirect evidence since indirect evidence is all that exists).

In lieu of formal inconsistency tests, we focused on between-study heterogeneity as a measure of variability in effects that could impact our confidence in the results. We employed a random-effects NMA model with a common between-study variance (τ²) across all comparisons. This model estimates a shared heterogeneity parameter, allowing each intervention effect to vary across studies around an overall mean effect. We quantified the total heterogeneity with metrics such as τ (the between-study standard deviation) and I² (the proportion of variability in effect estimates due to heterogeneity rather than chance). This global heterogeneity assessment served as a proxy check: if heterogeneity was extremely high, it might indicate underlying inconsistency or important effect modifiers. Conversely, low to moderate heterogeneity would increase confidence that the intervention effects are reasonably consistent across studies. As reported below, the selected model's heterogeneity was in an acceptable range (common τ² was moderate and I² was not excessive), suggesting that while studies differed to some extent, the variability in effects was adequately accounted for by the random-effects model. No concerning inconsistency was detected in the network, given the absence of conflicting direct evidence and the tolerable level of heterogeneity observed.

NMA Setup

Our NMAs were carried out within a Bayesian framework, utilizing Markov Chain Monte Carlo (MCMC) methods to estimate the posterior distributions of the effect sizes (Tonin et al. 2017). We adhered to a non-informative prior approach to ensure minimal prior influence on the results. We initialized four separate MCMC chains with different starting values to enhance the robustness and convergence of our simulations. Each chain underwent a substantial iteration process involving 100,000 iterations, with the first 10,000 discarded as burn-in to ensure stabilization of the chains. Convergence was checked using Gelman-Rubin-Brooks plots, providing a comprehensive assessment of stability across chains (see Appendix S1 for details).

We tested both fixed-effects and random-effects models, but ultimately used only a random-effects model. The selection criteria between these models were guided by the fact that random fits the data structure better. This was validated by the Deviance Information Criterion (DIC), with random effects models exhibiting a DIC lower by at least 10 points, indicating a significantly better fit and parsimony (Liew and Lee 2019).

Effect Size and Ranking Estimates

The probability of each intervention being the most effective was computed and visualized through rankograms and cumulative ranking probability plots. The overall efficacy and acceptability of the interventions were summarized using the Surface Under the Cumulative Ranking (SUCRA) curve, which provides a numerical value between 0% and 100%, where higher values denote superior effectiveness or acceptability (Chaimani et al. 2023). The SUCRA is created based on the rank probabilities generated from the MCMC iterations in gemtc (van Valkenhoef and Kuiper 2012). For each iteration, treatments are ranked, and these rankings are used to estimate the probability of each treatment being the best, second best, and so forth. The SUCRA is then calculated by dividing the ranks into quantiles and determining the cumulative probabilities that each treatment is within a specific rank. It reflects how consistently a treatment performs across iterations. The scatter plot of SUCRA scores illustrates that the average proportion of intervention being worse than the current treatment over the 101000 iterations (Chaimani et al. 2023).

Description of Studies

ROB in Included Studies

A comprehensive ROB assessment was conducted across five key methodological domains: confounder bias, selection bias, detection bias, outcome reporting bias, and outcome assessment bias. The findings indicate substantial methodological concerns in the body of included studies, which should be carefully considered when interpreting the synthesis of results.

In the domain of confounder bias, most studies (55.7%) were rated as having a high ROB, reflecting significant limitations in how confounding variables were identified, controlled, or reported. Only 44.2% of studies were assessed as low risk, highlighting a consistent pattern of insufficient adjustment for potential confounders that could affect internal validity.

For selection bias, a more balanced distribution was observed, with 56.9% of studies rated as medium risk, 37.5% as low risk, and only 5.6% as high risk. This suggests that while there are some concerns regarding participant selection processes (such as inadequate randomization or nonrepresentative sampling), the issue is less pronounced compared to confounding.

In terms of detection bias, which pertains to how outcomes were measured and whether these measurements were applied consistently across groups, over half of the studies (56.0%) were rated as high risk. An additional 21.5% were rated as medium risk, and only 18.9% were classified as low risk. This indicates a notable concern regarding the reliability and objectivity of outcome measurements, which may compromise the credibility of reported findings.

The domain of outcome reporting bias also demonstrated substantial risk, with 36.2% of studies rated as high risk and 45.2% as medium risk. Only 18.6% were assessed as low risk. This raises significant concerns about selective reporting of results, such as omission of unfavorable outcomes or post hoc changes to pre-specified outcomes, which may lead to a distorted evidence base.

Lastly, outcome assessment bias was predominantly rated as medium (46.0%) or high (34.0%) risk, with only 20% of studies considered low risk. This reflects issues such as a lack of blinding of outcome assessors or the use of subjective outcome measures that could introduce assessment-related variability or bias.

The summary of risk-of-bias assessment for the data set is given in the Table 7 below. Study-by-study details of the risk-of-bias for studies are provided in data set available on GitHub.

Synthesis of Results

We used a multilevel random effects model and tested the results for small study effects, as an indication of possible publication bias, whereby studies are more likely to be produced where they find desirable and significant effects and/or authors undertake multiple hypothesis testing (p-hacking) and report only those tests found to be statistically significant. The funnel plot (Figure 3) suggests asymmetry in the distribution of effect sizes by their standard errors (a measure of sample size), which is consistent with possible publication bias/small-study effects. To test for this formally, we conducted Egger's test, which includes the standard error of studies as a moderator variable in the meta-analysis model. The coefficient of the standard error is statistically significant at a 0.01% level of significance. We use the PEESE estimate to identify the “true” effect size by including the variance of the effect size as a moderator variable (Nakagawa et al. 2022). However, we also note that there is high between study heterogeneity in our meta-analysis model that includes all the interventions and effect sizes. Between study I ² ∼ 75% in our data (I ² above 50% is usually considered high) though this is not uncommon in the environmental literature (Nakagawa et al. 2022). Methods to test or adjust for publication bias in the presence of heterogeneity may not be powerful, as it is difficult to distinguish between genuine small-study bias and true heterogeneity in effect sizes. This is especially true when the meta-analysis is not large (Peters et al. 2010). The PET-PEESE approach that we employ here also performs poorly in case of high heterogeneity, though it has better properties than conventional meta-analysis (Stanley 2017). Though the size of our meta-analysis is favorably large, we are not able to necessarily distinguish between the size of publication bias and heterogeneity through statistical tests, and therefore expect the true estimate to be in the range defined by the estimates adjusted for publication bias and not adjusted for bias, as presented below.

OPEN IN VIEWER

Figure 3

Funnel plot showing the estimated effect sizes along with the standard error.

The average effect size across all interventions estimated using a multilevel random effects model is Cohen's d = 0.22 (95% CI = [0.18, 0.26]), which decreases to Cohen's d = 0.13 (95% CI = [0.09, 0.17]) on applying the PEESE adjustment for publication bias. The estimated average effect is both statistically significant and substantive across model specifications.

We assessed study quality by covering aspects of internal and external validity following the ROB framework recommended by CEE. Table 8 presents the results of the sensitivity analysis of the average effect size with respect to the assessed ROB. For each of the criteria, the average effect size is highest (between 0.25 and 0.32) for the group of studies assessed as having a high ROB, while the average effect size is comparatively smaller (between 0.11 and 0.23) for studies having a low ROB. While the magnitude of the average effect size is lower for studies with the highest quality, the effect is statistically significant for all estimated intervention effects.

Explaining Heterogeneity in Effect Sizes

The Q test indicated that there is statistically significant heterogeneity across the studies, even after accounting for small-study bias. The between-study heterogeneity and within-study heterogeneity baseline multilevel model is estimated to be approximately 75% and 25%, respectively. The diversity of interventions and their combinations used in our data set could be a source of heterogeneity in our analysis. We investigate such heterogeneity using two approaches: a meta-regression model and an NMA approach. In the following paragraphs, we first present estimates of the average effect sizes for the combinations of interventions reported in the literature.

Network Geometry, Transitivity, and Consistency Conditions for Using NMA

The primary studies included in our analysis were nearly always two-armed and compared a treatment or a combination thereof against a control. For example, if participants receive both feedback and monetary incentives, then the comparison would be feedback and monetary versus control. Since direct evidence (head-to-head comparison between interventions) was rare in our data set, our network has a star-shaped geometry (Figure 4). The predominance of a star-shaped geometry has two implications: first, as such the partial fulfillment of transitivity and second, consistency assumption does not have serious consequences for the NMA as there are not any sets to compare across and second, we do not expect the results of the NMA to differ significantly from a random effects meta-analysis because again there is only limited new information available. NMA does, however, provide an opportunity to probabilistically rank the effectiveness of interventions using SUCRA plots presented below. Additionally, the efficacy of NMA would also increase as more trials provide direct evidence between interventions.

OPEN IN VIEWER

Figure 4

Network diagram showing the number of effect sizes for each intervention combination. Combinations of interventions for which there were less than five observations have not been shown for brevity.

The first step to applying NMA was conducting a transitivity test to ensure that the studies included are similar enough so that their effect sizes are comparable. We qualitatively assessed transitivity by reviewing study characteristics across important effect modifiers identified by Khanna et al. (2021) to ensure no major systematic differences that would violate this assumption (e.g., all studies targeted a similar outcome under comparable conditions) (Lin et al. 2021). Table 9 shows the distribution of important moderator variables across the interventions. As can be seen from the table, importantly, most studies employ randomization across interventions. Across interventions, the proportion of studies with low ROB is also similar. Additionally, for most of the interventions, the predominant study design is difference-in-difference and employs means differences or OLS estimates. However, there is some variation in the geographical distribution of studies across interventions, but previous research only indicates minimal differences across geographies. Only monetary intervention studies tend to employ weather as a moderator variable. As such, we assess there is a moderate level of homogeneity in studies across interventions.

OPEN IN VIEWER

Table 9

Characteristics of the included studies across the studied interventions (selected).

No	Variable	Stats/values	Social comparison	Information	Feedback	Motivation	Monetary
1	Stats method	1. OLS	32 (66.7%)	39 (53.4%)	64 (52.9%)	3 (10.3%)	5 (16.7%)
		2. Means differences	4 (8.3%)	25 (34.2%)	29 (24.0%)	19 (65.5%)	12 (40.0%)
		3. Panel effects	12 (25.0%)	9 (12.3%)	28 (23.1%)	7 (24.1%)	13 (43.3%)
2	Study design	1. DID	28 (58.3%)	31 (42.5%)	72 (59.5%)	25 (86.2%)	15 (50.0%)
		2. Control-treatment	18 (37.5%)	19 (26.0%)	22 (18.2%)	2 (6.9%)	3 (10.0%)
		3. Pre-post	2 (4.2%)	23 (31.5%)	27 (22.3%)	2 (6.9%)	12 (40.0%)
3	Randomization	1. No	0 (0.0%)	7 (9.6%)	7 (5.8%)	2 (6.9%)	10 (33.3%)
		2. Yes	48 (100.0%)	66 (90.4%)	114 (94.2%)	27 (93.1%)	20 (66.7%)
4	Region	Asia	25 (52.1%)	31 (42.5%)	13 (10.7%)	12 (41.4%)	8 (26.7%)
		Europe inc. UK	20 (36.7%)	6 (8.2%)	55 (45.5%)	11 (37.9%)	4 (13.3%)
		North America	3 (6.2%)	18 (24.7%)	53 (43.8%)	6 (20.7%)	14 (46.7%)
		8. Others	0 (0.0%)	5 (5.8%)	0 (0.0%)	0 (0.0%)	4 (13.3%)
5	Weather	1. 0	38 (79.2%)	53 (72.6%)	84 (69.4%)	28 (96.6%)	15 (50.0%)
		2. 1	10 (20.8%)	20 (27.4%)	37 (30.6%)	1 (3.4%)	15 (50.0%)
6	ROB criterion: Confounder bias	Low	13 (27.1%)	29 (39.7%)	46 (38.0%)	15 (51.7%)	15 (53.6%)
		High	35 (72.9%)	44 (60.3%)	75 (62.0%)	14 (48.3%)	13 (46.4%)
7	ROB criterion: Selection bias	Low	18 (37.5%)	24 (32.9%)	56 (46.3%)	12 (41.4%)	11 (39.3%)
		Medium	29 (60.4%)	41 (56.2%)	57 (47.1%)	14 (48.3%)	16 (57.1%)
		High	1 (2.1%)	8 (11.0%)	8 (6.6%)	3 (10.3%)	1 (3.6%)
8	ROB criterion: Detection bias	Low	5 (10.4%)	16 (21.9%)	20 (16.5%)	6 (20.7%)	12 (42.9%)
		Medium	17 (35.4%)	20 (27.4%)	51 (42.1%)	3 (10.3%)	3 (10.7%)
		High	26 (54.2%)	37 (50.7%)	50 (41.3%)	20 (69.0%)	13 (46.4%)
9	ROB criterion: Outcome reporting bias	Low	12 (25.0%)	13 (17.8%)	9 (7.4%)	16 (55.2%)	11 (39.3%)
		Medium	35 (72.9%)	25 (34.2%)	74 (61.2%)	5 (17.2%)	6 (21.4%)
		High	1 (2.1%)	35 (47.9%)	38 (31.4%)	8 (27.6%)	11 (39.3%)
10	ROB criterion: Outcome assessment bias	Low	12 (25.0%)	7 (9.6%)	20 (16.5%)	16 (55.2%)	12 (42.9%)
		Medium	33 (68.8%)	40 (54.8%)	60 (49.6%)	5 (17.2%)	4 (14.3%)
		High	3 (6.2%)	26 (35.6%)	41 (33.9%)	8 (27.6%)	12 (42.9%)

Due to the star-shaped geometry (missing evidence on head-to-head comparisons between interventions), we are unable to validate the consistency assumption using the node-splitting method for our data structure (Liew and Lee 2019). Instead, consistency is assessed by global statistical heterogeneity quantified using the I² statistic, with a common heterogeneity parameter incorporated in the random-effects model (Fernández-Castilla and Van den Noortgate 2023). The global I² estimate for our network is 7%, which can be considered moderate heterogeneity in a meta-analysis context. This implies that there are some differences in effect size between studies—likely due to contextual or methodological diversity—but not so large as to undermine the overall findings of NMA. The random-effects model captured this heterogeneity, and the credible intervals for intervention effects were accordingly widened to reflect it. Importantly, incorporating this heterogeneity through the random-effects model adds confidence that the uncertainty is properly accounted for and the results are conservative.

The results for consistency checks confirmed that the direct and indirect evidence were in agreement (to the extent assessable). Because there were no actual loops in the evidence network, the consistency assumption cannot be violated in the usual sense. The node-splitting analysis we attempted yielded all NA results, and the global inconsistency chi-square test had between-design Q = 0 (with no degrees of freedom), indicating no detectable inconsistency. In practical terms, this means the NMA is essentially synthesizing parallel comparisons against control, and therefore the only considerations are heterogeneity and transitivity (discussed above), rather than inconsistency between different sources of evidence. The coherence of the network lends credibility to the comparative conclusions: the rank ordering and relative effects are driven by consistent patterns observed across studies rather than contradictory evidence. We emphasize, however, that “no inconsistency” here is largely a function of the network structure (and should not be interpreted as a confirmation of consistency beyond what the data allow). It simply reflects that there were no overlapping comparisons to test. Thus, our confidence in the NMA results comes primarily from the transitivity assumption holding and the reasonable level of heterogeneity, rather than from any statistical test of consistency.

Estimated Average Effect Sizes Across Intervention Combinations

The average effect sizes for intervention combinations estimated using the multilevel model (Figure 5) and NMA (Figure 6) are broadly in agreement with each other. Our analysis reveals differences in the average effect sizes across interventions. Studies that solely focus on monetary incentives have the highest average effect size (0.36; 95% CI = [0.23, 0.49]), followed by motivation (0.31; 95% CI = [0.18, 0.44]) and information (0.24; 95% CI = [0.15, 0.32]). Social comparison (0.13; 95% CI = [0.03, 0.23]) and feedback (0.10; 95% CI = [0.04, 0.17]) have similar, lower average effect size. The forest plots for each of the five interventions are provided in the Appendix S1.

OPEN IN VIEWER

Figure 5

Estimated average effect size using a multilevel random effects model for each combination of interventions studied in the literature (not adjusted for publication or risk of bias). (1) Dependent variable is the Cohen's d for each reported effect size across studies. Effect size > 0 implies reduction in energy consumption. (2) The number of effect sizes in the data set for each combination is shown in the square parentheses. Combinations of interventions for which there were less than five observations have not been shown for brevity. Estimates for all possible combinations are provided in additional Table 3. The estimated between study and within study I² from the multilevel model is 79% and 20%, respectively. Note that while global I² in network meta-analysis is a summary metric for all heterogeneity across the network, I² in multilevel meta-analysis partitions heterogeneity across different levels in the hierarchical data structure, providing targeted insights into sources of variance. Both statistics help interpret the reliability and consistency of meta-analytic results but are not directly comparable.

OPEN IN VIEWER

Figure 6

Estimated average effect size using a network meta-analysis model for each combination of interventions studied in the literature (not adjusted for publication or risk of bias). (1) Dependent variable is the Cohen's d for each reported effect size across studies. Effect size > 0 implies reduction in energy consumption. (2) The number of effect sizes in the data set for each combination is shown in the square parentheses. Combinations of interventions for which there were less than five observations have not been shown for brevity. Estimates for all possible combinations are provided in additional Table 3. The estimated global I² is 7%.

We find evidence that many interventions are complementary in that the effect of a combination of the interventions is higher than the effect of individual interventions. For example, studies that combine feedback and information, interventions are complementary, but the effect is not additive in that the combined effect is lower than the sum of individual effects. The effect size (0.29; 95% CI = [0.17, 0.42]) is higher than the effect size for information and feedback studies individually. Adding motivation strategies to most intervention packages seems to increase effectiveness. In other cases, the effect size of the combination is about the same as the individual effects, indicating little gain from combining the interventions. For example, the effect size of feedback and social comparison (0.13; 95% CI = [0.04, 0.22]) is about the same as social comparison and feedback individually.

An additional useful aspect of NMA is the ability to probabilistically compare the size of the interventions studied using the SUCRA score for each intervention combination. Figure 7 presents the SUCRA score for the studied intervention packages, along with the average effect sizes derived using the multilevel model. Two combinations of interventions, namely, information-social-motivation and feedback-information-motivation, have the highest SUCRA score, along with social-motivation, which is also ranked higher. Monetary interventions have a higher score than other standalone interventions, reconfirming the results from the NMA and the multilevel model.

OPEN IN VIEWER

Figure 7

The SUCRA score for each combination of interventions studied in the literature, as obtained from the network meta-analysis model, along with the estimated average effect size using a multilevel random effects model for each combination.

Summary of Main Results

Our meta-analyses of 192 studies and 663 effect sizes that provide estimates of the relationship between behavioral, monetary, and information interventions and reduction in energy consumption of households finds that overall, these interventions have a statistically significant, average effect size of Cohen's d = 0.22, or Cohen's d = 0.13 on applying the PEESE adjustment for publication bias.

Funder and Ozer (2019) provide updated benchmarks for interpreting effect sizes in psychology that are relevant here: an effect-size r of 0.05 (d = 0.10) indicates an effect that is very small for the explanation of single events but potentially consequential in the not-very-long run, an effect-size r of 0.10 (d = 0.20) indicates an effect that is still small at the level of single events but potentially more ultimately consequential, an effect-size r of 0.20 (d = 0.40) indicates an effect of medium size that is of some explanatory and practical use even in the short run and therefore even more important, and an effect-size r of 0.30 (d = 0.63) indicates an effect that is large and potentially powerful in both the short and the long run.

However, we also note that some intervention or their combinations yield substantially larger impacts—especially when considered at scale. We find that monetary incentives have a large average effect (0.36), along with motivation (0.31) and information (0.24). Combining monetary, behavioral, or information interventions can also increase effectiveness, even though the effect is not additive in that the combined effect is lower than the sum of individual effects. Combinations like information-social-motivation (0.38) and feedback-information-motivation (0.32) have high average effects. Among the behavioral interventions, while fewer studies have investigated motivation techniques as compared to social comparison and feedback, the results seem promising: motivation interventions by themselves or in conjunction with other interventions tend to have a higher impact. However, we should not just assume that more interventions mean bigger impacts, as the effect size of some combinations is about the same as individual effects.

What would the average effect reported in terms of Cohen's d above imply in terms of reductions in energy use? We try to provide a sense of this by calculating the average effect size both in terms of Cohen's d and percentage change in energy consumption for the subset of observations that directly report a reduction in energy consumption in percentage terms. For this subset of 397 observations, the average intervention effect is Cohen's d = 0.26 or 6.6% when expressed as a reduction in energy consumption. The average effect corrected for publication bias is Cohen's d = 0.15 or 4.6% when expressed as a percentage change in energy consumption.

Overall Completeness and Applicability of Evidence

This review searched for a wide sources of academic and nonacademic sources to create the most comprehensive data set of studies on behavioral, information, and monetary interventions for reducing energy consumption. The review also assesses how publication bias and reporting biases can affect the results obtained from studies. However, the language of the search query was English and tends to be predominantly from experiments performed in English-speaking countries. This could lead to biases if results in non-English speaking countries, importantly, Latin America, differ significantly from the countries included in our set.

Quality of the Evidence

The evidence synthesized in this review was of varying quality. Most of the studies that we identified implemented some form of randomization and used robust difference-in-difference study designs. However, the rigor of the statistical methods used varied across studies. Our ROB analysis also highlights the presence of methodological weaknesses across the included studies. Most notably, high ROB related to confounders, detection, and outcome reporting threatens the internal validity and reliability of synthesized findings (see Table 7). As such, interpretation of pooled results should be approached with caution. The potential for publication bias—where studies with null or negative findings are less likely to be published—further compounds these concerns and may skew the overall conclusions of the review. These limitations reinforce the need for transparent reporting, robust study designs, and standardized ROB assessments in future research.

Potential Biases in the Review Process

We were limited by time and financial resources in conducting this living review. While we try to ensure consistency and quality using a range of ML approaches and double-coding samples of our data set, we are unable to double-code all the studies, including the ROB assessments. We regularly checked for agreement between the different coders, but there could nevertheless be systematic differences in coding by different coders that can be addressed by double-coding the entire set of studies if resources become available in the future.

To use NMA to investigate the heterogeneity in effect sizes, we checked if our data met the transitivity and consistency assumptions. We assessed transitivity qualitatively by reviewing study characteristics across important effect modifiers identified by Khanna et al. (2021), and found moderate homogeneity in studies across interventions, though no major systematic differences were observed that would invalidate our results. Our star-shaped data set lacked any loops of evidence, so the usual statistical methods for checking consistency could not be applied. Instead, consistency was assessed by global statistical heterogeneity quantified using the I² statistic, with a common heterogeneity parameter incorporated in the random-effects model. The I² estimate for our network is 7%—likely due to contextual or methodological diversity—but is not so large as to undermine the overall findings.

Agreements and Disagreements With Other Studies or Reviews

The average effect size of the studies' interventions, along with intervention combinations, is broadly in line with those reported by Khanna et al. (2021), but provides a much more fine-grained comparison between various interventions and intervention packages. Unlike Delmas et al. (2013), who find that pecuniary feedback and incentives lead to a relative increase in energy usage rather than induce conservation, we find monetary incentives to be the most effective strategy. In this regard, the findings of this review are more akin to those of Buckley (2020). The review also confirms earlier findings by Delmas et al. (2013) that strategies providing information (including individualized audits and consulting) are comparatively more effective for conservation behavior than strategies that provide historical, peer comparison energy feedback. Javaid et al. (2022) have also found that behavioral interventions have a small to moderate effect in transport literature, and the need to complement behavioral interventions with other techniques (in this case, infrastructure). However, Lohmann et al. (2024) find that in the food domain, structural choice architecture interventions prove equally, if not more, effective than monetary incentives. The review also confirms publication bias in the literature reported by Buckley (2020) and adjusts the average effect sizes reported for publication bias, though we note that adjustment for publication bias is more difficult in the presence of high heterogeneity.

Implications for Practice and Policy

Our analysis finds that behavioral, monetary, and information interventions on average have a small to moderate effect, corresponding to 4%–6% of on energy consumption of households, though some intervention combinations have a larger impact. Ultimately, the practical consequences of the average effect sizes reported in this review, however, depend on at least three factors. First, consideration should be given to how often a person makes decisions that could be influenced by the interventions under investigation (Funder and Ozer 2019). For example, a widespread introduction of energy feedback would not only influence one purchase decision but could potentially have a cumulative effect on many different purchase decisions, which would significantly increase the actual expected effectiveness of an intervention. Second, the scalability and costs of interventions are important. A relatively cheap and easy-to-implement intervention that can be rolled out in such a way that it can influence the decisions of millions of people is to be evaluated differently from an intervention that is rather costly and resource-intensive to implement. Thirdly, interventions may differ strongly in their welfare effects (Allcott et al. 2022; Allcott and Kessler 2019), which, for example, incorporate the actual reduction in externalities, such as greenhouse gas emissions, achieved by the intervention, the cost that those who are affected by the intervention incur when changing their behavior, that is, the potentially lower comfort from curtailing energy usage in response to higher prices, potential effect heterogeneity and any other intervention effects that affect individual utility. Hence, the estimated meta-analytic average effects may serve as a good starting point for assessing the potential of interventions to promote the adoption of efficient appliances, but to increase the informative value of meta-analyses for such questions, as also recommended in Andor and Fels (2018), studies should be more transparent about the costs of interventions, scalability, and possibly welfare effects, so that these aspects can also be included in the quantitative research synthesis.

Implications for Research

Our findings support the idea that such interventions should not be looked at only individually but rather as packages to increase effectiveness. Smart packaging can ensure that the overall effect of a portfolio of well-integrated interventions is larger than the effect of individual interventions. While some mechanisms as to how various interventions interact have been suggested in the literature (Stern 2020; Wolske and Stern 2018), further research is needed to test these hypotheses, to harvest synergies and avoid possible trade-offs while combining interventions.

Our NMA was hindered by the star shaped geometry of the evidence. More studies that compare and contrast different interventions and their combinations directly, rather than always to a control group, can also help answer the question of which intervention combinations can enable more meaningful use of advanced synthesis methods like NMA that can then also help compare untested combinations of interventions.

The fast-growing literature on behavioral, information, and monetary interventions in household energy consumption makes this field a fitting case study for a “living” review assessment. Of the 663 effect sizes used for synthesis, about half come from studies produced after 2020 that were not included in previous reviews on the topic. We will continue to collect evidence and update the data set over the next 2 years as envisaged in the plan for the “living” review. However, there are significant challenges with consistently updating a review, most importantly, in terms of maintaining consistency in the identification and coding of studies, given resource constraints and changing personnel. Applying ML algorithms helped us significantly reduce the manual effort involved in identifying the relevant literature. Recent advances in AI and ML could, in the future, relieve reviewers from even more workload. However, they would need to be thoroughly evaluated, and reporting standards adjusted to avoid compromising review quality.