PROTOCOL: Housing improvements for health and associated socio-economic outcomes

Previous reviews on this topic

A number of reviews have examined the strength of association between housing specific hazards and health (Institute of Medicine 2004; Peat 1998; Rauh 2008; Raw 2001; Revie 1998; Wilkinson 1999). A recent review of reviews identified nine systematic reviews of housing related interventions which had examined impacts on health outcomes and health inequalities (Bambra 2008; Bambra 2010). Three of these reviews were of measures (including equipment and exercise regimes) to reduce falls at home amongst the elderly (Chang 2004; Gillespie 2003; McClure 2005); two reviews involved community and housing based interventions to reduce community and domestic injury (Nilsen 2004) and firearm injury (Hahn 2005); two reviews were of rental assistance programmes (Acevedo-Garcia 2004; Anderson 2003); one review examined UK investment in area based renewal, some of which included housing-led renewal (Thomson 2006); and one review assessed the health impacts of physical improvements to the housing fabric (Thomson 2001). These latter two reviews (Thomson 2001; Thomson 2006) were conducted by the authors of this protocol.

Other systematic reviews of housing interventions for health which we have identified, including Cochrane reviews, have focussed on equipment or behavioural interventions, or both, to reduce exposure to allergens amongst asthmatics (Gøtzsche 2008) and to reduce domestic injury and fires (DiGuiseppi 2000; Kendrick 2007Lyons 2006). Two further reviews conducted in the USA have been identified (Jacobs 2009; Saegert 2003). Neither of these reviews were systematic reviews, and they focussed on interventions aimed at minimising exposure to specific hazards, for example pest management, cleaning treatments, dehumidifiers and behavioural interventions to reduce domestic injury. In addition, both these reviews were limited to US literature.

The 2001 review by Thomson et al is the only international systematic review of improvements to the physical fabric of housing which has been identified to date (Thomson 2001). The review, conducted in 2000, included all quantitative studies of housing improvement, of any design, which took a measure of health, illness or wellbeing; 18 completed studies and 14 ongoing studies were identified. Of the 18 completed studies, eight were identified from electronic databases including databases of unpublished literature. The remaining 10 studies were identified through personal communication, conference attendance and handsearching bibliographies of books. Of these 10 studies, eight were conducted in the UK and two in the USA (Thomson 2002). Although this distribution between studies in the UK and the USA reflects the distribution of study locations identified through the electronic databases, it is possible that unpublished studies from beyond the UK were missed and this may have introduced some bias into the review. Many of the ongoing studies identified are now due for completion and an update to this review is required. Extra efforts to identify unpublished studies carried out beyond the UK will be made.

A Cochrane review of remediation of damp and mould to buildings, including housing, is also currently underway.

Types of studies

Prospective, retrospective, controlled, uncontrolled, randomised (including cluster randomised trials) and non-randomised studies of the health and social effects of housing improvements will be included in the review. Cross-sectional studies that do not investigate the effects of housing improvement will not be included, that is cross-sectional surveys reporting associations between housing conditions and health. Intervention studies reporting both quantitative and qualitative data will be reported in the review. The study designs and names used to describe study designs are defined in Appendix 1.

Types of participants

The review will not exclude any participants on the basis of family type, socio-economic status or other equity indicators such as race or ethnicity, occupation, education or religion. Studies from any region of the world and from both industrialised and non-industrialised countries will be included. Outcomes for both adults and children will be included in the review.

Included participants will be in receipt of a discrete programme of rehousing or housing improvement. Where households experience a change of housing conditions as an indirect result of some other life event, for example employment relocation, natural disaster, and the housing improvement is not part of a discrete programme these participants and the studies will not be included.

Types of interventions

All physical house types which are static (that is not caravans or house boats) will be included, this may include residential establishments providing permanent accommodation and sheltered housing. Housing interventions will be defined as rehousing and any physical change to housing infrastructure, for example heating installation, insulation, double glazing and general refurbishment where aspects of the housing fabric is improved. Physical improvements tailored to meet the needs of the resident will be included, for example medical priority housing. Where these improvements are limited to provision of indoor furniture or equipment, such as vacuuming, mattresses and air purifiers, these will be excluded. Studies which provide no specific information on the nature or extent of the physical housing improvement or focus on non-physical aspects of being rehoused will be excluded. For example, a study may report on the health effects of former residents of supported living quarters being relocated to live independently. It may be mentioned that the physical quality of the new housing is superior to previous accommodation but details of what the actual physical improvements are may be omitted as the intervention of interest to such a study is primarily the move to independent living. Such a study would be excluded.

Studies will be included if they have investigated changes in health, illness or wellbeing related outcomes among the residents following the delivery of a discrete housing improvement programme which has been delivered following and as a consequence of a natural disaster or labor migration. It is possible that following and as a consequence of such an event some of the population will live in improved housing. However studies will not be included where the study investigates the health and socio-economic effects of an event such as a natural disaster or economic migration but where no discrete programme of housing improvement has been delivered to the population.

Environmental studies of the adverse effects of lead, urea formaldehyde foam, air quality, allergens or radon will not be included. These studies assess the impact of exposure to the potential hazard rather than any impact of housing improvement. In addition, evidence of the harmful effects of radon, lead and asbestos are now accepted (Wilkinson 1999).

Types of outcome measures

Primary outcomes

Outcome measures will include any measure which can be interpreted as a direct measure of health, or mental and physical illness, general measures of self-reported wellbeing and quality of life measures.

Data on health service use will be extracted and reported but will not be included in the final synthesis. Studies which only report health service use outcomes will be reported to ensure provision of a comprehensive list of studies which have assessed heath related outcomes. These studies will not be included in the final synthesis.

There will be no minimum follow-up period to assess health effects. Where a study reports on health impacts at multiple time points all impacts will be extracted and reported. The final impact will be used as the study's findings. In the case where synthesis across more than one study is possible, the outcomes from the most similar time point across the studies will be used.

Secondary outcomes

Additional social and socio-economic outcomes which can be interpreted as determinants of health will be extracted, where reported; for example fuel costs, household income, measures of social contact, social exclusion, education, employment, time off work.

Electronic searches

The following electronic bibliographic databases will be searched with no restriction on language. The following includes databases considered to be relevant to the issue of health equity.

ASSIA (1987 to present) (CSA)

An example of the search strategy illustrating the search terms to be used is available in Appendix 2. The strategy and combination of terms used will be amended as required for each database. The search strategy will not be limited with respect to population characteristics such as age, gender, language, or race. The search strategy will include terms relating to public provision of housing aimed at low-income populations.

Searching other resources

Bibliographies of screened papers and identified reviews will be searched for eligible studies. Efforts to identify relevant grey literature will include contacting experts, searching SIGLE/COPAC, handsearching IDOX (formerly PLANEX), and searching relevant websites both within the UK and beyond.

A list of experts from the lead review authors' own contacts and authors of housing studies will be contacted by e-mail to request any information about completed or ongoing studies which might be relevant to the review.

Selection of studies

The results of the searches will be screened independently by two review authors to identify studies which meet the review's inclusion criteria. The initial screening will be based on study title and abstract. Where there is disagreement or ambiguity about inclusion the full reference will be obtained to allow further scrutiny of the eligibility of the study. The review authors will meet to discuss studies where there is disagreement over inclusion or exclusion of a study.

Data extraction and management

Citations will be stored in EndNote (bibliographic software). Assessment of risk of bias will be conducted by two review authors independently and disagreements resolved by discussion. The reported findings from each study will be extracted by one review author and checked by a second review author with disagreements or inaccuracies to be discussed between the authors. All data will be entered into an Access database and checked by a second review author. The final agreed data extraction will be entered into RevMan by one review author. A list of data extraction fields is available in Appendix 3.

The data extraction will include extraction of intervention context and the socio-demographic characteristics of the study sample, such as gender, race, age, and socio-economic status.

Assessment of risk of bias in included studies

Quantitative studies

We will complete the Cochrane risk of bias tool for each study. It is expected that this tool will not be sensitive to the variations in study quality across the various study designs included in this review, such as non-randomised studies and uncontrolled studies. For this reason, studies will also be assessed for risk of bias using a critical appraisal tool developed by a group of systematic reviewers in Hamilton, Canada (Hamilton Assessment Tool) (Thomas) and that has been recommended by the Cochrane Public Health Group for use in reviews of public health interventions where non-randomised studies are included (Armstrong 2008). We propose amending the Hamilton Assessment Tool to ensure that it is appropriate to studies of housing interventions, for example by including an assessment of key confounders accounted for beyond socio-demographics, such as eligibility for housing improvement and housing condition at baseline. Also, the Hamilton Assessment Tool (HAT) does not differentiate between prospective controlled study designs and other non-randomised study designs; we propose to amend the tool to allow distinctions between controlled and uncontrolled study designs. Our proposed amended HAT to assess risk of bias is presented in Appendix 4. Using this tool, each study will be assessed for the extent of bias introduced to the study with regard to selection of study population, study design, control for confounding, data collection measures and methods, blinding of assessor and participants, and withdrawals by final follow up. Each of these potential areas of bias will be graded as A, B, or C (A indicating minimal potential bias and C indicating considerable potential for bias) according to the criteria outlined in Appendix 4.

The quality assessment for each study will be carried out by two independent review authors and entered onto a Microsoft Access© database. Disagreements in any one of the six points of assessment (selection, study design, confounding, data collection, blinding, withdrawals) will be resolved through discussion between the two review authors.

Each study will be assigned to a summary category (A, B, or C) indicating the overall potential for bias. The criteria for this summary category are outlined in Appendix 4.

Qualitative studies

Qualitative studies, including studies reporting qualitative data supplementary to quantitative data, will be included in the review. There is much unresolved debate about appropriateness of assessments of the quality of qualitative studies and their data. Despite this, it is important to present details of the study design, sample, and data collection methods, as well as an indication of the review authors' appraisal of the validity of the reported findings and their interpretation.

Data on the study aims and methods, including sampling details and data collection methods, will be extracted and tabulated to provide an overview of the study design and methods. In addition, a critical appraisal tool developed for qualitative studies and previously recommended for use in systematic reviews will be used, such as the tool developed by the Joanna Briggs Institute.

Intervention implementation and performance bias

Variation in the ways in which an intervention is implemented may introduce bias and explain variance in the reported effects within a study (Type III error) (Dobson 1980). This may be referred to as performance bias.

It cannot be assumed that the housing improvements were implemented as originally planned, or that all recipients of the intervention used the intervention in the same way. Variation in intervention implementation may result in variation in exposure to the critical changes that the intervention aims to affect and will result in variation in the potential to benefit within a study. For example, the extent of housing improvement may be tailored according to individual household need and will vary, therefore, by the housing condition at baseline. In addition, delivery of a housing improvement may not result in exposure to improved housing conditions. For example, fear of costly fuel bills may prevent use of a new central heating system, or if an intervention is implemented without assessment of need there may be households where the potential to improve housing conditions is limited if housing conditions are satisfactory at baseline.

Included studies will be assessed for within study heterogeneity with respect to intervention implementation and also for heterogeneity in the extent of improvement in housing conditions actually experienced by participants (see Appendix 4).

Measures of treatment effect

Comprehensive Meta-Analysis (CMA) software© will be used to calculate standardised effect sizes for all health outcomes from controlled studies which report necessary data. These outcomes will be likely to include continuous and dichotomous variables and the standardised effect will be reported as an odds ratio (OR) and 95% confidence Interval (CI).

Unit of analysis issues

Housing interventions are allocated and implemented at a household level either to individually targeted households meeting pre-specified eligibility criteria or to all households within a targeted geographical area. However, health outcomes are assessed at an individual level.

In some studies health outcomes are only assessed for one occupant, and in others health outcomes are assessed for more than one or for all occupants (though these assessments may be made on behalf of other occupants by a nominated occupant). The sample type is likely to vary across the identified studies. We will extract all reported health and socio-economic outcomes for however many occupants have been included in the study. Where a study presents different data for different occupant types, the categories are likely to be adult or child; adult: gender, diagnosed with specific illness for example asthma or not diagnosed. Other subgroups of interest with respect to equity indicators are race or ethnicity, occupation, socio-economic status, education, religion. For the main analysis child and adult data will be reported and analysed separately. Data and analysis on other subgroups mentioned above, in particular those with equity implications, will be extracted, reported and synthesised where there are sufficient, similar data.

Dealing with missing data

We will contact authors of studies to obtain missing data. We will report withdrawals and levels of attrition for each study and incorporate these into the overall indication of study quality. CMA will be used to calculate standardised effect sizes for controlled studies which report the necessary data.

Assessment of heterogeneity

Statistical heterogeneity will be assessed using the Chi² and I² statistics. If appropriate, a meta-analysis of effect sizes will be conducted using a fixed-effect model, otherwise a random-effects model will be considered.

Heterogeneity within and between the studies will be investigated and reported with respect to study design, study quality, intervention, context, and implementation of the intervention. See the section on ‘subgroup analysis and investigation of heterogeneity’ for a more detailed description of how heterogeneity between the studies will be dealt with, and also ‘intervention implementation and performance bias’ for details of how heterogeneity with respect to implementation and performance bias will be assessed.

Assessment of reporting biases

We will investigate the impact of publication bias by preparing a funnel plot and calculating Egger's test if there are sufficient studies which report standard errors for the effect sizes.

Data synthesis

Quantitative data

Data from the better quality studies will be synthesised and the final synthesis will reflect the relative weight of evidence within each group of studies.

It is anticipated that there will be extreme levels of heterogeneity within the collection of studies identified. It has previously been recommended that measures to overcome heterogeneity should be taken, where possible, to facilitate a meta-analysis. These measures include calculation of standardised effect sizes, grouping of studies appropriately with respect to interventions and outcomes, and use of a random-effects model (Ioannidis 2008). Effect sizes from studies will be calculated for all prospective controlled studies identified using Comprehensive Meta-Analysis software©. Where data for similar outcomes following similar housing improvements (outcomes and interventions grouped as outlined in ‘subgroup analysis and investigation of heterogeneity’) are available these effect sizes will be pooled. Where the same continuous outcome is reported across more than one study a standardised mean difference will be computed (using the Comprehensive Meta-Analysis© software). These will be combined in a meta-analysis to calculate a weighted mean difference. Where a dichotomous variable is reported, an odds ratio will be calculated. Heterogeneity will be assessed using Chi² and I² statistics. If appropriate, a meta-analysis of effect sizes will be conducted using a fixed-effect model, otherwise a random-effects model will be considered for use. Where effect sizes are not amenable to pooling due to extreme heterogeneity, a forest plot will be used to illustrate the range of effect sizes. Where the outcomes within a category are similar but not the same, for example different measures of respiratory health, and they are a mix of continuous and dichotomous variables we will present effect sizes as odds ratios and present them in a forest plot to allow all the effect sizes to be shown together. The raw data from both dichotomous and continuous outcomes will be transformed using the Comprehensive Meta-Analysis© software. Results of experimental and quasi-experimental studies will be analysed separately.

For groups of studies where a statistical synthesis of the data is not appropriate the data will be synthesised narratively according to the ESRC guidance (Popay 2006). The main steps of the narrative synthesis will involve articulating a theory of how housing improvement might lead to health impacts (see ‘How the intervention might work’), conducting a preliminary synthesis to test the theory, exploring the relationships in the data (within and between similar studies), and assessing the robustness of the synthesis. The data from each study will be tabulated to provide a textual as well as a visual summary of the data; this will help facilitate the synthesis by illustrating emerging patterns with respect to reported impacts and study characteristics as well as improving the transparency of the synthesis.

To present a clear demonstration of what studies have been identified, how these studies compare to better quality studies, and why they have not been included in the final assessment, a narrative description of all studies, regardless of study design, will be included in the final review with an accompanying tabulation of all studies and reported impact data. This information will provide a definitive map of intervention research in this field and help to point to how future research design could be improved within this field.

Qualitative data

The synthesis of data from multiple qualitative studies has been contested as contrary to the qualitative methodological approach and epistemology. It has been argued that essential differences between studies with respect to theoretical and methodological foundations means that to synthesise data will overlook the strengths and values of the data that emphasise the importance of specific contexts, individual experiences and attached meanings. However, others argue that qualitative data can uncover impacts not predicted or detectable by quantitative studies and also shed light on important confounding factors and pathways which may help explain the variance in predicted health impacts. Importantly, these data may be generalisable to other similar contexts, populations and interventions.

It is expected that the qualitative studies will be heterogeneous with respect to intervention, context and population, as well as methodology and study quality. For these reasons we propose to conduct a narrative synthesis of the qualitative data in accordance with the ESRC guidance on narrative synthesis (Popay 2006). The findings from each study will be tabulated to provide a textual summary of the data. This will facilitate a thematic analysis and examination of emerging themes with respect to reported impacts, mediating factors, and pathways affecting health impacts.

Subgroup analysis and investigation of heterogeneity

We will assess heterogeneity for different aspects including statistical heterogeneity. With respect to heterogeneity of interventions, the synthesis will be carried out for groups of studies which include similar interventions, as described below.

Study heterogeneity: methods, intervention, population, context, and outcomes

The broad scope of this review will inevitably mean that the identified studies will display extreme variance in the methods used, the interventions being assessed, the study populations, and contexts in which the intervention is being implemented; and the potential range of illness, health and wellbeing outcomes being assessed. In addition to details of the intervention, study sample and study methods, and details of the local context such as rurality, slum conditions will be extracted, where available. We propose to group the interventions into broad categories of the type of housing intervention and, if appropriate and there is a sufficient number of studies, according to the context and population of the study. For example, studies relating to modern day housing interventions in wealthy countries, such as insulation and central heating, will be grouped and analysed separately from older studies of rehousing from slum neighbourhoods. The reported outcomes will also be grouped into broad categories, for example respiratory health, general health and mental health.

Data will be presented according to the appropriate groups for intervention, context and population, where possible. In addition, the reported data will be accompanied by an indication of study design, overall study quality, different aspects of potential for bias, and also an indication of intervention integrity (see above ‘Intervention implementation and performance bias’).

Statistical heterogeneity

Where there are substantial levels of statistical heterogeneity (> 50%) the data will be checked for accuracy. If statistical heterogeneity persists the data will be meta-analysed using a random-effects model. If substantial heterogeneity persists the standardised effect data will be presented on a forest plot but a meta-analysis will not be performed.

Investigation of equity and differential impacts across population subgroups

The studies in this review are expected to focus largely on low-income populations living in poor quality housing, including publicly provided housing. Knowledge of impacts on low-income populations is important with respect to improving the health of the worst off and may indicate the potential for housing improvements to impact on health inequalities. However, assessments of and data on variations in impact across different socio-economic groups are needed to confirm whether or not an intervention is likely to impact on the gap in health status between high and low income groups.

Where available, data for specific population subgroups will be extracted and reported separately, for example where impacts are reported by gender, socio-economic status, educational status, or religion. Where sufficient, similar data on specific subgroups are available we will consider synthesising and presenting these data separately to illustrate the differential effects for different subgroups.

Sensitivity analysis

Before making decisions about which studies should be included in the final syntheses, a sensitivity analysis will be conducted to examine variation in reported effects by study characteristics. This will be done for each outcome category (see ‘subgroup analysis and investigation of heterogeneity’) where data are available. The sensitivity analysis will require using the standardised effect measure and examination of heterogeneity and variation in heterogeneity when only studies with minimal risk of bias are included in the meta-analysis. The key study characteristics used for the sensitivity analysis will be the six potential sources of bias (study design, confounding, selection, data collection, blinding, and withdrawals) as well as the overall study quality grade assigned. Some other study characteristics may be identified for inclusion in the sensitivity analysis during the course of the review.