Evaluating urban metabolism assessment methods and knowledge transfer between scientists and practitioners: A combined framework for supporting practice-relevant research

Abstract

Recent years have seen a substantial growth in urban metabolism research, resulting from increasing scientific consensus that metabolic flow assessments can inform resource-efficient urban policy. However, only a few works report on retrospective evaluation of the relevance and impact of urban metabolism studies in urban planning. Practice-relevant urban metabolism research depends on the applicability of assessment methods as well as on the effectiveness of knowledge transfer between scientists and practitioners. This paper presents a retrospective evaluation of a collaborative urban metabolism project (EU-FP7 BRIDGE) conducted through empirical inquiry. The goal of the inquiry was to evaluate the applicability of the BRIDGE assessment method in urban planning and the effectiveness of the knowledge-transfer approach in one of the BRIDGE case-study cities (Helsinki, Finland) in a combined fashion. Through generalization of key findings on strengths and areas of improvement, a combined framework to evaluate both aspects in the design of urban metabolism projects is proposed. The framework aims at supporting scientists and practitioners in the development of collaborative research that can accommodate expectations as well as sustainability priorities and objectives of both parties.

Keywords

Urban metabolism project design Decision Support Systems Community of Practice sustainability knowledge exchange science–practice communication and collaboration

Introduction

Urban metabolism assessments and relevance for planning practice

Urban Metabolism (UM) research provides an interdisciplinary framework to investigate drivers of resource demand across urban scales, through quantification of energy, materials, water, and nutrient flows (Kennedy et al., 2007). UM assessments are key to gearing urban sustainability strategies and policy toward optimized urban resource management (Kennedy et al., 2011). Industrial ecology’s material flow analysis and energy accounting are the most used UM assessment methods (Cui, 2018; Newell and Cousins, 2014). However, only 42% of flow analyses and 31% of energy accounting studies specifically target planning practitioners and decision-makers, and the remaining majority is disseminated through publications for the scientific community (Beloin-Saint-Pierre et al., 2017). Shortcomings of UM methods that limit their applicability in urban planning and policy-making are increasingly investigated in the UM literature (Li and Kwan, 2018; Perrotti and Stremke, 2018). Since UM flow assessments primarily respond to the need to establish technical frameworks for resource accounting (Kennedy et al., 2011), additional expert interpretation is required to translate results into guidelines for urban planning. Material and energy assessments are only rarely combined with analysis using environmental pressure indicators such as Ecological and Carbon Footprints, which are closer to the language of planning practice (Galan and Perrotti, 2019). The limited applicability of UM studies in urban planning also results from the use of black-box models, which do not capture the spatial patterns of resource flow distribution in urban systems (Golubiewski, 2012). Several attempts to geo-reference disaggregated data and UM flows have been made in the last years (e.g. Pincetl et al., 2015; Xia et al., 2017). Spatially resolved studies can help express the impacts of urban forms on the resource-intensity of cities and can, therefore, facilitate the incorporation of UM thinking in sustainable urban planning practice.

UM knowledge transfer

The question of how results of UM studies can be transferred to urban planning agendas was at the starting point of the research presented in this paper. The process of UM knowledge transfer can be studied using insights from the wider field of sustainability science (Mauser et al., 2013). Research into sustainability knowledge transfer highlights the multiple challenges arising from transdisciplinary discussions among researchers as well as from the integration of governmental/professional stakeholder knowledge. The underlying assumption is that the incorporation of different communities of knowledge into the same analytical and/or problem-solving framework is necessary to: (i) gain an understanding of the drivers and impacts of the studied situation/problem; and (ii) the delivery of effective guidance for strategies and solutions at the local level (Lang et al., 2012). Among the several knowledge-transfer models discussed in the literature, there is a growing understanding of knowledge transfer as an iterative and reflexive cycle, as opposed to a linear, one-way science-to-practice process (Böcher and Krott, 2014). Mutual learning processes can lead to the co-generation of new solution-oriented knowledge and produce more practice-relevant results (Binder et al., 2015). The growth of retrospective evaluations of knowledge-transfer processes in recent years (Ugolini et al., 2018) reveals an increasing interest from both sides in bridging the science–practice communication gap in order to better face sustainability challenges. This trend is likely to grow in the future, in response to the strong emphasis on science–practice knowledge co-production and increasing demand for more participative research methods by the European Commission (e.g. Horizon 2020, Interreg, COST programs) and other funding agencies (Ugolini et al., 2015).

In UM research, early stage integration of stakeholder inputs and knowledge co-production can foster more versatile assessment frameworks, accommodating different stakeholder needs and levels of complexity (Dijst et al., 2018). Co-design of UM assessment objectives, methods, and tools with end-users can favor strong commitment from of all parties involved in sustainable UM transitions and stimulate more inclusive policy and action (Huang et al., 2015). It is therefore crucial to establish a comprehensive framework to investigate the conditions for effective knowledge transfer in UM research. Retrospective evaluation of existing knowledge-transfer approaches in collaborative UM projects can provide key insights on strengths and areas of improvement.

A combined evaluation of assessment methods and knowledge transfer

This article aims at proposing a combined framework to evaluate the conditions for both UM assessment applicability in planning practice and effective knowledge transfer in collaborative UM projects. The combined evaluation framework is proposed as a way forward to leverage more practice-relevant UM research. This paper builds on the results of research conducted in Helsinki, Finland (September 2015–July 2016). The research consisted of a retrospective evaluation of a neighborhood UM study in Helsinki, developed within the collaborative UM project BRIDGE (sustainaBle uRban plannIng Decision support accountinG for urban mEtabolism, EU-FP7). BRIDGE (2009–2011) was developed by an interdisciplinary consortium of scientists and planning practitioners in five case-study cities (Helsinki, Athens, London, Florence, and Gliwice) (see “Background” section). The main outcome of BRIDGE was a GIS-based decision support system (DSS) to inform sustainable urban planning based on UM assessments at the scale of municipalities, neighborhoods, and urban areas.

The research presented here aims to:

investigate the potential of the BRIDGE UM assessment method to inform urban planning orientations and sustainability policy by the City of Helsinki;

analyze the limits and potential of the knowledge-transfer approach adopted in BRIDGE to favor collaboration between scientists and practitioners.

The retrospective evaluation was conducted through a set of in-depth interviews with scientists and local practitioners having participated in the BRIDGE study of Helsinki, and triangulation with literature on the project and local planning documents. The research focused on BRIDGE, given its explicit ambition to improve the communication of scientific UM knowledge to the DSS end-users for the purpose of supporting sustainable urban planning and by means of science–practice collaborations (Chrysoulakis, 2015). Moreover, the BRIDGE DSS allowed geo-referencing of disaggregated data on energy, water, carbon, and pollutant fluxes (Mitraka et al., 2014), an essential aspect to enhance UM assessment applicability in planning practice (see above). Finally, BRIDGE was selected among other collaborative UM projects (e.g. EU-FP7 SUME, Schremmer et al., 2011) since it included an internal self-evaluation component on the knowledge transfer in the five case-study cities (Klostermann et al., 2015), the results of which were already published at the time the research was conducted.

The paper is structured as follows. The “Background” section provides a description of the approach to both UM assessment and knowledge transfer adopted in BRIDGE. In the “Method and materials” section, first, the reasons for selecting the Helsinki case study are described; then, the inquiry strategy and method employed to conduct the interviews and collect secondary data are presented. Subsequently, key findings of the inquiry are presented and discussed with respect to the impact of the BRIDGE UM assessment on urban planning orientations in Helsinki and the interviewees’ perceptions of the knowledge transfer in BRIDGE. Finally, following generalization of the key findings, the proposed combined evaluation framework is presented, including recommendations to parties undertaking future collaborative UM studies.

Background

The BRIDGE UM assessment method

The main goal of BRIDGE was to provide urban planning practitioners with a GIS-based DSS tool to assess urban planning interventions and visualize the results on maps. Three alternative planning proposals were assessed in each case-study city in terms of their impact on UM flows and related sustainability performance (Chrysoulakis et al., 2013). The main ambition and novelty of the project was to include both environmental and socioeconomic aspects of the UM in the assessment. These included: fluxes of energy, water, carbon, and pollutants (environmental); demographics, employment, human wellbeing, mobility/accessibility, social inclusion, and investment costs (socioeconomic). A core part of the project was the selection of a set of indicators for the DSS, to assess the planning alternatives against both aspects of the UM. First, the sustainability objectives of the local authorities were identified. Second, these objectives were translated into criteria to evaluate the sustainability of the planning alternatives in each city (e.g. air quality, energy/water balance, thermal comfort, economic viability, accessibility). Finally, environmental and socioeconomic indicators were established for each criterion (González et al., 2013). Examples of indicators include: pollutants/carbon concentration, energy usage in buildings and transportation, renewable energy potential, surface run-off/evaporation (environmental), number/type of dwellings, population growth, percentage of owned/rented dwellings, travel time to work, and use of public transport (socioeconomic). Consideration of all indicators in one single assessment framework was possible through the use of multicriteria analytical techniques (see methodological framework and flow diagram in Supplemental Figures S1 and S2). Indicator lists were initially established at the level of each city (see list for Helsinki in Supplemental Table S1). Subsequently, through comparison of the five indicator lists, a final set of “core” indicators (for all case-study cities) and “discretionary” indicators (city-specific) were identified and used to assess the planning alternatives (Supplemental Table S2). Spider diagrams were used to visualize and compare the score of each alternative against each objective (compared to a reference baseline), as well as an overall performance index for each alternative. The DSS also included a strategic scenarios component. Each planning alternative could be assessed against three different scenarios to 2030, expressing high environmental pressure (“climate change” scenario), economic constraints (“lack of energy” scenario), or the absence of both constraints (“BRIDGE in Wonderland” scenario) (see description in Supplemental Table S3). Assumptions on environmental conditions were based on the IPCC scenarios A2, A1F1, and B1 (Marques et al., 2015). Finally, to prioritize indicators according to specific sustainability policies in each city and the characteristics of each scenario (e.g. prioritizing environmental gains over socioeconomic benefits in the “climate change” scenario), indicators could be weighted through pair-wise comparison (Castro and Marques, 2015).

Knowledge-transfer approach

A Community of Practice (CoP) approach was adopted in BRIDGE in order to structure the collaboration and mutual learning process between environmental and social scientists and the DSS end-users (Klostermann et al., 2015). Two rounds of local CoP meetings were organized in each city with representatives of local planners, as learning spaces to bridge science–practice knowledge gaps. The first round of CoP meetings allowed building networks, discussing local urban development issues, and identifying sustainability objectives. Sites and existing planning proposals to develop the DSS were proposed. The second round of CoP meetings focused on the development of the indicator list at the level of each city. At the end of the local CoP meetings, two umbrella CoP meetings were organized with the entire BRIDGE consortium. In the first meeting, the final indicator set was agreed, and a beta version of the DSS tool was introduced to the end-users. The second umbrella CoP meeting focused on the strategic scenarios exercise and weighting system. A final project seminar was organized to test an updated draft of the DSS prototype with end-users and other interested parties, and to gather feedback before its final release. The DSS tested at this stage was indeed only partially developed; the final version was issued at the end of the project.

Method and materials

Selection of the case study

Helsinki was selected as a case study due to the nature and ambition of the City Council’s urban planning policy at the time the research was conducted. The “New Helsinki City Plan—Vision 2050” was drafted in 2013 and approved in 2016, in order to support long-term urban development, based on a predicted population increase of 40% by 2050 (compared to 2013). Helsinki in 2050 is described as an “urban, rapidly growing rail-transport network city.” The following urban development drivers are identified in the New Plan: new “local centers” developed through urban densification and clustering of housing and services in suburban areas, fast-rail network connections between the city’s inner-core and the new suburban centers, and enhanced accessibility to green spaces and the urban waterfront. In light of these ambitions, the research focused on the potential of the BRIDGE results to inform urban planning orientations and sustainability policy in the context of the implementation of the New Plan.

In Helsinki, the BRIDGE consortium concentrated on the analysis of urban densification strategies in Meri-Rastila, a low-density residential neighborhood in the East-Helsinki district of Vuosaari. Meri-Rastila had been classified as a priority area for mixed housing and service development in the former Helsinki City Plan 2002 and was subsequently identified as one of the city’s new local centers in the New Plan. In 2013, the neighborhood was characterized by poor architectural and urban quality and lack of services, although featuring an extensive urban forest punctuated by ice-age geological outcrops (Figure 1), and good public transport accessibility (Rastila metro station) (Nikinmaa and Vesala, 2010). Meri-Rastila concentrated the highest percentage of the nonnative Finnish population within Helsinki, equaling 30% of the neighborhood’s population, compared to an average 10% in the rest of the city (Vilkama, 2011). The BRIDGE UM study focused on the planning alternatives proposed by the City Planning Department for a housing development and new workspaces on a forested area along the western waterfront of Meri-Rastila (Meri-Rastila Länsiranta) (27 hectares forest, 7 hectares water, Figure 1). The question addressed in the BRIDGE Helsinki study was “What kind of neighborhood should be built within a walking distance (600 m radius) to the Rastila metro station?”

Figure 1.

(a) Location in Helsinki City and administrative boundary of Meri-Rastila neighborhood (light-shaded) and Meri-Rastila Länsiranta area (dark-shaded) (based on Orthophoto 2017 ©Helsinki City), (b) Meri-Rastila forest and ice-age rock outcrops (©author), and (c) Forest edge, Meri-Rastila residential neighborhood (©author).

In light of the urban planning orientations in Helsinki in 2015–2016, the research focused on the impact of the BRIDGE study on the planning process in Meri-Rastila Länsiranta, through analysis of primary and secondary sources.

Primary sources

The research involved a set of in-depth, semi-structured interviews conducted over six months (January–June 2016) with Helsinki-based practitioners and scientists having participated in the BRIDGE study of Meri-Rastila (12 interviewees in total). These included six practitioners at the City of Helsinki (five urban planners and one employee at the main, city-owned energy company in Helsinki) and six scientists at the University of Helsinki (or University former employees at the time BRIDGE was conducted). The interview questionnaire was developed based on the analysis of key secondary sources, including scientific/gray literature and planning documents (see “Secondary sources” section). The questions were clustered around four main themes, addressing one or both aspects of the evaluation (applicability of the BRIDGE UM assessment in the local planning process, and effectiveness of the CoP knowledge transfer in BRIDGE) (Table 1). The first thematic cluster (knowledge transfer) addressed the roles played by scientists and practitioners in the Helsinki CoP meetings, and the relations between the BRIDGE participants. The second cluster (UM assessment method) addressed the impact of the BRIDGE UM study on the planning process in Meri-Rastila Länsiranta, i.e. whether and to what extent the study had informed follow-up steps of the process. The third cluster (both aspects) focused on the strengths and limitations of the BRIDGE CoP approach and DSS tool. The interviewees were asked to suggest possible methodological improvements to enhance both project components. Finally, the fourth cluster (both aspects) was meant to record a more general appraisal of the added value of the BRIDGE approach (compared to other approaches the interviewees may have experienced/have knowledge of) and feedback to improve future collaborative UM projects. Additionally, the practitioners were asked to provide more specific suggestions for the development of practice-relevant UM assessment tools. Open questions were used throughout the questionnaire in order to better gather interviewees’ tacit knowledge and open the investigation to previously unforeseen perspectives. Supplemental Table S5 presents examples of questions in each thematic cluster.

Table 1.

The four thematic clusters in the interview questionnaire and aspect of the evaluation addressed in each cluster.

Thematic clusters of questions	Evaluation of UM assessment method	Evaluation of knowledge-transfer approach
1. Roles of practitioners and scientists in the BRIDGE knowledge-transfer process, relations between participants, and effectiveness of the process		X
2. Applicability of UM study in urban planning in Meri-Rastila Länsiranta	X
3. Strengths and limitations of CoP approach and DSS tool, suggested methodological improvements	X	X
4. General appraisal of the project, added value, and shortcomings of the BRIDGE approach	X	X

CoP: Community of Practice; DSS: decision support system; UM: urban metabolism.

Table 2.

Summary of key findings extrapolated from the qualitative content analysis of practitioners’ and scientists’ responses, according to thematic cluster of questions. For full Practitioners' and Scientists' quotes see Supplemental Table S5.

Cluster	Key findings from qualitative content analysis of
Cluster	Practitioners’ responses	Scientists’ responses
1. Roles of practitioners and scientists in the BRIDGE knowledge-transfer process, relations between participants, and effectiveness of the process	Conversations with scientists were key to practitioners’ understanding of UM concept and UM assessment methods.Establishment of indicator list was a crucial aspect in practitioners-scientists knowledge exchange. List established by scientists and then agreed by practitioners. Practitioners felt having limited technical knowledge to understand environmental indicators.	Knowledge co-production among different disciplines and between science and practice in CoP meetings was among the most valuable aspects of the whole BRIDGE.In identification of indicators, attempt to build Helsinki’s list in a cross-disciplinary way, and making sure practitioners had a general understanding of selected indicators.Despite knowledge gaps on specific UM aspects, collaboration with practitioners was particularly successful. Practitioners were described as “open-minded people,” ready to accept innovation.
2. Applicability of UM study in urban planning in Meri-Rastila Länsiranta	Final results of assessment were not precise, so incorporation into urban planning documents was difficult.DSS tool should have been designed for use in initial stages of the planning process (when major decisions are made), not to “polish design” of planning proposals.Almost no difference among alternative assessments was a clear limitation of UM method. Not possible to derive clear recommendations for planning process.UM method did not allow considering key challenges of urban densification in Meri-Rastila, e.g. strong controversies on Component Master Plan, forest urbanization. This limited applicability in planning process.Public consultation is a critical stage in the Finland planning system, so it is fundamental to communicate UM assessment results before this stage to inform public consultation on planning proposals (these were already published for consultation when DSS was developed).	Need to better harmonize/coordinate different timeframes of research and policy-making/planning practice when elaborating DSS.Results of alternative assessments (e.g. building energy demand for each urban form) should have been communicated at early stages of Component Master Plan. This limited applicability in planning process.Planning process characterized by strong public debate on forest urbanization. This made Meri-Rastila a relevant area to study tradeoffs between environmental/social aspects. This is, however, a very complex aspect of UM assessments, and time/budget in BRIDGE were limited.Assessment results showed no clear gains in urbanizing forest (no economic/environmental benefits, no improved neighborhood spatial quality). But results not considered in planning process (decision to urbanize/densify Meri-Rastila already taken by local authorities).
3. Strengths and limitations of CoP approach and DSS tool, suggested methodological improvements	More attention should be paid in CoPs to adapt tools to specific practitioners’ skills/expertise (can vary from country to country).Consideration of planners’ educational background is essential in UM knowledge transfer; no knowledge prerequisites should be taken for granted. In Finland most planners have architectural education. This influences UM methods (indicator choice/weighting system) and interpretation of environmental/socioeconomic data/results.All socioeconomic/environmental indicators in DSS were relevant for planning process. However, indicator ranges were very wide and this hindered (rather than enhanced) DSS use in planning. DSS applicability in planning process was limited by the too complex nature of tool.Scenarios exercise was the least successful component of DSS. Assessment of present situation in Meri-Rastila Länsiranta based on needs/opportunities would have been more valuable than long-term projections. The parameters of the three scenarios added further level of complexity/uncertainty instead of providing practical insights for decision-making.Weighting system was hard to use. Its understanding/use depends on practitioners’ background (e.g. if users have social-science background, social criteria are better understood and consequently higher valued than environmental). Local authorities’ political orientations can bias use of weighting system. Greater weights can be attributed to aspects that are central to political agendas in a specific situation. Political orientations change over time; planning decisions made today may not be shared by future decision-makers.Weighting system was even harder in scenarios exercise. Weights are attributed based on technical solutions and local authorities have to cope with problems (energy shortage/climate change) at the time of the assessment. However, technological progress is fast. A major problem today will be easily solved with technology available in the future (e.g. new renewable energy sources; more electric cars in the future will reduce CO₂ emissions).	Main strength of CoP meetings was they allowed discussions among scientists from different disciplines (Helsinki meetings) and with scientists from other cities (umbrella meetings).Practitioners’/Scientists’ time to be invested in the CoP was a problem (conflicting with other commitments).CoP is very relevant approach to develop DSS for decision-making in cities. However, more time and higher budget are needed if scientists have to perform in-depth analysis, and additionally communicate results/data to scientists in other fields as well as to practitioners.As for the DSS development, it is essential to better coordinate/synchronize timeframes of knowledge-transfer and planning processes.
4. General appraisal of the project, added value, and shortcomings of the BRIDGE approach	Opportunity to talk explicitly about UM for the first time was the most significant added value of BRIDGE. UM concept in BRIDGE allowed considering all environmental/socioeconomic impacts of urban planning together, which is key to sustainable planning. Normally, resource management strategies (water/energy/waste) are elaborated by distinct city council departments/decision-making sectors and are part of different planning documents.BRIDGE UM assessment (and UM frameworks in general) was too broad to produce results that can be easily translated into planning strategies.For UM frameworks to produce useful results for planning, project scopes must be narrowed down and number of criteria reduced (e.g. energy efficiency of planning proposal). Recommendations for planners can be more easily drawn when results are more precise.Qualitative data (e.g. people’s perception of gains/downsides of planning proposals, inputs from public consultation) are not part of UM assessments. This limits their relevance for real-world planning.To better inform planning, UM studies should engage more directly with the political space and relations of power in which planning processes are embedded.Rather than UM assessment results, bringing stakeholders with different backgrounds together and allowing conversations/interactions with scientists was the most successful aspect of BRIDGE.UM (as used in BRIDGE) is more an inspiring concept/way of thinking the city, than an operational framework for planning.	BRIDGE’s scope was very broad and general aims very ambitious. But this is the specific added value of UM compared to other approaches.Both budget and time scientists/practitioners could invest in project were too limited to establish a proper knowledge-transfer process and undertake investigations/measurements at appropriate levels for UM research. Due to limited time/budget, microclimatic data were not collected in Meri-Rastila (already available data from other sites were used).Use of holistic UM framework was among the strongest added value of BRIDGE. Compared to other approaches, UM frameworks provide an interdisciplinary perspective. This can help address new and otherwise neglected research questions that are critical for urban studies (e.g. functional interdependency/tradeoffs among socioeconomic/environmental aspects).Use of UM frameworks makes transdisciplinary cooperation an essential requirement to develop research on cities. Therefore, UM projects can prove the relevance/usefulness of transdisciplinary approaches to work on urban problems, and inspire similar attitudes in other research fields.

CoP: Community of Practice; DSS: decision support system; UM: urban metabolism.

Secondary sources

Analysis of the secondary sources allowed, in a first instance, to identify the four main themes around which the interview questions were clustered (Table 1). They were also used to triangulate the results of the primary data sourced through the interviews, by comparing evidence from the sources with interviewee claims. Three categories of secondary sources were used in the research: scientific and gray literature on the BRIDGE results produced by the consortium, and planning documents at the neighborhood and city scale by the Helsinki City Planning Department. Supplemental Table S4 provides sources, details, and description of documents in each category.

Data analysis technique

Qualitative techniques were preferred to quantitative methods in order to gain depth of meaning from the analysis of the data collected (Creswell, 2014). Transcripts of the interviews were researched by means of systematic qualitative content analysis, using iterative coding techniques (Miles and Huberman, 1994). The transcript texts were screened to identify content analytical units (e.g. key statements repeated throughout the texts and recurring expressions within the same cluster of questions) and to detect recurring patterns. To ensure internal validity, the results of the transcript content analysis were triangulated with data from the secondary sources (see above).

Results

The BRIDGE study of Meri-Rastila Länsiranta concentrated on three planning alternatives for housing development. The alternatives were elaborated prior to the start of BRIDGE, at an early stage of the local “Component Master Plan” (see description in Supplemental Table S4). The three planning alternatives differed in population density (500, 1500, 1800 inhabitants, respectively), urban form and building volumes, location of building units on site, and built/green area ratio. Details of each planning alternative are provided in Supplemental Figure S3. The first planning alternative with the lowest density was used as a default case (all indicator equal 1). As shown in the spider diagram (Figure 2(a)), assuming equal weights for all indicators, the three alternatives had similar average performance indexes. The second alternative scored slightly higher than the default case and the third alternative, although the water balance and mobility/accessibility criteria had lower scores. Moreover, the results of the strategic scenarios exercise showed little contrast between the performances of the alternatives in the three scenarios (Figure 2(b)) (Chrysoulakis et al., 2013). In the absence of both economic and environmental constraints (“BRIDGE in Wonderland” scenario), their performances were almost equal. In other words, the assessment showed no clear gains in increasing the neighborhood built areas when a long-term perspective is taken. The third alternative (higher-density) scored better in the “climate change” and “lack of energy” scenarios, even if only with a marginal advantage over the second alternative. The nearly undifferentiated outcomes of the assessment demonstrated the need for more accurate socioeconomic input data in order to predict the performance of planning proposals in the long term and to produce actionable results for planners. Hence the scenarios exercise revealed the limits of indicator-based evaluations in the definition of sustainable urban planning agendas on a long-term basis. Rather than representing an additional aspect of the quantitative assessment, the broader dimension brought by the scenarios exercise allowed confronting opinions on future strategic visions in the five cities (Marques et al., 2015).

Figure 2.

(a) Overall performance indexes of the three planning alternatives and spider diagram (using default weights for all indicators), with score 1 representing the baseline (alternative 1 = default case). The higher the score, the better the performance, and (b) scores of the three alternatives in each scenario (Source: Adapted from Chrysoulakis et al., 2013).

Key findings were extrapolated from the qualitative content analysis of the interview transcripts. They are summarized in Table 2 according to the questionnaire thematic cluster. They express the practitioners’ and scientists’ perceptions of the applicability of the BRIDGE UM assessment method/DSS tool and the effectiveness of the knowledge-transfer/CoP approach in BRIDGE. Key findings were derived from representative practitioners’ and scientists’ responses in each cluster, listed in Table 2 and Supplemental Table S5. They were interpreted in light of the specific context of the Finnish planning system and distinctive cultural aspects that may have influenced the development of the BRIDGE study in Helsinki (e.g. practitioners’ educational background, controversies on urbanization of Meri-Rastila forest, impacts of public consultation on Component Master Plan). Moreover, in line with the results of the BRIDGE self-evaluation of the CoP approach in all case-study cities (Klostermann et al., 2015), it should be mentioned that the practitioners’ perceptions of the DSS were influenced by the fact that the DSS version tested in the final BRIDGE seminar was not fully functional (see “Background” section). Therefore, it was essential to inquire whether the practitioners had subsequently made use of the full DSS version that was released at the end of the project. None of the practitioners, however, affirmed having used it in their practice.

Table 3.

Combined evaluation framework for the design of science–practice collaborative UM projects, illustrating the two main components of UM project design, and their first-, second-, and third-tier variables.

Components of UM project design	First-tier variables	Second-tier variables	Third-tier variables
Applicability of UM assessment method	1. UM concept and interpretation of sustainability	1.1. UM concept for scientists	1.1.1. Conceptual underpinnings based on specialist knowledge of disciplinary field
		1.2. UM concept for practitioners	1.2.1. Conceptual underpinnings based on educational background/previous experiences
		1.3. Relevance of UM approach for improving urban sustainability for scientists	1.3.1. Understanding of local sustainability issues/actionable strategies
			1.3.2. Understanding of limits of UM approach to address all aspects of urban planning
			1.3.3. Interest in integrating qualitative data (e.g. socioeconomic) in quantitative UM flows assessment
		1.4. Relevance of UM approach for improving urban sustainability for practitioners	1.4.1. Vision of local sustainability issues/actionable strategies
			1.4.2. Understanding of limits of UM approach to address all aspects of urban planning
			1.4.3. Need to integrate qualitative data (e.g. socioeconomic) in quantitative UM flows assessment
	2. Scientists’ versus Practitioners’ goals in achieving sustainability/sustainable UM	2.1. Added value of UM methods for scientists	2.1.1. Interest in developing transdisciplinary cooperation/applying holistic frameworks to better address existing research questions
			2.1.2. Interest in exploring new research questions emerging through transdisciplinary cooperation
			2.1.3. Interest in seeing proposed UM strategies implemented in planning process
		2.2. Planning authorities’ priorities in tackling local sustainability issues	2.2.1. Achievability of sustainability goals in planning timeframe (long/short term)
			2.2.2. Awareness of goals that can/cannot be achieved
			2.2.3. Synergies/tradeoffs between sustainability goals
			2.2.4. Links between sustainability goals and political agendas/orientations
	3. Actors involved in urban sustainability/planning policy development and implementation	3.1. Administrative structure of public sector	3.1.1. Competencies/tasks of involved planning departments/decision-making sectors (public works, parks, construction, transport, water, sewage, waste, etc.)
			3.1.2. Level of cooperation/communication among departments/sectors
			3.1.3. Barriers to cooperation/reasons for compartmentalization
		3.2. Engagement with private sector, socioeconomic stakeholders, and wider public (industry, NGOs, citizen associations, etc.)	3.2.1. Existing public–private partnerships and their nature
			3.2.2. Bottom-up versus top-down governance systems for resource-management (energy cooperatives, food co-ops, etc.)
			3.2.3. Property-right systems and impacts on resource-management/planning-policy implementation
	4. Required investment for local authorities	4.1. Equipment, infrastructure, training	4.1.1. Cost of equipment for data collection (purchase/maintenance)
			4.1.2. Cost of UM training/continuing education for staff
			4.1.3. Cost of non-open-source software required to use DSSs/other UM assessment tools
		4.2. Human resources	4.2.1. Practitioners’ time investment required to familiarize with DSSs/other UM assessment tools
		4.2. Human resources	4.2.2. Budget to hire new staff with relevant knowledge
	5. Limits of quantitative UM methods	5.1. Datasets	5.1.1. Availability
			5.1.2. Resolution
			5.1.3. Reliability of sources
		5.2. Selection of UM indicators	5.2.1. Socioeconomic indicators prioritized over environmental, or vice versa, to streamline analysis/results
		5.2. Selection of UM indicators	5.2.2. Indicator spectrum kept broad to test potential/benefits of UM frameworks
		5.3. Weighting systems	5.3.1. Bias due to end-users’ background knowledge/gaps
		5.3. Weighting systems	5.3.2. Bias due to end-users’ political orientation
		5.4. Long-term scenarios exercise	5.4.1. Limits of indicator-based evaluation when input data are not accurate
			5.4.2. Unforeseen technological innovation invalidating projections
			5.4.3. Change in decision-makers’ political orientation invalidating projections
Effectiveness of knowledge-transfer approach	6. Available knowledge and gaps	6.1. Scientists’ specialist knowledge	6.1.1. Disciplinary fields represented/required in UM study
			6.1.2. Attitude toward/experiences in transdisciplinary research and understanding of other disciplinary languages
			6.1.3. Limits of specialist knowledge and disciplinary bias
		6.2. Practitioners’ educational background and “know-how”/tacit knowledge acquired through experience	6.2.1. Knowledge gaps influencing decision on UM assessment method (quantitative/qualitative) and interpretation of environmental/socioeconomic data and assessment results
			6.2.2. Previous/current experiences that can compensate knowledge gaps
			6.2.3. Actions required to bridge knowledge gaps
	7. Knowledge/awareness of skills/competencies/expertise on other side of process	7.1. Scientists’ knowledge of planners’ work	7.1.1. Knowledge of local planning system/policy-making timeframe
			7.1.2. Knowledge of local administrative structure and competences/tasks of involved planning departments/decision-making sectors
			7.1.3. Knowledge of planners’ educational background
		7.2. Practitioners’ knowledge of scientists’ work	7.2.1. Knowledge of timeframes to assess sustainability performance/outcomes of planning practice
			7.2.2. Knowledge of research-process timeframes/constraints
			7.2.3. Knowledge of disciplinary languages/their differences
	8. Willingness to adapt to requirements of other side of process	8.1. Scientists’ adaptability	8.1.1. Willingness to adapt scientific assessment to involve planning levels/policy-making timeframe
		8.2. Practitioners/decision-makers’ adaptability	8.2.1. Willingness to adapt policy-making timeframes to research processes
			8.2.2. Willingness to adapt spatial scale of analysis
			8.2.3. Willingness to co-decide on planning levels to be involved in analysis
	9. Visions on role of each party and communication of knowledge transfer	9.1. Scientists’ visions of their role	9.1.1. Kicking-off/leading discussions based on previous research findings/experience
			9.1.2. Gathering feedback from practitioners at different project stages
			9.1.3. Implementing/facilitating two-way knowledge exchange to allow evidence-based planning and practice-oriented research
		9.2. Practitioners’ visions of their role	9.2.1. Using scientists’ inputs to understand/apply UM concept
			9.2.2. Using scientists’ findings to inform planning
			9.2.3. Sharing knowledge/competencies to co-produce findings
	10. Modes of knowledge transfer	10.1. Available models	10.1.1. Linear (one-way, science-to-practice)
			10.1.2. Functional (use of science to serve political interests; epistemic influence of science on political decisions)
			10.1.3. Cyclical (two-way: science-to-practice and practice-to-science, knowledge co-production)
		10.2. Types of knowledge transferred	10.2.1. Understanding of basic UM processes/dynamics
			10.2.2. Practitioners’ know-how/tacit knowledge on local conditions/issues
			10.2.3. Alternative methodologies to account for UM flows/visualization tools
			10.2.4. Datasets
			10.2.5. Application-ready tools
	11. Timing of process	11.1. Duration	11.1.1. Project timeframe and scientists’ time availability
		11.1. Duration	11.1.2. Policy-making timeframe and practitioners’ time availability
		11.2. Coordination/synchronization with other project WPs	11.2.1. Project start/end point and/or overlap with selected WPs
		11.2. Coordination/synchronization with other project WPs	11.2.2. Development throughout project/in parallel to all WPs

DSS: decision support system; UM: urban metabolism; WP: Work Package.

In general, all of the practitioners showed a high level of commitment throughout the interviewing process, providing extensive responses to all questions. They all acknowledged the relevance of a retrospective evaluation of the BRIDGE approach a few years after the end of the project, especially in terms of its applicability in their own planning practice. The scientists provided, on average, shorter responses than the practitioners and manifested a more moderate interest in engaging with the evaluation. At the time the interviews were conducted, they were all involved in other research projects (in some cases on UM-related topics) and some felt that BRIDGE was already “behind them.”

Discussion

UM assessment method

Generalization of the key findings from both scientists’ and practitioners’ responses pointed to the following main drivers of effective UM assessment methods: (i) clear connection between scientific research and problem-based applied approaches, in order to respond to local societal needs and practical aspects of urban planning; (ii) improved understanding of UM aspects by practitioners with no scientific background through training or regular exchange/face-to-face meetings with scientists; (iii) development of a common language across science and practice, accommodating priorities and concerns from both parties; (iv) preliminary consultation to make sure visions and objectives are shared between parties.

Two main areas of improvement were identified more specifically by practitioners for the BRIDGE-DSS and its applicability in the Meri-Rastila Länsiranta Component Master Plan: the strategic scenarios exercise and the weighting system. Beyond the observation that the DSS was not fully functional at the time the practitioners tested it (see “Results” section), it should be noted that these two components of the DSS were at the core of both local and umbrella CoP meetings (Klostermann et al., 2015). Hence, it is fundamental to analyze areas of improvement for the UM assessment method in parallel to the interviewees’ perceptions of the knowledge-transfer approach.

Knowledge transfer

Both scientists and practitioners expressed a strong interest in the BRIDGE CoP approach. Concerns were, however, raised on the limited time that both groups could invest in the CoP meetings. It is, therefore, crucial to design knowledge-transfer processes that are compatible with the time availability of both sides. Beyond actual availability of participants, previous knowledge-transfer studies report that knowledge sharing in science–practice collaborative research are often perceived as an additional burden, and can lead to fatigue and disengagement on both sides, especially when the added value and real-world impact of the process are unclear (Ugolini et al., 2015). Therefore, discussion on the expected impacts of UM projects in real-world practice (considering expectations on both sides of the knowledge transfer) and identification of the participant roles in achieving project goals are essential from the early stages of the project. Practitioners also expressed a strong need for transdisciplinary collaboration with scientists to be able to familiarize themselves with UM assessments due to the complexity of the UM framework and broad spectrum of technical competencies required. Beyond the Helsinki case, this can be explained by the increasing compartmentalization of public service departments and outsourcing of expertise, as observed in other studies (Ugolini et al., 2018).

The selection of a set of relevant indicators for Meri-Rastila proved to be the most intense aspect of the CoP meetings. Longer and more regular meetings would have been essential to engage properly in a knowledge co-production process focusing on such a broad spectrum of UM socioeconomic and environmental indicators. The communication of UM assessment results/delivery of DSS to end-users in early stages of the planning process was highlighted as a crucial factor that could have favored applicability in planning. In line with Binder et al. (2015) and Lang et al. (2012), this points to the significance of proper planning and scheduling the timing of the knowledge transfer, in terms of both harmonization with policy-making timeframes and synchronization with other steps/Work Packages of collaborative UM projects.

Combined evaluation framework

Building on the generalization of key findings from the Helsinki case, it is possible to outline an evaluation framework that can facilitate science–practice collaborative UM projects for effectively informing urban planning (Table 3). The framework is based on a three-tier model. The variables in each tier provide recommendations on aspects that scientists and practitioners should focus on in UM project design. The framework can support decision-making on both the UM assessment method (e.g. DSS tools) and knowledge-transfer approach (e.g. CoPs). The framework’s philosophy is that these two components of UM project design should be considered in a synergistic fashion. The variables of each component can be evaluated by themselves as well as in terms of their potential impacts (or tradeoffs) on variables of the other component. An example of cross-reading of variables in the two components is provided in Figure 3. The framework is grounded only on findings from the Helsinki research, and, therefore, does not address all aspects of collaborative UM projects (e.g. only limits of quantitative UM methods are listed). Hence the framework should be conceived as open-ended. Variables at all levels can be combined and/or further disaggregated based on results of future research, and new variables can be added if required, according to specific conditions and goals of UM projects.

Figure 3.

Cross-reading of variables in the two components of the framework: interdependency and feedback paths occurring between the first-tier variables “1. UM concept and interpretation of sustainability” (UM assessment method) and “6. Available knowledge and gaps” (knowledge transfer) through their respective second- and third-tier variables.

Conclusion

Investigation into applicability of UM assessments in urban planning and effectiveness of science–practice knowledge transfer is essential to foster more practice-relevant UM research. Therefore, these aspects should be addressed as two essential and intertwined criteria in the design of science–practice collaborative UM projects. Learning from retrospective evaluation of the BRIDGE Helsinki case study, this paper has demonstrated that UM assessment applicability and knowledge-transfer effectiveness can be evaluated in a synergistic fashion by taking stock of previous collaborative UM initiatives. The proposed combined evaluation framework provides recommendations on variables of both aspects that should be considered at the outset of collaborative projects and can help identify and assess areas of improvement for the two aspects in parallel. Based on the findings of the Helsinki study, these include, among others: acknowledgement of limits and potential of both scientists’ specialist knowledge and practitioners’ backgrounds/tacit knowledge, consideration of budget and time availability of both sides, proper embedding of knowledge transfer in the research timeframe, as well as the use of high-quality input data for long-term scenario assessments. Ultimately, the framework illustrates a way forward to catalyze discussion on shared visions and understanding of sustainable transition pathways through the use of the UM holistic concept.

Supplemental Material

Supplemental material for Evaluating urban metabolism assessment methods and knowledge transfer between scientists and practitioners: A combined framework for supporting practice-relevant research

Supplemental material for Evaluating urban metabolism assessment methods and knowledge transfer between scientists and practitioners: A combined framework for supporting practice-relevant research by Daniela Perrotti in EPB: Urban Analytics and City Science

Footnotes

Acknowledgements

The author is grateful to all planning professionals at the Helsinki City Planning Department and scientists at the University of Helsinki for their participation in the interviews, as well as to the anonymous reviewers of the journal for their insightful comments and thoughtful suggestions on the manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Koneen Säätiö (Kone Foundation), Finland, through a Researcher Grant awarded to the author (grant number 4eaf87).

Supplemental material

Supplemental material for this article is available online.

Daniela Perrotti is Professor of Landscape Architecture at Catholic University of Louvain, Faculty of Architecture, Architectural Engineering, and Urbanism. Her research interests cover applications of urban metabolism models in urban planning and design, with a focus on green infrastructure strategies to mitigate energy demand and the resource-intensity of cities. Daniela serves/has served as Principal Investigator and Co-Investigator for several research projects on integrated urban metabolism modeling approaches, in collaboration with local authorities, and academic and industry partners in Europe, China, and South America. Daniela has published over 30 peer-reviewed papers in international journals and books.

References

Beloin-Saint-Pierre

Rugani

Lasvaux

, et al. (2017) A review of urban metabolism studies to identify key methodological choices for future harmonization and implementation. Journal of Cleaner Production 163: S223–S240.

Binder

Absenger-Helmli

Schilling

(2015) The reality of transdisciplinarity: A framework-based self-reflection from science and practice leaders. Sustainability Science 10: 545–562.

Böcher

Krott

(2014) The RIU model as an analytical framework for scientific knowledge transfer: The case of the ‘decision support system forest and climate change’. Biodiversity Conservation 23: 3641–3656.

Castro

Marques

(2015) Combining environmental and socio-economic data. In: Chrysoulakis

de Castro

Moors

(eds) Understanding Urban Metabolism. A Tool for Urban Planning. New York: Routledge, pp.153–160.

Chrysoulakis

(2015) The BRIDGE approach. In: Chrysoulakis

de Castro

Moors

(eds) Understanding Urban Metabolism. A Tool for Urban Planning. New York: Routledge, pp.18–26.

Chrysoulakis

Lopez

San

Josè R

, et al. (2013) Sustainable urban metabolism as a link between bio-physical sciences and urban planning: The BRIDGE project. Landscape and Urban Planning 112: 100–117.

Creswell

(2014) Research Design. Qualitative, Quantitative and Mixed Methods Approaches. Thousand Oaks, CA: Sage.

Cui

(2018) How can cities support sustainability: A bibliometric analysis of urban metabolism. Ecological Indicators 93: 704–717.

Dijst

Worrell

Bocker

, et al. (2018) Exploring urban metabolism. Towards an interdisciplinary perspective. Resources, Conservation & Recycling 132: 190–203.

10.

Galan

Perrotti

(2019) Incorporating metabolic thinking into regional planning: The case of the Sierra Calderona Strategic Plan. Urban Planning 4(1): 152–171.

11.

Golubiewski

(2012) Is there a metabolism of an urban ecosystem? An ecological critique. Ambio 41(7): 751–764.

12.

González

Donnelly

Jones

, et al. (2013) A decision-support system for sustainable urban metabolism in Europe. Environmental Impact Assessment Review 38: 109–119.

13.

Huang

Cui

Yarime

, et al. (2015) Improving urban metabolism study for sustainable urban transformation. Environmental Technology & Innovation 4: 62–72.

14.

Kennedy

Cuddihy

Engel-Yan

(2007) The changing metabolism of cities. Journal of Industrial Ecology 11(2): 43–59.

15.

Kennedy

Pincetl

Bunje

(2011) The study of urban metabolism and its applications to urban planning and design. Environmental Pollution 159(8–9): 1965–1973.

16.

Klostermann

JEM

Groot

González

(2015) The use of communities of practice to involve stakeholders in the decision support system design. In: Chrysoulakis

de Castro

Moors

(eds) Understanding Urban Metabolism. A Tool for Urban Planning. New York: Routledge, pp.131–140.

17.

Lang

Wiek

Bergmann

, et al. (2012) Transdisciplinary research in sustainability science: Practice, principles, and challenges. Sustainability Science 7(1): 25–43.

18.

Kwan

(2018) Advancing analytical methods for urban metabolism studies. Resources, Conservation & Recycling 132: 239–245.

19.

Marques M, Castro E and Groot A (2015) Decision making under uncertainty. Use of foresight for assessing planning alternatives. In: Chrysoulakis N, de Castro E and Moors E (eds) Understanding Urban Metabolism. A Tool for Urban Planning. New York: Routledge, pp. 185–196.

20.

Mauser

Klepper

Rice

, et al. (2013) Transdisciplinary global change research: The co-creation of knowledge for sustainability. Current Opinion in Environmental Sustainability 5: 420–431.

21.

Miles

Huberman

(1994) Qualitative Data Analysis. Thousand Oaks, CA: Sage.

22.

Mitraka

Diamantakis

Chrysoulakis

, et al. (2014) Incorporating bio-physical sciences into a decision support tool for sustainable urban planning. Sustainability 6: 7982–8006.

23.

Newell

Cousins

(2014) The boundaries of urban metabolism: Towards a political–industrial ecology. Progress in Human Geography 39(6): 702–728.

24.

Nikinmaa

Vesala

(2010) Sustainable urban planning in Helsinki. In: Minutes of the 2nd CoP Meeting BRIDGE, Helsinki, 20 January 2010. Available at: http://www.bridge-fp7.eu/ (accessed 8 January 2019).

25.

Perrotti

Stremke

(2018) Can urban metabolism models advance green infrastructure planning? Insights from ecosystem services research. Environment & Planning B. Epub ahead of print 10 September 2018. DOI: 10.1177/2399808318797131.

26.

Pincetl

Graham

Murphy

, et al. (2015) Analysis of high-resolution utility data for understanding energy use in urban systems. The case of Los Angeles, California. Journal of Industrial Ecology 20(1): 166–178.

27.

Schremmer

, et al. (2011) Planning resource-efficient cities. The SUME synthesis report. 31 October. Available at: https://www.sume.at/project_downloads (accessed 8 January 2019).

28.

Ugolini

Massetti

Sanesi

, et al. (2015) Knowledge transfer between stakeholders in the field of urban forestry and green infrastructure: Results of a European survey. Land Use Policy 49: 365–381.

29.

Ugolini

Sanesi

Steidle

, et al. (2018) Speaking “green”: A worldwide survey on collaboration among stakeholders in urban park design and management. Forests 9: 458.

30.

Vilkama

(2011) City of Helsinki urban facts. Research Series 2011: 2.

31.

Xia

Zhang

Sun

, et al. (2017) Analyzing the spatial pattern of carbon metabolism and its response to change of urban form. Ecological Modelling 355: 105–115.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

23.20 MB