Abstract
OBJECTIVE:
The main purpose of the study was to investigate the usability of a newly developed instrument designed to assess the content and quality briefs (CQB-I) in programs for new healthcare environments. We studied the perception of using the instrument on a group of space planners.
BACKGROUND:
The study is part of a larger project designed to ensure the quality of the planning of new healthcare environments with a focus on documents (programs or briefs) created in the early phases of the planning process. In this study, we used an instrument that measures the clarity of the mission statement for the project based on user needs and care activities that will take place in the new healthcare environment. The instrument further evaluates whether there are clear patient-related outcome measures specified and whether the information in the documents is person-oriented and evidence-based.
METHODS:
The study used a mixed-method design where the relevance and usability of the instrument was estimated and a focus group interview was conducted.
RESULTS:
The study showed that a CQB-I is perceived to be a relevant instrument primarily as a guide for programming, but also for monitoring the produced programs.
CONCLUSIONS:
Instruments like CQB-I can help to bring research results into planning and contribute to dialogue and collaboration in the early phases of a planning process by orienting planning participants to user needs and facilitating the discussion and articulation of clear performance indicators.
Decisions about the design of new healthcare environments are extremely important because such decisions can influence people and work processes several years into the future and often require large financial commitments (Rybkowski, 2009). The implementation of new methods and models of care to achieve effective, safe, and person-centered care is largely dependent on the layout of the space (Henriksen, Isaacson, Sadler, & Zimring, 2007). The most critical decisions in the design of a new healthcare environment are made during the earliest stage of the planning process, that is, the pre-design phase (P-DP), before a detailed layout is presented.
The aim of the planning process is to create a healthcare environment that contributes to patient health and well being and that allows for high quality and safe care (Gesler, Bell, Curtis, Hubbard, & Francis, 2004; Huisman, Morales, van Hoof, & Kort, 2012). Healthcare environments must also be designed so that patients, families, and healthcare professionals feel welcome, visible, and included. Balancing and satisfying these different requirements can be challenging. P-DP for a new healthcare environment thus involves a thorough analysis of the project's patient-related objectives and the processes (care activities) and space conditions that are necessary to achieve these objectives (Curtis, Gesler, Priebe, & Francis, 2009; Steinke, Webster, & Fontaine, 2010; Ulrich, Berry, Quan, & Parish, 2010; Vischer & Zeisel, 2008). This approach illuminates the need for the analysis to be a shared responsibility among the various stakeholders, including those involved in the healthcare, architecture, planning, and construction facets of the project.
The most critical decisions in the design of a new healthcare environment are made during the earliest stage of the planning process, that is, the pre-design phase (P-DP), before a detailed layout is presented.
The P-DP is often led by a space planner (programmer) with support from the real estate organization. During this stage, an initial outline brief (or program) is written prior to the presentation of a detailed layout proposal. This outline brief is a key document for decision making in the P-DP process (Szigeti & Davis, 2002) and is considered essential for high quality construction projects; it should inform and control the project from the planning to construction phases (Ryd & Fristedt, 2007). Therefore, this document must be systematic, person-oriented (based on user requirements), and evidence-based (Preiser & Vischer, 2005; Vischer & Zeisel, 2008).
In this study, we explored the perceptions of a group of space planners who were using a new instrument designed to assess the content and quality of initial briefs created in P-DP for new healthcare environments. This study was part of a larger project designed to ensure the quality of the P-DP of new healthcare environments, with a focus on the outline brief created during P-DP.
We pre-tested an instrument called the Content and Quality in Briefs Instrument (CQB-I; See Table 1) to determine whether a project has a clear mission statement based on the needs of the users and the care activities that will occur in the new healthcare environment (Elf, Svedbo Engström, & Wijk, 2011). Furthermore, this instrument can be used to determine whether clear, patient-related outcome measures have been specified and whether the information in the documents is person oriented and evidence based. The CQB-I was developed through a deductive and iterative process. The items were generated through an extensive literature review and an expert evaluation on what should be included in briefs. The development of the instrument has been reported earlier by Elf, Svedbo Engström, and Wijk (2011).
Content and Quality in Briefs Instrument (CQB-I) and the Assessed Relevance Measured Using a Content Validity Index (CVI) (n = 9)
This instrument has been validated by experts (architects, space planners, and researchers) (Elf, Svedbo Engström, & Wijk, 2011, 2012); however, space planners' perceptions of the instrument's usability have not been studied. Therefore, we wanted to explore and pre-test the usability of the instrument among space planners with daily involvement in planning of new healthcare environments.
Background
The creation of the CQB-I arose from the expressed need to develop methods and processes for quality control in the early planning process of a new healthcare environment before a detailed layout is presented (Ryd & Fristedt, 2007; Szigeti & Davis, 2002). An effective and quality-driven planning process has been described as essential for achieving a good result—one that is functional, sustainable, meets the needs of the individuals in the environment, and is cost-effective over the long term (Ulrich et al., 2008). Research has revealed deficiencies in the quality of the planning process in several areas: combining knowledge from healthcare and architecture (evidence-based design), focusing on the needs of users (personal orientation) and clarifying patient-related outcome measures (PROMs) for later evaluations of the building (Preiser, 1995; Ulrich et al, 2008; Vischer, 2009). For example, in Sweden the National Board of Health, criteria for good healthcare—knowledge-derived from health research—can be transferred to the premises planning of new care environments to contribute to person-oriented, safe, and effective care (Henriksen, Isaacson, Sadler, & Zimring, 2007).
The Need to Assess Briefs
There is a need to assess the content and quality of briefs as part of the continuous quality improvement process (Vischer & Zeisel, 2008). Research in architecture, planning, and construction has demonstrated the importance of well-developed briefs that are user-oriented, evidence-based, and related to the strategic plan of the healthcare organization (Barrett & Balderly, 2003; Barrett, Hudson, & Stanley, 1999; Blyth & Worthington, 2001; Preiser & Vischer, 2005; Vischer & Zeisel, 2008). In general, the brief should include explicit information about the project objectives and specifications (functionality and the ability to support the activities to be performed in the new environment) based on end-user requirements and needs (Barrett & Stanley, 1999; Blyth & Worthington, 2001; Kelly & Hunter, 2005; Preiser & Vischer, 2005; Ryd & Fristedt, 2007). It is important that the initial brief relates to the strategic brief of the organization and describes how the design of the building will contribute to the organization's stated strategic goals (Kelly & Hunter, 2005; Blyth & Worthington, 2001). In addition, there is an increasing requirement that planning and written briefs include measurable outcomes and evidence-based information (Vischer, 2008). High quality briefs contain explicit outcomes that can serve as tools for evaluating the building when in use (post-occupancy evaluation) and are a natural component of an evidence-based approach (Baird, 2001).
An effective and quality-driven planning process has been described as essential for achieving a good result—one that is functional, sustainable, meets the needs of the individuals in the environment, and is cost-effective over the long term.
Despite numerous recommendations on what a brief should contain, there is a dearth of studies that have actually studied the content and quality of brief (Elf, Svedbo Engström, & Wijk, 2012). Elf and Malmqvist (2009) conducted a review of the information contained in briefs created during the planning process for healthcare environments. The results showed that only a few briefs incorporated explicit patient-focused goals for the project (2 out of 21 briefs), measurable patient outcomes (2 of 21), or references to new evidence (10 of 21). Less than half of the audited briefs contained a clear description of the objectives of the organization and the activities that were to occur in the planned healthcare environment. Furthermore, the results showed that the team perspective was frequently omitted, and the care process was often described solely from a medical perspective. This review was performed using a qualitative approach (content analysis) due to the absence of an objective evaluating instrument. Similar results were found in a recent nationwide study of briefs in Sweden (Elf, Svedbo Engström, & Wijk, 2012).
Briefs as Part of Evidence-Based Design
An appropriate brief provides a basis for an evidence-based design (EBD) process (Preiser & Vischer, 2005; Steinke, Webster, & Fontaine, 2010). EBD can be defined as a critical and reflective process in which decisions about the design of the environment are based on research, experience, and the user's experience of the building (Stankos & Schwartz, 2007; Vischer, 2009; Ulrich, Berry, Quan, & Parish, 2010). EBD is closely related to continuous quality improvement models, in which objectives within the healthcare environment are based on the best available research, knowledge, and experience and in which explicit evaluation measures are presented at the beginning of a project. The results of these evaluations are used as experience and input in the subsequent planning processes.
Presentation of the Instrument
The CQB-I can be used to determine whether the project includes a clear mission statement based on user needs, a description of the care activities that will occur in the new healthcare environment, clear patient-related outcome measures, and evidence-based information (Table 2). Furthermore, the instrument design incorporates a systems thinking approach, that is, it is important to first analyze the mission of the care and the types of activities that will occur in the environment and then to apply the results of this analysis when determining what types of facilities are required. The instrument includes a manual to support this assessment (Table 2).
Example of a Manual to Support Assessments with the CQB-I
Methods
A mixed-method design (Morse & Niehaus, 2007) was used to explore the perspectives of the space planners regarding the CQB-I and its usability. Mixed-methods research is defined as research in which the investigator collects and analyzes data, integrates the findings, and draws inferences using both qualitative and quantitative approaches and methods in a single study or a brief of inquiry (Creswell & Tashakkori, 2007).
Data were collected using observations and field notes, in addition to a questionnaire concerning the participants' perspectives on the relevance of the instrument and the accompanying manual by using Content Validity Index (CVI) (Polit & Beck, 2006). To obtain further information about the usability of the CQB-I, a focus group interview with space planners was conducted and videotaped.
Participants and Selection
The participants (n = 9) all had a position as a space planner, with backgrounds in nursing (n = 8) or engineering (n = 1), and were affiliated with nine different planning and construction organizations in Sweden. The participants formed a convenience sample (they were chosen because they were gathered for a space planner meeting). Informed consent was obtained before the study began.
Data Collection and Procedures
The data collection occurred at a regular meeting for planners of new healthcare environments. The participants were asked to individually review a pre-selected brief using the CQB-I and to then assess the relevance of the instrument and the related manual.
A data collector familiar with the CQB-I was present, observed the process and took field notes. The participants were able to ask questions throughout the review process. In particular, the data collector noted the length of time required to review the brief and whether the participants showed any uncertainty or hesitation about any of the questions in the CQB-I or the associated manual.
Outcomes and Instruments
The perceptions of the planners regarding the relevance of the instrument were assessed using a content validity index (CVI) (Polit & Beck, 2006), a method of assessing content validity.
Assessment of the Relevance of the Instrument
CVI is a guide for further analysis of the relevance of items and can be used on small groups (5–10 people are recommended). The important thing is that the group reflects experts in the field. The relevance of the entire instrument (Scale Content Validity Index, S-CVI) and of each item in the instrument (I-CVI) was ascertained. The participants were asked to individually specify the relevance of each sub-question on the instrument using a 4-point scale, where 0 = “not relevant,” 1 = “somewhat relevant,” 2 = “relevant,” and 3 = “very relevant.” In addition, the participants were asked to indicate the reasons for their judgments and were invited to provide comments, to suggest revisions to improve the instrument's content and clarity, and to note any questions that should be added.
Assessment of the Relevance of the Associated Manual
The manual that accompanies the CQB-I was also assessed using a questionnaire specifically developed for the assessment of manuals (Polit & Beck, 2006). The participants were asked to assess the extent to which the manual supported the evaluation of the brief by choosing between several response options. The scoring involved a 3-point scale, where 0 = “not at all,” 1 = “to a certain extent, but the difference between the response alternatives is unclear,” and 2 = “to a great extent; there is a clear difference between the response alternatives.” In addition, the participants were asked to provide any further explanations of their ratings, any comments and suggestions for revision, and how the differences between the alternatives might be made clearer.
Planners' Perceptions Regarding the Instrument
After the participants had tested the CQB-I, a semi-structured focus group interview using an interview guide with open-ended questions was conducted to assess the usefulness of the CQB-I. The questions were:
How and in what way do you think the CQB-I could be useful for your work? Are there any important components missing from the CQB-I?
The interview lasted for approximately 1 hour; it was videotaped and transcribed verbatim.
Data Analysis
The focus group interview and the field note data were analyzed according to the principles of qualitative content analysis with a manifest approach (Graneheim & Lundman, 2004). The quantitative data were analyzed using descriptive statistics.
The content analysis was performed in three stages by two independent researchers. An initial reading of the text was performed to obtain an overview of the content. Subsequently, the text was systematically read to identify units of greater meaning with common content. The next step was to systematize the data into a number of descriptive categories.
Analysis of the Relevance of the CQB-I
The relevance of the entire instrument (S-CVI) was calculated by adding the mean values for all of the questions and dividing the sum by the number of questions. The relevance of each item (I-CVI) (presumably, quite relevant or very relevant) was calculated by dividing the user assessment by the total number of participants (Polit & Beck, 2006). The limits were ≥ 0.78 for I-CVI and ≥ 0.90 for S-CVI.
Results
The analysis of the focus group interview results revealed three main categories:
Usability and relevance of the instrument, Limitations of the instrument, and Areas for improvement.
The findings from the focus group interview are presented together with the results from the questionnaires.
Usability and Relevance of the CQB-I
The participants expressed that the CQB-I could be relevant and valuable for their work. The instrument as a whole and the individual items were generally considered to be relevant (See Table 1, above). The planners indicated that the instrument could be applied as an assessment tool of the content and quality of the briefs but also as a guide in the planning process. They stated that the instrument could be especially valuable for highlighting the patient's perspective in a P-DP and the associated brief, as well as for helping to provide the stakeholders in the healthcare organization with valuable questions important to consider in a P-DP. Participants also expressed that the instrument could be an especially beneficial resource for a new planner gaining familiarity with a new role. They noted that the instrument could provide a basis for achieving a national consensus on the content in briefs for healthcare environments. The participants emphasized that the instrument could serve as a proactive framework used to prevent important issues in the process from being ignored, for example, by elucidating the healthcare environment objectives from the perspective of patients and their relatives. According to the space planners, healthcare organizations rarely consider the patient-based perspective when planning a new healthcare environment—as one participant expressed, “There is something else that causes a change, and then it is not the patient-based perspective that comes first; it is in the background.” Other concerns, such as the needs of professionals, infection control requirements, or the prevention of overcrowded units, typically dominate the healthcare environment planning discussion instead.
The participants considered that a self-audit of briefs (and, indirectly, of their planning process) would be key to ensuring important aspects of the process are not neglected and to maintaining focus on the patient-based perspective:
The instrument could be applied as an assessment tool of the content and quality of the briefs but also as a guide in the planning process.
As a checklist, when looking through a brief, we look at the perspectives of the patients and relatives, if available; to ensure that everything is the way it's supposed to be to help the business. We say that now you will have to provide figures, and you have to get to these parts so that all parts are included.
[The instrument] can be a great way to increase the quality of the briefs we develop. There may be a way to put a little more focus on the business and that they need to work more with their business, organization, etc.
“There is something else that causes a change, and then it is not the patient-based perspective that comes first; it is in the background.”
The planners scored several questions in the instrument to be less relevant, such as questions 3, 12, 13, and 14 (See Table 1, above). At the focus group interview, several of the group's members questioned whether the documents should include a description of the organization and the activities planned for in the environment. They considered that it was better and easier to refer to the strategic plan of the organization than mention this in the briefs. In addition, some planners expressed that the question concerning the quality indicators of good healthcare was irrelevant, and others questioned why the instrument included that item at all. The results were confirmed by estimating the relevance of the items using CVI (See Table 1, above).
Limitations of the Instrument
Although the group considered the instrument useful, they also noted several limitations, such as the risk that checklists may not be completed and used in their daily work. The group expressed workload comments, including, “You definitely need checklists for self-monitoring, but they are often not used.”
“You definitely need checklists for self-monitoring, but they are often not used.”
Additionally, it was mentioned that it is important to consider what type of brief is being evaluated, that is, at what point in the process is the evaluation being written and what is its purpose. The brief became increasingly detailed as the process continued, and the audit instrument then also needed more detailed questions:
Briefs must be examined from multiple perspectives because different professionals in the planning process focus on different details in a construction project. For example, healthcare professionals should perhaps study how health hygiene is considered in a brief; engineers may review briefs from a technology perspective.
The Practical Usability
The results from the field notes indicated that the participants did not experience difficulty assessing the briefs with the help of the CQB-I. The participants showed no uncertainty or disagreement over the use of the instrument and had no concerns during the assessment. Additionally, the majority of the participants used all questions in the instrument. The duration of the assessment was approximately 20 minutes. The subsequent focus group interview revealed a great commitment to the results and the group expressed that the CQB-I was easy to use, “just to look if the issues were or were not included.”
The participants reported that the manual provided support for using the instrument. Most of the questions in the manual were considered largely or very supportive.
Improvement Areas
Generally, the planners agreed that the instrument fulfilled its purpose, could contribute to the standardization of briefs, and could increase the overall quality of briefs, but that it lacked questions related to cost-benefit analyses and the impact on cost of various design proposals. Comments included, “It's all about money in the end,” and “Health economics is not just about saving money; it is also about using money in the right way.”
The test situation led to reflections on other types of audit instruments for various briefs:
You could compare various audit documents written from different perspectives if you have an audit instrument that you can quickly check that has questions that are absolutely important if you are a relative, or an instrument with the issues that are most important to patients, employees, technology, policy makers, or economists.
Decision makers need information on finances, goals, and benefits … they cannot make decisions without looking at them from the owner's perspective. If the brief is reasonable in terms of economics and technology, it will be as efficient as we have imagined.
There were also suggestions that brief must consider more “soft” dimensions in terms of interior design, environment, colors, and the atmosphere encountered upon entering a room—these issues should be addressed with an assessment instrument. The participants also expressed that the instrument failed to consider issues of technology solutions in a healthcare environment. Finally, the participants noted that the use of the instrument would be enhanced by its integration of a computer platform.
Discussion
This study showed that the CQB-I was perceived to be a relevant instrument primarily as a guide for P-DP but also for assessing the content of written briefs. The instrument was originally developed for a Swedish context but can be translated and adopted for international contexts. The study was small and had a limited selection of participants; therefore, the results cannot be generalized. By focusing on prospective space planners, that is, users of such instrument and their perceptions of the instrument, this study nevertheless provides valuable guidance for the further development of the CQB-I and similar instruments. Instrument development is a lengthy, iterative process (Polit & Beck, 2006) that requires several validity and reliability tests before the instrument can be used in practice; usability is a key consideration.
The CQB-I was designed primarily for assessing the content and quality written briefs created in the P-DP of new healthcare environments for a Swedish context, when the goal is to determine the needs and requirements and should be addressed in the briefs. That the participants thought that the instrument could act as a guide for the space planner or team planning for new healthcare environments was unexpected. The original idea behind CQB-I is that the instrument enables an assessment of whether the planning team has analyzed and formulated the goals and needs of the new healthcare environment based on user needs and evidence-based knowledge, which have been identified as essential for a planning process to achieve good results (Becker & Parson, 2007; Vischer, 2009; Yu, Shen, Kelly, & Hunter, 2007). The use of an instrument such as the CQB-I for briefs can guide discussions of the content and quality, and thus the planning process, toward a more user-oriented and knowledge-based process. There is no equivalent tool for the assessment of briefs, which suggests that such studies have not been the focus in the past.
The planners did not consider that a brief should include a description of the healthcare organization; instead, they thought that a reference to existing documents would be sufficient. Their argument was that briefs at this stage are overall descriptions and should only contain relevant information. While it is true that a brief should be short and concise, relevant information must be included. References to documents can be included and are valuable. However, if the descriptions of the organization and activities are not up to date or easily accessible, for example, problems can result, and therefore a warning flag should be raised. Planning for a new healthcare environment involves being a part of the strategic development of healthcare, and the goals and activities of the organization must be analyzed when planning the design of spaces (Becker & Parson, 2007; Blyth & Worthington, 2001). However, we need to explore the value of including a description of the organization in the briefs further on.
It was interesting that the planners reported that they often had to introduce the patient-based perspective to the planning process, although they considered this task to be the primary responsibility of the healthcare organization. The group stressed, however, that it was the responsibility of the planners to summarize the discussion and gather any information necessary for the continued planning and design process. Therefore, the planners need information about the patient group and the activities that will occur in the building. The group also stated that key issues discussed in the planning process were not always documented, which might have negative or even dangerous consequences for the entire construction process. Therefore, it might be beneficial to have a checklist to ensure that the key issues were addressed during the process.
The planners did not consider the quality indicators of good health worthy of mention in a brief. However, we emphasize that these indicators should be a basis for P-DP. These indicators were developed from research in healthcare, and their implementation in local planning is necessary for the process to be evidence-based. This finding may indicate that the space planners were not familiar with the indicators when the study was performed and therefore did not consider the indicators relevant. The issue of indicators arose during the subsequent focus group interview, and the participants changed their position slightly upon understanding the reasoning behind the indicators.
The group also discussed the difficulties of using a single audit tool for different stages of the planning process. Various real estate organizations design the planning and design process differently in terms of who writes the brief (a member of the healthcare organization or a planner employed by the real estate organization), which people are involved in the process, how the process is designed, the decision level addressed by the brief and the recipients of the brief. Although the group's concerns are relevant, the CQB-I is generic and indicates important areas to be included in the brief for patient healthcare spaces. The participants also remarked that there was no defined name for a brief produced in the early stages of the planning process. These documents are variously called “business descriptions for space designs,” “requirements analyses,” “feasibility studies,” and “functional briefs.” Thus, the names and contents of the briefs can vary. However, several studies have demonstrated that although briefs in the early stages have different names, their purposes and contents are relatively similar (Ryd, 2001).
Briefing in the early stages is a complex process, and one must always ask whether it is being performed efficiently. Research has shown that ensuring the quality of the early stages of the planning process is cost effective (Ulrich, Berry, Quan, & Parish, 2010). In addition to costs and benefits, society is increasingly focused on the importance of the implementation of evidence-based knowledge in healthcare, with a particular focus on factors that support patient-centered care. An instrument such as the CQB-I can be used to evaluate and develop briefs, thus making it more effective. The CQB-I may contribute to the continuous improvement process of briefs to identify areas in need of development. Assessing briefs with a valid instrument can make deficiencies in the process more visible. Briefs can be compared over time and between places and situations. The instrument clarifies the principles of quality management and an evidence-based approach to planning by enabling an assessment of the documents that result directly and indirectly from the process.
Conclusion
In summary, this study showed that the CQB-I satisfied its purpose and was easy to use. It is well known that it is difficult to transfer research results into healthcare services and into briefings. The CQB-I is an instrument that can help to bring research results into the planning process and contribute to discussions and collaboration among the stakeholders during the early stages. This instrument includes a user-oriented approach, enables clear discussions and documentation of outcomes, and consequently supports quality improvement in the planning process of new healthcare environments. However, this instrument should be developed and tested in larger groups.
Implications for Practice
There is a need to assess the content and quality of briefs as part of the continuous quality improvement process.
An instrument such as the CQB-I can be used to evaluate and develop briefs, by identifying areas in need of development. Assessing briefs with a valid instrument can make deficiencies in the planning process more visible.
Instruments like CQB-I can help to bring research results into planning and contribute to dialogue and collaboration in the planning process by orienting to the user needs, facilitating discussion and articulation of clear performance and related quality improvement indicators.
Footnotes
ACKNOWLEDGMENTS:
The study was partly funded by grants from SKL and Högskolan Dalarna. Thanks to Maria Svedbo Engström, PhD, student, School of Health and Social Studies, Falun, Sweden, for data collection.
