9. Assessments: Structure,concepts,and methods

Abstract

Assessments are an essential element of proper disaster management. Assessments help to define the damage and changes in functions at the time of the assessment. Assessments are transectional across the longitudinal phases of the disaster. Any intervention should be preceded by an assessment(s). The assessment process is deconstructed into a series of 10 steps: (1) need to know; (2) define the goal(s) and objectives(s) of an assessment; (3) select the appropriate indicators; (4) define the methods to be used for the assessment; (5) develop and test a plan for data collection; (6) train and brief data collectors; (7) gather (collect) the data; (8) synthesise the data and information collected; (9) output information for decision-making; and (10) compare findings with overarching goal and objectives. Steps 7–9 constitute a production process. Understanding this process is essential for identification of points of success and failure in achieving the desired assessment.

Assessments require careful selection of indicators. The selected indicators are used throughout the process. Currently, no standardised set of indicators has been validated. Criteria for the composition of assessment teams are provided and common sources of error are discussed. Prior to, during, and following disasters, assessments are directed by the appropriate coordination and control entity.

Keywords

Assessment basic societal functions benefits concepts data data collection demands effects errors goals impact indicators information methods objectives outcomes process Sphere project structure tools

Introduction

To assess means to estimate the size or quantity of something [1]; to determine the value, significance, or extent of; to appraise. An assessment is the act of assessing [2]; the product obtained from assessing; an interdisciplinary process that involves the collation, evaluation, interpretation of information from various sources concerning both direct and indirect losses, and short-, mid-, and long-term effects [3 p145]. The conduct and interpretation of assessments are keys to disaster management as well as research and evaluation. Assessments form the basis for all decision-making.

In the medical sciences, assessment refers to determining the functional status of each of the organ systems of a patient in an effort to identify the damage and its causes. In health disaster, assessments are used to determine the level of functioning of one or more of the basic societal functions (BSFs) or components of a BSF, and the resulting functional status of the society. Before an event takes place, assessments are used to define baseline levels of function, the levels of risk, and the operational capacity of the society at risk. These assessments are used for the development of mitigation, preparedness, and contingency plans, and to identify the most appropriate interventions in this context. Using agreed upon benchmarks, the effects of these interventions then must be assessed as to how they helped solving a problem (repair damage, restore/compensate for dysfunction) or improved the level of preparedness of a community at risk.

Assessments define the situation of a society by examining the functional status of the BSFs or their components at a specific point in time; they are the first and crucial part of the pathway through data collection, information production, planning, and tasking that lead to an intervention or set of interventions (responses). Assessments are repeated to monitor any changes in the damage to structure and/or functions due to the passage of time, the continuation of the event (e.g. flooding, drought), and the effects of an intervention. All decisions must be based on the assessment(s), the relationship of the findings to previous assessments, and the resources that are available. The pre-event assessment serves as the point of reference to define structural and functional damage. Likewise, the first assessment conducted during and/or immediately following an event, in addition to providing the information required for defining the immediate needs of the stricken population, serves as the point of reference for all of the interventions implemented. Assessments mirror the societal situation at that point in time when the last of the data were collected. Consequently, any delay between the collection of data, making an inference from the data, and communicating the information derived, is unfortunate and may result in outdated information. The completion of immediate assessments in urgent situations (e.g. following an earthquake) may require hours or days. Nevertheless, depending on the nature of the event and the urgency associated with it (drought, high winds), assessments may require weeks or even months to complete.

The data/information acquired from the conduct of pertinent assessments that will be used for decision-making requires special knowledge and skills. The accuracy of the assessments depends on the abilities of the assessors to collect the data and the interpreters to synthesise the data into useful information.

Assessments form the basis for the selection of interventions, and interventions always must be based on the needs defined from the assessments. As described previously, needs are comprised of what it will take to prevent further deterioration of status of the society and of what is needed to return the functions of the BSFs to their pre-event levels. Once the needs have been determined from a process of synthesis of many factors, a strategic planning process ensues that is directed at meeting the defined needs. The results of this process are transformed into a strategic plan. The strategic plan includes the overarching goals for which interventions (responses) will be aimed and describes the mechanisms that will be used to meet the defined needs. From the different options for interventions, the ones that are most likely to meet the needs, in whole or in part, are selected and implemented. Hopefully, each intervention produces the intended effects for (output), and that this output is beneficial to the society affected (outcome). Interventions also may produce other effects that may or may not have been anticipated prior to the implementation of the intervention. Such effects may be beneficial to the affected society, or they may be detrimental. Regardless of the effects, every intervention implemented must be evaluated for its efficiency, effectiveness, efficacy, outcome (benefits/impacts), and costs to the affected society, including the positive and negative other effects of the intervention [3 pp102–12].

Each of the steps of the assessment process is outlined in Figure 9.1. How the assessment process finally results in an intervention is discussed in detail in Chapters 10–14.

Figure 9.1.

The assessment process.

The assessment process

The process used for conducting assessments has been deconstructed into its component parts. Although many assessments are conducted based on intuition and experience, in order for such assessments to be evaluated, the processes used, whether deliberate or intuited, must be understood. Without a clear definition of the process, it is difficult to identify any critical points of failure or success, and difficult, if not impossible, to improve the quality of the assessment process the next time it is performed. The process proposed to be used in the conduct of an assessment is illustrated in Figure 9.1. This process is generic, although, in this figure, it specifically addresses assessments used for the analysis of the health effects of disasters and the interventions implemented. The process also applies to interventions used to augment preparedness.

The 10 steps are used, sometimes subconsciously, to conduct all assessments. When the assessments acquired do not provide the information that was being sought, it is essential that the critical point(s) of failure are identified by using the steps in the process outlined. This should facilitate the conduct of more productive assessments in the future.

1. Need to know

Assessments are performed in order to gather information that is required, but is not available (gaps). There must be a specific reason to collect the data/information. Collecting data for no specific, defined reason is inappropriate, time consuming, and expensive; such data generally are useless and have a high opportunity cost. However, unexpected findings during assessments should be collected and reported if deemed appropriate for potential operations and/or preparedness initiatives.

2. Define goal and objectives

The value of the data collected will be determined relative to the overall goal and specific objectives that must be attained in order to achieve all or part of the goal for which they were collected. This is analogous to defining the goal and objectives of an intervention prior to its approval and implementation.

3. Select appropriate indicators

The data to be collected is in the form of indicators. Therefore, an essential element of all assessments will be the selection of common indicators so that collected data will be identical in format and thus, will facilitate comparisons. The selection or development of appropriate indicators for specific components of each BSF is essential for the conduct of such assessments. This is a complicated process, and, ideally, should be done as part of preparedness. All indicators selected must have good construct validity and accurately reflect the information that is being sought. If no valid indicators are available, new ones must be developed and their construct validity must be tested. Ultimately, a standardised set of indicators should evolve, some of which may be event-type specific and others that will be generalisable across all disasters. Indicators must be validated according to their specificity and sensitivity. The crude mortality rate (number of deaths per day per 10000 people) is neither a sensitive nor specific indicator, but is an overarching indicator of the health status of a population.

Indicators may be quantitative, qualitative, or a combination of both; some qualitative indicators may be scaled [3 pp118–22]. Indicators selected must be understood, practical to collect, and reproducible. During the last decade, many organisations have put forth sets of health indicators; some of these are discussed below. The cited sets of indicators/instruments are not intended to be all inclusive.

The Sphere Project began in 1997 with the charge to develop a set of “universal minimum standards in core areas of humanitarian assistance”. The goals of the Project were to “improve the quality of humanitarian assistance…and to enhance the accountability of the humanitarian system in disaster response”. The first edition of the Humanitarian Charter and Minimum Standards in Disaster Response was published in 2000. It provided minimum standards in four areas important to health: (1) water, sanitation, and hygiene; (2) food security, nutrition, and food aid; (3) shelter, settlements, and non-food items; and (4) health systems and infrastructure. Initially, the minimum standards were provided mainly for camps for refugees and internally displaced persons [4]. But, in the 2004 edition, the standards and indicators were more broadly focused. In the health section, it provides minimum standards for: (1) prioritising health services; (2) supporting national and local health systems; (3) coordination; (4) primary health care; (5) clinical services; and (6) health information systems. Some of the indicators used are quantitative and others are qualitative [4]. In the 2011 edition, this has been expanded further.

Prior to 2002, there existed protocols for the conduct of epidemiological assessments from at least eight organisations (WHO, UNHCR, UNICEF, IFRC, SPHERE, MSF, OFDA, and the US Centers for Disease Control and Prevention) in addition to some epidemiological centres (“epicentres”). In 2002, Bradt and Drummond synthesised the assessment protocols of these nine organisations into one simplified instrument to be used for the rapid epidemiological assessment of health in displaced populations [5]. They proposed that this instrument was the beginning of a standardised, minimum, essential data set, and that indicators incorporated could be used for initial assessments as well as for monitoring progress. With some minor modifications, the indicators derived from the synthesis are provided in Appendix 9.1. The indicators were placed into nine categories: (1) population; (2) security; (3) site management; (4) water; (5) sanitation; (6) food; (7) non-food; (8) shelter; and (9) medical – very much in line with the BSFs that first were described in the conceptual framework of these Guidelines [3 pp72–5]. The indicators chosen are associated primarily with public health. Unfortunately, to date, there have not been publications in which the use of this exact set of indicators has been used.

Since the earthquake and tsunami that devastated many areas in South East Asia on 26 December 2004, at least four additional attempts have been made to identify a universal set of health indicators for use in assessing/describing/evaluating disasters. These include, but are not limited to the: Initial Rapid Assessment Tool [6]; the Tsunami Recovery Impact and Monitoring System [7]; the Health Resources Availability Mapping System [8, 9]; and the Health Cluster Guide [10].

The Initial Rapid Assessment Tool (IRA) has 11 sections containing a total of 86 tables: The sections include: (1) general; (2) general situation; (3) health; (4) water; (5) basic and environmental sanitation; (6) food and nutrition; (7) shelters; (8) livelihoods (early recovery); (9) protection; (10) organisation and coordination; and (11) education [6].

The Tsunami Recovery Impact and Monitoring System (TRIAMS) has four sections including: (1) vital needs; (2) basic social services; (3) infrastructure; and (4) livelihoods. This tool was used for an analysis of the damage and changes in functions of four districts in Aceh Province of Indonesia following the 2004 earthquake and tsunami in South East Asia. It incorporates 40 indicators. Importantly, TRIAMS has incorporated a new method for displaying the indicators. An example is provided in Figure 9.2 [7].

Figure 9.2.

Number and percentage of houses destroyed by the earthquake and tsunami in four districts of Aceh Province, Indonesia.

The Health Resources Availability Mapping System (HeRAMS) is the most recently devised set of indicators that document the levels of care available by subsectors of health facility/mobile clinic/community-based interventions at each point of delivery. It consists of three major sections: (1) community care; (2) primary care; and (3) secondary and tertiary care. The community care section includes eight “sectors” and 14 indicators; the primary care section contains nine sectors and 13 indicators; and the tertiary care section contains four sectors and 10 indicators [8]. The HeRAMS instrument recently was used in South Sudan, Darfur, and Kivo [9].

Although the recently published Health Cluster guide is broader than the HeRAMS, it incorporates the HeRAMS. It consists of the same three levels of care as defined in the HeRAMS, and uses the following categories of indicators: (1) health resources availability (eight indicators); (2) health services coverage (six indicators); (3) risk factors (six indicators); and (4) health outcomes (five indicators) [10].

Each of these instruments uses a slightly different set of indicators; almost all of the indicators are public health related, although they differed substantially. Each of the indicators used meet the Specific, Measurable, Attainable, Realistic, and Time-bound (SMART) criteria for indicators [11, 12]. While these instruments are relatively new and their value has not been validated, hopefully, their use will help to cull out the indicators that have the greatest value. At the time of this writing, no data were available that compared the uses and relative value of these instruments in specific settings.

One additional set of indicators has been used by the Pan-American Health Organization (WHO-AMRO) for the evaluation of the vulnerability of hospitals in emergencies and disasters [13]. This instrument uses three sets of indicators: (1) structural; (2) nonstructural; and (3) functional, and is used to score the relative vulnerability of the hospitals relative to their ability to continue to function during or following an event. Its use allows hospitals to be categorised in one of three levels of safety: (A) facilities deemed able to protect the lives of their occupants and likely to continue functioning in a disaster situation; (B) facilities that can resist a disaster, but in which equipment and critical services are exposed; or (C) health facilities in which the lives and safety of the occupants are threatened during a disaster. Use of this instrument has been tested repeatedly and has met with substantial success.

Currently, there is an abundance of indicators and tools that can be applied to disaster research and evaluation. Regardless of the indicators selected, efforts should be made to assure, to the extent possible, that the indicators selected are not duplicative of other indicators. Those indicators that apply to more than one BSF should be discussed between the BSFs involved. Information on these indicators usually need only be collected once, not separately, for each of the stakeholders or BSFs.

The selected indicators should be identical whether used pre-event, during, or after a disaster to document the damage, levels of functions, and the effects of the interventions used. If additional indicators are used in the research/evaluation, their respective baseline, pre-event status should be sought. Without a baseline (point of reference), the current status has little or no meaning.

4. Define methods

Once the most appropriate and practical indicators have been selected, methods for collecting the data must be developed and validated. Appropriate data collection tools must be developed, or ones already validated must be adapted for use in the current assessments. There still is no universal consensus on any of the currently available tools (with the exception of the Safe Hospital Inventory) [13]. The forms used for data collection often are inconsistent, there is no agreed structure, and the questions posed are far from uniform. For the most part, each organisation charged with the responsibility for the collection of data and information relative to damage and functional states uses its own system and data collection forms. The use of such diverse methods for data collection confounds the external validity of the conclusions drawn from the data collected. Furthermore, these inconsistencies prohibit combining the results from different assessments by different organisations into more valid and strong assessments and hypotheses. It also impedes/prohibits the development of the science of the medical care and public health aspects of disasters; commensurable assessments should form the platform for research and evaluation on disasters and the responses to them. Thus, there is a need for the development and use of standardised tools for the conduct of assessments. This applies to interventions aimed at enhancing preparedness as well as those directed at attenuating the effects of the event and returning the damaged society to its pre-event state. The methods and tools selected must be practical and within the means (financial, personnel, timelines, etc.) of those doing the assessments and the synthesis. When possible, untested tools with unknown construct validity should be avoided [3 pp102–12].

The testing and subsequent enhancement of the currently available standardised tools and data collection sheets will facilitate the assessments of damage and the functional status of the BSFs and their respective components. Documents on available methodologies to assist with assessments have been published [3 pp72–5, 5–10]. For the most part, these documents are directed towards rapid health assessments and are a mix of damage and functional status assessments. They also are used to collect information relative to the public health aspects and the functional status of many of the other BSFs that bear directly, or indirectly, upon health. However, for the most part, these tools either have not been tested adequately, and/or have not been accepted universally [14]. Parts of already existing assessment tools could be abstracted and amalgamated into a generic, universally endorsed platform for assessments, as done by Bradt and Drummond [5]. This not only will structure and promote appropriate and rapid responses, but also will facilitate future research and evaluation. All generic components in assessments, including endorsed indicators, should be integral parts of the pre-event inventory of a society.

5. Develop and test data collection (operational) plan

Once the methods have been selected, a data collection plan must be developed. This plan must include how the data will be collected and by whom. Once the plan has evolved, when possible, its implementation should be piloted using a small sample. Pilot testing of new plans generally saves time in the long run. Potential barriers most often are detected during such pilot efforts, and appropriate corrections can be made. However, in urgent circumstances, pilot testing may not be feasible. Selection of the sampling method to be used is part of this planning process and must be a part of the assessment plan. All plans must be approved by the coordination and control centre (CCC).

6. Train/brief data collectors

Data collection requires special expertise. Even persons experienced in data collection must be oriented, trained, and/or briefed in the elements of the data collection plan and the instruments to be used for data collection. Furthermore, data collection must be done in concert with the culture of the affected society. It is inappropriate to impose oneself into a population without understanding its culture, religions, customs, conventions, and languages. In some instances, interpreters may be required. Even the abstraction of data from municipal or medical records requires expertise. All data collectors must be oriented to the culture and techniques to be validated.

7. Gather/collect data (implementation)

Using the instruments devised and the data collection plan, the trained/educated data collectors implement the data collection process in accordance with the plan. Data collection includes not only the gathering of the data, but also inputting the data into the database defined in the plan. Data collection must be coordinated and facilitated by the CCC. During disasters, the data collected are not the sole property of the agency collecting the data, but must be shared with the most appropriate level of coordination and control. The data collected represent the input into the information production process.

In addition, there is a danger that too many assessments will be made in the same geographical area. Often, multiple observations have been conducted in the same area by different persons, as occurred following the 2004 earthquake and tsunami in South East Asia [15]. This can result in accounting the same damage as multiples, and consequently, overestimation of the resources required to halt the deterioration or to return the society to its pre-event state.

8. Synthesis of the data

The synthesis of the data into the information for which the assessments are being conducted is a transformation process. This transformation of data into useful information requires education, training, and experience. It is this process that determines the validity of the product that results from the assessment. It is at this step that the process is most vulnerable. No matter how sound the data, interpretation of the data is crucial to give them meaning. Interpretations of the data only will be as good as is the expertise of the persons performing the data transformations.

9. Information

The output of this transformation process consists of the information that was needed – the reason the assessment was being conducted. It is this information that is used for decision-making. During disasters, this information must be shared with all of the stakeholders through the CCC.

10. Comparison with goal and objectives

The output from the assessment must fill the gap(s) of information defined in steps 1 and 2. The information that results may contribute to definition of the pre-event status, to the types and amounts of damage sustained, to the changes in functions that have resulted from the damage, or from the interventions implemented to meet the defined needs.

Assessment teams

Assessments are done best by well-trained and experienced personnel using standardised data collection forms and techniques. Unfortunately, such personnel are difficult to identify and, for international disaster management, not universally credentialed. Currently, available tools are being used and validated for use, accuracy, construct validity, and efficiency. Ideally, personnel who are expert in at least one of the BSFs should be charged with the responsibility for the collection of those data relative to their respective area(s) of expertise.

Health assessments should be done by teams of persons trained to conduct the assessments using standardised instruments. A competent team must be multidisciplinary, since problems may call for persons knowledgeable about nutrition, logistics, environmental health, and/or engineering, as well as experts from other societal functional systems required in the provision of health services (the medical care and public health BSFs). The composition of the teams may vary according to the type of hazard involved, the character of its onset, the scope of the damage, the longitudinal phase of the disaster, and the setting in which the disaster has occurred (culture, religion, climate, etc.). As an absolute minimum, at least the leader of any assessment team should be experienced in the conduct and processing of the assessments in disaster settings. The World Health Organization (WHO) has listed more specific demands that ideally should apply to all of the assessment team members; These include:

familiarity with the region and the culture and language of the population affected

knowledge of and experience with the type [nature] of the disaster for which the assessments are being conducted

personal qualities, such as endurance, motivation, personal health, and the capacity for teamwork, as well as local acceptability for team members recruited abroad

analytical (interpretive) skills, particularly the ability to identify trends and patterns [16].

These are stringent demands – the lack of these qualifications may contribute to incorrectly interpreted observations, as has occurred during previous disasters [17]. The responsibility for these problems reflects back to the selection of the team members by the respective organisations involved in disaster management. However, for most regions, countries, and especially districts, it currently is impossible to find all of these qualities, competencies, and proficiencies in each team member. But as a minimum, all requirements must be met by the group as one entity.

Assessing damage, changes in functions, and available resources requires training. Even though substantial knowledge on the conduct of assessments has been demonstrated, standardised methodologies for accurate and reliable assessments have not been properly mainstreamed in the relevant professional courses for education and training. Consequently, there is no standardised training for assessment teams, and there are no standards for team composition. This makes the efforts to develop programmes to educate and train persons to perform these assessments very difficult. Guidelines are required that include the conduct of assessments of a single element to the complexities associated with more complete assessments of larger parts of society.

Role of the CCC

All of the data to be collected by the assessment teams must be defined by the CCC either in plans developed before the event occurs or prior to the beginning of the definitive assessments, or preferably, both. No generic plan, worked out in advance, is likely to be appropriate without being modified according to the current scenario. The CCC must be in charge of the assessments. This coordinated effort is essential to: (1) identify the most appropriate teams to conduct the assessments; (2) obtain the information necessary for decision-making and the planning and implementation of the intervention; and (3) prevent unnecessary duplication and the problems associated with such overassessments. The need for reliability and reproducibility of the findings (e.g. confirmation through triangulation)¹ should not be confused with duplication.

All of the information acquired by the assessors must be funnelled through the most relevant CCC. As noted, during a disaster, all of the data/information gathered are the property of the CCC and do not remain the sole property of the assessors. This latter issue became a significant problem following the earthquake and tsunami that devastated parts of South East Asia in 2004. Some of the nongovernmental organisations (NGOs) believed that the data collected by their organisations belonged to them, and did not share them with the CCC [18]. At least one organisation had not shared the data collected 3 months after the assessment was completed. It is in the CCC that the results of the damage assessments, assessments of changes in functions, available resources, and local culture are synthesised into needs. These needs are assigned the appropriate priorities and are integrated into plans for and selection of interventions.

Many organisations dispatch their own assessment teams into the disaster area. It is clear that the greater the number of assessment teams in the disaster area, the greater the likelihood of a duplication of efforts [18]. The number of such teams should be kept to an absolute minimum, but not so limited that it increases the likelihood that some BSFs will not be evaluated adequately. It is unlikely that one organisation is capable of studying all of the BSFs and their respective components properly – indeed, not even all of the components of one BSF.

It frequently has been emphasised that the stricken society can best define its real needs. Research, however, does not fully support this concept [19]. Although self-assessment is important, relying exclusively on self-assessments results in an absence of quality control of the assessments and has the potential to confuse needs with demands.

Assessments, together with planning, selection of appropriate interventions, implementation of the selected interventions, and evaluation of the effects, therefore, should be coordinated through a central body responsible for coordination and control [20]. It has become clear from the dissection of Hurricanes Georges and Mitch in Central America in 1998 [14], and during the WHO-sponsored conference on the 2004 tsunami convened in Phuket, Thailand, that in all of these situations, there was no single focal point for data coordination [21]. Several organisations were assigned this task, but there was no universally accepted mandate or authority. Clearly, a lead agency is essential and must be given, through previous planning, the mandate, authority, and resources required to provide the coordination. Thus, assessments to be used in the definitions of needs require a coordination and control structure for the coordination of efforts and to assure completion of the assigned tasks. Local and national governments must be involved in this process [22]. Following the international Humanitarian Response Review of 2005 [23], important developments have taken place in the area of humanitarian coordination and control that may provide appropriate mechanisms to address these challenges. For example, the Clusters approach, that distributes humanitarian responsibilities in a structure that essentially is consistent with the BSFs, goes a long way to meet the needs for centralised, common information management, strategic decision-making, and distribution of operational responsibilities [24].

The defined needs always should represent the actual needs of the population affected and not the needs of the organisation doing the assessment or the sponsor of the assessment. Some responding organisations do their own assessments and mount their own responses based on these assessments without coordinating their activities through the single, responsible CCC [25]. To the extreme, such assessments may be directed to what an organisation is capable of delivering, and less to what the population actually needs. Such self-serving assessments must be controlled by the lead, coordinating agency.

Administrative issues

Administrative arrangements are crucial for the completion of successful assessments. For international responses, the list of administrative criteria obtained from the WHO includes:

obtaining authorisations for travel and security clearances

organising transportation and other logistics (e.g. vehicles, fuel)

setting up a communication system and informing the authorities in the affected area of a timetable for conducting the assessment

organising other equipment, such as computers, height boards, scales, and checklists

ensuring the safety and security of team members from violence, infections (e.g. vaccinations and prophylactic treatment), or other hazards in the emergency-affected area.

In addition, all such teams must be self-sufficient so as not to place an additional burden on the already overburdened population being assessed. This must include sufficient shelter, food, beverages, and prescribed drugs and healthcare devices [26].

From this list, it seems evident that the conduct of assessments by assessors from within the affected nation usually would be the least complicated. Internal assessments also could be completed much faster than those conducted by an international team being forced to abide by all regulations, and whose members may be unfamiliar with the affected area. On the other hand, a completely objective assessment may require some emotional distance to the object or situation being observed. This is one of the advantages associated with enlistment of one or more experienced international assessment teams (especially at the regional level). A combination of these options may be the most advantageous approach.

Assessments create expectations! Any assessment or attempted evaluation is unlikely to be accepted by any group of disaster victims unless it is followed by a response(s). Furthermore, it is inappropriate to have the same questions asked of the same person(s) by multiple survey teams. This problem was stressed by Chief Segun Olesula during the 1997 Gothenburg Conference; Knut Ole Sundnes (Medical Operations and Planning Officer, UNPROFOR/UNPF)observed this in the Bihac “pocket” during the Yugoslavian conflict 1992–1995; and it was also observed following the earthquake and tsunami in South East Asia in 2004 [16].

However, while response(s) should be a mandatory and logical consequence(s) of assessment(s), information from the assessments may indicate that a response is not needed. This is unlikely to be accepted by the victimised people, and may lead to complications and frustrations.

The literature, standing orders, and guidelines from organisations such as the UN-OCHA, WHO, PAHO, IFRC, ICRC, and the US Centers for Disease Control and Prevention make it clear that some relevant knowledge about assessment does exist. However, the assessments not always are conducted by people with the necessary experience [21]. On some occasions, high officials with little or no insight into disaster problems and their management constitute the prime evaluation team. This may divert both resources and interest away from the real assessments; their participation may distort the conclusions drawn, and correct assistance may be delayed unnecessarily. Also, they may constitute safety problems, and thus, consume protection resources that, for example, have resulted in temporarily closed airports that postpone relief efforts.² Hopefully, standardisation of these assessments and processes may be one of the outcomes from the use of these Guidelines.

Documentation

An important part of each assessment is documentation. All of the findings upon which decisions are made must be documented using standardised forms, photographs, video recorders, taped interviews, notes, logbooks, etc. Documentation (records) is essential for evaluation and for modifying and improving future assessments and processes. Otherwise, future evaluations of the responses will prove very difficult. Adequacy of documentation is part of an evaluation process. Use of the currently available institutionalised, standard, data collection instruments must be evaluated and their value validated.

Sources of error

There are many potential sources of error in the conduct of assessments and the processes that result in defining the needs and implementing plans to address them. The following list of potential sources of error is partly abstracted from the WHO Needs Assessment documents [27, 28]:

Logistical

Transportation and fuel are insufficient for the conduct of assessment

Communications between field and regional and national levels are inadequate; the authorities in charge of the area(s) targeted by the assessment are not informed on time, and are not ready to assist.

Organisational

A lead organisation is not designated, the responsibilities of the various organisations are not well defined, and a team leader is not appointed

Key decision-makers and potential donors either are not informed that an assessment is being undertaken, or feel pressured to respond to political demands before the findings are known, resulting in inappropriate assistance

The assessment is conducted too late, or takes too long

Information is collected that is not needed for planning of the responses.

Technical

Specialists with the appropriate skills and experience are not involved in the assessment

Programmes that could be implemented immediately, on the basis of past experience, are delayed unnecessarily until the assessment is completed

Assessment conclusions are based on data that do not represent the true needs of the affected population (e.g. from nonrepresentative surveys)

Information received from field workers and official interviews is taken at face value without cross-checking all sources

A surveillance system is developed too slowly, thus preventing monitoring and evaluation of the responses.

These are sources of error that have been identified throughout all of the longitudinal phases of a disaster, and it is an objective of these Guidelines to analyse what worked, what could have worked better, what went wrong, the sources of error, and to identify potential solutions. Otherwise, we will continue to suffer from disaster mismanagement. In the Rwanda crisis, the UNHCR and UNICEF operated with different lists of assessed priorities [3 p148, 29].

Use of assessments during disasters

Assessments define the current state at the time the assessment is completed. Such operations require expert human input and thought. Some of the specific uses of assessments include identification of: (1) damage; (2) levels of functions; (3) demands; (4) inventories; (5) costs; and (6) outcomes (benefit). To be meaningful, assessments during disasters must be related to some baseline(s).

Establishing baselines

Assessments are required to establish baseline levels of functions from which changes/deviations will be judged. Assessments of the pre-event status of a BSF or of its components serve as points of reference to describe changes in levels of function due to damage. Similarly, assessments are used to establish control levels of functioning prior to the initiation of any intervention and any changes related to the intervention. Choosing the most appropriate indicators to be used in the assessments is of critical importance. These same indicators must be used in all subsequent assessments. Indicators for the objectives (outputs) have to differ from the indicators for the goals (outcomes/benefits).

Damage

Damage is the negative result from the impact of an event [3 p148]. Damage can occur to living beings, the built environment, the natural environment, and/or the economics (financial structure). Damage always relates to physical damage and does not include the changes in functions that result from the damage. Therefore, damage assessments must be related to the status of persons (living beings), the built environment, and the natural environment before the occurrence of the event or before implementation of interventions directed at recovery (restoration, repair, reconstruction, rehabilitation). If the damage does not result in changes in function, then the society was able to buffer the damage (buffering capacity) and repairing the damage has low priority.

Functions

Assessments of the functional status of the BSFs have great value in disasters. As has been noted, changes in function are determined by assessments in relation to the pre-event state and in relation to the status determined by the previous assessment. Again, the indicators selected are key and must be carried through as part of all subsequent assessments. Assessments of function also relate to the functional threshold for the indicators used in the assessment, as well as for the critical threshold, if one exists for that function. Thus, assessments of the levels of functions are used to determine the functional deficits that exist. The level of function is not described in terms of available supplies. Unless they are the effects of an intervention, assessments of levels of functions only indirectly define needs.

Demands

Assessments should take into consideration the demands by the affected population. The demands of the affected population have been shown to differ substantially from the needs estimated by the expatriate workers [19]. The affected population tends to overestimate their needs while the responders tend to underestimate the actual needs of the population. The assessments of demands generally entail perceptions; such perceptions must be considered when attempting to identify and prioritise needs.

Inventories

Inventories are an accounting of the various goods and services that are available. Inventories can be, for example, the accounting of supplies, equipment, personnel, and their types and levels of expertise. Inventories are a form of assessment that is dynamic and requires tracking or repeated assessments.

Costs

Costs are an expenditure of resources [3 p148]. Assessments that relate to costs may focus on economic, material, environmental, human, and/or opportunity costs. Assessments of costs may involve actual accounting procedures or estimates of the costs required to provide relief and/or promulgate recovery. Often, the media describes the impact of a disaster in terms of “estimated costs” (e.g. “cost are estimated to exceed $1 billion”). Estimates for replacement or repair of damaged structures and costs to repair damages to the environment sometimes are cited. Indicators of human and opportunity costs are more difficult. Indicators frequently used for human costs include the number of “lives lost”, while the years of life lost, earnings and tax revenue not achieved, the number of injured, the opportunity costs associated with the injuries, number of orphaned children, deterioration of mental health [30], etc., often have not been included. Little has been done to account or estimate the losses due to injuries. For example, there is little information about the costs associated with those injured in the Iraq war, both military and civilian [31]. The costs of the Iraqi conflict to the Iraqi civilians have been staggering. In addition, little has been done in defining indicators for the opportunity costs for humans or materials. This must be addressed.

Outcome/benefit

An outcome is an effect, a result or influence [1], the result of a specific intervention(s) or project(s) relative to their pre-established goals and objectives [3 p155]. Outcomes could be accrued for the individual, family, work force, community, state, and/or country, and may be positive of negative. A benefit is whatever is for the good of a person or a thing [3 p146]. The benefits are a measure or judgment of value and should describe the value of the changes resulting from an intervention to the recipients of the intervention. Generally, “benefit” is a positive term; “outcome” may be either positive or negative. Currently, there is no codified set of indicators of benefit or outcome. However, the expected benefits and potential outcome(s) to be accrued should be described in the goals for the intervention. Indicators of benefits and/or impact most often are qualitative, but can be quantitative (i.e., money donated, crude mortality rate). The level of “satisfaction” is an indicator.

Summary

Assessments constitute an essential element of proper disaster management. Assessments help to define the damage and changes in functions at the time the assessment is completed. Assessments are transectional across the longitudinal phases of the disaster. Any initiative, regardless of being part of improving preparedness of a society or actions in case of disasters, should be preceded by an assessment. The generic elements imbedded in the assessment process are the same, but will be conducted under very different time limits. Agreement on certain elements in advance (i.e. representative indicators, methods, and team composition) will shorten such planning when circumstances so dictate.

A plan that is not based on proper assessments is likely to fail when challenged. This applies to pre-event planning and preparedness, as well as for disaster management.

Assessments are the responsibility of the CCC, both for planning, preparedness, and disaster responses. It requires special insights and skills and, in case of assessments used to identify needs during sudden onset disasters, must be relevant to field proficiencies in order to collect the most necessary data in shortest possible time.

Footnotes

Appendix

Appendix 9.1.

One suggestion for a minimum dataset of variables and indicators for assessing and monitoring of displaced persons.

Date
Assessor
Disaster name
Disaster type
Site name
Location
Population
Registration	Y/N
Total population
No. of households
Arrivals/week
Departures/week
Demographics (by gender and age, percentage or actual number)
Under 1 year 5%, women 15–44 years
Under 5 years 20%, men 15–44 years
5–14 years 35%
≥45 years 15%
Typical livelihood
Vulnerable groups
Security
Officer in charge
Camp leader
Indicators
Incidents at site	Y/N
Type
Issues
Site management
Lead agency
Contact
Phone/fax
Indicators
Original site use
Area (m²)
Area (m²/person) (>30 m²/person)
Road access	OK/not OK/problem
Water access	OK/not OK/problem
Drainage	OK/not OK/problem
Building repair	OK/not OK/problem
Electricity	OK/not OK/problem
Issues
Water
Lead agency
Contact
Phone/fax
Indicators
Water source
Litres/person/day (>20)
No. of reservoirs
Condition at base
m from home (<100)
No. of taps
Running hours/day
Persons/tap (<200)
Home source
Jerry cans	Y/N
Turbid	Y/N
Colour	Y/N
Odour	Y/N
Chlorination	Y/N
Boiling	Y/N
Coliforms/dl (<10)
Issues
Sanitation
Lead agency
Contact
Phone/fax
Indicators
No. of latrines
Latrine type
Persons/latrine (<20)
Squat plate	Y/N
Water seal	Y/N
Percentage blocked (0)
m from water (>100)
m from home (<30)
Water at latrines	Y/N
Hot water	Y/N
Soap g/person/month (>500)
Cleaning supplies	Y/N
Maintenance teams	Y/N
Printed health messages	Y/N
Clean latrines	Y/N
Vermin/vectors	Y/N	Vermin type (none)
Wash bucket	Y/N
Showers	Y/N	Persons/shower (<20)
Waste drums	Y/N
Waste pits	Y/N	Persons/pit (<500)
Issues
Food
Lead agency
Contact
Phone/fax
Indicators
Self-preparation	Y/N
Cooking equipment	Y/N
Cooking fuel	Y/N
Communal kitchen	Y/N
Warehouse food storage	Y/N
Food distribution	Y/N
Supplement feeding	Y/N
Food security	Y/N
Staples kcals/person/day (>2100)
Issues
Non-food
Lead agency
Contact
Phone/fax
Indicators
Mats/mattresses	Y/N
Blankets	Y/N
Bed nets	Y/N
Hygiene parcels	Y/N
Warehouse storage	Y/N
Issues
Shelter
Lead agency
contact
Phone/fax
Indicators
No. of tents
No. of buildings
Building materials
Sheeting	Y/N
Space partitions	Y/N
Shelter m²/person (>4)
Issues
Medical
Lead agency
Contact
Phone/fax
Indicators
Clinic
On site	Y/N
Distance from camp
Hours open
Structure OK	Y/N
No. of doctors
No. of nurses
Fees	Y/N
Running water	Y/N
Toilet	Y/N
Electricity	Y/N
No. of examination tables
ORS corner	Y/N
IV fluids	Y/N
Dispensary	Y/N
X-ray	Y/N
Overnight stay	Y/N
Standard case definition	Y/N
Treatment protocols	Y/N
Total visits/week
Active case finding	Y/N
Percentage total population/day (<1)
Total deaths/week:
Active death finding	Y/N
Deaths/10k person/day (<1)
Total referrals/week
Referral destination
Incidence watery diarrhoea
Case definition
Past week
Treatment for watery diarrhoea
ORS prep demonstrated	Y/N
Dysentery visually confirmed	Y/N
Acute respiratory infections	Y/N
Diagnosis of pneumonia by X-ray	Y/N
Measles immunisation campaign	Y/N
Cold chain intact	Y/N
Malaria microscopically confirmed	Y/N
Falciparum	Y/N
Epidemics	Y/N
Type epidemic
Control plan	Y/N
Malnutrition
Therapeutic feeding	Y/N	Type
Trauma type
Somatic/Physical
Psych fear in population	Y/N	Reason
Provider-stated needs
Issues

Modified from: Bradt DA and Drummond CM. Rapid epidemiological assessment of health status in displaced populations – an evolution towards standardised minimum essential data sets. Prehosp Disaster Med 2002;17:178–85.

Notes

References

Thompson

(ed). The concise Oxford dictionary of current English, 9th ed. Oxford: Clarendon Press, 1995.

Pickett

(exec ed). The American heritage college dictionary, 4th ed. New York: Houghton Miflin Co, 2002.

Task Force on Quality Control of Disaster Management, World Association for Disaster and Emergency Medicine and Nordic Society for Disaster Medicine. Health disaster management: guidelines for evaluation and research in the Utstein style. Sundnes

Birnbaum

(eds). Prehosp Disaster Med 2003;17(Suppl 3).

The Sphere Project: Humanitarian charter and minimum standards in disaster response. Oxford: Oxfam Publishing, UK, 2000, 2004.

Bradt

Drummond

CM.

Rapid epidemiological assessment of health status in displaced populations – an evolution towards standardized minimum, essential data sets. Prehosp Disaster Med 2002;17:178–85.

IASC Health Cluster, IASC Nutrition Cluster and IASC WASH Cluster. Initial rapid assessment (IRA) assessment tool. Geneva: WHO, 2007.

UN, WHOUN, WHO and IFRC: Tsunami recovery impact assessment and monitoring system. Geneva: WHO. 2006.

Health Resources Availability Mapping System (HeRAMS). Geneva, WHO, 2009. Global Health Cluster. Available at: http://www.who.int/hac/network/global_health_cluster/herams_services_checklist_eng.pdf (accessed June 2011).

Zagaria

: Application of the Global Health Cluster HeRAMS Tool in South Sudan, Darfur, and North Kivu. Presentation at the Twelfth Annual Meeting of the Inter-Agency Working Group on Reproductive Health in Crisis. Health Action in Crisis: WHO: Santo Domingo, 2010.

10.

WHO, UN-OCHA IASC-, Global Health Cluster: Health Cluster Guide. WHO: Geneva. 2009.

11.

United Nations Development Programme. Handbook on Planning, Monitoring and Evaluating for Development Results. New York, 2002. Available at: http://web.undp.org/evaluation/handbook/ch2-4.html (accessed December 2013).

12.

World Bank Group. Selecting Indicators. Washington DC. Available at: https://saeguide.worldbank.org/sites/worldbank.org.saeguide/files/documents/Developing%20Indicators.pdf (accessed December 2013).

13.

World Health Organization (WHO) and WHO-Western Pacific Regional Office (WPRO). Safe hospitals in emergencies and disasters. Geneva and Manila: 2010. Available at: http://www.wpro.who.int/emergencies_disasters/documents/SafeHospitalsinEmergenciesandDisastersweboptimized.pdf (accessed December 2013).

14.

WHO-PAHO. Evaluation of preparedness and response to Hurricanes Georges and Mitch: conclusions and recommendations. Prehosp Disaster Med 1999;14:53–65.

15.

Deitchman

What have we learned? – needs assessment. Prehosp Disaster Med 2005;20:468–70.

16.

World Health Organization. Protocol for assessing national surveillance and response capacities for the International Health Regulations (2005) in accordance with annex 1 of the IHR. A guide for Assessment Teams. Geneva: WHO, December 2010. Available at: www.who.int/ihr/publications/who_hse_ihr_201007_en.pdf (accessed June 2011).

17.

Aga Khan

. Improving the disaster management capabilities of the United Nations. New York: UNA-USA, 1988.

18.

McGarry

Emerson

Fink

. Health aspects of the Tsunami disaster in Asia. Prehosp Disaster Med 2005;20:368–77.

19.

Rubin

Heuvelmans

Tomic-Cica

. Health-related relief in the former Yugoslavia: needs, demands, and supplies. Prehosp Disaster Med 2000;15:1–11.

20.

World Health Organization and European Regional Office. Community health needs assessment. WHO Euro, Copenhagen, 2001. Available at: www.euro.who.int/__data/assets/pdf_file/0018/102249/E73494.pdf (accessed June 2011).

21.

Emergency and Humanitarian Action, World Health Organization and Regional Office for South East Asia. Tsunami 2004: a comprehensive analysis, NLM classification number HC79.D45. India: SEARO, 2013.

22.

Smith

Fink

Hansch

. Health services delivery: a critical review of experience. Prehosp Disaster Med 2005;20:389–92.

23.

Oyegbite

What have we learned? – coordination. Prehosp Disaster Med 2005;20:471–4.

24.

International Council of Voluntary Organizations. OCHA’s global humanitarian response review 2004–2005. Geneva: ICVA, 2005. Available at: https://icvanetwork.org/doc00001369.html (accessed June 2011).

25.

UN Office for the Coordination of Humanitarian Affairs, Inter-Agency Standing Committee, Global Health Cluster and Subgroup on Management and Coordination. Gap guidance materials: assisting the health sector coordination mechanism to identify and fill gaps in the humanitarian response. Geneva: WHO, Global health Cluster, 2007. Available at: http://www.who.int/hac/network/global_health_cluster/gap_guidance_assisting_the_health_sector_coordination.pdf (accessed December 2013).

26.

Aldis

Rockenschaub

Gorokhovich

. Assessing impact and needs. Prehosp Disaster Med 2005;20:396–8.

27.

United Nations Office for Coordination of Humanitarian Affairs. OCHA disaster response preparedness toolkit. New York: OCHA: 2008. Available at: https://business.un.org/en/documents/186 (accessed December 2013).

28.

World Health Organization and Health Action in Crisis. Global assessment of national health sector emergency preparedness and response. Geneva: WHO: 2008. Available at: http://www.who.int/hac/about/Global_survey_inside.pdf (accessed December 2013).

29.

Domres

Mang

The flight from Rwanda in 1994: what were (are) the priorities. Prehosp Disaster Med 1997;12:47–51.

30.

Bolton

Tang

AM.

Using ethnographic methods in the selection of post-disaster mental-health interventions. Prehosp Disaster Med 2004;19:97–101.

31.

Siegler

Roberts

Balch

. Media coverage of violent deaths in Iraq: an opportunistic capture-recapture assessment. Prehosp Disaster Med 2008;23:369–71.