Developing a new project risk ranking model by means of jackknife resampling method consider interval analysis

Abstract

Since the successful performance of the projects results from their complicated nature and internal non-reliability, risk evaluation for gaining success in management and project performance, is a required factor. In reality, due to the complicated nature of the projects and lack of specialized manpower, attaining comprehensive information in connection with project risks shall be difficult, while there is a little information about them. The Jackknife resampling method is therefore used meticulously to solve such problems and estimate the society parameter. In this paper, non-parameter Jackknife resampling method has been used for interval estimation of risks, considering the concept of the confidence interval and opinions of the specialists. Then evaluation and classification of major project risks have been made, using interval analysis method. A case study has also been presented to prove the efficiency of the proposed method and finally the comparison of the results of this method with those of Bootstrap sampling methods were presented. The results of this research have indicated that the proposed method not only plays an important role in lowering the expenses and is timely economical, but also gives more meticulous results in proportion to the Bootstrap sampling method.

Keywords

Project risks ranking jackknife resampling method interval analysis construction project

1 Introduction

Projects will generally have successful performance when implemented on time, considering the budget and satisfaction of the beneficiaries [11]. Performance and success of the major projects are affected by the complexity and internal non-reliability of the projects. This complexity and non-reliability comes from the external environment and intricate structure of the project itself [18, 25]. Therefore, risk management is a significant factor in major projects. Risk in a project shall be considered as an indefinite incident or conditions that if happens, effects objectives of the project (such as limits, expense, safety, environment and quality) [1, 16, 1, 16]. Project risk management is to challenge activities, particularly in primary phases [10] and usually includes four steps of risk identification, risk assessment, reaction to risk and risk supervision [16]. Project risk assessment comprises the following objectives [2]:

Presenting a general view of public level in accordance with the project

Identification of risks; focus of management on risks and their factors

Assistance in decision-making on urgent measures and planning for further measures

Facilitating resource allocation process

Previous studies on risk management have been concentrated on in major projects. Faucher and Fitzgibbons [6] made a statistical analysis of technical specifications of risks in 79 major projects in Canada, including energy, communications, transportation and defense divisions. Miller and Lessard [14] classified major projects risks in three major groups. They classified demand, financial and procurement risks under the title of market risks; technical, construction and performance risks under the title of execution risks and management risks, social acceptability and power, under the title of existing risks in the organization.

Hastak and Baim [9] defined risks that are effective in cost management, performance, maintenance and repairs as well as the time during which risk factors will affect the relevant facilities cost. Miller and Floricel [7], by presenting a conceptual framework of strategic methods for solving chaos in engineering projects, developed and improved this system. They indicated that rapid preparation for avoiding predicted risks and improving the ability to react when confronting incidents is a useful and lasting event.

Zhang [28], considering role of risks in construction projects, presented a new method of risk management. Matawan et al. [19] presented a fuzzy system for estimation and assessment of risk of changes in construction projects. Tse et al. [27] classified risk breakdown structure on a hierarchy basis. Zayed et al. [28] introduced a risk index that could do estimation and assessment of risk resources and grading of major projects with regard to the grade and size of the company as well as size and level of the projects. Fang [5] also introduced a method based on Bayes’ Theorem to get to know the significance of critical path from the viewpoint of system theory. Ebrahimnejad et al. [4] identified effective standards for risk and presented a fuzzy MCDM model for risk assessment. Mojtahedi et al. [16] presented a better concept for risk diagnosis and assessment, considering the concept of safety. They considered health, safetyand environmental standards, in addition to time and cost and finally used nominal group technique (NGT) and multiple attribute group decision making (MAGDM) techniques for risk diagnosis and assessment of gas refinery construction.

Zhou and Zhang [30] established an active risk management system for major projects in China that included six phases of risk database, identification and determination of risk; risk assessment; risk pre-monitoring and risk tracking. Makui et al. [12] presented a methodology for diagnosis and analysis of engineering projects, using Fuzzy team decision-making method.

Tavakkoli-Moghaddam et al. [26] made a highway projects risk assessment using the jackknife technique. They calculated an average of risk probability and risk shock intensity upon highway project risk identification by a sort of Jackknife technique, without any consideration to the concept of confidence intervals. Then upon calculation of risk factor by multiplying of average probability and average shock intensity, made an assessment of highway project risk.

Various techniques have been introduced for risk management and assessment in engineering projects, but we are still in need for a simple and systematic technique to determine and classify major risks among a set of alternatives. It has been found by studying previous researches that those studies focused on a statistical parametric framework. The hypothesis in recent studies is that there is enough society risk data and that the size of the available society for extracting sample data to attain the results is big enough, but it may be possible that in real decision-making, these hypotheses do not apply, due to lack of professional expertise and or time limits [16]. Accordingly, resampling techniques with less data may be used for identification of risks in major projects. Determination of exact risk quantity is though very difficult in reality. That’s why interval risk estimation, with regard to the concept of the confidence interval (point estimation), may not be equal to the society parameter, when the number of samples grows [8] and experts’ opinion may increase the accuracy of anticipation [21]. The goal of this paper is to use the Jackknife Resampling method and interval analysis for risk identification and grading. Meanwhile, the results of this technique have been compared to another resampling technique (Bootstrap). It should be noted that using resampling techniques, while being economical, makes the calculation more rapid for collecting data. Using resampling techniques shall be useful when having less data or when calculation by traditional statistical methods is more complicated and difficult. On the other hand, answering to questions that cannot be responded by parametric methods, shall be possible by these resampling techniques (like Jackknife technique) [31].

The statistical method introduced in this paper, was innovated by Quenouille in the year 1956 for testing hypotheses and estimating confidence interval, which could not be calculated by older statistical methods and was then named Jackknife by Tukey in the year 1958 [23].

The remainder of the paper proceeds as follows: Section 1, talks about collecting risk data, applying Jackknife Resampling Method and calculation of confidence interval for risk standards and then goes into grading of the existing risks by applying interval analysis. Section 2, the structure of the research method a three-step method includes a collection of risk data, Jackknife Resampling Method and interval analysis. Section 3, the application of this method in the bridge construction project is presented as a case study followed by final risk grading. The fourth section goes to the results of calculations and a comparison of the results when using Jackknife Sampling Method and Bootstrap. Finally, the results of this research and certain proposals for making more researches are presented in the fifth part.

2 Methodology

Nonparametric Jackknife Method is one of the resampling methods in which by certain observations as original sample, a proper method for estimation of society variance and calculation of confidence interval is given when there is not a lot of data concerning the society [23]. In this paper, risks happen in three phases of identification, estimation and grading. In the first phase risk data are collected. Identification and collecting risk data shall be made by means of methods like an interview, questionnaire, brainstorming and historical data [4, 15].

In the first phase, after identification of risks, two criteria of risk assessment including risk probability and risk impact shall be calculated. The definitions of two commonly used criteria of risk assessment are as follows [16, 22].

Probability. Risk probability assessment investigates likelihood that each specific risk will occur.

Impact. Risk impact assessment investigates potential effects on a project objective, including time, cost, quality and HSE.

Upon identification of potential risks, in the second phase, since determination of exact risk quantity is difficult and complicated, the confidence interval will be calculated by means of the Jackknife Resampling Method, in which the collected data shall be used as original sample, as follows [23]:

Step 1) Determination of observations and defined risk data as original sample. In this paper, the risk data is utilized in terms of the impacts on a five-point descriptive scale (Table 3). Original sample X = {x₁, x₂, …, x_n}

Step 2) Determination of Jackknife Samples (in this sample, an observation i has been omitted from the original sample) =X_-i

Step 3) Calculation of average Jackknife Samples (samples from which one of the observations have been omitted) =θ_i

Step 4) Estimation of average ( ${\hat{θ}}^{*}$ ) using θ_i and Equation (1) $\overset{⌢}{θ^{*}} = \frac{1}{n} \sum θ_{i}$ (1)

Step 5) Calculation of standard deviation of Jackknife Samples average using Equation (2) $S^{*} = \sqrt{\frac{1}{n - 1} \sum (θ i - \overset{⌢}{θ^{*}})^{2}}$ (2)

Step 6) Calculating confidence interval using Equation (3), in which α is reliability and t with (n - 1) is Degree of Freedom. $\overset{⌢}{θ^{*}} \pm t_{α / 2} . (S^{*} / \sqrt{n})$ (3)

Phase three, itself is divided into two steps. In the first step of phase 3, compute interval risk score (IRS).

$\begin{matrix} IRS & = & P \times I = [p^{L}, p^{U}] \times [i^{L}, i^{U}] \\ = & [min (p^{L} i^{L}; p^{L} i^{U}; p^{U} i^{L}; p^{U} i^{U}), \\ max (p^{L} i^{L}; p^{L} i^{U}; p^{U} i^{L}; p^{U} i^{U})] \end{matrix}$ (4) in which L is lower bound and U upper bound of the interval. An important risk which has crucial impact on the project objective, such as scope, time, cost, quality and HSE, are taken into account. A definition of multiplication for interval numbers for each criterion, represented as pairs of real numbers, follows from elementary property of the inequality relation [17]. Final risk grading shall be done in the second step of phase 3. In order to rank the risks, we need to compare IRSs. Suppose that ${IRS}_{1} = [{rs}_{1}^{L}, {rs}_{1}^{U}]$ and ${IRS}_{1} = [{rs}_{2}^{L}, {rs}_{2}^{U}]$ are two interval risk scores (IRS) that we want to calculate their minimum IRS. We have the four following situations [24]:

If interval risk scores (IRSs) do not have any intersection, minimum IRS shall be the interval with lower quantities. In other words, if ${rs}_{1}^{U} \leq {rs}_{2}^{L}$ , we choose IRS₁ as minimum IRS.

Final risk grading shall be done in the second step of phase 3.

If both IRSs are the same, both have equal priority.

When ${rs}_{1}^{L} \leq {rs}_{2}^{L} < {rs}_{2}^{U} \leq {rs}_{1}^{U}$ , then minimum IRS shall be chosen as follows:

If γ

({rs}_{2}^{L} - {rs}_{1}^{L}) \geq (1 - γ) ({rs}_{1}^{U} - {rs}_{2}^{U})

, then IRS₁ will be our minimum; otherwise, IRS₂ will be equal to minimum IRS.

When ${rs}_{1}^{L} < {rs}_{2}^{L} < {rs}_{1}^{U} \leq {rs}_{2}^{U}$ , If $γ ({rs}_{2}^{L} - {rs}_{1}^{L}) \geq (1 - γ) ({rs}_{2}^{U} - {rs}_{1}^{U})$ , then IRS₂ will be minimum IRS and otherwise, the minimum will be IRS₁.

Here γ determines the level of expert or decision-maker optimism and its quantity is 0 < γ ≤ 1.

3 Experimental study

In this section, the study is used application of Jackknife Resampling Method and interval analysis for risk estimation of a bridge construction project to illustrate the proposed methods.

Step 1. Collection risk data

Collecting risk data includes identification of potential risks of the bridge construction project. In the first step, the project manager, design engineers team and risk management specialists have participated for identifying risks and drawing risk breakdown structure. Figure 1 shows the calculated breakdown structure for design step.

In the next step, risks with high probability rate and vital shock intensity in the goals of the bridge construction project, will be chosen by the experts and other defined risks with minimum effects shall not be considered. A list of important risks is presented in Table 1.

Opinions of four experts or decision-makers (DM) concerning estimation of risk criteria (P and I) is presented in Table 4. Since this data includes language terminology, first this language terminology shall be changed to numerical quantities by the relevant ratio in the Tables 2 and 3 [16].

By changing the language terminology in Table 4 using Tables 2 and 3, the results shall be considered as original sample. These results have been indicated in Table 5.

Step 2. Using resampling technique for determination of confidence interval

Using the Jackknife method at this stage (Equations 1–3), a 95% distance (α= 0.05) for risk standards (P and I) has been calculated. Considering n = 4, the intervals calculated for P and I have been indicated in the Table 6.

Step 3. Determination of risk interval score and risks grading

Risk interval scores have been calculated in Step 1 of phase 3, using Equation (4) and the results are indicated in Table 7.

Then considering the given solution in Step 2 of phase 3 as well as (γ= 0.8), final risk grading for all decision-makers shall be: $R_{9} > R_{3} > R_{8} > R_{7} > R_{2} > R_{6} > R_{5} > R_{1} > R_{4}$

4 Comparison of risk assessment results by jackknife and bootstrap sampling method

Standard deviation of original samples has been calculated using Equations (5) and (6) and Jackknife Sampling Method has also been applied for P and I standards. The calculated standard deviations together with a Bootstrap samples standard deviation for 500 repetitions have been indicated in the Tables 8 and 9 [21]. $S_{o} = \sqrt{\frac{1}{n - 1} \sum {(X_{i} - θ_{o})}^{2}}$ (5) $S^{*} = \sqrt{\frac{1}{n - 1} \sum (θ i - \overset{⌢}{θ^{*}})^{2}}$ (6)

In Equation (5), S₀ is the standard deviation of the original sample and θ₀ is the average of the original sample. In equation (6)S^* is Jackknife standard deviation of the sample mean. The standard deviation decrease percentage shall be calculated using Equations (7) and (8) for Bootstrap and Jackknife samples that are indicated in the Tables 10 and 11 [21]. ${SD}_{red} % = \frac{{SD}_{original} - {SD}_{Bootstrap}}{{SD}_{original}} \times 100$ (7) ${SD}_{red} % = \frac{{SD}_{original} - {SD}_{jackknife}}{{SD}_{original}} \times 100$ (8)

As indicated by the comparison of Tables 10 and 11 as well as Figs. 2 and 3, standard deviation of Jackknife Method samples are less than the standard deviation of original samples and Bootstrap samples. This shows that accuracy of the Jackknife Sampling Method is more than that of the Bootstrap method; accordingly, it presents more accurate grading and shall be used when accurate risk estimation and grading is required.

5 Conclusion

Risk assessment is essential for success in major projects management and performance. This paper presents a solution for risk grading of major projects, using Jackknife Resampling Method in three phases including collection of risk data, calculation of confidence intervals by Jackknife Method (for determination of exact risk quantity is difficult and complicated in reality) and finally calculation of risk interval scores. Application of this method in a bridge construction project was also studied and finally the results of Jackknife and Bootstrap sampling methods were compared. By this comparison it was found that jackknife resampling method presents more decrease percentage in standard deviation than Bootstrap method and will therefore give more accurate risk grading. It shall then be used when accurate risk estimation and grading is required. Furthermore, Jackknife method shall be a useful one when there are less data of the society (particularly because we it is the case in reality). It shall be proposed that a comparison of this method with other sampling methods be made in future researches.

References

California Department of Transportation (Caltrans), Project Risk Management Handbook, fourth ed. Caltrans, Office of Project Management Process Improvement, Sacramento, CA, 2007.

Carr

and Tah

J.H.M.

, A fuzzy approach to construction project risk assessment and analysis: Construction risk management system, Advances in Engineering Software32 (2001), 847–857. DOI: 10.1016/S0965-9978(01)00036-9

Ebrahimnejad

, Mousavi

S.M.

and Mojtahedi

S.M.H.

, A fuzzy BOT project risk evaluation model in Iranian power plant industry, Singapore, In: Proc, 5th IEEE Int, Conf on Ind Eng Eng Management, 2008, pp. 1038–1042.

Ebrahimnejad

, Mousavi

S.M.

and Mojtahedi

S.M.H.

, A fuzzy decision making model for risk ranking with application to the onshore gas refinery, International Journal of Business Continuity and Risk Management1 (2009), 38–66. DOI: 10.1504/IJBCRM.2009.028950

Fang

, Bayesian theory and robust control strategy in risk management oflarge-scale engineering project, In: Proc, Int Conf Manage Ser Sci, IEEE, 2009, pp. 1–7. DOI: 10.1109/ICMSS.2009.5301136

Faucher

and Fitzgibbons

, Public demand and the management of technological risk in large-scale projects, Science and Public Policy20(3) (1993), 173–185. DOI: 10.1093/spp/20.3.173

Floricel

and Miller

, Strategizing for anticipated risks and turbulence in large-scale engineering projects, International Journal of Project Management19(8) (2001), 445–455. DOI: 10.1016/S0263-7863(01)00047-3

Freund

and Walpole

, Mathematical Statistics, Prentice Hall, Third Edition, 1980.

Hastak

and Baim

E.J.

, Risk factors affecting management and maintenance cost of urban infrastructure, Journal of Infrastructure Systems7(2) (2001), 67–76. DOI: 10.1061/(ASCE)1076-0342

10.

Lee

, Park

and Shin

J.G.

, Large engineering project risk management using a Bayesian belief network, Expert Systems with Applications36 (2009), 5880–5887. DOI: 10.1016/j.eswa.2008.07.057

11.

Long

N.D.

, Ogunlana

, Quang

and Lam

K.C.

, Large construction projects in developing countries: A case study from Vietnam, International Journal of Project Management22(7) (2004), 553–561. DOI: 10.1016/j.ijproman.2004.03.004

12.

Makui

, Mojtahedi

S.M.H.

and Mousavi

S.M.

, Project risk identification and analysis based on group decision making methodology in a fuzzy environment, International Journal of Management Science and Engineering Management5(2) (2010), 108–118. DOI: 10.1080/17509653.2010.10671098

13.

Malekly

, Mousavi

S.M.

and Hashemi

, A fuzzy integrated methodology for evaluating conceptual bridge design, Expert Systems with Applications37 (2010), 4910–4920. DOI: 10.1016/j.eswa.2009.12.024

14.

Miller

and Lessard

, Understanding and managing risks in large engineering projects, International Journal of Project Management19(8) (2001), 437–443. DOI: 10.2139/ssrn.289260

15.

Mojtahedi

S.M.H.

, Mousavi

S.M.

and Makui

, Risk identification and analysis concurrently: Group decision making approach, In: Proc, 4th IEEE Int, Conf Manag Innov Technol (ICMIT), Thailand, 2008, pp. 299–304.

16.

Mojtahedi

S.M.H.

, Mousavi

S.M.

and Makui

, Project risk identification and assessment simultaneously using multi-attribute group decision making technique, Safety Science48(4) (2010), 499–507. DOI: 10.1016/j.ssci.2009.12.016

17.

Moore

R.E.

, Methods and Applications of Interval Analysis, Society for Industrial and Applied Mathematics, Philadelphia, PA, 1979.

18.

Morris

P.W.G.

and Pinto

J.K.

, The Wiley Guide to Managing Projects, John Wiley & Sons, Chichester, UK, 2004.

19.

Motawa

I.A.

, Anumba

C.J.

and El-Hamalawi

, A fuzzy system for evaluating the risk of change in construction projects, Advances in Engineering Software37 (2006), 583–591. DOI: 10.1016/j.advengsoft.2006.01.006

20.

Mousavi

S.M.H.

, Malekly

, Hashemi

and Mojtahedi

S.M.H.

, A two-phase fuzzy decision making methodology for bridge scheme selection, In: Proc 5th IEEE Int, Conf Ind Eng Eng Manag, Singapore, 2008.

21.

Mousavi

, Tavakkoli-Moghaddam

, Hashemi

S.M.H.

and Mojtahedi , A novel approach based on non-parametric resampling with interval analysis for large engineering project risks, Safety Science49 (2011), 340–1348. DOI: 10.1016/j.ssci.2011.05.004

22.

Project Management Institute, A Guide to the Project Management Body of Knowledge (PMBOK Guide), fourth ed, Proj Manage Inst, Newton Square, PA, 2008.

23.

Ramachandran

K.M.

and Tsokos

C.P.

, Mathematical statistics with applications, Academic Press, Amsterdam, Boston, 2009.

24.

Sayadi

M.K.

, Heydari

and Shahanaghi

, Extension of VIKOR method for decision making problem with interval numbers, Applied Mathematical Modelling33(5) (2009), 2257–2262. DOI: 10.1016/j.apm.2008.06.002

25.

Tah

J.H.M.

and Carr

, Towards a framework for project risk knowledge management in the construction supply chain, Advances in Engineering Software32 (2001), 835–846. DOI: 10.1016/S0965-9978(01)00035-7

26.

Tavakkoli-Moghaddam

, Azaron

, Mousavi

S.M.

, Mojtahedi

S.M.H.

and Hashemi

, Risk assessment for highway projects using jackknife technique, Expert Systems with Applications38 (2011), 5514–5524. DOI: 10.1016/j.eswa.2010.10.085

27.

Tse

C.K.

, Wong

W.M.

and Chu

C.H.

, Risk consideration in the handling of a large scale engineering project, In: Proc, 4th Int Struct Eng Const Conf (ISEC4), 2008, pp. 1501–1506.

28.

Zayed

, Amer

and Pan

, Assessing risk and uncertainty inherent in Chinese highway projects using AHP, International Journal of Project Management26(4) (2008), 408–419. DOI: 10.1016/j.ijproman.2007.05.012

29.

Zhang

, International Project Risk Management, Master Degree Thesis. Professional Management Science and Engineering Unit of Tianjin University, 2004.

30.

Zhou

H.-B.

and Zhang

, Dynamic risk management system for large project construction in china, In: Proc, Geo Florida Conf Adv Anal Model Des, 2010, pp. 1992–2001. DOI: 10.1061/41095(365)202

31.

Zoubir

A.M.

and Boashash

, The bootstrap and its application in signal processing, IEEE Signal Processing Magazine15(1) (1998), 56–76. DOI: 10.1109/79.647043