An integrated technology pushing and requirement pulling model for weapon system portfolio selection in defence acquisition and manufacturing

Abstract

An integrated technology pushing and requirement pulling model is presented to address the problem of selecting a suitable portfolio from several candidate multi-function weapon systems in the defence acquisition and manufacturing process. The optimal weapon system portfolios both satisfy the technical effects and compound capability requirements with relative acquisition and manufacturing constraints. In the first stage, multiple criteria analysis is employed to build a portfolio value hierarchal structure from two main perspectives: a five-level set of technology pushing measures and a two-level set of requirement pulling measures. The developmental maturity level of a technology, its functionality, systems, capability, and portfolio are all captured by technology pushing measures, based on an additive multiple criteria model. The requirement satisfaction level of a system or portfolio is obtained by requirement pulling measures based on linear programming optimization and the first-ignorance model, which is founded on expert opinion, using pairwise comparisons. In the second stage, by means of manufacturing cost consumption and capability requirement constraint assumptions, the dominance structure of the portfolios is designed and all the technology and requirement portfolios are generated as a set of Pareto-optimal solutions, through multi-objective integer programming. An illustrative case study is presented to validate the efficiency of the proposed model, and the computational results are further analysed and discussed.

Keywords

Weapon system portfolio selection technology pushing and requirement pulling model portfolio value analysis multi-objective programming

Introduction

In an annual research report, Washington announced a marginal increase in less than 1% in its annual military budget, which resulted in a total annual military budget of approximately US$623 billion in fiscal year 2013. However, the US military budget has been above US$600 billion for 6 years, from 2008 to 2013.¹ The sustained annual defence spending planned by the Department of Defense (DoD) has been used to improve weapon system products research, development, production, and manufacturing capabilities to a far greater extent than any other organizations in the world.² As a national decision, weapon system selection (WSS) in defence acquisition and manufacturing is vital for improving a nation’s ability to defend its domestic and international security and interests. It can even be considered as crucial as the outcome of a real war³ Moreover, defence investment budgets have been increased in many countries for new weapon system development and manufacturing, with the aim of reducing the countries’ reliance on foreign weapons procurement in more areas as their defence manufacturing capabilities and technologies mature.^2,3

Presently, with an increasing military requirement for joint operations, the ubiquitous interdependence and interaction among systems led the WSS to become System of Systems (SoS) engineering by considering the overall benefit of the selected systems. The traditional emphasis, however, has primarily been on increasingly sophisticated, high-tech system production processes in a certain field, rather than a systematic high-level planning decision.⁴ Once a new system is manufactured and added to the existing weapon SoS, the traditional development mode in many cases results in subsequent modifications and revisions that waste the defence acquisition budget and slow the weapon SoS construction process. The system portfolio effect is becoming increasingly profound such that it cannot be ignored in the establishment and development of the weapon SoS project especially in the acquisition and manufacturing process, which is highly sensitive to variations in cost and time. Therefore, portfolio decision-making is required in the acquisition and manufacturing of multiple weapon systems for weapon SoS development.

A portfolio decision analysis (PDA) is a combination of decision analysis and portfolio management for assisting decision makers (DMs) to make an informed evaluation and selection of a subset of competing alternatives, with the aim of contributing to the attainment of multiple, incommensurate, and often conflicting objectives, and in the presence of limited resources and other constraints.⁵ Many of the portfolio decision-making problems facing the DMs in military organizations are similar to those of civil and commercial organizations. Often, they face constraints such as limited defence budgets and multiple, incommensurate, or even conflicting objectives.^2,5 However, the defence acquisition and manufacturing procedure, simplified as a weapon system portfolio selection (WSPS) problem in this study, is more challenging because of (1) safety reasons, as it is often difficult to test military systems (e.g. weapon of mass destruction) and, therefore, to ascertain their value in realistic operational conditions;⁶ (2) the strong relationship between the continual development of advanced technology related to the manufacturing process and predicting future requirements of the military; and (3) the hostile environment, which makes human factors especially important since levels of personal decision risk are often much higher than would be acceptable in civil applications.⁶ Therefore, with regard to WSPS, it is necessary to conduct a sound, evidence-based value analysis and take full advantage of the suggestions from DMs in the multi-objective portfolio decision process. This is also the research motivation of this article.

The main contribution of this study is to model and solve the WSPS problem with an integrated technology pushing and requirement pulling (TPRP) methodology framework, which comprises two stages: weapon system portfolio value modelling and WSPS solving based on multi-objective programming. Emphasizing the inherent value structure of a weapon systems portfolio from the technology push and requirement pull perspectives, this approach involves first designing a hierarchal TPRP value model with a five-level set of technology pushing measures (TPMs) and a two-level set of requirement pulling measures (RPMs). With the hierarchal TPRP value model, it accounts for the assessment of the portfolio readiness level (PRL) of a weapon system portfolio through the technology readiness level (TRL)/integration readiness level (IRL) of the related manufacturing technology and the acquisition of the portfolio satisfaction level (PSL)/system satisfaction level (SSL) through experts’ judgments. Interval probability is used to determine SSL from DMs, which will permit key stakeholders to interact independently and simultaneously to achieve more realistic, comfortable, and more sound decisions, even if the future capability requirement is not initially clear. Besides, the approach aims at solving the WSPS problem and determining the best options at the second stage, called technology and requirement (T&R) portfolios in this study, through multi-objective programming. Some key components that are necessary in this stage include system manufacturing cost and capability constraints, a feasible portfolio generation method, and a dominance structure design.

The remainder of this article is organized as follows. Section ‘Literature review’ presents the literature review, previous theories and approaches, and their relationship with our methodology. Section ‘Methodological development’ presents the portfolio selection methodology. Section ‘Weapon system portfolio value modelling’ describes weapon system portfolio value modelling from two perspectives: TPMs and RPMs. Section ‘WSPS problem solving based on multi-objective programming’ introduces WSPS problem solving based on multi-objective programming, which comprises constraint assumptions and feasible portfolio generation, dominance structure design, and computation of T&R portfolios. An illustrative case study with T&R portfolio analysis is conducted in section ‘An illustrative case study ’. Finally, the main conclusions are presented in section ‘Conclusion’.

Literature review

The portfolio selection theory proposed by Markowitz⁷ heralded a new era in the study of finance using mathematical tools. It has been subsequently applied by many researchers to select a single optimal alternative out of many, in fields such as research and development (R&D) project selection,⁸ capital budgeting in healthcare,⁹ and defence acquisition and manufacturing.^{2,3,6,10–17} With regard to the relevant literature on WSS in defence acquisition and manufacturing, weapon system screening and weapon system ranking enjoy more enduring interest than WSPS. To the best of our knowledge, portfolio selection theory was first applied in the military context by Buede and Bresnick¹⁰ for military project selection. Subsequently, several studies on military portfolio selection emerged, and portfolio selection theory began to be widely applied in the defence acquisition and manufacturing field.

The past 20 years of research on military portfolio selection has essentially focused on WSPS. Although it is frequently termed as military project selection, the project concerns the selection of a new weapon system or the development of an army installation portfolio. The most common analysis techniques include multiple-objective analysis,^2,6,10–16 multiple criteria analysis (MCA),¹⁷ value analysis,^11–16 optimization,^3,13–17 cost-efficiency analysis,^16,17 and expert judgments.^10,12,16 Additionally, there are other, less frequently utilized techniques such as the Monte Carlo technique,¹⁵ risk analysis,^13–16 Pareto analysis,^13,14,17 and resource allocation techniques.^10,14 Clearly, with an emphasis on mathematical and quantitative elements, the foregoing military portfolio selection processes usually rely heavily on the balance of constraints and value optimization, in which the budget and the additive value model are the most common options. It is clear that with so many existing methods, extensive effort has been incorporated into portfolio maximization addressing multiple objectives. However, as shown in Table 1, the representative contributions made by past research deserve more attention.

Table 1.

Representative portfolio selection models for weapon product acquisition and manufacturing.

Authors and year	Analysis techniques	Value model	Portfolio value calculation	Constraints	Main contribution
Buede and Bresnick¹⁰	Multiple-objective; value analysis; resource allocation techniques; decision conferencing	Balance beam technique used to holistically assess value	Sum of project value	Budget	The earlier military application of portfolio analysis in the past decades
Parnell et al.^13,14	Multiple-objective; value analysis; Pareto analysis; risk analysis	Valued from multiple perspectives	Optimization model to maximize the sum of the project value	Budget	Pareto analysis, optimization model, and future capability needs are considered first
Parnell et al.¹⁵	Multiple-objective; value analysis; Monte Carlo risk analysis; optimization; value-focused thinking model	A space technology value model to identify the best S&T projects	Nonlinear aggregation model of project value into portfolio value.	Budget	Monte Carlo risk analysis and technologies contained in the projects are considered primarily as value measures
Buckshaw et al.¹⁶	Multiple-objective value analysis; risk analysis; optimization; cost–benefit analysis	Expert opinions from various disciplines	Quantitative risk assessment model	Budget	Cost–benefit analysis of weapon system is considered first
Kangaspunta et al.¹⁷	Multiple criteria analysis; optimization; cost-efficiency analysis	Additive value model with four impact attributes	Optimization model to maximize the efficiency and minimize the cost	Budget	Combat simulation is primary to estimate the dependencies among the weapon systems

S&T: science and technology.

First, while multiple objectives are often transformed into a single objective by using the multiple-objective decision-making (MODM) method, Parnell et al.¹³ extended the multiple-objective analysis using the Pareto method and the optimization model to maximize the project’s additive value for future needs. In the past, studies have contributed to some relatively efficient methods^2,3,17,18 to search the entire Pareto-optimal set from the feasible portfolio space. Because of the need for multiple objectives, the computation of non-dominated portfolios is often performed by identifying all Pareto-optimal solutions of a multiple-objective integer programming problem. However, the pairwise comparisons in the search process will exponentially increase, thereby making the computation more time-consuming when the number of feasible system portfolios is large. A sorting strategy with regard to cost, presented by Kung et al.¹⁹ and Deb,²⁰ has proven to be efficient in identifying the Pareto-optimal solutions from vast amounts of feasible options,¹⁷ which is also employed in this study.

Second, the Monte Carlo risk analysis and related technologies^14,15 are first considered as value measures in military portfolio selection. The technology effect is an important factor in determining which military portfolio meets the technique-feasibility objective as regards the defence budget constraint. A common term, TRL,^21,22 has been used across many US government agencies to assess the developmental maturity of evolving technologies before incorporating them into a system manufacturing process.^23,24

Third, Buckshaw et al.¹⁶ proposed a methodology based on the judgment of experts from various disciplines, coupled with a cost–benefit analysis of the weapon systems. Cost–benefit analysis of a weapon system is considered first in WSS. With the capability-based planning (CBP) concept experiencing considerable attention in applications for defence and military acquisitions over the past few years, especially in the planning of weapon systems procurement and manufacturing,^25–27 a weapon system that meets future military requirements is considered vital. Capability is defined as the ability to achieve a desired effect under specified conditions, through a combination of systems with corresponding functions that have been created by technology integration, to perform a set of tasks.^28,29 Moreover, with regard to the judgment of experts from various disciplines, many of the scores of MCA from the literature are, to some extent, inaccurate and tend to encompass more qualitative comparison judgments than the conclusions of a precise point-valued estimation.^30,31

Recently, Kangaspunta et al.¹⁷ identified the value of a weapon system portfolio through combat simulation, to estimate interdependencies. Interdependencies among the systems in a portfolio for defence acquisition and manufacturing can be considered as the integration of technologies based on the ‘techniques view’, and the integration of functions based on the ‘capability view’. IRL³² (which, like TRL, is denoted as a metric) was first proposed by Gove and colleagues^33,34 to evaluate the integration readiness between the two technologies and can be used with a TRL to determine the system readiness level (SRL). Therefore, in this study, an SRL that incorporates the current TRL and IRL scales is employed to measure the technology push effect on a system. However, the multi-function trait of the current system makes the analysis of SRL even more complex. Ramirez-Marquez and Sauser^{22–24,32,33,35} have elaborated extensively upon the rationale behind the SRL. For a system with multiple functions and multiple capabilities (MFMC), Tan et al.^24,35 proposed a hierarchical SRL, where the SRL is defined at three different levels: capability-based SRL (SRL_C), function-based SRL (SRL_F), and whole system-based SRL (composite SRL).

Above all, the literatures are devoid of solutions for measuring the inherent value of a weapon system portfolio by the static property of the system portfolio, rather than with merely subjective criteria.^10,13,16 Even when the capability needs and technologies of a military are considered in WSPS by researchers,^13–15 the two factors are not given enough weightage to adequately support the entire decision-making process. In addition, benefiting from advanced technologies and integrations, many multi-function weapon systems have been developed and utilized to meet the demand for multi-capability products. It is easy to see that the groundwork of the valuation of weapon systems lies in the maturity of technological development and the achievement of desired military capabilities. Moreover, the traditional approaches failed to recognize the relationship between the manufacturing cost super-addition and SRL, called SSL improvement, considering the cost as only comprising the weapon system price. Finally, an ignorance of the ambiguous information on future capability requirements leads to the low credibility of the point-value scores obtained by experts’ judgment.

Methodological development

The overall TPRP model framework for the WSPS problem in defence acquisition and manufacturing is shown in Figure 1. It clearly reveals that the framework comprises two stages: weapon system portfolio value modelling and WSPS solving based on multi-objective programming. Then, the illustration of the main components of the framework and the reasons for choosing it to solve this problem are given as follows:

1. Weapon system portfolio value modelling.

The military application of PDA often employs value instead of utility, as a measure of the system or portfolio,^11–16 the definition of value is determined by the decision objectives. We formulate the weapon system portfolio value model, TPRP, from two perspectives: first, from the technology pushing perspective, and second, from the requirement pulling perspective. A hierarchical TPRP value model is composed of five-level TPMs and two-level RPMs, which can also be primarily classified into two levels: system level and portfolio level. During the five-level TPMs, the TRL and IRL at the same level are the foundations for acquiring the other levels’ measures (function readiness level (FRL), SRL, capability readiness level (CRL), and PRL). TRL and IRL can be obtained as scores through a comparison between the current technology and integration readiness state, and fixed standards. Multiple criteria decision analysis (MCDA) and the additive model are employed in the calculation of FRL, SRL, CRL, and PRL. Without loss of generality, the weights of each sub-criterion at the same level are considered the same. With respect to the two-level requirement pull measures (RPMs), they depend on experts’ judgment to such an extent that the point value and additive average models cannot adequately describe the SSL and its relationship with the PSL. Therefore, interval probability is employed to describe SSL for a certain capability, and the three integration models, IM-Median, IM-Best, and IM-Worst, are built based on three different attitudes to calculate the SSL of all the systems in a portfolio into a PSL.

2. WSPS solving based on multi-objective programming.

WSPS solving based on multi-objective programming in this study is not focused on converting multiple objectives into a single one, but use Pareto analysis to obtain non-dominated portfolios because several recommendations for portfolios are more practical and useful for stakeholders than a single optimal portfolio is, in defence acquisition and manufacturing. During the acquisition and manufacturing process, the constraints of the weapon system portfolio, which are the manufacturing cost and capability types in this study, are used to generate a set of feasible system portfolios from candidate weapon systems. The dominance structure defines the dominated portfolios, which may be discarded from the candidate portfolios, retaining only the non-dominated ones that are referred to, in this study, as T&R system portfolios. The generation of a T&R system portfolio is obtained by maximizing the measures of the system portfolio, subject to the given capability requirement and manufacturing cost constraints.

Compared with the aforementioned solutions and models in the literature review, our portfolio selection methodology offers the following special abilities. First, it explicitly considers the current technology developments (TDs) and future military requirement factors in defence acquisition and manufacturing. Moreover, the impacts of these factors are obtained through not only impersonal TRL/IRL but also the DM’s simultaneous judgments. Second, the generation of a T&R system portfolio is obtained by maximizing the measures of the system portfolio, subject to the given capability requirements and manufacturing cost constraints. This multi-objective programming model and solving algorithm are generic so that the non-dominated system portfolio can be offered even with some other measures and manufacturing constraints. Third, our methodology accommodates an abundant and valuable T&R portfolio analysis including PRL/PSL comparison at different cost levels, key system portfolio analysis, and cost strategy influence analysis on weapon system acquisition and manufacturing. Finally, the methodology can optimize the available information to achieve an almost accurate decision when the future military application is ambiguous. This is very important and useful because, in practice, experts are more comfortable when expressing their preference than when given a precise result.

Weapon system portfolio value modelling

TPRP value model based on MCDA

A TPRP value model based on an MCDA, shown in Figure 2, depicts the measures of a system and a system portfolio from two different perspectives: technology push and requirement pull. The technology pushing and requirement pulling (TPRP) model extends the SRL hierarchy to two levels: portfolio level and system level. Thus, it is evident that the WSPS problem should balance the PRL and PSL of a candidate weapon system portfolio during the entire selection process. An ideal system portfolio is one that optimizes both the maturity and the extent of satisfaction with a certain capability requirement.

As discussed, the TPRP value model revealed that one should consider the following issues when obtaining the criteria value for each level.

TRL and IRL are foundational measurements. Because the FRL is influenced by low-level technologies and their integrations, it can be considered as a conjunct result of the TRL and IRL. Its definition and calculation are based on an additive value function, which is similar to that in the extant literature^23,32,33 about $SRL_C_{fk}$ . If there are $n'$ kinds of technologies contained in candidate systems, the TRL and IRL can be given in mathematical representations: $TR L_{i}$ is the TRL of technology i, where $i = 1, 2, \dots, n'$ , and $IR {L_{i}}_{j}$ is the IRL of technologies i and j. Considering that there are $n'$ technologies to form a function r, the readiness level of function r is denoted as $FR L_{r}$ , where $f, r \in N^{+}, r \leq f$ .

The calculation of SRL by FRL involves two assumptions: (1) the system developmental level is determined by the developmental level of the multiple functions it contains and (2) the different FRL values stem from the fact that individual functions have different effects on a system. Calculating an SRL, this process should take into account all the functions, and an additive value function is used in this process, which is similar to the definition of $SRL_F_{f}$ given by the literature.^22,34 Assume that m is the number of all the candidate systems and there are $f'$ functions within a system, $f' \in N^{+}, f' \leq f$ . Similarly, we select the first $f'$ statistics from all the f functions, as those contained in a system $S_{t}$ , $m, t \in N^{+}, t \leq m$ .

The understanding of the relationship between systems and capabilities in this study is similar to the definition in Department of Defence Architecture Framework Version 2.0 (DoDAF V2.0),²⁸ but different from Tan’s consideration in the hierarchical SRL.³⁴ When a portfolio is selected, its fitness is determined by the utilities supplied by a series of multi-function systems, based on various technologies and integrations. Calculating a CRL, this process should take into account all the functions containing a certain capability as its ‘subsystem’ measurement; the additive value function is also used in this process. The capability is a high-level emergence behaviour of a certain number of functions supplied by the corresponding systems. Thus, the CRL of a capability is determined by the FRL of all the constitutional functions within a capability. Assume that a capability comprises the first $f "$ functions, $f ", f \in N^{+}, f " \leq f$ .

There is only one final measure, PRL, to indicate the value of a weapon system portfolio from the technology pushing perspective. Obviously, the PRL can be measured by its ‘subsystem’ measurement, SRL, and its definition and calculation are similar with the composite SRL given in the literature.^24,35 Assume that p ( $p \in N^{+}$ ) is the number of all the possible system portfolios and there are $m'$ systems within a system portfolio, where $m' \in N^{+}, m' \leq m$ . We still select the first $m'$ statistics from all the m systems as those in a portfolio $P_{s}$ , where $s, p \in N^{+}, s \leq p$ .

SSL depends on ambiguous military requirements and experts’ judgments to such an extent that the point value and additive average models cannot adequately describe the SSL and its relationship with the PSL. Therefore, the interval probability is employed to describe SSL for a certain capability, and three integration models, IM-Median, IM-Best, and IM-Worst, are built based on three different attitudes to calculate the SSL of all the systems in a portfolio into a PSL. Considering that there are k ( $k \leq m$ ) systems in a set of candidate systems $S^{*} = {S_{t}, t = 1, 2, \dots, m}$ , having the corresponding functions to satisfy a certain capability requirement, $C_{u}$ , assume that there are $v$ kinds of capability requirements that should be satisfied by the possible weapon system portfolio, $u, v \in N^{+}, u \leq v$ .

The definition of PSL in a function form represents the extent of a weapon system portfolio to satisfy a certain capability. If there are multiple capability requirements, the number of PSL results of a weapon system portfolio is equal to that of the capability requirements. It worth noting that the component abilities of the multi-capability requirements, which represent the expected effects that the system portfolio should achieve in future operations and support, are coexistent with the others, but independent of each other. Therefore, we do not intend to integrate the different PSLs for multi-capability requirements into a single one. $PSL (P_{s}, C_{u})$ is an integration value of all the $SSL (S_{t}, C_{u})$ of the systems included in portfolio $P_{s}$ .

Here, a value model with $v + 1$ measured at the portfolio level is given with a hierarchy tree structure. The portfolio-level measures are obtained through the calculation of those measures at a system level. In this study, the value of a systems portfolio is indicated by the $v + 1$ measures in portfolio level: $PR L_{s}$ , $PSL (P_{s}, C_{1})$ , $PSL (P_{s}, C_{2})$ , ..., $PSL (P_{s}, C_{u})$ , ..., $PSL (P_{s}, C_{v})$ .

Five-level technology push measures via TRL and IRL

Consequently, the effect of technology pushing in this study is assessed by five technology pushing measures (TPMs): TRL, FRL, SRL, CRL, and PRL (see Table 2). The TRL and IRL are assumed to have the following properties:^{21–24,32–37}

A system has n technologies and any two technologies have potential for integration.

The TRL for any technology has 10 potential integral values, 0–9.

The IRL for any integration has 10 potential integral values, 0–9.

Table 2.

Five-level TPMs and their definitions.

Level	TPMs	Definition	Remark
I	TRL and IRL	An index of maturity from 0 to 9 applied at the technology level	Traditional definition
II	FRL	An index of maturity from 0 to 1 applied at the function level	Our definition
III	SRL	An index of maturity from 0 to 1 applied at the system level	Traditional definition
IV	CRL	An index of maturity from 0 to 1 applied at the capability level	Our definition
V	PRL	An index of maturity from 0 to 1 applied at the portfolio level	Our definition

TPM: technology push measure; TRL: technology readiness level; IRL: integration readiness level; FRL: function readiness level; SRL: system readiness level; CRL: capability readiness level; PRL: portfolio readiness level.

Assuming there are n technologies contained within the candidate systems, mathematically, the procedure for obtaining a PRL based on the five-level TPMs is as follows:

1. TRL is defined in equation (1) as a vector, and the value of $TR L_{i}$ is normalized to be between 0 and 1²⁴

\begin{matrix} {[TRL]}_{n \times 1} = [\begin{matrix} TR L_{1} \\ TR L_{2} \\ ⋮ \\ TR L_{n} \end{matrix}] \overset{Normalize}{\to} \\ {[TRL']}_{n \times 1} = \frac{TRL}{9} = [\begin{matrix} TR {L'}_{1} \\ TR {L'}_{2} \\ ⋮ \\ TR {L'}_{n} \end{matrix}] \end{matrix}

(1)

2. IRL is defined in equation (2) as a matrix, where $IR L_{ij}$ is the IRL between the technology i and technology j and the value of $IR L_{ij}$ is normalized to be between 0 and 1

\begin{matrix} {[IRL]}_{n \times n} = [\begin{matrix} IR L_{11} & IR L_{12} & \dots & IR L_{1 n} \\ IR L_{21} & IR L_{22} & \dots & IR L_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ IR L_{n 1} & IR L_{n 2} & \dots & IR L_{nn} \end{matrix}] = \overset{Normalize}{\to} \\ {[IRL']}_{n \times n} = [\begin{matrix} IR {L'}_{11} & IR {L'}_{12} & \dots & IR {L'}_{1 n} \\ IR {L'}_{21} & IR {L'}_{22} & \dots & IR {L'}_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ IR {L'}_{n 1} & IR {L'}_{n 2} & \dots & IR {L'}_{nn} \end{matrix}] \end{matrix}

(2)

Note that $IR L_{ii}$ denotes the hypothetical integration of the technology i to itself and $IR L_{ii} = 9$ .

3. To calculate the FRL of each function, we assume that there are f functions supplied by all the technologies and their integrations and the FRL of the function r is denoted as $FR L_{r}$ , where $f, r \in N^{+}, r \leq f$ .

At first, the product of the TRL and IRL matrices is denoted as ITRL matrix given as the following formula

ITRL = Norm \times IRL' \times TRL'

(3)

where Norm is a diagonal matrix $diag [1 / m_{1}, 1 / m_{2}, \dots, 1 / m_{i}, \dots, 1 / m_{n}]$ and m_i is the number of integrations of technology i with itself and all other technologies, with the diagonal matrix Norm, a normalized and consistent $ITR L_{i}$ scaled $[0, 1]$ , can be gained from the original with a $[0, m_{i}]$ scale.^22–24,32

We then assume that the technologies and integrations contained in a function are only subsets of all the technologies, set $T^{*}$ , and all the integrations, set $I^{*}$ . Specifically, the number of the technologies contained in $F_{r}$ is $n'$ and $n', n \in N^{+}, n' \leq n$ . Without loss of generality in the illustration of $F_{r}$ , we simply select the first $n'$ technologies as the technologies contained in function $F_{r}$ . So the $ITRL_F_{r}$ matrix, denoted as the ITRL of the function r, is the product of the ${[IRL'_F_{r}]}_{n' \times n'}$ and ${[TRL'_F_{r}]}_{n' \times 1}$ matrices

ITRL_F_{r} = {[Norm]}_{n' \times n'} \times {[IRL'_F_{r}]}_{n' \times n'} \times {[TRL'_F_{r}]}_{n' \times 1}

(4)

where ${[Norm]}_{n' \times n'}$ , ${[TRL'_F_{r}]}_{n' \times 1}$ , and ${[IRL'_F_{r}]}_{n' \times n'}$ are sub-matrices of ${[Norm]}_{n \times n}$ , ${[TRL']}_{n \times 1}$ , and ${[IRL']}_{n \times n}$ , respectively. Thus, equation (4) can be rewritten as follows

\begin{matrix} ITRL_F_{r} = [\begin{matrix} ITR L_{1} \\ ITR L_{2} \\ \dots \\ ITR L_{n'} \end{matrix}] \\ = [\begin{matrix} 1 / m_{1} & 0 & \dots & 0 \\ 0 & 1 / m_{2} & \dots & 0 \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 0 & 0 & \dots & 1 / m_{n'} \end{matrix}] \\ \times [\begin{matrix} IR {L'}_{11} & IR {L'}_{12} & \dots & IR {L'}_{1 n'} \\ IR {L'}_{21} & IR {L'}_{22} & \dots & IR {L'}_{2 n'} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ IR {L'}_{n' 1} & IR {L'}_{n' 2} & \dots & IR {L'}_{n' n'} \end{matrix}] \times [\begin{matrix} ITR L_{1} \\ ITR L_{2} \\ \dots \\ ITR L_{n'} \end{matrix}] \\ = [\begin{matrix} (IR {L'}_{11} TR {L'}_{1} + IR {L'}_{12} TR {L'}_{2} + \dots + IR {L'}_{1 n'} TR {L'}_{n'}) / m_{1} \\ (IR {L'}_{21} TR {L'}_{1} + IR {L'}_{22} TR {L'}_{2} + \dots + IR {L'}_{2 n'} TR {L'}_{n'}) / m_{2} \\ \dots \\ (IR {L'}_{n' 1} TR {L'}_{1} + IR {L'}_{n' 2} TR {L'}_{2} + \dots + IR {L'}_{n' n'} TR {L'}_{n'}) / m_{n'} \end{matrix}] \end{matrix}

(5)

Definition 1

Considering that there are $n'$ technologies to form a function r, the readiness level of function r, denoted as $FR L_{r}$ , can be defined by the following formula

FR L_{r} = \frac{1}{n'} \sum_{i = 1}^{n'} ITR L_{i}

(6)

4. Calculate the SRL of a system. The SRL of the system $S_{t}$ , denoted as $SR L_{t}$ , is the average of all the $f'$ FRL, defined as follows

SR L_{t} = \frac{1}{f'} \sum_{r = 1}^{f'} FR L_{r} = \frac{1}{f'} \frac{1}{n'} \sum_{r = 1}^{f'} \sum_{i = 1}^{n'} ITR L_{i}

(7)

5. Calculate the CRL of a capability.

Definition 2

The readiness level of capability $C_{u}$ , denoted as $CR L_{u}$ , can be defined by the formula

CR L_{u} = \frac{1}{f "} \sum_{r = 1}^{f "} FR L_{r} = \frac{1}{f "} \frac{1}{n'} \sum_{r = 1}^{f "} \sum_{i = 1}^{n'} ITR L_{i}

(8)

To the best of our knowledge, this definition and formula for the CRL are proposed by our study first. The core thoughts and calculation are also based on an additive value function and multiple criterion decision.

6. Calculate the PRL of a portfolio.

Definition 3

The readiness level of portfolio $P_{s}$ , denoted as $PR L_{s}$ , is the average of all the $t'$ SRLs and is formulated as follows

PR L_{s} = \frac{1}{m'} \sum_{t = 1}^{m'} SR L_{t} = \frac{1}{m'} \frac{1}{f'} \frac{1}{n'} \sum_{t = 1}^{m'} \sum_{r = 1}^{f'} \sum_{i = 1}^{n'} ITR L_{i}

(9)

The five-level TPM is a comprehensive and useful index to indicate where a weapon system portfolio is, in its developmental cycle. Corresponding to the SRL scale definitions in much of the literature,^23,24,35 the common definitions of SRL, PRL, and the corresponding terms are provided in Table 3.

Table 3.

The scale of SRL, PRL, and their definitions.

SRL (PRL)	Definition
0.90–1.00	Operation and support (OS)
0.80–0.89	Production and deployment (PD)
0.60–0.79	Engineering and manufacturing development (EMD)
0.40–0.59	Technology development (TD)
0.10–0.39	Concept refinement (CR)

SRL: system readiness level; PRL: portfolio readiness level.

Two-level RPMs via SSL and PSL

In this section, a two-level requirement pulling measures (RPMs) is proposed to indicate the overall satisfaction level of the system with multi-capability requirements, with two components: SSL and PSL. Presented first are the definitions of SSL and the procedure to obtain an SSL from expert opinions on system pairwise comparisons. Second, the PSL is defined and three integration models are proposed, corresponding to three different decision-making attitudes. Through the two-level RPMs, the satisfaction of a system portfolio with a certain capability can be acquired to indicate its value from a requirement pull perspective.

Since the SSL is a measure corresponding to a system and a certain capability, its expression is given as a function with two parameters. PSL is proposed in the same way.

Definition 4

$SSL (S_{t}, C_{u})$ is a system-level index indicating the possibility of a system, $S_{t}$ , satisfying a capability, $C_{u}$ , a requirement when the system is developed and utilized in a combat operation context.

The capability requirement is an expected result of the systems portfolio to execute a series of future tasks. Therefore, $SSL (S_{t}, C_{u})$ is an estimated value, based on the experiential knowledge of experts. Because of the unavoidable uncertainty of the prediction, $SSL (S_{t}, C_{u})$ is denoted as an interval number as follows

SSL (S_{t}, C_{u}) = [L_{t}^{{C_{u}}^{-}}, L_{t}^{{C_{u}}^{+}}]

(10)

where the upper and lower bounds of $SSL (S_{t}, C_{u})$ are $L_{t}^{{C_{u}}^{+}}$ and $L_{t}^{{C_{u}}^{-}}$ , restricted by the following inequalities

0 \leq L_{t}^{{C_{u}}^{-}} \leq L_{t}^{{C_{u}}^{+}} \leq 1

(11)

It is clear that $SSL (S_{t}, C_{u}) \in [0, 1]$ . If $L_{t}^{{C_{u}}^{-}} = L_{t}^{{C_{u}}^{+}}$ , $SSL (S_{t}, C_{u})$ degenerates into a real number. Furthermore, the centre and width of the interval probability, $SSL (S_{t}, C_{u})$ , are, respectively, defined as follows³⁸

m (SSL (S_{t}, C_{u})) = \frac{1}{2} (L_{t}^{{C_{u}}^{-}} + L_{t}^{{C_{u}}^{+}})

(12)

w (SSL (S_{t}, C_{u})) = L_{t}^{{C_{u}}^{+}} - L_{t}^{{C_{u}}^{-}}

(13)

For all the k systems, there are interval probability sets containing k elements $SSL {(S, C_{u})}^{*} = {SSL (S_{t}, C_{u}) = [L_{t}^{{C_{u}}^{-}}, L_{t}^{{C_{u}}^{+}}], t = 1, 2, \dots, k}$ , which represent all the possible $SSL (S_{t}, C_{u})$ of the candidate systems with the requirement for capability $C_{u}$ . Therefore, Definition 5 is given as follows:^31,39,40,47

Definition 5

For $\forall L_{t}^{C_{u}} \in [L_{t} {^{C_{u}}}^{-}, L_{t} {^{C_{u}}}^{+}]$ , there is an equation, $\sum_{t = 1}^{k} L_{t}^{C_{u}} = 1$ .

Theorem 1

The interval set $SSL {(S, C_{u})}^{*}$ satisfies Definition 5 if, and only if, the following conditions hold^31,41

\begin{matrix} L_{t}^{{C_{u}}^{+}} + L_{1}^{{C_{u}}^{-}} + \dots + L_{t - 1}^{{C_{u}}^{-}} \\ + L_{t + 1}^{{C_{u}}^{-}} + \dots + L_{k}^{{C_{u}}^{-}} \leq 1 \forall t = 1, 2, \dots, k \\ L_{t}^{{C_{u}}^{-}} + L_{1}^{{C_{u}}^{+}} + \dots + L_{t - 1}^{{C_{u}}^{+}} \\ + L_{t + 1}^{{C_{u}}^{+}} + \dots + L_{k}^{{C_{u}}^{+}} \geq 1 \forall t = 1, 2, \dots, k \end{matrix}

(14)

See Appendix 1 for the proof.

Definition 6

The first-ignorance of $SSL {(S, C_{u})}^{*}$ , denoted as $I^{1} (SSL {(S, C_{u})}^{*})$ , is defined by the average of the width of the intervals as follows^31,41

\begin{matrix} I^{1} (SSL {(S, C_{u})}^{*}) & = \frac{1}{k} \sum_{t = 1}^{k} w (SSL (S_{t}, C_{u})) \\ = \frac{1}{k} \sum_{t = 1}^{k} (L_{t}^{{C_{u}}^{+}} - L_{t}^{{C_{u}}^{-}}) \end{matrix}

(15)

Similar definitions and theorems have been used in the literature^39,40 as the constraints and operations of the interval probability.

When there is little information available for the experts to predict $SSL (S_{t}, C_{u})$ , a precise estimation of $L_{t}^{{C_{u}}^{-}}$ and $L_{t}^{{C_{u}}^{+}}$ is difficult to achieve. In fact, an expert is more comfortable stating a personal preference towards a set of systems by means of pairwise comparison, and determining which one has more possibility to satisfy a certain capability requirement. Many studies have already been conducted on handling preference information.^31,42,43 Wang and Elhag³⁸ introduce a goal programming model to obtain interval weights from imprecise preferences in MCDA. Guo and Tanaka³¹ elicit the interval-valued probabilities, based on a linear and quadratic programming model, from subjective pairwise comparisons for the likelihood among several events. The first-ignorance model and linear programming approach are employed in this study to minimize the imprecision of pairwise comparisons.

Considering the pairwise comparison process for each pair of candidate systems in a finite set $S^{*} = {S_{t}, t = 1, 2, \dots, m}$ , the possible judgment score from experts on systems $S_{t}$ and $S_{h}$ ( $t, h \in N^{+}, t, h \leq m$ ) is denoted as a_th, which is an integer number in the interval [1, 9].

As shown in Figure 3, an axis for a_th with a number 1–9 depicts the possible judgment scores from experts. a_th = 1 represents that S_t and S_h have the same possibility to satisfy a certain capability requirement, a_th = 3 indicates that S_t is fairly more likely to satisfy a certain capability requirement than S_h, a_th = 5 means that S_t is a little more likely to satisfy a certain capability requirement than S_h, a_th = 7 denotes that S_t is much more likely to satisfy a certain capability requirement than S_h, and when a_th = 9, S_t is most likely to satisfy a certain capability requirement. The other numbers, that is, 2, 4, 6, and 8 are used analogically. Additionally, an assumption must be noted to explain the relationship between a_th and a_ht.

Assumption 1

The comparison results a_th and a_ht hold the condition a_th*a_ht = 1.

This means that when an expert makes a comparison, it is not possible that in the first comparison S_t will be more likely to satisfy a certain capability requirement than S_h, but in the second comparison, S_h and S_t have the same possibility to satisfy a certain capability requirement.

Next, we have a $k \times k$ comparison matrix $A (C_{u})$ with regard to k systems, which can satisfy a certain capability, $C_{u}$ , requirement as follows^17,34

A {(c_{j})}_{k \times k} = [\begin{matrix} 1 & a_{12} & \dots & a_{1 k} \\ 1 / a_{12} & 1 & \dots & a_{2 k} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 1 / a_{1 k} & 1 / a_{2 k} & \dots & 1 \end{matrix}]

(16)

Assumption 2

Even if there are multiple experts, there is only one $k \times k$ comparison matrix $A (C_{u})$ for a certain capability, $C_{u}$ , which represents the integrated preference information after a conference discussion.

The interval ratio $SSL (S_{t}, C_{u}) / SSL (S_{h}, C_{u})$ can be calculated by interval arithmetic as follows^31,44,48

\frac{SSL (S_{t}, C_{u})}{SSL (S_{h}, C_{u})} = [\frac{L_{t}^{{C_{u}}^{-}}}{L_{h}^{{C_{u}}^{+}}}, \frac{L_{t}^{{C_{u}}^{+}}}{L_{h}^{{C_{u}}^{-}}}]

(17)

Assumption 3^31,44,45,48

The given pairwise comparison a_th should belong to the estimated interval ratio $SSL (S_{t}, C_{u}) / SSL (S_{h}, C_{u})$ , that is

a_{th} \in [\frac{L_{t}^{{C_{u}}^{-}}}{L_{h}^{{C_{u}}^{+}}}, \frac{L_{t}^{{C_{u}}^{+}}}{L_{h}^{{C_{u}}^{-}}}]

(18)

\Leftrightarrow \frac{L_{t}^{{C_{u}}^{-}}}{L_{h}^{{C_{u}}^{+}}} \leq a_{th} \leq \frac{L_{t}^{{C_{u}}^{+}}}{L_{h}^{{C_{u}}^{-}}}

(19)

\begin{matrix} \Leftrightarrow & {\begin{matrix} L_{t}^{{C_{u}}^{-}} - a_{th} L_{h}^{{C_{u}}^{+}} \leq 0 \\ L_{t}^{{C_{u}}^{+}} - a_{th} L_{h}^{{C_{u}}^{-}} \geq 0 \\ L_{t}^{{C_{u}}^{-}} \geq ε \end{matrix} \end{matrix}

(20)

where $ε$ is a very small positive real number.

To determine the interval set $SSL {(S, C_{u})}^{*}$ with the smallest first-ignorance (Definition 3), obtaining the interval probabilities from the expert opinion can be derived by the following optimization model^31,41,48

\begin{matrix} min_{L_{t} {^{C_{u}}}^{+}, L_{t} {^{C_{u}}}^{-}} I^{1} (SSL {(S, C_{u})}^{*}) = \frac{1}{k} \sum_{t = 1}^{k} (L_{t}^{{C_{u}}^{+}} - L_{t}^{{C_{u}}^{-}}) \\ s . t . L_{t}^{{C_{u}}^{+}} + L_{1}^{{C_{u}}^{-}} + \dots + L_{i - 1}^{{C_{u}}^{-}} \\ + L_{i + 1}^{{C_{u}}^{-}} + \dots + {L_{k}}^{{C_{u}}^{-}} \leq 1 \forall t = 1, 2, \dots, k \\ L_{t}^{{C_{u}}^{-}} + L_{1}^{{C_{u}}^{+}} + \dots + L_{t - 1}^{{C_{u}}^{+}} \\ + L_{t + 1}^{{C_{u}}^{+}} + \dots + L_{k}^{{C_{u}}^{+}} \geq 1 \forall t = 1, 2, \dots, k \\ L_{t}^{{C_{u}}^{-}} - a_{th} L_{h}^{{C_{u}}^{+}} \leq 0 \forall (t = 1, 2, \dots, k, h > t) \\ L_{t}^{{C_{u}}^{+}} - a_{th} L_{h}^{{C_{u}}^{-}} \geq 0 \forall (t = 1, 2, \dots, k, h > t) \\ L_{t}^{{C_{u}}^{-}} \geq ε \forall t = 1, 2, \dots, k \\ L_{t}^{{C_{u}}^{+}} - L_{t}^{{C_{u}}^{-}} \geq 0 \forall t = 1, 2, \dots, k \end{matrix}

(21)

Definition 7

The PSL, $PSL (P_{s}, C_{u})$ , is a portfolio-level index indicating the possibility of a portfolio, $P_{s}$ , satisfies a capability, $C_{u}$ , a requirement when the portfolio is developed and utilized in a combat operation scenario.

$PSL (P_{s}, C_{u})$ is an integration value of all the $SSL (S_{t}, C_{u})$ of the systems included in portfolio $P_{s}$ . For the stability of the integration model, and considering any interactions among weapon systems in a portfolio, three possible criteria are assumed to indicate three different decision-making attitudes towards the integration process, which are the median criterion, best criterion, and worst criterion.¹⁸ The three different criteria are formulated as three integration models, IM-Median, IM-Best, and IM-Worst, which are as follows

[IM-Median Model]

\underset{IM - Median}{PSL (P_{s}, C_{u})} = \frac{\sum_{t = 1}^{\underset{P_{s}}{N (s)}} m (SSL (S_{t}, C_{u}))}{\underset{P_{s}}{N (s)}}

(22)

[IM-Best Model]

\underset{IM - Best}{PSL (P_{s}, C_{u})} = Max {L_{t}^{{C_{u}}^{+}}, t = 1, 2, \dots, \underset{P_{s}}{N (s)}}

(23)

[IM-Worst Model]

\underset{IM - Worst}{PSL (P_{s}, C_{u})} = Min {L_{t} {^{C_{u}}}^{-}, t = 1, 2, \dots, \underset{P_{s}}{N (s)}}

(24)

where $\underset{P_{s}}{N (s)}$ is the number of the weapon systems in a portfolio $P_{s}$ . Finally, the $PSL (P_{s}, C_{u})$ of a portfolio $P_{s}$ to capability $C_{u}$ is given as three decision proposals for DMs, based on the above models.

In fact, equations (23) and (24) define the upper and lower bounds of a $PSL (P_{s}, C_{u})$ with regard to all DMs. Therefore, $PSL (P_{s}, C_{u})$ can also be denoted as an interval number as follows

PSL (P_{s}, C_{u}) = [P {L_{s}}^{{C_{u}}^{-}}, P {L_{s}}^{{C_{u}}^{+}}]

where the upper and lower bounds, $P L_{s}^{{C_{u}}^{+}}$ and $P L_{s}^{{C_{u}}^{-}}$ , hold the following conditions

\begin{matrix} 0 \leq P L_{s}^{{C_{u}}^{-}} \leq P L_{s}^{{C_{u}}^{+}} \leq 1 \\ P L_{s}^{{C_{u}}^{-}} = \underset{IM - Worst}{PSL (P_{s}, C_{u})} \\ P L_{s}^{{C_{u}}^{+}} = \underset{IM - Best}{PSL (P_{s}, C_{u})} \end{matrix}

(25)

WSPS problem solving based on multi-objective programming

In this section, the WSPS problem is solved through the use of a multi-objective integer programming model. First, the constraint assumptions are given and a set of feasible portfolios are generated. The dominance structure is then defined in order to identify which weapon system portfolios are T&R portfolios (non-dominated). Finally, multi-objective integer programming is conducted to obtain the T&R portfolios.

Constraint assumptions and feasible portfolios

Another important component in the TPRP model is concerned with addressing the constraints of the weapon system portfolio during the acquisition and manufacturing process, which can be used to generate a set of feasible system portfolios from candidate weapon systems. Generally, a portfolio, $P_{s}$ , is a subset of the candidate weapon system, $S^{*}$ , and thus, the set of all possible portfolios is denoted as

P^{*} = {P_{s} | P_{s} \subseteq S^{*}, s \in N^{+}}

(26)

The set of feasible portfolios $P_{F}^{*} \subseteq P^{*}$ can be restricted by different kinds of constraints such as the availability of resources, budget, and the demands of the capability.^11–17 In this study, the manufacturing costs of weapon system portfolios are simplified and calculated by the cost of maturing the level of the technologies and integrations. Once a technology or integration appears in a function, a certain manufacturing cost is produced. The cost of a weapon system is accelerated through the combination of several component functions, which is the same as that of a weapon system portfolio through the gathering of several component systems. Additionally, the number of multi-capability requirements that the PRL must meet is a fundamental constraint and qualification of any portfolio. It means that once the multi-capability requirements are ascertained, the systems that can supply corresponding functions are almost clear. The different capability requirements correspond to the different scopes of candidate systems.

Therefore, a weapon system portfolio is feasible if its cost does not exceed the specified budget constraints and the portfolio can completely satisfy $v$ kinds of capability requirements. These constraints are often modelled through a set of linear inequalities so that

P_{F}^{*} = {P_{s} \in P^{*} | C (P_{s}) \leq B, \underset{P_{s}}{N (C)} = v, s \in N^{+}}

(27)

where the matrices $C \in R^{q \times \underset{P_{s}}{N (s)}}$ and $B \in R^{q}$ , $\underset{P_{s}}{N (C)}$ represents the number of capabilities that the weapon system portfolio, $P_{s}$ , can supply.

Dominance structure

Generally, dominated portfolios may be discarded from the candidate portfolios, retaining only the non-dominated ones, which are referred to, in this study, as T&R system portfolios. To identify the relationship between a pair of system portfolios, three relative definitions are given to determine the whole set of T&R portfolios.

Definition 8

Define the relationship $≻$ on $PSL (P_{s}, C_{u})$ , as follows

If {\begin{matrix} P {L_{s}}^{C_{u} -} \geq P {L_{s'}}^{C_{u} -} and P {L_{s}}^{C_{u} +} \geq P {L_{s'}}^{C_{u} +}, \forall P_{s}, P_{s'} \in P^{*} \\ P {L_{s}}^{C_{u} -} \geq P {L_{s'}}^{C_{u} +}, \exists P_{s}, P_{s'} \in P^{*} \end{matrix}

then we can say $PSL (P_{s}, C_{u}) ≻ PSL (P_{s'}, C_{u})$ .

Definition 9

Let $P_{s'}, P_{s} \in {P_{F}}^{*}$ , portfolio $P_{s}$ dominates $P_{s'}$ , denoted by $P_{s} ≻ P_{s'}$ , if, and only if, $PR L_{s} \geq PR L_{s'}$ , $C (P_{s}) \leq C (P_{s'})$ , and $PSL (P_{s}, C_{u}) ≻ PSL (P_{s'}, C_{u})$ , for all capabilities $C^{*} = {C_{u}, u = 1, 2, \dots, v}$ .

Definition 10^17,18,46

A feasible $P_{s} \in {P_{F}}^{*}$ is a T&R portfolio if, and only if, $P_{s'} \in {P_{F}}^{*}$ does not exist, such that $P_{s'} ≻ P_{s}$ .

Computation of T&R portfolios

The generation of a single T&R weapon system portfolio is easily realized through maximizing the measures of the system portfolio, subject to the given capability requirement and cost constraints. However, the computation of all the T&R portfolios is complex and time-consuming process in which the $v + 2$ objective integer programming model is formulated as follows

\underset{P_{s} \in P_{F}^{*}}{v - max} [PR L_{s}, \underset{v}{\underset{︸}{PSL (P_{s}, C_{1}), \dots, PSL (P_{s}, C_{u}), \dots, PSL (P_{s}, C_{v})}}, - C (P_{s})]

(28)

Therefore, a portfolio, $P_{s} \in {P_{F}}^{*}$ , is T&R if, and only if, another portfolio, $P_{s'} \in {P_{F}}^{*}$ , does not exist, such that

[P R L_{s^{'}}, \underset{v}{\underset{︸}{P S L (P_{s^{'}}, C_{1}), \dots, P S L (P_{s^{'}}, C_{u}), \dots, P S L (P_{s^{'}}, C_{v})}}, - C (P_{s^{'}})] ≩ [P R L_{s}, \underset{v}{\underset{︸}{P S L (P_{s}, C_{1}), \dots, P S L (P_{s}, C_{u}), \dots, P S L (P_{s}, C_{v})}}, - C (P_{s})]

(29)

where the relationship ≩ between the two vectors is consistent with the dominance structure given by Definition 9 and Definition 10. Equation (29) can be used to ascertain the T&R portfolio through pairwise comparisons between any pair of weapon system portfolios. The algorithm is given as follows, based on a sorting strategy to construct a T&R portfolio set:¹⁷

Step 1: assume that the number of the feasible portfolios in $P_{F}^{*}$ is M, $M \leq p \in N^{+}$ . The feasible portfolio sorted in descending order with regard to cost is enumerated as $P_{F}^{*} = {P_{1}, P_{2}, \dots, P_{M}}$ , which holds that $C (P_{s}) \geq C (P_{s'})$ if $s < s'$ .

Step 2: the initialization of the T&R portfolios set, s. $P_{TR}^{* 0} \leftarrow ϕ$ and $s \leftarrow 0$ .

Step 3: the iterations are carried out through all the portfolios $P_{s} \in P_{F}^{*}$ in an increasing order of index s.

While $s < M$ ,

The index s increases to s + 1.

If $\exists P_{s'} \in P_{ND}^{* s - 1}$ such that $P_{s'} ≻ P_{s}$ , then set $P_{TR}^{* s} \leftarrow P_{TR}^{* s - 1}$ and go to Step 3.

Set $P_{TR}^{* s} \leftarrow {P_{s}} \cup {P_{s'} \in P_{TR}^{* s - 1} | P_{s} ⊁ P_{s'}}$ and go to Step 3.

Step 4: set $P_{TR}^{*} \leftarrow P_{TR}^{* M}$ . $P_{TR}^{*}$ is the final set of T&R portfolios.

The algorithm reduces the number of pairwise comparisons by sorting the feasible portfolios in descending order with regard to their cost. It is worth noting that the algorithm used in this study is a bit different from those commonly found in the literature.¹⁷ We use a feasible portfolio set as the initial solution space, instead of the entire possible portfolio. This difference makes the algorithm work more efficiently by reducing the number of pairwise comparisons.

An illustrative case study

Case overview

Candidate weapon system description: based on the scenario carried out by Kangaspunta et al.,¹⁷ the illustrative example, which is adopted from the case examined in Sauser et al.²² and Tan et al.²⁴ to show the SRL approach, demonstrates the application of the proposed TPRP model for WSPS. The goal of the case study is to analyse and assess which of various combinations of indirect fire systems would make the optimal T&R portfolios by supporting the future mechanized infantry forces to fulfil a series of capabilities.

More specifically, the candidate weapon systems indexed by $i = 1, 2, \dots, 10$ are a variety of unmanned vehicles, intelligence and reconnaissance systems, information systems, command and control systems, and indirect (or direct) fire systems, denoted by a set, $S^{*} = {S_{1}, S_{2}, S_{3}, S_{4}, S_{5}, S_{6}, S_{7}, S_{8}, S_{9}, S_{10}}$ .

In total, the technologies and their integrations contained in 10 candidate weapon systems are put together to form a candidate weapon systems diagram using a technological perspective. As shown in Figure 4, there are 20 technologies and 21 integrations in 10 separate weapon systems.

2. Constraint assumption: the TPRP model considers both the manufacturing cost and the required capabilities as constraints of the feasible portfolios. Therefore, with regard to the first constraint, Tables 4 and 5 show the cost for maturing the technologies and their integrations. It is worth noting that most of the original levels of every technology and integration given by Tan et al.²⁴ are so low that the systems and portfolio cost very little, which leads to an effective reduction in the manufacturing cost constraints and a lower weapon system portfolio maturity level. Therefore, we adopt the TRL and IRL of every technology and integration with an increase of 1 in Figure 4.

Table 4.

The cost for TRL upgrade.

Technology	T₁	T₂	T₃	T₄	T₅	T₆	T₇	T₈	T₉	T₁₀
TRL level	Cost
1	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
2	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
3	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
4	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
5	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
6	$ 0	$ 0	$163	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
7	$ 0	$275	$181	$ 0	$944	$305	$ 0	$ 0	$ 0	$ 0
8	$ 0	$481	$279	$227	$992	$820	$ 0	$665	$348	$ 0
9	$446	$623	$872	$100	$310	$446	$844	$726	$669	$114
Technology	T₁₁	T₁₂	T₁₃	T₁₄	T₁₅	T₁₆	T₁₇	T₁₈	T₁₉	T₂₀
TRL level	Cost
1	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
2	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
3	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
4	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
5	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
6	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
7	$173	$ 0	$ 0	$630	$ 0	$ 0	$224	$ 0	$ 0	$ 0
8	$442	$ 0	$376	$602	$ 0	$282	$541	$ 0	$406	$ 0
9	$470	$128	$585	$694	$604	$171	$668	$221	$822	$160

TRL: technology readiness level. $: US$.

Table 5.

The cost for IRL upgrade.

Integration	1, 2	2, 3	2, 4	3, 4	4, 5	4, 19	5, 6	6, 7	6, 8	6, 9	8, 10
IRL level	Cost
1	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
2	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
3	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
4	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
5	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
6	$ 0	$ 0	$ 0	$ 0	$ 0	$952	$ 0	$ 0	$ 0	$ 0	$ 0
7	$ 0	$ 0	$ 0	$ 0	$648	$743	$ 0	$900	$ 0	$ 0	$988
8	$820	$ 0	$ 0	$613	$743	$832	$828	$859	$ 0	$172	$434
9	$407	$867	$213	$881	$465	$424	$401	$661	$179	$556	$861
Integration	9, 11	16, 18	17, 11	17, 12	17, 13	17, 14	17, 15	17, 4	18, 20	19, 17	20, 4
IRL level	Cost
1	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
2	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
3	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
4	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
5	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0	$ 0
6	$714	$ 0	$543	$ 0	$ 0	$ 0	$ 0	$370	$ 0	$ 0	$ 0
7	$311	$ 0	$511	$800	$ 0	$ 0	$ 0	$351	$ 0	$ 0	$280
8	$935	$ 0	$165	$771	$ 0	$510	$ 0	$558	$651	$681	$321
9	$732	$942	$933	$504	$634	$506	$937	$919	$507	$563	$859

IRL: integration readiness level. $: US$.

As shown in Tables 4 and 5, the cost is given in thousands of dollars (e.g. US$1000). Taking the costs of technology 2, technology 3, and their integrations as an example, it will take US$623,000 and US$872,000 for the upgrade of technologies 2 and 3, respectively, from levels 5 to 9, but US$867,000 is required for their integration.

With regard to the capability requirement constraint, the weapon system portfolio should meet the four capability requirements in future operations. The required capabilities, indexed by $u = 1, 2, 3, 4$ , are reconnaissance and intelligence capability ( $C_{1}$ ), orientation capability ( $C_{2}$ ), command and decision capability ( $C_{3}$ ), and action capability ( $C_{4}$ ).

3. Weapon system functions: as illustrated in the case study by Tan et al.,³⁵ only the shaded technologies and their corresponding integrations are taken into account in evaluating the maturity of the corresponding capability of a function. According to Definition 2 in this study, the emerging capabilities are the results of system functions, rather than the causes. Hence, we adopted Tan’s definition of function and its corresponding capability in the candidate weapon system diagram for the technological perspective. The shaded technologies and their corresponding integrations are considered when evaluating the maturity of the corresponding function of a weapon system. A simple illustration of $F_{1}, F_{2}, F_{3}, F_{4}$ is shown in Figure 5.

As depicted in Table 6, there are 28 functions, formed by the 20 technologies and their integrations, by the terms of our definition. The 10 weapon systems supplying 28 functions can satisfy four of the capability requirements. The functions supplied by each weapon system are shown in Table 7, and the relationships between the four capabilities and their causal functions are shown in Table 8. For example, the development of function F₄ requires technology {T₁, T₂, T₄, T₁₇, T₁₁} and the integrations among them {I₁₂, I₂₄, I_(4)(17), I_(17)(11)}. I₁₂ represents the integration between T₁ and T₂, while I_(17)(11) represents that between T₁₇ and T₁₁. The weapon systems that possess the function F₄ are S₁, S₄, and S₅. However, only capability C₁ can be obtained through the function F₄.

4. $SSL (S_{t}, C_{u})$ judgment matrices: in what follows, we give expert opinions through pairwise comparisons on a pair of weapon systems to determine which one is more likely to satisfy a certain capability requirement. The four judgment matrices are C_1(10×10), C_2(9×9), C_3(9×9), and C_4(9×9), which are gained through pairwise comparisons on C₁, C₂, C₃, and C₄ from an expert conference discussion, respectively. It is worth noting that because of the limitations of the three functions F₁, F₂, and F₄, controlled by S₁, it is impossible for S₁ to supply corresponding functions for capabilities C₂, C₃, and C₄. Therefore, the first judgment matrix, C_1(10×10), contains the pairwise comparison results of all the 10 weapon systems, while there are only nine rows and columns in the other three satisfaction matrices

\begin{matrix} C_{1 (10 \times 10)} = [\begin{matrix} 1 & 4 & 7 & 3 & 2 & 2 & 2 & 1 / 9 & 1 / 9 & 1 / 9 \\ 1 / 4 & 1 & 5 & 1 / 2 & 1 / 2 & 2 & 5 & 1 / 9 & 1 / 9 & 1 / 9 \\ 1 / 7 & 1 / 5 & 1 & 1 / 7 & 1 / 8 & 1 / 3 & 3 & 1 / 9 & 1 / 9 & 1 / 9 \\ 1 / 3 & 2 & 7 & 1 & 1 / 4 & 2 & 5 & 1 / 9 & 1 / 9 & 1 / 9 \\ 1 / 2 & 2 & 8 & 4 & 1 & 3 & 5 & 1 / 9 & 1 / 9 & 1 / 9 \\ 1 / 2 & 1 / 2 & 3 & 1 / 2 & 1 / 3 & 1 & 4 & 1 / 9 & 1 / 9 & 1 / 9 \\ 1 / 2 & 1 / 5 & 1 / 3 & 1 / 5 & 1 / 5 & 1 / 4 & 1 & 1 & 1 & 1 \\ 9 & 9 & 9 & 9 & 9 & 9 & 1 & 1 & 1 & 1 \\ 9 & 9 & 9 & 9 & 9 & 9 & 1 & 1 & 1 & 1 \\ 9 & 9 & 9 & 9 & 9 & 9 & 1 & 1 & 1 & 1 \end{matrix}] \\ C_{2 (9 \times 9)} = [\begin{matrix} 1 & 1 & 9 & 1 & 1 & 1 & 9 & 9 & 9 \\ 1 & 1 & 9 & 1 & 1 & 1 & 9 & 9 & 9 \\ 1 / 9 & 1 / 9 & 1 & 1 / 9 & 1 / 9 & 1 / 9 & 1 / 3 & 5 & 9 \\ 1 & 1 & 9 & 1 & 1 & 1 & 9 & 9 & 9 \\ 1 & 1 & 9 & 1 & 1 & 1 & 9 & 9 & 9 \\ 1 & 1 & 9 & 1 & 1 & 1 & 9 & 9 & 9 \\ 1 / 9 & 1 / 9 & 3 & 1 / 9 & 1 / 9 & 1 / 9 & 1 & 2 & 6 \\ 1 / 9 & 1 / 9 & 1 / 5 & 1 / 9 & 1 / 9 & 1 / 9 & 1 / 2 & 1 & 3 \\ 1 / 9 & 1 / 9 & 1 / 9 & 1 / 9 & 1 / 9 & 1 / 9 & 1 / 6 & 1 / 3 & 1 \end{matrix}] \\ C_{3 (9 \times 9)} = [\begin{matrix} 1 & 9 & 9 & 9 & 9 & 9 & 9 & 1 / 9 & 9 \\ 1 / 9 & 1 & 1 / 3 & 1 / 5 & 1 / 2 & 2 & 1 / 2 & 2 & 7 \\ 1 / 9 & 3 & 1 & 1 / 2 & 1 / 3 & 4 & 3 & 1 / 3 & 9 \\ 1 / 9 & 5 & 2 & 1 & 7 & 9 & 3 & 1 / 3 & 9 \\ 1 / 9 & 2 & 3 & 1 / 7 & 1 & 4 & 8 & 1 / 8 & 9 \\ 1 / 9 & 1 / 2 & 1 / 4 & 1 / 9 & 1 / 4 & 1 & 4 & 1 / 4 & 2 \\ 1 / 9 & 2 & 1 / 3 & 1 / 3 & 1 / 8 & 1 / 4 & 1 & 1 & 7 \\ 1 / 9 & 4 & 1 / 6 & 1 / 5 & 1 / 5 & 3 & 1 / 4 & 4 & 3 \\ 1 / 9 & 1 / 7 & 1 / 9 & 1 / 9 & 1 / 9 & 1 / 2 & 1 / 7 & 7 & 1 \end{matrix}] \\ C_{4 (9 \times 9)} = [\begin{matrix} 1 & 9 & 9 & 1 & 1 & 1 & 9 & 9 & 9 \\ 1 / 9 & 1 & 2 & 1 & 1 & 1 & 4 & 6 & 8 \\ 1 / 9 & 1 / 2 & 1 & 1 & 1 & 1 & 1 / 3 & 2 & 5 \\ 1 & 1 & 1 & 1 & 1 & 1 & 9 & 9 & 9 \\ 1 & 1 & 1 & 1 & 1 & 1 & 9 & 9 & 9 \\ 1 & 1 & 1 & 1 & 1 & 1 & 9 & 9 & 9 \\ 1 / 9 & 1 / 4 & 3 & 1 / 9 & 1 / 9 & 1 / 9 & 1 & 3 & 6 \\ 1 / 9 & 1 / 6 & 1 / 2 & 1 / 9 & 1 / 9 & 1 / 9 & 1 / 3 & 1 & 4 \\ 1 / 9 & 1 / 8 & 1 / 5 & 1 / 9 & 1 / 9 & 1 / 9 & 1 / 6 & 1 / 4 & 1 \end{matrix}] \end{matrix}

For example, as shown in C_1(10×10), the experts believe that system S₁ is significantly more likely to satisfy capability C₁ requirements than S_3, and more likely than S₄. However, when compared with S₈, S₉, and S₁₀, it is least likely to satisfy the capability C₁ requirements.

Table 6.

Technologies and the integrations for functional development.

Function	Technologies and the integrations to form a function
F₁	{T₁, T₂, T₄, T₅, T₆, T₇}, {I₁₂, I₂₄, I₄₅, I₅₆, I₆₇}
F₂	{T₁, T₂, T₄, T₅, T₆, T₈, T₁₀}, {I₁₂, I₂₄, I₄₅, I₅₆, I₆₈, I_(8)(10)}
F₃	{T₁, T₂, T₄, T₅, T₆, T₉, T₁₁}, {I₁₂, I₂₄, I₄₅, I₅₆, I₆₉, I_(9)(11)}
F₄	{T₁, T₂, T₄, T₁₇, T₁₁}, {I₁₂, I₂₄, I_(4)(17), I_(17)(11)}
F₅	{T₁, T₂, T₄, T₁₇, T₁₂}, {I₁₂, I₂₄, I_(4)(17), I_(17)(12)}
F₆	{T₁, T₂, T₄, T₁₇, T₁₃}, {I₁₂, I₂₄, I_(4)(17), I_(17)(13)}
F₇	{T₁, T₂, T₄, T₁₇, T₁₄}, {I₁₂, I₂₄, I_(4)(17), I_(17)(14)}
F₈	{T₁, T₂, T₄, T₁₇, T₁₅}, {I₁₂, I₂₄, I_(4)(17), I_(17)(15)}
F₉	{T₁, T₂, T₄, T₁₉, T₁₇, T₁₁}, {I₁₂, I₂₄, I_(4)(19), I_(19)(17), I_(17)(11)}
F₁₀	{T₁, T₂, T₄, T₁₉, T₁₇, T₁₂}, {I₁₂, I₂₄, I_(4)(19), I_(19)(17), I_(17)(12)}
F₁₁	{T₁, T₂, T₄, T₁₉, T₁₇, T₁₃}, {I₁₂, I₂₄, I_(4)(19), I_(19)(17), I_(17)(13)}
F₁₂	{T₁, T₂, T₄, T₁₉, T₁₇, T₁₄}, {I₁₂, I₂₄, I_(4)(19), I_(19)(17), I_(17)(14)}
F₁₃	{T₁, T₂, T₄, T₁₉, T₁₇, T₁₅}, {I₁₂, I₂₄, I_(4)(19), I_(19)(17), I_(17)(15)}
F₁₄	{T₁, T₂, T₄, T₂₀, T₁₈, T₁₆}, {I₁₂, I₂₄, I_(4)(20), I_(20)(18), I_(18)(16)}
F₁₅	{T₁, T₂, T₃, T₄, T₅, T₆, T₇}, {I₁₂, I₂₃, I₃₄, I₄₅, I₅₆, I₆₇}
F₁₆	{T₁, T₂, T₃, T₄, T₅, T₆, T₈, T₁₀}, {I₁₂, I₂₃, I₃₄, I₄₅, I₅₆, I₆₈, I_(8)(10)}
F₁₇	{T₁, T₂, T₃, T₄, T₅, T₆, T₉, T₁₁}, {I₁₂, I₂₃, I₃₄, I₄₅, I₅₆, I₆₉, I_(9)(11)}
F₁₈	{T₁, T₂, T₃, T₄, T₁₇, T₁₁}, {I₁₂, I₂₃, I₃₄, I_(4)(17), I_(17)(11)}
F₁₉	{T₁, T₂, T₃, T₄, T₁₇, T₁₂}, {I₁₂, I₂₃, I₃₄, I_(4)(17), I_(17)(12)}
F₂₀	{T₁, T₂, T₃, T₄, T₁₇, T₁₃}, {I₁₂, I₂₃, I₃₄, I_(4)(17), I_(17)(13)}
F₂₁	{T₁, T₂, T₃, T₄, T₁₇, T₁₄}, {I₁₂, I₂₃, I₃₄, I_(4)(17), I_(17)(14)}
F₂₂	{T₁, T₂, T₃, T₄, T₁₇, T₁₅}, {I₁₂, I₂₃, I₃₄, I_(4)(17), I_(17)(15)}
F₂₃	{T₁, T₂, T₃, T₄, T₁₉, T₁₇, T₁₁}, {I₁₂, I₂₃, I₃₄, I_(4)(19), I_(19)(17), I_(17)(11)}
F₂₄	{T₁, T₂, T₃, T₄, T₁₉, T₁₇, T₁₂}, {I₁₂, I₂₃, I₃₄, I_(4)(19), I_(19)(17), I_(17)(12)}
F₂₅	{T₁, T₂, T₃, T₄, T₁₉, T₁₇, T₁₃}, {I₁₂, I₂₃, I₃₄, I_(4)(19), I_(19)(17), I_(17)(13)}
F₂₆	{T₁, T₂, T₃, T₄, T₁₉, T₁₇, T₁₄}, {I₁₂, I₂₃, I₃₄, I_(4)(19), I_(19)(17), I_(17)(14)}
F₂₇	{T₁, T₂, T₃, T₄, T₁₉, T₁₇, T₁₅}, {I₁₂, I₂₃, I₃₄, I_(4)(19), I_(19)(17), I_(17)(15)}
F₂₈	{T₁, T₂, T₃, T₄, T₂₀, T₁₈, T₁₆}, {I₁₂, I₂₃, I₃₄, I_(4)(20), I_(20)(18), I_(18)(16)}

Table 7.

Relationships between 10 weapon systems and their functions.

Weapon system	The functions supplied by a weapon system
S₁	F₁, F₂, F₄
S₂	F₆, F₁₄, F₁₅
S₃	F₅, F₇, F₁₄, F₁₆, F₁₈, F₂₆
S₄	F₃, F₄, F₅, F₆, F₇, F₈, F₁₁, F₁₄
S₅	F₄, F₅, F₆, F₁₂, F₁₄, F₂₅
S₆	F₇, F₉, F₁₀, F₁₅, F₁₇, F₁₈, F₁₉, F₂₇
S₇	F₁₈, F₂₁, F₂₃, F₂₆
S₈	F₃, F₈, F₉, F₁₀, F₁₄
S₉	F₁₂, F₁₅, F₁₇, F₂₂, F₂₄
S₁₀	F₁₃, F₂₀, F₂₂, F₂₃, F₂₆, F₂₈

Table 8.

Relationships between four capabilities and their causal functions.

Capability	The functions to support a capability requirement
C₁	F₁, F₂, F₃, F₄, F₅, F₆, F₇, F₁₁, F₁₄, F₁₅, F₁₈, F₂₂, F₂₄, F₂₆
C₂	F₆, F₇, F₈, F₉, F₁₀, F₁₁, F₁₂, F₁₃, F₁₄, F₁₅, F₁₇, F₂₁, F₂₂, F₂₃
C₃	F₆, F₇, F₈, F₁₂, F₁₄, F₁₅, F₁₆, F₁₇, F₁₈, F₁₉, F₂₀, F₂₁, F₂₂, F₂₃
C₄	F₃, F₅, F₇, F₁₁, F₁₄, F₁₅, F₁₈, F₂₂, F₂₃, F₂₄, F₂₅, F₂₆, F₂₇, F₂₈

T&R portfolio analysis

Based on the TPRP model, 55 T&R weapon system portfolios are selected from the 1022 feasible portfolios. It is easy to see that about 95% of feasible portfolios are discarded because at least one of the T&R weapon system portfolios is higher than those in the 95% on all the five measures: $PR L_{s}$ , $PSL (P_{s}, C_{1})$ , $PSL (P_{s}, C_{2})$ , $PSL (P_{s}, C_{3})$ , and $PSL (P_{s}, C_{4})$ and lower in $C (P_{s})$ .

The PRL values of all the T&R weapon system portfolios are shown in Figure 6, where the 55 T&R portfolios are marked in red. All the PRL values are dispersed in the interval $[0.3899, 0.4299]$ , while the boundary between the concept refinement (CR) and TD lies between 0.39 and 0.4. Moreover, 82% of all the T&R portfolios are concentrated in the PRL 0.4–0.4299 range. Since the readiness level of some key technologies and integrations is still around 7, the TD state of a majority of system portfolios is at a low level, even for those that exceed the CR state.

Regarding the PRL boundary of all the T&R portfolios, portfolio P₇ = {S₅} ranks the highest and P₂ = {S₇, S₉} ranks the lowest. Since there is only one system, S₅, in portfolio P₇, the PRL₇ is completely determined by SRL₅. Similarly, P₂ comprises S₇ and S₉. SRL₇ and SRL₉ are so low, relatively, that their average stays at the bottom of the PRL value.

The CRL₁–CRL₄ values of the T&R system portfolios are illustrated in Figure 7 and can be easily sorted into two types, although the values generally fluctuate around 0.4. On the one hand, the CRL values for both the reconnaissance and intelligence capability (C₁) and the action capability (C₄) are almost equivalent among all the T&R system portfolios, with only a slight fluctuation. On the other hand, the CRL values of the orientation capability (C₂) and the command and decision capability (C₃) obviously vary in the T&R system portfolios, both showing a similar recurrent tendency towards change. Portfolio P₇ has outstanding performances in CRL for all four capabilities. Next, portfolio P₃₄ = {S₁, S₈}; the CRL₃₄ ranks top in the orientation capability and in the command and decision capability. Meanwhile, portfolio P₂₁ has a high readiness level in orientation capability because of its component systems S₃ and S₄.

PSL(P_s, C_u) value for C₁–C₄ of all the T&R weapon system portfolios is shown in Figure 8. With regard to PSL(P_s, C_u) value in C₁–C_4, the maximum of the T&R portfolios is approximately 0.45 and the median values of PSL(P_s, C_u) in C₁–C₄ are almost 0.25. For instance, PSL(P₇, C_u) values in C₁–C₄ have the point value 0.448, which is nearly the highest. Portfolio P₂₇ = {S₂, S₆} and portfolio P₂₈ = {S₂, S₅} have the same PSL(P_s, C_u) values in C₁–C₄, but the uncertainties of these values are vastly different. The values of PSL(P_s, C₂) and PSL(P_s, C₄) are exactly 0.25, but PSL(P_s, C₃) ranges widely [0.05, 0.45]. Portfolio (P₃₂ = {S₂, S₃, S₆, S₉}) has the lowest median value and the PSL(P₃₃, C_u) values in C₂–C₄ are clear point values.

Figure 9 shows the relationship between PSL(P_s, C_u) and CRL_u in C₁–C₄ of all the T&R weapon system portfolios. Although the value of CRL_u in C₂–C₄ increases from 0.38 to 0.44, the median values of PSL(P_s, C_u) in C₂–C₄, for the most part, stay at 0.25. Thus, if the values of the CRL_u in C₂–C₄ do not thoroughly exceed the development phase, a slight increase cannot sufficiently improve the satisfaction level of the capability requirements. However, with the increased CRL₁ value, the median value of PSL(P_s, C₁) increases sharply from 0.125 to 0.45.

A majority of the cost values of all the T&R weapon system portfolios lie between approximately US$ 0.4 × 10⁸ and 1.0 × 10⁸, as shown in Figure 10. The developmental cost of portfolio P₃₂ = {S₂, S₃, S₆, S₉} ranks the highest, with US$126,700,000, and portfolio P₂₂ = {S₂} ranks at the bottom, with US$16,270,000; meanwhile, portfolio P₇ = {S₅} has a relatively low cost value, with US$27,252,000.

The values of CRL_u and PSL(P_s, C_u) in C₁–C₄, with the increase in cost consumption, are shown in Figures 11 and 12, respectively. Overall, the influence of the increasing cost consumption from US$16,270,000 to US$126,700,000 by the improvement of CRL_u and PSL(P_s, C_u) is inconspicuous. In Figure 12, some values of PSL(P_s, C_u) in C₁–C₄ even decrease with an increasing cost consumption, from US$16,270,000 to US$39,340,000. Therefore, it is easy to see that inefficient investments and unwise allocations cannot match up with the capability requirements. Instead, it leads to a decrease in the weapon system PSL.

Discussion

The TPRP model focuses on the effects of technology push and requirement pull when DMs determine whether or not a weapon system portfolio should be reserved or discarded during the defence acquisition and manufacturing process. With an illustrative example in a combat scenario, 55 combinations of indirect fire systems were selected and evaluated, to identify the best among them (T&R portfolios) in supporting the mechanized infantry forces to fulfil a series of capabilities: reconnaissance and intelligence, orientation, command and decision, and action capabilities. Some highlighted results of this case study can be summarized as follows:

Effect of key technologies and integration push: the additive value model is used universally in the definitions of FRL, SRL, CRL, and PRL, which leads to average results for these variables. However, the multiplicative operation between TRL and IRL influences the FRL with a germination effect, especially for some key technologies and integrations. For example, T₂ appears in the technology sets of all the functions and, caused by its relatively low TRL value (TRL₂ = 7), the FRL of all the functions cannot rank high. Besides, there are some other key technologies and integrations with low readiness levels, as shown in Figure 5, such as T₆ = 7, T₁₇ = 7, I₄₍₁₇₎ = 6, and I₄₍₁₉₎ = 6. Therefore, even if most of the TRL and IRL values are above 6 after our modification to the original data, the SRL and PRL of the T&R weapon system portfolios are still relatively low because of the key technologies and integration push effect.

Effect of capability requirement pull: the four capability requirements assumed in this study are independent of each other and are of equal importance. These four requirements (reconnaissance and intelligence, orientation, command and decision, and action capabilities) are the constituent abilities that cannot be absent in a standard operation procedure. Therefore, we do not integrate the four capabilities into a single entity. However, the independence of all the capability requirements is common in the practical analysis and utilization. Moreover, with the increasing number of requirements, the aggregation of the capability requirement is becoming more necessary. Generally, it can be proven that the four capability requirement measures used in this study are stricter than the one capability requirement measure, gained by aggregation. Therefore, the effect of the capability requirement pull in the case study is comparatively successful.

Role of the capability number constraint: the number of the feasible indirect fire system portfolios is almost equivalent to that of the possible portfolios; however, as a distinctive component of the TPRP model, the capability number constraint is still valuable and significant in practical utilization. Once the capability requirements are ascertained, it is easy for experts to discard any worthless weapon system portfolios with fragmentary inherent functions. Because of the wide application of multiple functions weapon systems in this case study, systems may even have eight functions, as is the case in S₆ and S₄. These weapon system portfolios easily provide complete functionality and satisfy the four capability requirements. Therefore, the role of the capability-complete constraint is not completely considered.

Role of the cost constraint: in the case study, once a technology or integration appears in a function, a certain manufacturing cost is produced. The cost is accelerated in a system through the combination of several component functions; the cost of a portfolio is the same throughout the gathering of its component systems. Since technologies and integrations are treated equally, the cost of one system does not have precedence over another. In fact, with regard to cost consumption, the allocation programming and control of available costs are of great importance in many fields. As a result, even if the cost level increases, there is a clear decrease in the PSL(P_s, C_u) values in C₁–C₄, as shown in Figure 12. Therefore, avoiding inefficient cost allocation is very necessary in WSPS.

Conclusion

The TPRP model for WSPS accounts for the inherent value of weapon system portfolios from the technology push and requirement pull perspectives in defence acquisition and manufacturing. It explicitly notes that the ‘value’ of a system is not only related to its availability but also to its capability. In other words, value is achieved when the component technologies of the system are maturely developed and integrated with each other, as well as when the system can meet with multiple capability requirements. In terms of design, the TPRP model provides a reasonable and comprehensive approach to carrying out the selection of weapon systems. The developmental maturity level of a technology, function, system, capability, or portfolio is captured by five-level TPMs, based on an additive multi-criterion model. The requirement satisfaction level of a system or portfolio is obtained by two-level RPMs, based on a linear programming optimization and first-ignorance model founded on expert opinions on pairwise comparisons. In addition to the T&R system portfolios, the TPRP model also shows the PRL, PSL, and the costs of all the T&R system portfolios, as well as how these variables influence each other.

Although the portfolio value structure modelling and multi-objective integer programming are complex and time-consuming, the TPRP model offers several advantages. First, it emphasizes the innovative outlook that the current developmental state of component technologies and integrations strongly influence whether a weapon system portfolio should be reserved or discarded, and whether it can satisfy the focal military capability requirement. Second, the number of multi-capability requirements is considered a fundamental constraint of the weapon system portfolio in a standard operational procedure. It directly reduces the scale of the set of feasible portfolios at the beginning of the computation of the T&R portfolio. Finally, the TPRP model accommodates an uncertain satisfaction level with the interval probability elicited from expert opinions, which is significant and valuable in practical decision-making.

Improvements can be made for future studies in the following ways. First, more attention should be given to the universal application of the additive model in the definitions of FRL, SRL, CRL, and PRL, and the aggregation model can be more flexible and innovative. Second, we will investigate the different weights of each capability requirement on how to influence the final T&R system portfolio set. Finally, the set of the T&R system portfolios is obtained by pairwise comparisons between portfolios; thus, it is necessary to persevere to reduce the solution time of the dynamic programming algorithms and improve quality.

Footnotes

Appendix 1 Acknowledgements

We are grateful to the anonymous reviewers for their valuable comments and construction criticism.

Declaration of conflicting interests

The authors declare that there is no conflict of interest.

Funding

This work was supported by the National Natural Science Foundation of China: Reasoning and Learning Approach to Evidential Network with Application under Grant no. 71201168 and Hunan Provincial Innovation Foundation for Postgraduate under Grant no. CX2013B023.

References

Department of Defense. Fiscal Year (FY) 2013 President’s Budget Submission, Defense Advanced Research Projects Agency, Department of Defense, Washington, DC, February 2012.

Greiner

Fowler

Shunk

. A hybrid approach using the analytic hierarchy process and integer programming to screen weapon systems projects. IEEE T Eng Manage 2003; 50: 192–203.

Lee

Kang

S-H

Rosenberger

. A hybrid approach of goal programming for weapon systems selection. Comput Ind Eng 2010; 58: 521–527.

Zhou

Jiang

Yang

. A hybrid approach for multi-weapon production planning with large-dimensional multi-objective in defense manufacturing. Proc IMechE, Part B: J Engineering Manufacture 2014; 228: 302–316.

Hipel

Fang

. An interactive portfolio decision analysis approach for system-of-systems architecting using the graph model for conflict resolution. IEEE T Syst Man Cy A. Epub ahead of print 9 April 2014. DOI: 10.1109/TSMC.2014.2309321.

Burk

Parnell

Portfolio decision analysis: lessons from military applications. In: Salo

Keisler

Morton

(eds) Portfolio decision analysis: improved methods for resource allocation. New York: Springer, 2011, pp. 333–358.

Markowitz

Portfolio selection. J Financ 1952; 7(1): 77–91.

Stummer

Heidenberger

Interactive R&D portfolio analysis with project interdependencies and time profiles of multiple objectives. IEEE T Eng Manage 2003; 50: 175–183.

Peacock

Richardson

Carter

. Priority setting in health care using multi-attribute utility theory and programme budgeting and marginal analysis (PBMA). Soc Sci Med 2007; 64: 897–910.

10.

Buede

Bresnick

Applications of decision analysis to the military systems acquisition process. Interfaces 1992; 22(6): 110–125.

11.

Deviren

Yavuz

Kılınç

Weapon selection using the AHP and TOPSIS methods under fuzzy environment. Expert Syst Appl 2009; 36: 8143–8151.

12.

Sullivan

MJ.

Best practices: an integrated portfolio management approach to weapon system investments could improve DOD’s acquisition outcomes. Report no. GAO-07-388, 30 March 2007. Washington, DC: United States Government Accountability Office.

13.

Parnell

Gimeno

Westphal

. Multiple perspective R&D portfolio analysis for the National Reconnaissance Office’s technology enterprise. Mil Oper Res 2001; 6(3): 19–34.

14.

Parnell

Bennett

Engelbrecht

. Improving customer support resource allocation within the National Reconnaissance Office. Interfaces 2002; 32(3): 77–90.

15.

Parnell

Burk

Schulman

. Air Force Research Laboratory space technology value model: creating capabilities for future customers. Mil Oper Res 2004; 9(1): 5–18.

16.

Buckshaw

Parnell

Unkenholz

. Mission oriented risk and design analysis of critical information systems. Mil Oper Res 2005; 10(2): 19–38.

17.

Kangaspunta

Liesiö

Mild

. Cost-efficiency analysis of weapon system portfolios. Eur J Oper Res 2012; 223: 264–275.

18.

Zhou

Huang

Teng

. Project selection of robust portfolio models with incomplete information. J Financ Invest Anal 2012; 1(2): 157–199.

19.

Kung

Luccio

Preparata

On finding the maxima of a set of vectors. J Assoc Comput Mach 1975; 22(4): 469–476.

20.

Deb

Multi-objective optimization using evolutionary algorithms. Chichester: John Wiley & Sons, 2001.

21.

Mankins

JC.

Technology readiness levels, a white paper. 6 April 1995. Washington, DC: NASA.

22.

Sauser

Ramirez-Marquez

Magnaye

. A systems approach to expanding the technology readiness level within defense acquisition. Int J Def Acquis Manag 2008; 1: 39–58.

23.

Ramirez-Marquez

Sauser

System development planning via system maturity optimization. IEEE T Eng Manage 2009; 56(3): 533–548.

24.

Tan

Sauser

Ramirez-Marquez

Analyzing component importance in multifunction multicapability systems developmental maturity assessment. IEEE T Eng Manage 2011; 58: 274–294.

25.

Neaga

Henshaw

Yue

. The influence of the concept of capability-based management on the development of the systems engineering discipline. In: Presented at the 7th annual conference on systems engineering research, Loughborough University, Loughborough, 20–23 April 2009. Loughborough University, Loughborough: Research School of Systems Engineering.

26.

Hipel

Yang

. A data-centric capability-focused approach for system-of-systems architecture modeling and analysis. Syst Eng 2013; 16(3): 363–377.

27.

Dong

Wang

Chen

Domain-specific modeling and verification for C4ISR capability requirements. J Cent South Univ T 2012; 19: 1334–1340.

28.

DoD Architecture Framework Working Group. DoD architecture framework (version 2.0). Washington, DC: DoD, 2009.

29.

C4ISR Architecture Working Group. C4ISR architecture framework (version 2.0). Washington, DC: DoD, 1997.

30.

Cano

Moral

Using probability trees to compute marginals with imprecise probabilities. Int J Approx Reason 2002; 29: 1–46.

31.

Guo

Tanaka

Decision making with interval probabilities. Eur J Oper Res 2010; 203: 444–454.

32.

Sauser

Forbes

Long

. Defining an integration readiness level for defense acquisition. In: Proceedings of the 19th international symposium of the international council on systems engineering (INCOSE), Singapore, 20–23 July 2009.

33.

Gove

Sauser

Ramirez-Marquez

Integration maturity metrics: development of an integration readiness level. Inf Knowl Syst Manag 2010; 9: 17–46.

34.

Gove

Development of an integration ontology for system operational effectiveness. PhD Thesis, Stevens Institute of Technology, Hoboken, NJ, 2007.

35.

Tan

Ramirez-Marquez

Sauser

A probabilistic approach to system maturity assessment. Syst Eng 2011; 14(3): 279–293.

36.

Phaal

Palmer

PJ.

Technology management – structuring the strategic dialogue. EMJ: Eng Manag J 2010; 22(1): 64–74.

37.

Tan

Sauser

Ramirez-Marquez

. Monte-Carlo simulation approach for system readiness level estimation. In: Proceedings of the 19th international symposium of the international council on systems engineering (INCOSE), Singapore, 20–23 July 2009.

38.

Wang

Elhag

A goal programming method for obtaining interval weights from an interval comparison matrix. Eur J Oper Res 2007; 177: 458–471.

39.

Weichselberger

Pohlmann

A methodology for uncertainty in knowledge-based systems (Lecture Notes in Artificial Intelligence). Berlin, Heidelberg: Springer-Verlag, 1990.

40.

Klir

Uncertainty and information: foundations of generalized information theory. Hoboken, NJ: Wiley Interscience 2006.

41.

Guo

Tanaka

Interval regression analysis and its application, ISI invited paper (IPM30: Interval and Imprecise Data Analysis). In: Proceedings of the 56th session, bulletin of the International Statistical Institute, Lisboa, 22–29 August 2007.

42.

Camerer

Weber

Recent development in modeling preference: uncertainty and ambiguity. J Risk Uncertainty 1992; 5: 325–370.

43.

Figueira

Greco

Slowinski

Building a set of additive value functions representing a reference preorder and intensities of preference: GRIP method. Eur J Oper Res 2009; 195: 460–486.

44.

moore

Methods and applications of interval analysis. Philadelphia, PA: SIAM, 1979.

45.

Sugihara

Ishii

Tanaka

Interval priorities in AHP by interval regression analysis. Eur J Oper Res 2004; 158: 745–754.

46.

Liesiö

Mild

Salo

Robust portfolio modeling with incomplete cost information and project interdependencies. Eur J Oper Res 2008; 190(3): 679–695.

47.

Berleant

Cozman

Kosheleva

. Dealing with imprecise probabilities: interval-related talks at ISIPTA’05. Reliab Comput 2006; 12: 153–165.

48.

Dou

Zhang

Jiang

. MCDM based on reciprocal judgment matrix: a comparative study of E-VIKOR and E-TOPSIS algorithmic methods with interval numbers. Appl Math Inf Sci 2014; 8: 1401–1411.

An integrated technology pushing and requirement pulling model for weapon system portfolio selection in defence acquisition and manufacturing

Abstract

Keywords

Introduction

Literature review

Methodological development

Weapon system portfolio value modelling

TPRP value model based on MCDA

Five-level technology push measures via TRL and IRL

Definition 1

Definition 2

Definition 3

Two-level RPMs via SSL and PSL

Definition 4

Definition 5

Theorem 1

Definition 6

Assumption 1

Assumption 2

Assumption 331,44,45,48

Definition 7

WSPS problem solving based on multi-objective programming

Constraint assumptions and feasible portfolios

Dominance structure

Definition 8

Definition 9

Definition 1017,18,46

Computation of T&R portfolios

An illustrative case study

Case overview

T&R portfolio analysis

Discussion

Conclusion

Footnotes

Appendix 1

Acknowledgements

Declaration of conflicting interests

Funding

References

Assumption 3^31,44,45,48

Definition 10^17,18,46