Capability performance analysis for processes with multiple characteristics using accuracy and precision

Introduction

Manufacturing processes are subject to the influences of multiple factors, all of which increase the difficulty of the process quality evaluation. In recent years, many novel evaluation methods have been introduced to enhance process analysis and problem solving through definition of an effective production capability index. A considerable amount of research has focused on process capability indices (PCIs) as convenient and effective tools for measuring process performance to assist in quality assurance and purchasing decisions.^1–6 Using PCIs, production departments can trace and improve quality characteristics. Consequently, these indices have been widely applied in the automated manufacturing, semiconductor, and packaging industries.

In the last two decades, numerous statisticians and quality engineers have investigated PCIs with the aim of improving the assessment of both process potential and performance.^7–18 These indices, including the basic forms C p , C pk , C pm , and C pmk , are appropriate measures for processes with two-sided specifications and have become popular unitless measures for determining whether a process is capable of reproducing items that meet the quality requirements preset by the product designer. The rule for the general form is larger the better. In view of this, Vännman and Kotz ¹⁹ proposed a class of capability indices depending on two nonnegative parameters, u and v , defined as follows

where μ is the process mean, σ is standard deviation (overall process variability), d = ( USL − LSL ) / 2 is the half-length of the specification interval, T is the target value, and u , v ≥ 0 . Thus, the above-mentioned four basic indices can be considered as special cases of equation (1). It is easy to verify that C p ( 0 , 0 ) ( μ , σ ) = C p , C p ( 1 , 0 ) ( μ , σ ) = C pk , C p ( 0 , 1 ) ( μ , σ ) = C pm , and C p ( 1 , 1 ) ( μ , σ ) = C pmk .

Some of the desirability of these indices originates from their unitless nature. In terms of management and marketing, this characteristic of PCIs provides production departments with a simple and convenient basis for determining process performance. As a result, they can serve as a common language among management departments, sales departments, and customers, thereby facilitating product sales and the receipt of orders. However, there is a significant amount of process information that cannot be conveyed with a single index. When requirements specified by customers have not been met, a single index does not have the capacity to represent distinct influences on process performance or to provide the production department with a sufficient reference to make improvements.

This same point was made by Deleryd and Vännman, ²⁰ who stressed that PCIs, such as that depicted in equation (1), which combine information about closeness to a target (accuracy) and process spread (precision) into representation by a single number, pose major drawbacks. For instance, if the process is found noncapable, quality engineers must know whether this noncapability is caused by an inaccurate process mean, too much deviation, or a combination of these two factors. Thus, considering δ as the accuracy index, which measures the degree of process centering, and γ as the precision index, which measures the degree of process deviation, we define the process as follows

Based on the suggestions of Deleryd and Vännman, ²⁰ a number of researchers have converted PCIs into functions of the parameters δ and γ to measure and evaluate process capability.^21–26 Their objective was to assist production departments in quickly analyzing the process yield of various quality characteristics, the degree to which they deviated from target values, and their degree of variance, so that personnel could effectively impose improvement measures. To this end, equation (1) can be redefined and rewritten as

Similarly, the basic indices C p , C pk , C pm , and C pmk can be considered as special cases of the index in equation (3) by letting u = 0 or 1, and v = 0 or 1. In recent years, the six-sigma quality management model has effectively reduced the defect rates of product manufacturing. At a six-sigma quality level, the number of defective products reduces to only 3.4 in every million products. For this reason, the model has been widely applied in the manufacturing industry as a method of cost reduction. Previous studies^27–30 employed different capability indices to derive the relationship between process capability and six-sigma quality levels to assess and determine process performance. However, when the values of different PCIs are equal, the process capabilities and quality levels that they imply for the quality characteristics differ, which can inconvenience product departments during application.

In summary, accuracy and precision indices have an advantage over conventional PCIs in the evaluation of process levels. Therefore, we applied the concept of quality levels in the six-sigma model to develop a process quality-level analysis chart (PQLAC) based on precision index γ and accuracy index δ . The proposed approach enables plotting several nominal-the-best quality characteristics on a single analysis graph. This graphic representation clearly shows the extent to which the quality characteristics have deviated from their target values (accuracy) and the degree of variance in the quality characteristics (precision). In addition to creating a chart to enable direct evaluation of the process quality levels of different quality characteristics, we collected reference material from expert personnel to assist in the improvement of the process capabilities. Finally, we developed a pre- and post-improvement hypothesis testing model with which production departments can assess the effectiveness of their improvement efforts.

Relationship among quality level, δ, γ, and process yield

As mentioned above, the accuracy index δ primarily measures the extent to which the process mean μ has deviated from target value T. δ closer to 0 indicates a smaller deviation in the process mean and a higher degree of accuracy. The precision index γ represents the degree of variance in the process; a smaller value indicates a lower degree of standard deviation and a higher degree of precision. Definitions of the two indices are presented in equation (2). Pearn and Chen ³¹ and Chen et al. ³² stated that to achieve a six-sigma quality level, the process must present a mean deviation of 1 . 5 σ from the target value and a standard deviation of σ = d / 6 . It is apparent that when the quality level of the process reaches six sigma ( 6 σ ) and if the process mean displays a left offset of 1 . 5 σ from the target value and d = 6 σ , then δ = − 1 / 4 . Thus, equation (2) can be expressed as

For the sake of generality, let δ k denote the process accuracy when the quality level reaches k sigma and let γ k represent the process precision when the quality level reaches k sigma. Therefore, when the quality level of the process reaches k sigma, δ k and γ k can be expressed as follows

Thus, in addition to using equations (5) and (6) to evaluate accuracy and precision of the process for quality characteristics, we can also determine whether the process attains the k sigma quality-level set by the production department. The constraint for the quality-level measurement region is therefore

For processes with two-sided specification limits, the process yield can be calculated as F ( USL ) − F ( LSL ) , where F ( · ) is the cumulative distribution function of the quality characteristic X , USL is the upper specification limit, and LSL is the lower specification limit. On the assumption of normality, process yield can be expressed as follows

By substituting equations (5) and (6) into equation (8), we can derive the relationship between process yield and quality levels

A comprehensive view of the content above shows that when production departments set a stronger regulation of quality levels, process yield increases (as shown in Figure 1), and accuracy and precision are greater. Table 1 presents the relationships among quality level, δ , γ , and process yield %.

OPEN IN VIEWER

Figure 1.

Relationship between quality level and process yield.

OPEN IN VIEWER

Table 1.

Relationships among quality level, δ , γ , and process yield.

Quality level	γ k	δ k	Yield %
6 sigma	0.166667	− 0 . 25 to 0 . 25	0.999996602
5.5 sigma	0.181818	− 0 . 27 to 0 . 27	0.999968329
5 sigma	0.200000	− 0 . 3 to 0 . 3	0.999767371
4.5 sigma	0.222222	− 0 . 33 to 0 . 33	0.998650101
4 sigma	0.250000	− 0 . 375 to 0 . 375	0.993790316
3.5 sigma	0.285714	− 0 . 43 to 0 . 43	0.977249581
3 sigma	0.333333	− 0 . 5 to 0 . 5	0.933189401

PQLAC

As discussed above, the accuracy index δ and the precision index γ can effectively present the extent to which process quality characteristics have deviated from target values and their degree of variance. Smaller values in both indices indicate that the process yield and quality levels of the quality characteristic are better. To simultaneously evaluate the process performance and quality levels of multiple nominal-the-best quality characteristics, we developed a PQLAC with the accuracy index δ on the horizontal axis and the precision index γ on the vertical axis, as shown in Figure 2. The PQLAC can be used to determine the process performance of various quality characteristics and allows production departments to track and improve quality characteristics with poor process capabilities, thereby allowing the enhancement of overall production quality to meet customer needs.

OPEN IN VIEWER

Figure 2.

Process quality-level analysis chart (PQLAC).

In this section, we detail the application of the PQLAC. We first assume that the process performance of a given quality characteristic has already achieved the three-sigma quality level, but it does not exceed the four-sigma quality level. Obviously, at present, the process yield of the quality characteristic is greater than 0.933189 and smaller than 0.993790 (i.e. 0 . 933189 ≤ Yield % ≤ 0 . 993790 ). When the factory produces 1 million products every time, the number of the defective products ranged between 6210 and 66810 based on the above equation. Hence, the production costs and materials of these defective products not only incur considerable losses but also increase rework costs for enterprises, further undermine the market competitiveness of the company. For this reason, the production department must actively analyze and identify the causes of this poor process performance and then adopt relevant improvement measures to enhance the process level so that the quality characteristic can be increased to the ideal six-sigma quality level, as shown in Figure 3, allowing for the reduction of the defect rate and consequent costs. Following analysis and identification, improvements in each quality-level region are carried out according to the reference provided by expert personnel. These recommendations are displayed in Table 2.

OPEN IN VIEWER

Figure 3.

Application and analysis of PQLAC.

OPEN IN VIEWER

Table 2.

Database of expert advice

Quality area	State of performance	Possible cause	Recommended improvement	Improvement cost
A and B	Sufficient process precision but insufficient accuracy	Improper operation	Review operation procedures and establish standard operating procedures (SOPs)	Low
		Undue processing	Intensify personnel training and selection procedures and establish a reward and penalty system
		Incorrect machine parameter settings	Adjust machine parameters and institute regular checks
C	Sufficient process accuracy but insufficient precision	Poor quality of raw materials	Enhance inspection and control of raw materials and select suppliers with quality guarantees	High
		Unstable processing quality	Regularly inspect processing tools and confirm their processing performance and lifespan
		Old processing machines	Regularly replace old machines
E and F	Both accuracy and precision insufficient	See defects in A, B, and C	See recommendations for A, B, and C	Bigger than A and B

Hypothesis testing model for process improvement effectiveness

The PQLAC can assist the production department in the identification of critical-to-quality characteristics with poor process capabilities. Based on the accuracy index δ and the precision index γ , factors influencing the performance of the process can be systematically analyzed, thereby enabling the improvement of quality levels. However, once corresponding improvement mechanisms have been implemented, the production department must confirm whether the improvements were as effective as anticipated and the target quality levels demanded by customers have been reached. We developed the hypothesis testing method used in the field of statistics through the derivation of a joint confidence interval for δ and γ based on Boole’s inequality. The ( 1 − α 1 − α 2 ) × 100 % joint confidence interval for δ and γ can be expressed as

When α 1 = α 2 = α / 2 , we can obtain the Cartesian product from the inequality above, which defines set region S as

where

Using the four values above, we construct the ( 1 − α ) × 100 % joint confidence region for δ and γ , that is, set region S . The schematic diagram is shown in Figure 4.

OPEN IN VIEWER

Figure 4.

Schematic diagram of set region S .

In terms of practical application, the ( δ , γ ) of each quality characteristic has a ( 1 − α ) × 100 % probability of falling within the joint confidence region. In fact, the distance d between ( δ , γ ) and the origin represents the quality level of the quality characteristic. Therefore, the closer δ and γ are to 0, the smaller d is and the higher the precision and accuracy of the process. This also implies that the quality level of the quality characteristic in the process is higher. In contrast, the further δ and γ are from 0, the lower the quality levels. For the sake of generality, we set the six-sigma quality level as the measurement standard for improvement effectiveness. Using mathematical programming, we obtained the upper and lower limits of a ( 1 − α ) × 100 % confidence interval for δ and γ to develop a statistical hypothesis testing model to determine the effectiveness of improvement measures.

We employed the distance between ( δ , γ ) and the origin as the target function, as shown in equation (12), and the set region S as the region of feasible solutions to derive the maximum and minimum values and thus the upper and lower limits of the ( 1 − α ) × 100 % confidence interval, as shown in Figure 5.

OPEN IN VIEWER

Figure 5.

Schematic diagram to solve upper and lower limits of the ( 1 − α ) × 100 % confidence interval for δ and γ .

Case study

In this section, we consider a case study to demonstrate the proposed PQLAC and the testing model. The selected case is the manufacturing process of a five-way pipe, which is an important component in the production of bicycles. The structure of a specific five-way pipe is shown in Figure 6, and the specifications of its quality characteristics are presented in Table 3; data were taken from a factory located in Taichung, Taiwan. All of the key quality characteristics of the five-way pipe are nominal-the-best specifications.

OPEN IN VIEWER

Figure 6.

Structure of five-way pipe.

OPEN IN VIEWER

Table 3.

Specifications of quality characteristics for five-way pipe.

Characteristic	LSL	Target	USL
Inner diameter	ϕ 43 . 9	ϕ 44 . 0	ϕ 44 . 1
Outer diameter	ϕ 48 . 8	ϕ 49 . 0	ϕ 49 . 2
Left crank	9.65 mm	9.75 mm	9.85 mm
Right crank	9.65 mm	9.75 mm	9.85 mm
Length	67.77 mm	68 mm	68.23 mm
Gap between bearings	0.9 mm	1.0 mm	1.1 mm

USL: upper specification limit; LSL: lower specification limit.

We considered a random number of samples—100—from a stable (under statistical control) process and measured six product quality characteristics. For the sample, the Kolmogorov–Smirnov test for normality confirmed a p - value > 0 . 15 . That is, it is reasonable to assume that the process data collected from the factory are normally distributed. The calculated sample mean; sample standard deviation; accuracy index δ ; precision index γ ; points of joint confidence interval, a 1 , a 2 , b 1 , b 2 ; and 95% lower confidence limit and upper confidence limit are presented in Table 4. Five pairs of δ and γ values are plotted on the proposed PQLAC in Figure 7, clearly representing the process status of each in-process quality characteristic in a single tool. As can be seen from this figure, it is easy to determine their quality level and process performance.

OPEN IN VIEWER

Table 4.

Quality-related variables of five-way pipe.

Code	Characteristic	μ	σ	δ	γ	a 1	a 2	b 1	b 2	d L	d U
I	Inner diameter	44.014	0.0163	0.144	0.163	0.103	0.185	0.140	0.193	0.174	0.268
II	Outer diameter	48.955	0.0349	−0.226	0.175	−0.270	−0.182	0.151	0.207	0.236	0.341
III	Left crank	9.768	0.0159	0.182	0.159	0.141	0.222	0.137	0.188	0.197	0.291
IV	Right crank	9.774	0.0153	0.242	0.153	0.203	0.281	0.132	0.182	0.242	0.334
V	Length	67.934	0.0292	−0.287	0.127	−0.319	−0.255	0.110	0.151	0.277	0.353
VI	Gap between bearings	1.0054	0.0313	0.054	0.313	−0.025	0.134	0.270	0.372	0.270	0.396

OPEN IN VIEWER

Figure 7.

PQLAC for five-way pipe.

The PQLAC allows for quick evaluation of whether or not the quality performance meets the required minimum five-sigma level. The five-sigma level is within the contours of − 0 . 3 ≤ δ ≤ 0 . 3 and 0 ≤ γ ≤ 0 . 2 . Clearly, the plotted points I, II, III, IV, and V lie inside the area of Q L 5 . In contrast, the plotted point VI lies outside the set area. The excessive process variance in point VI is, therefore, the primary cause of insufficient process levels. Thus, quality improvement efforts focused on point VI should be based on the reduction of process variance. In this endeavor, the production department can refer to the expert recommendations in Table 2 to effectively enhance the performance of the process associated with the characteristic “gap between bearings.”

After the production department implemented the recommended improvements, we once again selected 100 samples, calculating the mean gap between bearings to be 1.004 with a standard deviation of 0.018. Therefore, we derived an accuracy index δ of 0.044 and a precision index γ of 0.181, thereby indicating that the improved process performance for the gap between bearings fell within the five-sigma quality-level region ( Q L 5 ) and satisfied customer requirements. To further confirm the effectiveness of the improvements, we then calculated the upper and lower limits of the ( 1 − α ) × 100 % confidence interval for δ and γ , resulting in D ′ = [ d ′ L , d ′ U ] = [ 0 . 156 , 0 . 233 ] . As seen in Table 4, the pre-improvement upper and lower limits of the ( 1 − α ) × 100 % confidence interval for δ and γ are D = [ d L , d U ] = [ 0 . 270 , 0 . 396 ] . Thus, based on the evaluation criteria presented in Section “Testing model and evaluation criteria,” we can derive D ∩ D ′ = ϕ and d ′ U < d U , indicating a significant improvement in quality following the implementation of the expert recommendations. This result demonstrates the effectiveness of the accuracy index δ , precision index γ , and the proposed testing model in the process performance evaluation.

Conclusion

The application of the accuracy index δ and precision index γ enables users to effectively investigate the extent to which quality characteristics have deviated from target values, as well as their degree of variance. For this reason, in this study, we incorporated the accuracy index δ , precision index γ , and the quality-level concept of the six-sigma model to propose a novel PQLAC, which is capable of analyzing the capability performance for processes with multiple characteristics. The cause of unsatisfactory process capabilities in terms of to poor quality characteristics can also be analyzed and adequate measures can be taken to improve quality and establish a quality improvement database. Hence, the PQLAC can not only directly identify the quality levels for various quality characteristics but also provide recommendations for improvement of all quality-level regions to serve as a reference for production departments. Clearly, the approach proposed in this article is superior to previous applications of PCIs for exploring process capability, including C p , C pk , C pm , and C pmk . Furthermore, to assist production departments in confirming the effectiveness of implemented improvements, we derived the upper and lower limits of the ( 1 − α ) × 100 % confidence intervals for δ and γ by using mathematical programming to develop a statistical hypothesis testing model.

A case study involving a process for a five-way pipe, which has six quality characteristics with a nominal-the-best specifications type, is discussed to demonstrate the applicability of the proposed PQLAC and testing model in practice. According to the sample data taken from a stable and in-control process during the period time, we obtained the relevant process information for each quality characteristic, as summarized in Table 4. We found that the quality characteristic “gap between bearings” has a poor process capability based on PQLAC analysis when customers expect the quality performance to meet the five-sigma level. The primary cause of insufficient process levels was the excessive process variance. After the production department implemented the recommended improvements in Table 2, the precision performance of the “gap between bearings” characteristic improved from 0.313 to 0.181. Finally, we calculated the pre-improvement and post-improvement upper and lower limits of the ( 1 − α ) × 100 % confidence interval for δ and γ , that is, [ 0 . 270 , 0 . 396 ] and [ 0 . 156 , 0 . 233 ] , respectively, to further confirm that the post-improvement process capability met customer requirements. Most notably, the process performance of the “gap between bearings” characteristic has been upgraded based on the statistical hypothesis testing model. As mentioned above, the approach proposed in this article can effectively assist managers in measuring and monitoring process performance in a timely manner to ensure that the quality levels of their products meet customer demands.