Particle swarm optimization algorithm with multi methods argument

Abstract

Particle swarm optimization (PSO) is well known for dealing with complex nonlinear problems. In recent years, many researchers developed improved PSO algorithms to enhance the search and convergence ability. However, when dealing with the engineering control problems, the goal function is usually unknown and discrete. Thus, an algorithm with good search ability and fast convergence speed is required. This paper presents a new development algorithm called multi methods argument particle swarm optimization (MMAPSO). This algorithm uses an argument strategy to draw the merits of some search methods, such as chaotic search, cloud search and gradient descent search. This strategy debates the search method according to the best position of particle and average convergence speed. The experiments are conducted on uni-modal functions, multi-modal functions and noisy functions. The results demonstrate the superiority of MMAPSO algorithm on twelve functions when compared with other six algorithms.

Keywords

Particle swarm optimizer (PSO)chaotic search cloud search gradient descent

1. Introduction

Particle swarm optimization (PSO) [5] is well known for dealing with complex mathematics functions and nonlinear problem and is applied in various engineering problems. For example, hybrid power system [15], wind energy control [7], optimal placement of wind turbines [18], open vehicle routing [12], intelligent identity [2] and clustering [4]. Over the last two decades, many improved algorithms have been proposed.

The main problem of PSO algorithm is premature convergence and low convergence speed at later stage. Premature will lead to fall into the local optimal position.

To prevent the particle from falling into the local, a global search method, chaotic search [26], was introduced. Comprehensive learning particle swarm optimization (CLPSO) [10] and simulated annealing particle swarm optimization (SAPSO) [21] were proposed, by exchanging the best position where the particle experienced. These algorithms focused on enlarging the search space at early stage and decreasing the space at late stage. However, for some uni-model functions, the convergence speed of these methods was slow. Since they cost great global search computing at early stage. On the contrary, linear decreasing inertia weight PSO (LDIWPSO) [19] and gray PSO [8] were proposed to improve PSO performance by changing the inertia weight. Nevertheless, in order to guarantee the algorithm’s convergence, the inertia weight has a boundary, which decides each particle’s step size should be in a limit area. Hence, this idea still can not get rid of falling into the local with some probability.

To speed up the convergence, the gradient descent [14] and SQP [26] methods were introduced. However, these methods were just suitable for monotonous, continuous functions.

A non-uniform mutation [25] was introduced into simulated annealing (SA) algorithm. This mutation enlarged search space at early stage and decreased the search space with the iteration numbers, which accelerated the convergence at late stage. However, for the large dimension problems, the convergence speed is slow.

In this paper, we combine some search methods, such as gradient descent search, cloud search and chaotic search, through argument in different cases, to get a better performance algorithm, which is Multi Methods Argument Particle Swarm Optimization Algorithm (MMAPSO). Experiments are conducted on uni-modal functions, multi-modal functions and noisy functions. The results demonstrate the convergence superiority of MMAPSO algorithm on twelve out of thirteen functions compared with other six algorithms. The engineering control example illustrates this algorithm’s application value.

2. PSO and improved PSO

2.1. PSO algorithm

The optimization problem is assumed to: $\begin{matrix} min f (x) = f (x_{1}, x_{2}, \dots, x_{n}) \end{matrix}$ s.t. $\begin{matrix} x_{i} \in [L_{i}, U_{i}] . \end{matrix}$

If the particle is expressed as $X_{i} = (x_{i 1}, x_{i 2}, \dots, x_{i D})$ , the best position it experienced (the best fitness value $p_{best}$ ) is $P_{i} = (p_{i 1}, p_{i 2}, \dots, p_{i D})$ . The index of best position experienced in particle group is $P_{g}$ , or $g_{best}$ . The speed of particle i uses $V_{i} = (v_{i 1}, v_{i 2}, \dots, v_{i D})$ to express, for each generation, its $d (1 ⩽ d ⩽ D)$ dimension iteration functions are below: $\begin{array}{l} (1) & \begin{matrix} v_{i d} (t + 1) = & v_{i d} (t) + c_{1} r_{1} (p_{i d} (t) - x_{i d} (t)) \\ + c_{2} r_{2} (p_{g d} (t) - x_{i d} (t)), \end{matrix} \\ (2) & x_{i d} (t + 1) = x_{i d} (t) + v_{i d} (t + 1), \end{array}$ where, $c_{1}$ and $c_{2}$ are all positive constants, called learning factor; $r_{1}$ and $r_{2}$ are random numbers ranging from 0 to 1; $v_{i d}$ is the d dimension speed of each particle. The first term of the right of the equal sign in Eq. (1) is the previous particle’s velocity; the second term is the direction of the particle’s historical best position; the last term is the direction of the swarm’s global best position.

2.2. Improved PSO

Basically, the improved PSOs can be divided into three classifications. The first one is to change the parameters of PSO, such as, binary particle swarm optimization algorithm (BPSO) [6], linear decrease inertia weight (LDIWPSO) [19,20], hybrid Fletcher–Reeves PSO [1] and Grey relational analysis PSO [8]; the second one is to introduce new conception and distribution to PSO, for example, proposing a concept of diversity to avoid the particle swarm premature [23]; fitness direction ratio particle swarm optimization (FDRPSO) [17], using the ratio of the relative fitness and the distance of other particles to determine the direction; simulated annealing particle swarm optimization (SAPSO) [21], introducing simulated annealing idea to help particle jump out of the local; adding a chaotic search to search the global area [11]; combining SQP algorithm [26] or gradient search [14] to accelerate convergence; the last one is to propose novel learning strategy, such as, comprehensive learning particle swarm optimization (CLPSO) [10], orthogonal experimental design learning strategy (OLPSO) [27], hybridizing cellular learning (CPSO) [9], synchronous and asynchronous learning PSO [24], dynamic neighborhood learning (DNLPSO) [13] and self regulating learning (SRPSO) [22].

3. Multi Methods Argument Particle Swarm Optimization Algorithm (MMAPSO)

In this paper, we combine three search methods i.e. adaptive gradient descent search (AGS), cloud search (CS) and piece-wise chaotic search (PCS) into LDIWPSO [19], then argue them according to the change of the best position (Pg), and finally decide when to use the reasonable method, shown in Fig. 1.

Fig. 1.

The scheme of MMAPSO.

3.1. Adaptive gradient descent search (AGS)

The equation of gradient descent method is: $\begin{matrix} (3) & x_{k + 1} = x_{k} - λ \cdot grad, \end{matrix}$ where, λ is the step size. It influences the result accuracy, grad is the gradient of the goal function at $x_{k}$ point. To avoid solving the huge Hesse matrix of the gradient, we use Eq. (4) to replace the grad. In this algorithm, set $x_{k}$ as $Pg$ . $\begin{array}{l} {grad}_{j} = & (f (x_{1}, \dots x_{j} + Δ x_{j}, \dots, x_{n}) \\ (4) & - f (x_{1}, \dots, x_{j}, \dots, x_{n})) / Δ x_{j}, \end{array}$ where, $Δ x_{j} = ‖ Pg - {Pg}^{'} ‖_{2}$ , $λ = 0.5$ , ${Pg}^{'}$ is the latest best position, $Δ x_{j}$ stands for the distance between the two best points of the different generations. Hence, the gradient of $Pg$ will change adaptively according to the convergence velocity.

The AGS procedure is that:

Calculate the sub-gradient of $Pg$ according to Eq. (4).

Generate $x_{k + 1}$ according to Eq. (3).

Compare $fitness (Pg)$ with fitness ( $x_{k + 1}$ ), if the later is smaller, replace ${Pg}^{'}$ with $Pg$ and $Pg$ with $x_{k + 1}$ and calculate the speed with Eqs (13), (14), else break.

Run Step 1 to Step 3 for M times.

3.2. Cloud search (CS)

The AGS method is only used for local search of the best point ( $Pg$ ) in MMAPSO. However, when the goal function is non-differentiable and discontinuous, this search method will mislead the $Pg$ point. At this time, the cloud search (CS) method should be used.

The main aim of introducing CS is to do a further local search in case that the AGS method is work-less. If the CS and AGS search method both work, the convergence speed of them will be compared and the better one will be chosen. The idea of CS is mainly to generate some new particle around the best position by the rule of cloud model, then find the new best position.

The cloud model presents a concept according to human thought by using three numerical characteristics ( $Ex$ , $En$ , $He$ ), denoted by $C (Ex, En, He)$ [9]. A cloud model can be seen as a particle swarm, and one particle of them is called cloud drop. In MMAPSO, we propose a cloud rule (Eqs (5) and (6)). This rule generates a new cloud model according to the particle’s best position ( $Pg$ ). $\begin{array}{l} (5) & \begin{matrix} a_{i} = norm (En, He), \\ n = \frac{ρ_{1} \sqrt{(Up - {Pg}_{j}) / 3}}{2^{i}}, \\ He = \sqrt[4]{En}, if i ⩽ N / 2, \\ En = \frac{ρ_{1} \sqrt{({Pg}_{j} - Low) / 3}}{2^{i}}, \\ He = \sqrt[4]{En}, if i > N / 2, \end{matrix} \\ (6) & \begin{matrix} b_{i, j} = norm ({Pg}_{j}, a_{i}), \\ b_{i, j} = b_{i, j} + 2 ({Pg}_{j} - b_{i, j}), \\ if i ⩽ N / 2 and b_{i, j} < {Pg}_{j}, \\ b_{i, j} = b_{i, j} - 2 (b_{i, j} - {Pg}_{j}), \\ if i > N / 2 and b_{i, j} > {Pg}_{j}, \end{matrix} \end{array}$ where, $norm (x, u)$ is normal random number which uses x and u to represent the expectation and the variance, respectively. $ρ_{1}$ is the radius factor decided by the convergence situation. $\begin{matrix} (7) & ρ_{1} = {‖ Pg - {Pg}^{'} ‖}_{2}, \end{matrix}$ where, ${Pg}^{'}$ is the last best position of the particle swarm, and the $En$ and $He$ decide the width and thickness of the cloud. Hence, this realizes a local search near $Pg$ . For example, we suppose that $Pg$ is $(1, 1)$ and ${Pg}^{'}$ is $(2, 2)$ , the boundary is $[- 10, 10]$ , the number of could drop (N) is 20, to generate a could model. Figure 2 shows the distribution of the points. This illustrates that the distribution is mainly suitable for local search and the few far away points can implement a global search with small probability.

Fig. 2.

Distribution in cloud model.

The CS procedure is:

Generate a new particle swarm b according to Eqs (5), (6), (7).

For $i = 1$ to N, compare $fitness (Pg)$ with each $fitness (bi)$ . If the later is smaller, replace ${Pg}^{'}$ with $Pg$ , ${Pg}^{'}$ with $bi$ , and calculate the speed using Eqs (13), (14). End for i.

3.3. Piece-wise chaotic search with varying neighbourhood (PCS)

If the best position of the particle swarm does not update for several times, it maybe fall into a local. The chaotic search will be used to help the particle swarm jump out of the local.

The chaotic search procedure is:

Generate a new rand particle swarm C form 0 to 1 according to Eq. (8).

Generate a new point $C^{'}$ by chaotic mapping Eq. (9), and replace C with $C^{'}$ .

According to different radius $ρ_{2}$ , generate different $y_{i}$ by Eqs (10), (11).

Compare $fitness (Pg)$ with each fitness ( $y_{i}$ ), if the later is smaller, replace ${Pg}^{'}$ and $Pg$ with $Pg$ and $y_{i}$ , respectively.

Next search starting with Step 2.

Because

ρ_{2}

changes from 0 to 1 as the particle index number i increases,

c_{i}^{'}

has the chaotic feature which covers the area

[0, 1]

but never repeat itself, and

y_{i}

centres

x_{i}

and changes radius following

ρ_{2}

increasingly within the boundary, the new particle

y_{i}

can cover each corner, especially when

x_{i}

gathers in a small local area at late stage. This feature guarantees PCS to jump out of the local. Figure 3 shows the chaotic map (Eq. (9)) feature with 1000 points.

\begin{array}{l} (8) & \begin{matrix} c_{i, j} = {rand}_{i, j}, \\ i = 1, 2, \dots, n, j = 1, 2, \dots, D, \end{matrix} \\ (9) & c_{i, j}^{'} = \{\begin{matrix} 4 \cdot u \cdot c_{i, j} \cdot (0.5 - c_{i, j}), \\ 0 ⩽ c_{i, j} < 0.5, \\ 1 - 4 \cdot u \cdot (c_{i, j} - 0.5) \cdot (1 - c_{i, j}), \\ 0.5 ⩽ c_{i, j} ⩽ 1, \end{matrix} \\ (10) & y_{i, j} = (Up - Low) \cdot ρ_{2} \cdot c_{i, j}^{'} \cdot 0.5 + x_{i}, \\ (11) & ρ_{2} = (exp (i / N) - 1) / (exp (1) - 1) . \end{array}

Fig. 3.

When $u = 4$ , the feature of chaotic map.

3.4. Multi method argument strategy

The main idea of MMAPSO algorithm is that: first, a best point $Pg$ is gained, which is converged by the method AGS or CS. If the offspring’s fitness is not smaller than the parent’s, then it means Pg does not update, we will suspect this point falling into a local, and try to use PCS with some probability ( $pcc$ ). All these search methods are parallel with LDIWPSO.

When and how to use PCS? We use $pc$ and $pcc$ to stand for the continuous updating times of $Pg$ and the probability of PCS, respectively. If $Pg$ updates, $pc = 0$ ; else $pc = pc + 1$ , and if $pc > 10$ , set $pc = 10$ . The Eq. (12) is used to calculate the probability of PCS. $\begin{matrix} (12) & pcc = 0.05 + 0.9 * (exp (pc) - 1) / (exp (10) - 1), \end{matrix}$ $p c$ changes from 1 to 10 which decides the $pcc$ changing from 0.05 to 0.95. That means if the $Pg$ does not update continuously, PCS will be used with greater probability.

When and how to use AGS or CS? The two search methods are mainly used for local search. The probability depends on their convergence speed for different functions, and is calculated by Eqs (13) and (14). $\begin{array}{l} (13) & \begin{matrix} {SpeedCS}_{t} = \{\begin{matrix} | F_{t} - F_{t}^{'} | / n, & chosen, \\ 1 / 6, & unchosen, \end{matrix} \\ {SpeedAGS}_{t} = \{\begin{matrix} | F_{t} - F_{t}^{'} | / M, & chosen, \\ 1 / 6, & unchosen \end{matrix} \end{matrix} \\ (14) & \begin{matrix} SpeedCS = \frac{1}{t} \sum_{t = 1}^{t} {SpeedCS}_{t}, \\ SpeedAGS = \frac{1}{t} \sum_{t = 1}^{t} {SpeedAGS}_{t}, \end{matrix} \end{array}$ where, t is the iteration time of the algorithm; n is the particle size; M is the gradient descent iteration time. If a search method is chosen, the value, the fitness of the original best position subtracts the fitness of the new best position and is divided by n or M, will be the convergence speed of this search method for one iteration (epoch), else set the speed with $1 / 6$ . Figure 4 shows the flowchart of MMAPSO, where, MaxDT is the maximum epoch time.

Fig. 4.

Flowchart of MMAPSO.

4. Analysis of experimental data

To prove the algorithm superiority, we compare MMAPSO algorithm with other six algorithms, PSO [5], LDIWPSO [19], HCPSO [11], GPSO [14], CLPSO [10] and SAPSO [25]. Their setting parameters are shown in Table 1.

Table 1
Setting parameters

Algorithm Parameters

PSO $ω = 0.7$ , $c 1 = c 2 = 1.4962$

LDIWPSO $ω = 0.9 - 0.5 t / MaxDT$ , $c 1 = c 2 = 1.4962$

HCPSO $ω = 0.7$ , $c 1 = c 2 = 1.4962$

GPSO $ω = 0.7$ , $c 1 = c 2 = 1.4962$

CLPSO $ω = 0.9 - 0.5 t / MaxDT$ , $c 1 = c 2 = 1.4962$

SAPSO $ω = 0.9 - 0.5 t / MaxDT$ , $c 1 = c 2 = 1.4962$

MMAPSO $ω = 0.9 - 0.5 t / MaxDT$ , $c 1 = c 2 = 1.4962$

Algorithm	Parameters
PSO	$ω = 0.7$ , $c 1 = c 2 = 1.4962$
LDIWPSO	$ω = 0.9 - 0.5 t / MaxDT$ , $c 1 = c 2 = 1.4962$
HCPSO	$ω = 0.7$ , $c 1 = c 2 = 1.4962$
GPSO	$ω = 0.7$ , $c 1 = c 2 = 1.4962$
CLPSO	$ω = 0.9 - 0.5 t / MaxDT$ , $c 1 = c 2 = 1.4962$
SAPSO	$ω = 0.9 - 0.5 t / MaxDT$ , $c 1 = c 2 = 1.4962$
MMAPSO	$ω = 0.9 - 0.5 t / MaxDT$ , $c 1 = c 2 = 1.4962$

The test parameters are shown in Table 2, the seven algorithms in twelve functions (Table 3) contain uni-modal, multi-modal and noise styles.

Table 2

Test parameters

Dimension	Particle size	Epoch	Sample
10	20	3000	30
20	30	3000	30
30	40	3000	30
100	40	3000	30

Table 3

Benchmarks for simulations

NO.	Func. name	Expression	Box constraint	Style
F1	Sphere	$f (x) = \sum_{i = 1}^{n} x_{i}^{2}$	$[- 10, 10] D$	uni
F2	Rosenbrock	$f (x) = \sum_{i = 1}^{n - 1} 100 {(x_{i + 1} - x_{i}^{2})}^{2} + {(x_{i} - 1)}^{2}$	$[- 10, 10] D$	uni
F3	Diff power	$f (x) = \sum_{i = 1}^{n} {\| x_{i} \|}^{2 + a_{i}}$ , where $a_{i} = 10^{\frac{i - 1}{n - 1}}$	$[- 10, 10] D$	uni
F4	Step	$f (x) = \sum_{i = 1}^{n} {(x_{i}^{} + 0.5)}^{2}$	$[- 10, 10] D$	uni
F5	Ellipse	$f (x) = \sum_{i = 1}^{n} {(a_{i} x_{i})}^{2}$ , $a_{i} = 20^{\frac{i - 1}{n - 1}}$	$[- 10, 10] D$	uni
F6	Schaffer	$f (x) = 0.5 + \frac{(sin \sqrt{x^{2} + y^{2}}) - 0.5}{{[1 + 0.001 (x^{2} + y^{2})]}^{2}}$	$[- 10, 10] D$	multi
F7	Rastrigin	$f (x) = \sum_{i = 1}^{n} x_{i}^{2} - 10 cos 2 π x_{i} + 10$	$[- 5, 5] D$	multi
F8	Ackley	$f (x) = 20 + e - 20 * exp (- 0.2 \sqrt{\frac{1}{n} \sum_{i = 1}^{n} x_{i}^{2}}) - exp (\frac{1}{n} \sum_{i = 1}^{n} cos (2 π x_{i}))$	$[- 10, 10] D$	multi
F9	Griewank	$f (x) = 1 + \frac{\sum_{i = 1}^{n} {(x_{i} - 100)}^{2}}{4000} - \prod_{i = 1}^{n} cos (\frac{x_{i} - 100}{\sqrt{i}})$	$[- 10, 10] D$	multi
F10	2D minima	$f (x) = 78.332331408 + \sum_{i = 1}^{n} (x_{i}^{4} - 16 x_{i}^{2} + 5 x_{i}) / n$	$[- 10, 10] D$	multi
F11	Salomon	$f (x) = 1 - cos (2 π \sqrt{\sum_{i = 1}^{n} x_{i}^{2}}) + 0.1 \sqrt{\sum_{i = 1}^{n} x_{i}^{2}}$	$[- 10, 10] D$	multi
F12	Noise quadric	$f (x) = \sum_{i = 1}^{n} i x_{i}^{4} + random (0, 1)$	$[- 10, 10] D$	Noise

4.1. Comparison on uni-modal functions

We test these algorithms with different dimensions in five uni-modal functions and run 30 times for each computing. The t-test of MMAPSO compared with other algorithms is based on the fitness value. The sample size is 30 and the Significance level is 0.05. Values +, − and = in the column ‘h’ denote MAMPSO performs significantly better or worse than compared algorithm. The results are shown in Tables 4, 5 and 6.

Table 4
Result for uni-modal functions

Algorithm D F1 Algorithm D F2

min mean std h min mean std h

PSO 10 0.0138 0.363 0.2 + PSO 10 1.1079 172.2422 1.40E+05 +

20 0.2018 3.5874 7.6482 + 20 52.12 856.42 5.96E+05 +

30 2.8615 8.4326 24.3968 + 30 601.6 1174 1.96E+05 +

100 75.3908 134.5173 791.146 + 100 2.96E+04 5.03E+04 3.17E+08 +

LDIWPSO 10 2.11E−06 0.001 1.65E−06 + LDIWPSO 10 6.91E−01 108.97 7.21E+04 +

20 0.0136 3.5329 332.2464 + 20 15.66 136.84 1.23E+04 +

30 0.0393 0.7468 0.8382 + 30 1.8408 116.53 8.68E+03 +

100 22.3151 152.0785 1976 + 100 624.46 2.91E+03 2.15E+06 +

HCPSO 10 0.0022 0.1167 0.0164 + HCPSO 10 1.0623 387.68 3.30E+06 +

20 0.3838 4.2344 8.7976 + 20 184.2 1303 2.32E+06 +

30 2.5091 6.9689 14.5434 + 30 525.22 1.63E+03 8.68E+05 +

100 113.89 145.69 1233 + 100 2.54E+04 3.02E+04 3.15E+07 +

GPSO 10 3.31E−318 5.61E−304 0 = GPSO 10 1.0415 2.92E+02 6.69E+05 +

20 1.48E−314 9.14E−304 0 = 20 413.94 3.05E+03 2.96E+07 +

30 4.21E−310 1.37E−303 0 = 30 816.65 1.89E+03 7.24E+05 +

100 2.11E−285 9.77E−279 0 = 100 6.09E+04 8.07E+04 8.23E+07 +

CLPSO 10 0.0348 0.293 0.1693 + CLPSO 10 1.6774 26.4676 763.64 +

20 1.0267 3.0429 1.6811 + 20 198.67 4.86E+02 6.55E+04 +

30 5.3244 8.92E+00 12.715 + 30 612.83 1.04E+03 2.98E+05 +

100 106.66 1.21E+02 134.09 + 100 3.55E+04 6.75E+04 6.75E+08 +

SAPSO 10 0.0016 0.3681 0.0106 + SAPSO 10 1.1111 71.3757 3.96E+04 +

20 0.0353 0.8583 1.5454 + 20 1.7879 3.30E+02 1.23E+05 +

30 0.5604 3.293 17.0892 + 30 24.69 1.57E+03 8.52E+06 +

100 1.8969 64.8743 1298 + 100 2.39E+03 9.28E+03 8.27E+07 +

MMAPSO 10 5.10E−205 3.40E−194 0 MMAPSO 10 1.00E+00 1.00E+00 9.81E−27

20 5.51E−196 3.28E−188 2.98E−219 20 1.00E+00 1.00E+00 2.32E−06

30 2.24E−188 1.89E−182 0 30 1.00E+00 1.01E+00 1.90E−05

100 5.81E−171 1.36E−158 9.29E−316 100 1.08E+00 1.11E+00 1.30E−03

Algorithm	D	F1	Algorithm	D	F2
PSO	10	0.0138	0.363	0.2	+	PSO	10	1.1079	172.2422	1.40E+05	+
20	0.2018	3.5874	7.6482	+	20	52.12	856.42	5.96E+05	+
30	2.8615	8.4326	24.3968	+	30	601.6	1174	1.96E+05	+
100	75.3908	134.5173	791.146	+	100	2.96E+04	5.03E+04	3.17E+08	+
LDIWPSO	10	2.11E−06	0.001	1.65E−06	+	LDIWPSO	10	6.91E−01	108.97	7.21E+04	+
20	0.0136	3.5329	332.2464	+	20	15.66	136.84	1.23E+04	+
30	0.0393	0.7468	0.8382	+	30	1.8408	116.53	8.68E+03	+
100	22.3151	152.0785	1976	+	100	624.46	2.91E+03	2.15E+06	+
HCPSO	10	0.0022	0.1167	0.0164	+	HCPSO	10	1.0623	387.68	3.30E+06	+
20	0.3838	4.2344	8.7976	+	20	184.2	1303	2.32E+06	+
30	2.5091	6.9689	14.5434	+	30	525.22	1.63E+03	8.68E+05	+
100	113.89	145.69	1233	+	100	2.54E+04	3.02E+04	3.15E+07	+
GPSO	10	3.31E−318	5.61E−304	0	=	GPSO	10	1.0415	2.92E+02	6.69E+05	+
20	1.48E−314	9.14E−304	0	=	20	413.94	3.05E+03	2.96E+07	+
30	4.21E−310	1.37E−303	0	=	30	816.65	1.89E+03	7.24E+05	+
100	2.11E−285	9.77E−279	0	=	100	6.09E+04	8.07E+04	8.23E+07	+
CLPSO	10	0.0348	0.293	0.1693	+	CLPSO	10	1.6774	26.4676	763.64	+
20	1.0267	3.0429	1.6811	+	20	198.67	4.86E+02	6.55E+04	+
30	5.3244	8.92E+00	12.715	+	30	612.83	1.04E+03	2.98E+05	+
100	106.66	1.21E+02	134.09	+	100	3.55E+04	6.75E+04	6.75E+08	+
SAPSO	10	0.0016	0.3681	0.0106	+	SAPSO	10	1.1111	71.3757	3.96E+04	+
20	0.0353	0.8583	1.5454	+	20	1.7879	3.30E+02	1.23E+05	+
30	0.5604	3.293	17.0892	+	30	24.69	1.57E+03	8.52E+06	+
100	1.8969	64.8743	1298	+	100	2.39E+03	9.28E+03	8.27E+07	+
MMAPSO	10	5.10E−205	3.40E−194	0		MMAPSO	10	1.00E+00	1.00E+00	9.81E−27
20	5.51E−196	3.28E−188	2.98E−219		20	1.00E+00	1.00E+00	2.32E−06
30	2.24E−188	1.89E−182	0		30	1.00E+00	1.01E+00	1.90E−05
100	5.81E−171	1.36E−158	9.29E−316		100	1.08E+00	1.11E+00	1.30E−03

Except for the function Sphere (F1) and $D = 10$ in function Step (F4), MMAPSO has the best min, mean and STD value. For the function Sphere, MMAPSO has not significant difference with GPSO. Because the gradient search method has the fastest convergence speed for function Sphere, and MMAPSO cost some computational resource on other search method, the result of GPSO is smaller than MMAPSO. For the function Step, when $D = 10$ , GPSO is even significantly better than MMAPSO. However, GPSO does not always have the second best result, which illustrates the gradient descent search method (AGS) is not always suitable for all uni-modal functions. The results of the function F2, F3 and F5 illustrate that the AGS does not have the fastest convergence speed for uni-modal functions either. Because of the work of CS, MMAPSO can converge faster than GPSO. Hence, MMAPSO has the better adaptability for each function. Figure 5 shows the convergence performance in function Ellipse. In this comparison, MMAPSO starts at the best position and converges with the fastest speed.

Table 5

Result for uni-modal functions

Algorithm	D	F3				Algorithm	D	F4

		min	mean	std	h			min	mean	std	h
PSO	10	1.27E−04	26.92	7.36E+03	+	PSO	10	0.0112	0.3475	0.3565	+
	20	0.1252	348.22	4.38E+05	+		20	0.9327	3.983	7.4547	+
	30	1.2352	3.23E+03	5.69E+07	+		30	4.1815	9.0948	16.0504	+
	100	2.13E+04	4.31E+04	7.13E+07	+		100	122.1728	186.6175	2.31E+03	+
LDIWPSO	10	1.35E−13	6.23E+03	3.50E+08	+	LDIWPSO	10	2.49E−06	0.0069	2.90E−04	+
	20	1.92E−05	911.22	1.58E+06	+		20	0.1756	0.7074	0.6855	+
	30	0.0084	2.54E+04	4.26E+09	+		30	0.3563	1.6801	2.8031	+
	100	1.03E+03	5.32E+06	6.15E+09	+		100	68.2084	160.42	3.86E+03	+
HCPSO	10	9.50E−06	3.3341	64.28	+	HCPSO	10	0.0112	0.3475	0.3565	+
	20	1.58E+00	43.0655	1.88E+03	+		20	1.71E+00	4.6619	3.9919	+
	30	1.83E+00	72.61	1.10E+04	+		30	6.2183	10.2153	14.3053	+
	100	4.98E+03	6.91E+04	1.81E+10	+		100	81.7479	107.5526	671.3986	+
GPSO	10	2.16E−14	3.69E+02	1.19E+06	+	GPSO	10	0	0	0	−
	20	5.90E−12	6.27E+04	1.96E+05	+		20	0	6.59E−13	1.30E−23	=
	30	1.35E−03	8.52E+04	5.61E+05	+		30	0	0	0	=
	100	6.06E+02	6.58E+05	6.57E+07	+		100	0	0	0	=
CLPSO	10	1.29E−04	1.0236	6.6064	+	CLPSO	10	0.0314	0.2688	0.0454	+
	20	9.49E−02	1.90E+01	518.44	+		20	1.5325	3.5954	2.1895	+
	30	1.55E+01	2.61E+03	6.17E+07	+		30	3.2373	7.23E+00	5.3999	+
	100	2.38E+03	5.56E+03	4.52E+06	+		100	87.9228	1.15E+02	598.62	+
SAPSO	10	1.97E−08	475	6.78E+06	+	SAPSO	10	0.0045	2.49E−01	0.0739	+
	20	7.40E−01	2.45E+03	2.10E+07	+		20	0.7565	1.29E+01	1.21E+03	+
	30	1.67E−01	6.35E+03	2.68E+08	+		30	3.1968	1.57E+01	1.26E+03	+
	100	1.07E+03	1.19E+04	1.27E+04	+		100	45.3806	9.00E+01	2.35E+03	+
MMAPSO	10	1.47E−25	1.91E−16	7.76E−31		MMAPSO	10	1.12E−02	3.48E−01	3.57E−01
	20	2.28E−15	5.16E−12	5.61E−23			20	0	0	0
	30	5.73E−12	2.42E−11	3.51E−22			30	0	0	0
	100	1.13E−10	2.85E−10	1.09E−20			100	0	0	0

Table 6

Result for uni-modal functions

Algorithm	D	F5

		min	mean	std	h
PSO	10	2.7632	73.61	1.61E+04	+
	20	28.54	198.03	1.68E+04	+
	30	101.12	619.86	1.24E+06	+
	100	3.29E+03	5.43E+03	4.37E+06	+
LDIWPSO	10	1.74E−02	128.72	6.66E+04	+
	20	24.74	629.72	7.90E+05	+
	30	113.95	1.51E+03	1.62E+06	+
	100	3.68E+03	7.73E+03	7.95E+05	+
HCPSO	10	2.7632	73.6181	1.61E+04	+
	20	39.1033	180.6093	3.50E+04	+
	30	55.8824	350.15	7.74E+04	+
	100	3.25E+03	4.82E+03	1.90E+06	+
GPSO	10	5.3879	1.12E+02	6.92E+04	+
	20	3.7114	3.83E+02	1.60E+05	+
	30	84.42	2.34E+02	1.17E+04	+
	100	3.90E+03	5.25E+03	1.07E+06	+
CLPSO	10	1.5007	10.3153	207.92	+
	20	32.7231	80.5833	967.19	+
	30	146.21	2.62E+02	4.67E+03	+
	100	3.33E+03	4.50E+03	5.33E+05	+
SAPSO	10	0.1737	1.19E+02	4.61E+04	+
	20	23.91	4.48E+02	3.32E+05	+
	30	86.05	1.46E+03	4.56E+06	+
	100	479.06	4.49E+03	2.87E+07	+
MMAPSO	10	2.73E−21	7.36E−04	6.54E−19
	20	4.46E−04	2.20E−03	1.14E−06
	30	1.30E−03	3.80E−03	5.32E−06
	100	3.34E−02	4.28E−02	2.47E−04

4.2. Comparison on multi-modal functions

For multi-modal functions, the solutions are easy to fall into the local. We use SR to denote the solution’s successful convergence rate. Each algorithm runs for 30 times. If one solution of the algorithm is below the accuracy level, the algorithm successfully converges for one time; otherwise, the algorithm will be considered falling into a local for one time. We set the different accuracy level according to different function solutions, shown in Table 7.

Tables 8, 9, 10 show the solutions of multi-modal functions. Because MMAPSO introduces CS method, which enlarges the search space, the best successful rate (SR) is gained. It means MMAPSO can jump out of the local well, both for the low dimension and the high dimension. Except 100 dimension test in function Griewank (F9) and 30, 100 dimension test in function 2D minima (F10), MMAPSO has significantly better solutions than the other algorithms. Because of the fast convergence of AGS, GPSO gain the second best solutions, however, comparing with MMAPSO, the SR of GPSO is smaller than MMAPSO. that means CS and PCS method works better.

Figures 6–8 show the convergence situation in function Rastrigin (F7), Ackley (F8) and 2D minima (F10), respectively. At the early stage, because of the gradient local search, the convergence performance for MMAPSO is much faster than other algorithms; at the late stage, MMAPSO can jump out of the local well too.

4.3. Comparison on noisy functions

The noisy function has the random and uncertain-able characteristics. Table 11 shows the solutions of the noisy function tested by the seven algorithms. because of the discrete characteristic the AGS method will be work-less, GPSO does not have the second best solutions. CS and PCS method will play the main role, and the MMAPSO has the best solutions, which illustrates that MMAPSO can be suitable for discrete problems.

4.4. Discussion of each search methods

In order to analyse the use of each search method, Table 12 shows the average numbers of use in MMAPSO for the twelve functions.

For some functions such as F1, F4, the numbers of use of AGS are much more than CS. This means the gradient method is quite effective. As the dimension increases to 100, the space becomes large, and CS will work more or less.

For some functions such as F2, F3, F5, F7 and F8, AGS has almost the same numbers of use with CS. This means AGS does not work well, and CS plays a main role for searching the solutions. However, the search speed of CS is not fast either, besides, their speeds are both lower than the default speed, hence, the two search methods have a fair probability.

For some multi modal functions such as F6, F9 and F10, PCS has much more numbers of use than other methods. This means the best position (Pg) of particle swarm does not update usually during the solution process. Hence, we will consider Pg falling to a local, the global search will be used necessarily.

For noisy function F12, the numbers of use for AGS are much more than others. It seems that AGS plays the main role. However, if the numbers of use for CS and PCS is zero, this algorithm will be equal to GPSO. Besides, the solution of GPSO in Table 7 shows that this algorithm does not work well as MMAPSO, because the random factor in the function misleads AGS. Hence, CS method is the real key role to improve the solution in this function.

To sum up, MMAPSO can adapt different functions by choosing different search methods flexibly.

Fig. 5.

Comparison on ellipse.

5. Application for control systems

5.1. Sliding mode control parameters optimization for hydraulic AGC system

Hydraulic AGC (Automatic Gap Control) system mainly uses the servo valve to control the position of hydraulic cylinder [3].

Servo valve port flow equation is: $\begin{matrix} (15) & Q_{L} = C_{d} ω x_{v} \sqrt{\frac{P_{s} - P_{L} sgn (x_{v})}{ρ}} . \end{matrix}$ Hydraulic cylinder flow equation is: $\begin{matrix} (16) & Q_{L} = A \overset{∙}{x} + C P_{L} + \frac{V}{4 β_{e}} \overset{∙}{P_{L}} . \end{matrix}$ The hydraulic cylinder force equation is: $\begin{matrix} (17) & A P_{L} = M \overset{∙ ∙}{x} + B \overset{∙}{x} + K x + F_{L} . \end{matrix}$ Equations (15) to (17) are written in the form of non-linear equations: $\begin{matrix} (18) & \begin{matrix} \overset{∙}{x_{1}} = x_{2}, \\ {\overset{∙}{x}}_{2} = x_{3}, \\ \overset{∙}{x_{3}} = a_{1} x_{1} + a_{2} x_{2} + a_{3} x_{3} + a_{4} g (x_{v}) u + d, \\ y = x_{1} . \end{matrix} \end{matrix}$ Where, $\begin{array}{l} a_{1} = - \frac{4 β_{e} C K}{M V}; \\ a_{2} = - \frac{K}{M} - \frac{4 β_{e}}{M V} (A^{2} + C B); \\ a_{3} = - \frac{B}{M} - \frac{4 β_{e} C}{V}; \\ a_{4} = \frac{4 β_{e} A C_{d} ω}{M V \sqrt{ρ}} K_{p} K_{s v}; \\ g (x_{v}) = \sqrt{P_{s} - P_{L} sgn (x_{v})}; \\ d = - \frac{F_{L}}{M} - \frac{4 β_{e} C}{M V} F_{L}; \end{array}$ $β_{e}$ is the hydraulic oil elastic modulus; C is total leakage coefficient of the hydraulic cylinder; K is load stiffness; M is the load weight; V is the volume of the hydraulic oil; A is effective area of hydraulic cylinder; B is damping coefficient; ρ is the hydraulic oil density; $C_{d}$ is valve port flow coefficient; ω is the opening gradients of the servo valve; $P_{s}$ is hydraulic pump outlet pressure; $F_{L}$ is the load; $K_{p}$ is servo amplifier proportion coefficient; $K_{s v}$ is servo valve proportional coefficient.

Table 7
Accuracy level for functions

Function F6 F7 F8 F9 F10 F11

Level 1.00E−03 1.00E−02 1.00E−01 1 1.00E−03 1.00E−03

Function	F6	F7	F8	F9	F10	F11
Level	1.00E−03	1.00E−02	1.00E−01	1	1.00E−03	1.00E−03

Table 8

Result for multi-modal functions

Algorithm	D	F6					Algorithm	D	F7

		SR	min	mean	std	h			SR	min	mean	std	h
PSO	10	0	0.0372	0.0627	0.001	+	PSO	10	0	0.7027	4.4666	20.5774	+
	20	0	0.0372	0.1134	0.0015	+		20	0	2.3363	10.3147	21.0943	+
	30	0	0.0782	0.138	0.0021	+		30	0	6.4053	11.8686	17.9388	+
	100	0	0.2277	0.2693	0.0023	+		100	0	115.82	141.19	640.03	+
LDIWPSO	10	0	0.0097	0.0599	0.002	+	LDIWPSO	10	0.4	4.92E−04	0.4858	6.69E−01	+
	20	0	0.0372	0.1087	0.0022	+		20	0	1.0931	1.2318	0.2396	+
	30	0	0.0782	0.1476	0.0029	+		30	0.03	0.0284	2.0705	1.1126	+
	100	0	0.1782	0.2763	0.0039	+		100	0	65.4	224.06	1.62E+04	+
HCPSO	10	0	0.0372	0.0627	0.001	+	HCPSO	10	0	0.7027	4.4666	20.5774	+
	20	0	0.0372	0.0675	0.0013	+		20	0	3.2769	9.0411	10.1988	+
	30	0	0.0372	0.0895	0.0013	+		30	0	9.4935	16.7747	36.1069	+
	100	0	0.1782	0.2504	0.0039	+		100	0	77.1193	151.88	2.43E+03	+
GPSO	10	0	0.0097	6.47E−02	0.0011	+	GPSO	10	0.63	3.55E−15	6.65E−01	1.254	+
	20	0	0.0782	1.28E−01	0.0011	+		20	0.33	1.07E−10	2.60E+00	12.56	+
	30	0	0.0782	1.53E−01	0.0013	+		30	0.37	5.84E−13	6.54E−01	0.3425	+
	100	0	2.73E−01	3.11E−01	6.64E−04	+		100	0.17	3.16E−13	1.42E+00	2.3942	+
CLPSO	10	0	0.0097	0.0676	9.16E−04	+	CLPSO	10	0	0.1356	1.2106	0.6742	+
	20	0	0.0782	0.148	0.0018	+		20	0	3.3679	4.8895	1.3006	+
	30	0	0.127	1.37E−01	4.66E−04	+		30	0	5.1052	1.03E+01	7.5378	+
	100	0	0.2727	3.04E−01	3.10E−04	+		100	0	91.3251	1.20E+02	506.24	+
SAPSO	10	0	0.0097	3.36E−02	1.10E−03	+	SAPSO	10	0.03	0.0169	5.87E−01	0.3597	+
	20	0	0.0372	8.16E−02	2.40E−03	+		20	0	0.7105	3.09E+00	11.5949	+
	30	0	0.0372	1.04E−01	2.20E−03	+		30	0	1.3464	7.75E+00	56.6497	+
	100	0	0.1782	2.53E−01	2.60E−03	+		100	0	14.1556	1.32E+02	1.77E+04	+
MMAPSO	10	1	0.00E+00	0.00E+00	0.00E+00		MMAPSO	10	1	0.00E+00	1.56E−11	6.22E−20
	20	1	0.00E+00	4.44E−17	1.92E−33			20	1	4.26E−14	4.26E−08	1.80E−14
	30	1	5.55E−17	1.27E−16	4.14E−33			30	1	3.19E−14	2.15E−05	4.63E−09
	100	0.6	1.11E−16	3.90E−03	2.83E−05			100	1	9.16E−11	2.30E−07	9.80E−14

Table 9

Result for multi-modal functions

Algorithm	D	F8					Algorithm	D	F9

		SR	min	mean	std	h			SR	min	mean	std	h
PSO	10	0	1.7941	3.4349	1.2959	+	PSO	10	1	0.0165	0.1723	0.0138	+
	20	0	2.6454	4.5691	1.3514	+		20	1	0.164	0.3328	0.0205	+
	30	0	3.5087	4.551	0.4165	+		30	1	0.1997	0.3966	0.0205	+
	100	0	5.9007	6.3144	0.0898	+		100	1	0.8195	0.9359	0.0044	+
LDIWPSO	10	0.2	7.10E−03	1.8582	2.77E+00	+	LDIWPSO	10	1	1.48E−02	0.1467	1.15E−02	+
	20	0.17	0.237	2.866	5.2028	+		20	1	0.0196	0.0837	0.0047	+
	30	0	1.4105	2.8574	1.0159	+		30	1	0.0025	0.1012	0.0246	+
	100	0	5.7219	6.6881	0.3815	+		100	1	0.5603	0.7209	0.0392	+
HCPSO	10	0	1.7941	3.4349	1.2959	+	HCPSO	10	1	0.0165	0.1723	0.0138	+
	20	0	2.8606	4.3266	0.9028	+		20	1	0.0761	0.2908	0.012	+
	30	0	2.8886	4.7581	0.6788	+		30	1	0.3096	0.4044	0.0086	+
	100	0	4.7323	5.5093	0.3239	+		100	1	0.8567	0.9178	0.0043	+
GPSO	10	0.4	0.0176	1.34E+00	1.7767	+	GPSO	10	1	2.22E−16	9.50E−02	0.0052	+
	20	0.1	0.1104	3.11E+00	2.6084	+		20	1	0	8.60E−03	8.10E−05	+
	30	0	2.3162	5.45E+00	3.068	+		30	1	2.22E−16	6.40E−03	8.10E−05	+
	100	0	6.34E+00	7.85E+00	1.3677	+		100	1	2.77E−15	4.70E−15	2.41E−30	=
CLPSO	10	0.1	0.3064	1.7128	0.3742	+	CLPSO	10	1	0.0197	0.0995	0.003	+
	20	0	2.6278	3.0129	0.0402	+		20	1	0.1506	0.2257	0.0048	+
	30	0	2.5306	3.35E+00	0.2669	+		30	1	0.2216	3.62E−01	0.0156	+
	100	0	5.4457	5.70E+00	0.0398	+		100	1	0.8201	8.74E−01	0.0037	+
SAPSO	10	0.5	0.033	9.89E−01	0.4619	+	SAPSO	10	1	0.0622	2.66E−01	0.022	+
	20	0	1.1211	2.23E+00	0.577	+		20	1	0.0216	1.14E−01	0.0148	+
	30	0	1.1313	2.48E+00	1.5993	+		30	1	0.0601	1.86E−01	0.0333	+
	100	0	4.5088	6.07E+00	2.0714	+		100	1	0.2004	7.42E−01	0.1021	+
MMAPSO	10	1	2.38E−11	1.24E−06	1.09E−11		MMAPSO	10	1	0.00E+00	4.81E−17	4.83E−33
	20	1	1.40E−03	3.60E−03	5.18E−06			20	1	0	4.99E−16	1.45E−31
	30	1	2.00E−03	5.80E−03	4.32E−06			30	1	2.22E−16	6.43E−16	9.80E−32
	100	1	1.14E−02	1.47E−02	1.05E−05			100	1	3.44E−15	5.61E−15	3.15E−30

Table 10

Result for multi-modal functions

Algorithm	D	F10					Algorithm	D	F11

		SR	min	mean	std	h			SR	min	mean	std	h
PSO	10	0	4.4537	15.1737	17.1748	+	PSO	10	0	0.2999	0.5899	0.0327	+
	20	0	8.8498	18.309	35.4495	+		20	0	0.5999	0.8999	0.0667	+
	30	0	19.7939	22.128	3.0101	+		30	0	0.7999	1.0199	0.0196	+
	100	0	32.642	34.3937	1.6142	+		100	0	1.6999	2.1199	0.087	+
LDIWPSO	10	0	2.83E+00	13.023	2.41E+01	+	LDIWPSO	10	0	9.99E−02	1.4199	8.70E−02	+
	20	0	8.7661	16.3964	12.0206	+		20	0	0.1999	0.5099	4.99E−02	+
	30	0	9.33	16.5298	21.2942	+		30	0	0.1999	0.6299	0.0601	+
	100	0	21.7923	41.8143	1.51E+03	+		100	0	0.8999	1.7999	0.4	+
HCPSO	10	0	4.4537	15.1737	17.1748	+	HCPSO	10	0	0.2999	0.5899	0.0327	+
	20	0	12.8673	16.4087	7.7703	+		20	0	0.4999	0.6499	0.0161	+
	30	0	14.365	18.5	4.7107	+		30	0	0.5999	0.8699	0.0423	+
	100	0	29.4513	33.461	5.3178	+		100	0	0.9999	1.4199	0.087	+
GPSO	10	0	5.7547	1.28E+01	11.1505	+	GPSO	10	0	0.2999	5.57E−01	0.0515	+
	20	0	4.241	1.20E+01	17.8752	+		20	0	0.5999	8.30E−01	0.0201	+
	30	0.2	4.57E−10	6.79E+00	27.5937	=		30	0	0.6999	9.50E−01	0.0206	+
	100	0.57	4.57E−10	5.49E+00	56.5007	=		100	0	1.70E+00	2.08E+00	0.102	+
CLPSO	10	0	0.0356	1.0329	1.6933	+	CLPSO	10	0	0.1999	0.3865	0.0136	+
	20	0	0.6369	1.8492	0.8392	+		20	0	0.0399	0.5899	0.0099	+
	30	0	1.7474	3.09E+00	1.1323	+		30	0	0.5999	8.20E−01	0.0129	+
	100	0	13.4371	1.68E+01	9.2671	+		100	0	1.8999	2.06E+00	0.013	+
SAPSO	10	0	2.8949	1.02E+01	22.0981	+	SAPSO	10	0	0.0999	2.50E−01	0.0488	+
	20	0	11.2995	1.44E+01	4.4219	+		20	0	0.1999	5.50E−01	0.1383	+
	30	0	10.1725	1.63E+01	11.9832	+		30	0	0.0999	5.60E−01	0.1649	+
	100	0	24.5275	2.77E+01	12.943	+		100	0	1.2999	1.70E+00	0.175	+
MMAPSO	10	0.23	4.57E−10	8.58E+00	3.72E+01		MMAPSO	10	0.9	1.62E−11	1.00E−02	9.28E−04
	20	0.3	4.57E−10	6.22E+00	3.56E+01			20	0.8	4.47E−04	2.50E−03	9.66E−04
	30	0.13	4.57E−10	6.13E+00	36.5646			30	1	8.35E−04	2.50E−03	2.07E−06
	100	0.5	4.57E−10	1.58E+00	7.0586			100	0	9.99E−02	9.99E−02	0.00E+00

The sliding face is: $\begin{matrix} (19) & S = c_{1} x_{1} + c_{2} x_{2} + x_{3} + \int (x_{1} - x_{d}) d t . \end{matrix}$

If $\dot{S} = 0$ , then $\begin{array}{l} \tilde{u} = & \frac{1}{g (x_{v})} [- \sum_{i = 1}^{3} {\tilde{a}}_{i} x_{i} - c_{1} x_{2} - c_{2} x_{3} - c_{3} x_{1} \\ (20) & + c_{3} x_{d} - d] . \end{array}$

The uncertainty is $\begin{matrix} (21) & \sum_{i = 1}^{3} Δ a_{i} x_{i} + Δ d \end{matrix}$ set $\begin{matrix} (22) & | \sum_{i = 1}^{3} Δ a_{i} x_{i} + Δ d | ⩽ q, q > 0 . \end{matrix}$

The control method is: $\begin{matrix} (23) & u = \tilde{u} + Δ u . \end{matrix}$ Where, $\begin{array}{l} Δ u = - \frac{1}{g (x_{v})} (q * sat (s / ξ) + γ * s), \\ sat (s / ξ) = \{\begin{matrix} 1, & s > ξ \\ s / ξ, & s ⩽ ξ \end{matrix} (ξ > 0) . \end{array}$

It is easy to prove $s \dot{s} = - s * sat (s / ξ) - γ s^{2} < 0$ , which illustrates this method is convergence.

From $\dot{s} = 0$ , we do the Laplace transform, to get $x_{1} = \frac{c_{3}}{s^{3} + c_{2} s^{2} + c_{1} s + c_{3}} x_{d}$ . Next, through reasonable pole assignment, the value of $c_{1}, c_{2}, c_{3}$ can be obtained. Finally, we choose three poles as K1, K2 + K3j, K2 − K3j, the ranges of K1, K2, K3 are $[- 100, 100000]$ , $[- 100, - 10000]$ , $[100, 10000]$ , respectively.

We use PSO algorithm to calculate the reasonable $c_{1}, c_{2}, c_{3}, h$ value, to make the goal function F minimalism.

After calculating the system parameter, we get $\begin{array}{l} a_{1} = - 619.25; a_{2} = - 9.5 e 6; \\ a_{3} = - 1.5 e 3; a_{4} = 4; q = 9 e 5 . \end{array}$

The objective function F expression is: $\begin{matrix} (24) & F = max e_{j}^{2} + ω \sum e_{j}^{2}, j \in [i, i + t] . \end{matrix}$

Fig. 6.

Comparison on Rastrigin.

Fig. 7.

Comparison on Ackley.

Fig. 8.

Comparison on 2D minima.

At this time, the problem is transfer to: $\begin{array}{l} min (F), \\ s.t. - 10000 < K 1 < - 100, \\ - 10000 < K 2 < - 100, \\ 100 < K 3 < 10000 . \end{array}$

The parametric settings are that: the particle swarm size is 10, the iteration time is 30, the results optimized by PSO and MMAPSO are shown in Table 13. The step response is shown in Fig. 9. It can illustrate that the system performance optimized by MMAPSO is better than the one of PSO algorithm.

Table 11

Result for noisy functions

Algorithm	D	F12

		min	mean	std	h
PSO	10	0.0115	1.9317	8.5093	+
	20	4.8738	62.1786	2.45E+03	+
	30	60.22	328.15	4.90E+04	+
	100	1.32E+04	1.74E+04	7.56E+06	+
LDIWPSO	10	1.07E−01	0.5764	8.02E−02	+
	20	0.1335	1.3378	1.0022	+
	30	1.1863	1.00E+03	9.99E+06	+
	100	97.12	6.90E+04	2.00E+10	+
HCPSO	10	0.0115	1.9317	8.5093	+
	20	6.9289	49.9169	4.60E+03	+
	30	82.9325	161.73	2.87E+03	+
	100	5.09E+03	1.22E+04	7.02E+07	+
GPSO	10	0.0021	2.65E+00	40.9559	+
	20	9.0672	3.62E+01	577.64	+
	30	75.0773	1.72E+02	9.14E+03	+
	100	6.37E+03	2.34E+04	1.31E+08	+
CLPSO	10	0.2067	1.3585	2.3185	+
	20	3.8585	20.0923	534.6548	+
	30	35.3174	1.03E+02	5.23E+03	+
	100	6.32E+03	1.83E+04	9.85E+07	+
SAPSO	10	0.4589	3.30E+01	2.96E+03	+
	20	118.18	5.91E+02	1.66E+05	+
	30	543.22	6.94E+03	8.67E+07	+
	100	7.49E+04	1.73E+05	2.83E+10	+
MMAPSO	10	8.07E−02	5.32E−01	8.62E−02
	20	1.18E−02	3.96E−01	7.52E−02
	30	8.32E−02	5.05E−01	0.1016
	100	5.38E−02	4.16E−01	0.1457

5.2. Optimization for PID controller

Another example is to correct the steel plate by hydraulic Straightener machine [16]. The transfer function model of the system is: $\begin{matrix} (25) & \frac{17.424}{s (\frac{s^{2}}{208^{2}} + \frac{0.4}{208} s + 1) (\frac{s^{2}}{500^{2}} + \frac{s}{500} + 1)} . \end{matrix}$

The PID discretion control equation is: $\begin{array}{l} Kp * e (k) + Ki * \sum_{i = 0}^{k} e (i) \\ (26) & + Kd [e (k) - e (k - 1)] . \end{array}$

We hope that the system can reach the steady state quickly and accurately. Hence, the objective function is: $\begin{array}{l} T = & \int_{0}^{\infty} (ω 1 \cdot | e (t) | + ω 2 \cdot u^{2} (t)) d t \\ (27) & + ω 3 \cdot t_{u}, \end{array}$ where, $| e (t) |$ is the error; $t_{u}$ is the rise time; $u (t)$ is the output of the controller; $ω 1$ , $ω 2$ , $ω 3$ are the weight. In order to prevent the overshoot problem of system, we use the punish function; once the system’s overshoot happens, the objective function will be: $\begin{array}{rcl} T & = & \int_{0}^{\infty} (ω 1 | e (t) | + ω 2 u^{2} (t) + ω 4 | e y (t) |) d t \\ (28) & + ω 3 \cdot t_{u}, \end{array}$ where, $ω 4 ≫ ω 1$ , $e y (t) = y (t) - y (t - 1)$ , $y (t)$ is the output of the object controlled. We set $ω 1 = 0.5$ , $ω 2 = 1$ , $ω 3 = 1$ , $ω 4 = 200$ , the range of $Kp$ , $Ki$ , $Kd$ is $[0.2, 10]$ , $[1, 50]$ and $[1 E - 7, 1 E - 1]$ respectively, and the overshoot should be smaller than 20%. Therefore, we get the problem: $\begin{array}{l} min T \\ s.t. 1 e - 2 ⩽ Kp ⩽ 2 \\ 0.1 ⩽ K i ⩽ 10 \\ 1 e - 10 ⩽ Kd ⩾ 1 e - 1 . \end{array}$

Noticed that the goal function is dynamic and discrete, we choose CLPSO, PSO and MMAPSO to optimize this problem and compare them. The parametric settings are: swarm size $N = 30$ , function evaluations $FEs = 3000$ . Table 14 shows the results which illustrates that the results optimized by MMAPSO have the smallest goal function value, and the mean error. Hence, MMAPSO is more efficient.

6. Conclusion

We combine LDIWPSO algorithm with some other search methods such as AGS, CS and PCS, to propose a hybrid algorithm MMAPSO. AGS and CS are used for local search, and PCS is used for global search. These methods can improve the convergence speed and the ability of jumping out of the local. For each function, MMAPSO can adapt different functions choosing different search method with the argument strategy. After the numerical experiments, the MMAPSO algorithm has the best convergence speed in dealing with uni-modal functions, the best successful rate in dealing with multi-modal functions and the best solution in dealing with noisy functions. The application examples of control system prove this algorithm can gain a better optimistic result than PSO. The future work is to compare this method to other algorithms mentioned in this paper and apply this algorithm to other fields.

Table 12

Numbers of use for search method in MMAPSO

Dimension	AGS	CS	PCS	Dimension	AGS	CS	PCS	Dimension	AGS	CS	PCS
F1				F2				F3

10	960	15	525	10	291	291	918	10	709	708	83
20	1133	7	360	20	693	693	114	20	689	688	123
30	1153	9	338	30	692	692	116	30	688	685	127
100	1129	123	248	100	557	557	386	100	683	680	137
F4				F5				F6

10	306	14	1180	10	694	696	130	10	143	14	1343
20	364	11	1125	20	684	686	130	20	172	4	1324
30	364	11	1125	30	684	686	130	30	172	4	1324
100	400	34	1066	100	683	685	132	100	208	64	1228
F7				F8				F9

10	316	285	899	10	653	654	193	10	92	45	1363
20	696	694	110	20	694	695	111	20	116	86	1298
30	696	694	110	30	694	695	111	30	116	86	1298
100	732	650	118	100	684	685	131	100	209	191	1100
F10				F11				F12

10	432	437	631	10	681	683	136	10	1326	172	2
20	120	117	1263	20	684	685	131	20	1241	257	2
30	120	117	1263	30	684	685	131	30	1241	257	2
100	162	139	1199	100	611	613	276	100	1109	389	2

Table 13

Comparison on pole parameters

Parameter	PSO	MMAPSO
K1	−1214	−8914
K2	−2732	−1058
K3	3406	258

Fig. 9.

Step response of sliding mode control by PSO and MMAPSO algorithm.

Table 14

Comparison on PID optimization

Variables	CLPSO	PSO	MMAPSO
$Kp$	3.0934	2.619	4.2937
$Ki$	17.2368	16.217	23.96
$Kd$	0.042	7.33E−05	0.0956
overshoot	10.79%	11.93%	10.95%
Mean error	0.0099	0.0101	0.0084
T	2261	2271	2226

Authors statements

The authors declare that there is no interest conflict between them.

References

Agrawal and

Silakari, Fletcher–Reeves based particle swarm optimization for prediction of molecular structure, Journal of Molecular Graphics and Modelling 49 (2014), 11–17. doi:10.1016/j.jmgm.2014.01.001.

Alfi and

M.-M.

Fateh, Intelligent identification and control using improved fuzzy particle swarm optimization, Expert Systems With Applications 38 (2011), 12312–12317. doi:10.1016/j.eswa.2011.04.009.

Cerman and

Hušek, Adaptive fuzzy sliding mode control for electro-hydraulic servo mechanism, Expert Systems With Applications 39 (2012), 10269–10277. doi:10.1016/j.eswa.2012.02.172.

Cura, A particle swarm optimization approach to clustering, Expert Systems With Applications 39 (2012), 1582–1588. doi:10.1016/j.eswa.2011.07.123.

Kenndy and

Eberhart, Particle swarm optimization, in: Proceedings of IEEE International Conference on Neural Networks, 1995, pp. 1942–1948. doi:10.1109/ICNN.1995.488968.

Kennedy and

R.C.

Eberhart, A discrete binary version of the particle swarm algorithm, in: Systems, Man, and Cybernetics, Computational Cybernetics and Simulation, IEEE International Conference on, IEEE, 1997, pp. 4104–4108.

Kongnam and

Nuchprayoon, A particle swarm optimization for wind energy control problem, Renewable Energy 35 (2010), 2431–2438. doi:10.1016/j.renene.2010.02.020.

M.-S.

Leu and

M.-F.

Yeh, Grey particle swarm optimization, Applied Soft Computing 12 (2012), 2985–2996. doi:10.1016/j.asoc.2012.04.030.

Li, The cloud control method and balancing patterns of triple link inverted pendulum systems, Chinese Engineering Science 1 (1999), 41–46.

10.

J.J.

Liang,

A.K.

Qin,

P.N.

Suganthan and

Baskar, Comprehensive learning particle swarm optimizer for global optimization of multimodal functions, Evolutionary Computation, IEEE Transactions on 10 (2006), 281–295.

11.

Liu,

Wang,

Y.-H.

Jin,

Tang and

D.-X.

Huang, Improved particle swarm optimization combined with chaos, Chaos, Solitons and Fractals 25 (2005), 1261–1271. doi:10.1016/j.chaos.2004.11.095.

12.

MirHassani and

Abolghasemi, A particle swarm optimization algorithm for open vehicle routing problem, Expert Systems With Applications 38 (2011), 11547–11551. doi:10.1016/j.eswa.2011.03.032.

13.

Nasir,

Das,

Maity,

Sengupta,

Halder and

P.N.

Suganthan, A dynamic neighborhood learning based particle swarm optimizer for global numerical optimization, Information Sciences 209 (2012), 16–36. doi:10.1016/j.ins.2012.04.028.

14.

M.M.

Noel, A new gradient based particle swarm optimization algorithm for accurate computation of global minimum, Applied Soft Computing 12 (2012), 353–359. doi:10.1016/j.asoc.2011.08.037.

15.

Paliwal,

Patidar and

Nema, Determination of reliability constrained optimal resource mix for an autonomous hybrid power system using particle swarm optimization, Renewable Energy 63 (2014), 194–204. doi:10.1016/j.renene.2013.09.003.

16.

X.-r.

Peng, The high-precision control of the large hydraulic straightening machine, in: 2013 Fourth International Conference on Digital Manufacturing and Automation (ICDMA), IEEE, 2013, pp. 1464–1467. doi:10.1109/ICDMA.2013.350.

17.

Peram,

Veeramachaneni and

C.K.

Mohan, Fitness-distance-ratio based particle swarm optimization, in: Swarm Intelligence Symposium, 2003. SIS’03, Proceedings of the 2003, IEEE, 2003, pp. 174–181. doi:10.1109/SIS.2003.1202264.

18.

Pookpunt and

Ongsakul, Optimal placement of wind turbines within wind farm using binary particle swarm optimization with time-varying acceleration coefficients, Renewable Energy 55 (2013), 266–276. doi:10.1016/j.renene.2012.12.005.

19.

Shi and

Eberhart, A modified particle swarm optimizer, in: Evolutionary Computation Proceedings, IEEE World Congress on Computational Intelligence, The 1998 IEEE International Conference on, IEEE, 1998, pp. 69–73.

20.

Shi and

R.C.

Eberhart, Empirical study of particle swarm optimization, in: Evolutionary Computation, CEC 99. Proceedings of the 1999 Congress on, IEEE, 1999.

21.

H.-L.

Shieh,

C.-C.

Kuo and

C.-M.

Chiang, Modified particle swarm optimization algorithm with simulated annealing behavior and its numerical verification, Applied Mathematics and Computation 218 (2011), 4365–4383. doi:10.1016/j.amc.2011.10.012.

22.

Tanweer,

Suresh and

Sundararajan, Self regulating particle swarm optimization algorithm, Information Sciences 294 (2015), 182–202. doi:10.1016/j.ins.2014.09.053.

23.

J.S.

Vesterstrøm and

Riget, A diversity-guided particle swarm optimizer – The ARPSO, Evalife Technical Report, 2002.

24.

Voglis,

K.E.

Parsopoulos and

I.E.

Lagaris, Particle swarm optimization with deliberate loss of information, Soft Computing 16 (2012), 1373–1392. doi:10.1007/s00500-012-0841-5.

25.

Xinchao, Simulated annealing algorithm with adaptive neighborhood, Applied Soft Computing 11 (2011), 1827–1836. doi:10.1016/j.asoc.2010.05.029.

26.

Xu,

Geng,

Zhu and

Gu, A piecewise linear chaotic map and sequential quadratic programming based robust hybrid particle swarm optimization, Information Sciences 218 (2013), 85–102. doi:10.1016/j.ins.2012.06.003.

27.

Z.-H.

Zhan,

Zhang,

Li and

Y.-H.

Shi, Orthogonal learning particle swarm optimization, Evolutionary Computation, IEEE Transactions on 15 (2011), 832–847.

Particle swarm optimization algorithm with multi methods argument

Abstract

Keywords

1. Introduction

2. PSO and improved PSO

2.1. PSO algorithm

2.2. Improved PSO

3. Multi Methods Argument Particle Swarm Optimization Algorithm (MMAPSO)

3.2. Cloud search (CS)

4.3. Comparison on noisy functions

4.4. Discussion of each search methods

5.1. Sliding mode control parameters optimization for hydraulic AGC system

Table 7 Accuracy level for functions Function F6 F7 F8 F9 F10 F11 Level 1.00E−03 1.00E−02 1.00E−01 1 1.00E−03 1.00E−03

6. Conclusion

Authors statements

References

Table 7
Accuracy level for functions

Function F6 F7 F8 F9 F10 F11

Level 1.00E−03 1.00E−02 1.00E−01 1 1.00E−03 1.00E−03