Game theoretic-based optimal load frequency control of power systems with network-induced delays

Abstract

This paper presents a game theoretic-based load frequency control (LFC) scheme for power systems with network-induced delays. First, a dynamic model of two-area LFC systems is developed under consideration of bounded network-induced delays. Second, the optimal control problem of power systems with network-induced delays is formulated as a stochastic linear quadratic game. Then, by using differential games and Lyapunov theories, the game theoretic-based optimal load frequency controllers are designed for the system under consideration, and an algorithm is presented to make the desired optimal LFC gains solvable. Finally, a case study is carried out to show the effectiveness of the proposed method.

Keywords

Load frequency control modelling Nash equilibrium network-induced delays power systems

Introduction

The control of power plants and power systems has received much attention in the literature (Bahgaat et al. 2014; Huang et al. 2017; Ma et al. 2017; Zhang and Xu 2017). Load frequency control (LFC) is a very important issue in the dynamical operation of interconnected power systems (Shayeghi et al. 2009). The main objective of LFC is to maintain the stability of frequency by balancing the generation and loads (Mi et al. 2013; Peng et al. 2018). LFC for multi-area power systems is a new challenge in smart grids. Especially in interconnected power systems, when a control area is not able to maintain stability of frequency by itself, other control areas will offer temporary power support. In multi-area power systems, control areas have different LFC capacities. For example, the control areas with a strong LFC capacity will contribute more than those with weak LFC capacities. However, it is still difficult to quantify or compensate extra costs of temporary power support (Dou and Liu 2013; Islam et al. 2017; Snodgrass 1990). In general, different control areas have different objectives during the operation process. Thus, the optimal running of the multi-area power systems is a kind of optimal decision-making process for the multiple stakeholders.

Game theory is an effective optimization tool to solve the optimal decision-making problem for multiple stakeholders. In this situation, game theory is one of the key mathematical tools in the analysis and design of future smart grids, as well as other complex systems. In the past few decades, game theory has been adopted in some disciplines such as economics and military studies. In recent years, game theory has also been an important mathematical tool in the analysis and design of power systems (Fang et al. 2012; Saad et al. 2012), and some good results have been achieved. For example, game theoretic energy consumption scheduling was designed by Mohsenian-Rad et al. (2010) for demand-side management in terms of minimizing energy costs; a game theoretic approach dealing with the control decision process of individual sources and loads in small-scale and DC power systems was proposed by Weaver and Krein (2009); differential game theory was utilized by Ye et al. (2011) to solve the coordination problem between primary and secondary frequency controls; and a cooperative control scheme using differential games as a possible solution to study load frequency control of multi-area power systems was investigated by Chen et al. (2015). Since the system components in smart grids are connected by communication networks, the non-ideal quality of services of communication networks, such as network-induced delays, cannot be neglected in practical engineering. However, the effects of the communication networks are not well considered in the above-mentioned works. Although network-induced characteristics are taken into account by Pal and Negi (2017), Peng et al. (2017a), Peng et al. (2017b) and Wang and Han (2018); Wang et al. (2018, 2017, 2016a,b) – refer to the survey papers by Gupta and Chow (2010), Zhang et al. (2017a) and Zhang et al. (2017b) for recent developments – the investigated control plants are general network-based systems instead of power systems.

The communication network is a very important element in complex power grids. There are some novel results available discussing the effects of a communication network. For example, the delay-dependent stability of a power system was studied by Duan et al. (2017); under consideration of signal transmission delays, a wide-area measurements-based two-level control design problem was investigated by Dotta et al. (2009); based on Lyapunov theory and linear matrix inequalities, the delay-dependent stability of the LFC scheme was studied by Jiang et al. (2012); and a delay-dependent robust approach was presented by Zhang et al. (2013) for analysis/synthesis of a traditional PID(proportional-integral-derivative)-type LFC scheme. Note that Nash equilibrium of game theory is a suitable scheme to coordinate the common and contradictory objectives between different control areas (Chen et al. 2015). However, network-induced delays are not well addressed in the LFC of interconnected power systems. With the development of wireless networks and large-scale complex power systems, how to propose effective schemes to reduce the effects of network-induced delays in power systems is of paramount importance. This motivates the current study.

This paper presents a game theoretic-based optimal LFC scheme for a two-area power system with network-induced delays. The cooperative control problem of the two-area power system with communication networks is formulated as a stochastic linear quadratic game, and then used to coordinate the common and contradictory objectives between different control areas. The performance analysis demonstrates that the stability and dynamic performance of the two-area power system can be guaranteed by utilizing the designed load frequency controller. The main contributions of this paper are summarized as follows:

A discrete-time augmented model for the two-area power system is well constructed under consideration of bounded network-induced delays. This model reflects more realistic dynamical behaviours of power systems with network-induced delays.

The optimal control problem of the two-area power system with network-induced delays is formulated as a stochastic linear quadratic game.

The game theoretic-based optimal load frequency controllers are designed for the studied system withnetwork-induced delays, while achieving the Nash equilibrium and guaranteeing the exponential mean-square stability. The designed controllers can facilitate coordinating the common objectives and contradictory objectives among different control areas.

Notation: throughout this paper, $N$ denotes the set of positive integers, $R^{n}$ denotes the n-dimensional Euclidean space, $R^{m \times n}$ denotes the set of $m \times n$ real matrices, $O_{n}$ is a $n \times n$ zero matrix and $I_{n}$ is an $n \times n$ identity matrix.

Dynamic modelling and problem formulation

The purpose of this section is to develop a dynamic LFC model for the power system with network-induced delays, and provide an optimal cost function to find the Nash equilibrium solutions for every area of the power system. A continuous-time model is introduced for the two-area LFC power system with network-induced delays, then the constructed model is converted to a discrete-time model; following this, the optimal cost function is provided for the solution of a stochastic linear quadratic differential game.

Two-area LFC modelling with network-induced delays

A two-area LFC system considering network-induced delays is given in Figure 1. The two-area power system consists of generators, turbines and governors, where $Δ {\bar{X}}_{gi}$ , $Δ {\bar{P}}_{gi}$ , $Δ {\bar{P}}_{di}$ , and $Δ {\bar{f}}_{i}$ with $i = 1, 2$ denote the deviation of valve position, the generator mechanical output, the load, and the frequency of the ith-area LFC power system, respectively. $T_{gi}$ , $T_{ti}$ , and $T_{pi}$ are time constants of the governor, the turbine and the electric system of the ith-area LFC power system, respectively. $K_{pi}$ , $r_{i}$ and $T_{12}$ are the electric system gain, the speed drop and the tie-line synchronizing coefficient, respectively.

Inspired by Peng and Zhang (2016), the continuous-time linear dynamic system in Figure 1 can be described as:

\overset{\cdot}{\bar{x}} (t) = A \bar{x} (t) + B_{1} {\bar{u}}_{1} (t) + B_{2} {\bar{u}}_{2} (t) + \tilde{E} Δ {\bar{P}}_{d} (t),

(1)

where

\begin{matrix} \bar{x} (t) = [Δ {\bar{f}}_{1} (t) Δ {\bar{P}}_{g 1} (t) Δ {\bar{X}}_{g 1} (t) Δ {\bar{f}}_{2} (t) Δ {\bar{P}}_{g 2} (t) Δ {\bar{X}}_{g 2} (t) \\ Δ {\bar{P}}_{tie} (t) Δ {\bar{P}}_{c 1} (t) Δ {\bar{P}}_{c 2} (t {)]}^{T} \end{matrix}

\bar{u} (t) = [{\bar{u}}_{1} (t) {\bar{u}}_{2} (t {)]}^{T}

\begin{matrix} A = [\begin{matrix} \frac{- 1}{T_{p 1}} & \frac{K_{p 1}}{T_{p 1}} & 0 & 0 & 0 & 0 & \frac{- K_{p 1}}{T_{p 1}} & 0 & 0 \\ 0 & \frac{- 1}{T_{t 1}} & \frac{1}{T_{t 1}} & 0 & 0 & 0 & 0 & 0 & 0 \\ \frac{- 1}{r_{1} T_{g 1}} & 0 & \frac{- 1}{T_{g 1}} & 0 & 0 & 0 & 0 & \frac{1}{T_{g 1}} & 0 \\ 0 & 0 & 0 & \frac{- 1}{T_{p 2}} & \frac{K_{p 2}}{T_{p 2}} & 0 & \frac{- K_{p 2}}{T_{p 2}} & 0 & 0 \\ 0 & 0 & 0 & 0 & \frac{- 1}{T_{t 2}} & \frac{1}{T_{t 2}} & 0 & 0 & 0 \\ 0 & 0 & 0 & \frac{- 1}{r_{2} T_{g 2}} & 0 & \frac{- 1}{T_{g 2}} & 0 & 0 & \frac{1}{T_{g 2}} \\ T_{12} & 0 & 0 & T_{12} & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{matrix}] \\ B_{1} = {[\begin{matrix} 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 \end{matrix}]}^{T} \\ B_{2} = {[\begin{matrix} 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \end{matrix}]}^{T} \\ \tilde{E} = {[\frac{- K_{p 1}}{\begin{matrix} T_{p 1} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & \frac{- K_{p 2}}{T_{p 2}} & 0 & 0 & 0 & 0 & 0 \end{matrix}}]}^{T} Δ {\bar{P}}_{d} (t) = [Δ {\bar{P}}_{d 1} Δ {\bar{P}}_{d 2}]^{T} \end{matrix}

The power perturbation $Δ {\bar{P}}_{di}$ in Figure 1 is assumed to be a known constant obtained by forecasting technology. Therefore, $Δ {\bar{P}}_{di}$ is a known step perturbation. One can adopt the method presented by Fosha and Elgerd (1970) and Pham et al. (2016) to eliminate $\tilde{E} Δ {\bar{P}}_{d} (t)$ by redefining the vectors of state, control and disturbance as follows:

\begin{matrix} x (t) = \bar{x} (t) - \tilde{x} (t) \\ u (t) = \bar{u} (t) - \tilde{u} (t) \\ Δ P_{d} (t) = Δ {\bar{P}}_{d} (t) - Δ {\tilde{P}}_{d} (t), \end{matrix}

where $\tilde{x} (t)$ , $\tilde{u} (t)$ and $Δ {\tilde{P}}_{d} (t)$ are the steady-state values of state, control and disturbance, respectively, while

\begin{matrix} \tilde{x} (t) = [0 Δ {\tilde{P}}_{g 1} (t) Δ {\tilde{X}}_{g 1} (t) 0 Δ {\tilde{P}}_{g 2} (t) Δ {\tilde{X}}_{g 2} (t) 0 Δ {\tilde{P}}_{c 1} (t) Δ {\tilde{P}}_{c 2} (t {)]}^{T} \\ \tilde{u} (t) = [0 0]^{T}, Δ {\tilde{P}}_{d} (t) = [Δ {\tilde{P}}_{d 1} (t) Δ {\tilde{P}}_{d 2} (t {)]}^{T} \\ Δ {\tilde{P}}_{gi} (t) = Δ {\tilde{X}}_{gi} (t) = Δ {\tilde{P}}_{ci} (t) = Δ {\tilde{P}}_{di} (t), i = 1, 2 . \end{matrix}

Based on the above given formulas, one can construct a discrete-time model for the two-area power system.

Networked discrete-time modelling of the two-area power system

This paper aims to design the game theoretic optimal load frequency controllers under consideration of communication networks. Since network-induced delays are unavoidable, the sampled state data cannot be directly used by system (1).

To facilitate modelling, one can assume that (1) sensors are clock-driven with a constant sampling period T; (2) controllers and actuators are event-driven; (3) the network-induced delays are bounded, and the network-induced delays of each controller are mutually independent; and (4) the initial state of the system is deterministic.

It is assumed that network-induced delays $τ_{k}$ are upper bounded with $τ_{k} < hT$ , where h is a positive integer. During a sampling period, the actuator will keep the latest available control input until another control input arrives at the actuator.

Based on the above description, one can see that the control inputs $u_{i} (t)$ available to the controlled plant should be piecewise constant, and there are at most $h + 1$ various values during every sampling period, where $i = 1, 2$ . The $u_{i} (t)$ updates at random time points $kT + t_{j}^{k}, j = 0, 1, \dots, h$ , and $t_{j}^{k} > t_{j + 1}^{k}$ ; see Figure 2 for more details. From system (1), one obtains

x_{k + 1} = A_{d} x_{k} + \sum_{j = 0}^{h} B_{1 j}^{k} u_{1, k - j} + \sum_{j = 0}^{h} B_{2 j}^{k} u_{2, k - j},

(2)

where $A_{d} = e^{AT}$ , $B_{ij}^{k} = \int_{t_{j}^{k}}^{t_{j - 1}^{k}} e^{A (T - s)} ds B_{i}$ , $t_{h}^{k} = 0$ , $t_{- 1}^{k} = T$ , $i = 1, 2$ .

Defining $z_{k} = [x_{k}^{T}, u_{1, k - 1}^{T}, \dots, u_{1, k - h}^{T}, u_{2, k - 1}^{T}, \dots, u_{2, k - h}^{T}]$ , one can obtain the following model:

z_{k + 1} = \bar{A} z_{k} + {\bar{B}}_{1} u_{1, k} + {\bar{B}}_{2} u_{2, k}

(3)

where

\begin{matrix} \bar{A} = [\begin{matrix} A_{d} & A_{12} & A_{13} \\ 0 & 0 & 0 \\ 0 & A_{32} & 0 \\ 0 & 0 & 0 \\ 0 & 0 & A_{53} \end{matrix}] \\ {\bar{B}}_{1} = {[\begin{matrix} B_{10}^{kT} & I_{m}^{T} & \underset{2 h - 1}{\underset{︸}{0, \dots, 0}} \end{matrix}]}^{T} \\ {\bar{B}}_{2} = {[\begin{matrix} B_{20}^{kT} & \underset{h}{\underset{︸}{0, \dots, 0}} & I_{m} & \underset{h - 1}{\underset{︸}{0, \dots, 0}} \end{matrix}]}^{T} \\ A_{12} = [B_{11}^{k}, \dots, B_{1 h}^{k}] \\ A_{13} = [B_{21}^{k}, \dots, B_{2 h}^{k}] \\ A_{32} = [\begin{matrix} I_{m} & \dots & 0 & 0 \\ ⋮ & ⋱ & ⋮ & ⋮ \\ 0 & 0 & I_{m} & 0 \end{matrix}] \\ A_{53} = [\begin{matrix} I_{m} & \dots & 0 & 0 \\ ⋮ & ⋱ & ⋮ & ⋮ \\ 0 & 0 & I_{m} & 0 \end{matrix}] \end{matrix}

Based on the discrete-time model of equation (3), one can investigate the game theoretic LFC.

Game theoretic-based optimal cost function

In this subsection, an optimal cost function is constructed to achieve the Nash equilibrium. Under the Nash equilibrium, if any subsystem unilaterally deviates from the control scheduling, a worse value of the cost function will be obtained.

The Nash equilibrium of the stochastic linear quadratic games can be described as follows:

J_{1}^{N} (u_{1}^{*}, u_{2}^{*}) \leq J_{1}^{N} (u_{1}, u_{2}^{*}), J_{2}^{N} (u_{1}^{*}, u_{2}^{*}) \leq J_{2}^{N} (u_{1}^{*}, u_{2}),

(4)

where $(u_{1}^{*}, u_{2}^{*})$ is the Nash equilibrium solution, and the cost functions are given as

J_{i}^{N} = min_{u_{i}} E [z_{k}^{T} Q_{i 0} z_{k} + \sum_{k = 0}^{N - 1} (z_{k}^{T} Q_{i 1} z_{k} + u_{k}^{T} Q_{i 2} u_{k})], i = 1, 2,

(5)

where $Q_{i 0}$ , $Q_{i 1}$ and $Q_{i 2}$ are positive definite weighting matrices with appropriate dimensions.

The cost function for the ith control area is written as

J_{i}^{N} = min_{u_{i, k}} E [z_{k}^{T} {\bar{Q}}_{i 0} z_{k} + \sum_{k = 0}^{N - 1} (z_{k}^{T} {\bar{Q}}_{i 1} z_{k} + {\bar{u}}_{i, k}^{T} {\bar{Q}}_{i 2} {\bar{u}}_{i, k})], i = 1, 2,

(6)

where

\begin{matrix} {\bar{Q}}_{10} = [\begin{matrix} Q_{10} & 0 & \dots & 0 & 0 \\ 0 & [1 / (1 + 1)] Q_{12} & \dots & 0 & ⋮ \\ ⋮ & ⋮ & ⋱ & ⋮ & ⋮ \\ 0 & 0 & \dots & [1 / (h + 1)] Q_{12} & 0 \\ 0 & \dots & \dots & 0 & O_{h * h} \end{matrix}] \\ {\bar{Q}}_{11} = [\begin{matrix} Q_{11} & 0 & \dots & 0 & 0 \\ 0 & [1 / (1 + 1)] Q_{12} & \dots & 0 & ⋮ \\ ⋮ & ⋮ & ⋱ & ⋮ & ⋮ \\ 0 & 0 & \dots & [1 / (h + 1)] Q_{12} & 0 \\ 0 & \dots & \dots & 0 & O_{h * h} \end{matrix}] \\ {\bar{Q}}_{12} = Q_{12} / (h + 1) \\ {\bar{Q}}_{20} = [\begin{matrix} Q_{20} & 0 & \dots & \dots & 0 \\ 0 & O_{h * h} & \dots & \dots & 0 \\ ⋮ & ⋮ & [1 / (1 + 1)] Q_{22} & \dots & 0 \\ ⋮ & ⋮ & ⋮ & ⋱ & ⋮ \\ 0 & 0 & 0 & \dots & [1 / (h + 1)] Q_{22} \end{matrix}] \\ {\bar{Q}}_{21} = [\begin{matrix} Q_{21} & 0 & \dots & \dots & 0 \\ 0 & O_{h * h} & \dots & \dots & 0 \\ ⋮ & ⋮ & [1 / (1 + 1)] Q_{22} & \dots & 0 \\ ⋮ & ⋮ & ⋮ & ⋱ & ⋮ \\ 0 & 0 & 0 & \dots & [1 / (h + 1)] Q_{22} \end{matrix}] \\ {\bar{Q}}_{22} = Q_{22} / (h + 1) \end{matrix}

Now, we are in a position to design a game theoretic-based optimal LFC for system (3) to achieve Nash equilibrium while ensuring that system (3) is exponentially mean-square stable. This can be stated as follows:

Problem 1. Consider networked system (3). This paper aims to

present methods to design game theoretic-based optimal load frequency controllers for system (3) and achieve Nash equilibrium while guaranteeing exponential mean-square stability;

design an algorithm to make the coupled LFC gains solvable under finite and infinite horizon game cases.

Design of game theoretic optimal LFC scheme for power systems

This section aims to tackle Problem 1. The following lemmas and definition will be utilized to solve this problem.

Lemma 2. (Astrom 2012) Assume that the function $l (x, y, u)$ has a unique minimum with respect to $u \in U$ for all $x \in X$ and $y \in Y$ . $u^{0} (x, y)$ denotes the value of u for which the minimum is achieved. Then

min_{u (x, y)} El (x, y, u) = El (x, y, u^{0} (x, y)) = E min_{u} l (x, y, u)

For convenience of understanding, such linear transformations $ζ_{K}$ , $ξ_{K}$ , $ξ_{* i}$ : $M^{n + hm} \to M^{n + hm}$ are introduced to investigate infinite horizon networked control systems (NCSs) stochastic differential games, where

\begin{matrix} ζ_{K} P = E {Ψ_{K, 0}^{T} P Ψ_{K, 0}} \\ ξ_{K} P = ζ_{K} P + Q_{1} + K^{T} Q_{2} K \\ ξ_{*} P = ξ_{K} P, P \in M^{n + hm}, \end{matrix}

with $K = (Q_{2} + E (B^{T} PB))^{- 1} E (B^{T} PA)$ .

Lemma 3. (Zhu et al. 2004) If system (9) is mean-square stable, then $P_{i} = lim_{N \to + \infty} ξ_{* i} θ$ exists, and $P_{i}$ is the minimum positive definite solution of the equation $P_{i} = ξ_{* i} P_{i}$ .

Definition 4. System (3) is exponentially mean-square stable if the following condition holds for every initial state $(x_{0}, t_{0})$

E (| | x_{k} | |^{2} | x_{0}, t_{0}) \leq β γ^{k} | | x_{0} | |^{2},

where $β > 0, 0 < γ < 1$ .

Based on the above lemmas and definition, we are now in a position to derive the finite and infinite horizon game theoretic load frequency controllers for system (3).

Finite horizon game theoretic LFC

With the given finite horizon cost function (equation (5)), the game theoretic-based LFC gain matrices can be designed based on the following theorem while guaranteeing that system (3) is exponentially mean-square stable.

Theorem 5. The finite horizon game theoretic LFC gain matrices $K_{1, k}$ and $K_{2, k}$ , which minimize criterion (5) for system (3), are given as

u_{i, k} = - K_{i, k} z_{k}, i = 1, 2,

(7)

where

K_{i, k} = [{\bar{Q}}_{i 2} + E ({\bar{B}}_{i}^{T} P_{k + 1} {\bar{B}}_{i})]^{- 1} E ({\bar{B}}_{i} P_{k + 1} {\bar{A}}_{i}),

(8)

with

{\begin{array}{l} P_{i k} = \begin{array}{l} E [(\bar{A} - \sum_{p = 1}^{2} {\bar{B}}_{p} K_{p, k}^{T}) P_{i (k + 1)} (\bar{A} - \sum_{p = 1}^{2} {\bar{B}}_{p} K_{p, k})] \\ + K_{i, k} {\bar{Q}}_{i 2} K_{i, k} + {\bar{Q}}_{i 1} \end{array} \\ P_{N} = {\bar{Q}}_{i 0} \\ {\bar{A}}_{1} = \bar{A} - {\bar{B}}_{2} K_{2, k} \\ {\bar{A}}_{2} = \bar{A} - {\bar{B}}_{1} K_{1, k} \end{array}

Moreover, with control law (7), system (3) is exponentially mean-square stable.

Proof. If there exist a group of solutions $(P_{1 k}, K_{1, k}, P_{2 k}, K_{2, k})$ for the above coupled equations, then one can construct $u_{2, k}^{*} = - K_{2, k} z_{k}$ , and by substituting $u_{2, k}^{*}$ into system (3), one has the following system:

z_{k + 1} = {\bar{A}}_{1} z_{k} + {\bar{B}}_{1} u_{1, k},

(9)

where

{\bar{A}}_{1} = \bar{A} - {\bar{B}}_{2} K_{2, k} .

Cost function (6) is derived from cost function (5). The following Bellman equation can be obtained by repeatedly utilizing Lemma 2 to cost function (6):

J_{1}^{k} = min_{u_{1, k}} E (z_{k}^{T} {\bar{Q}}_{10} z_{k} + u_{k}^{T} {\bar{Q}}_{10} u_{k} + J_{1}^{k + 1} | z_{k})

(10)

First, we assume that the solution of Bellman equation (10) exists with such a form

J_{1 i} = E (z_{i}^{T} P_{1 i} z_{i}),

(11)

where $P_{1 k}$ is symmetric, and the initial value of Bellman equation (10) is

J_{1}^{N} = min_{u_{1, k}} E (z_{N}^{T} {\bar{Q}}_{10} z_{N}) = E (z_{N}^{T} {\bar{Q}}_{10} z_{N}) .

(12)

We prove that the assumption is apparently correct when $i = N$ and $P_{N} = {\bar{Q}}_{10}$ . If the assumption holds when $i = k + 1$ , we will prove that the assumption still holds when $i = k$ .

\begin{matrix} J_{1}^{k} = min_{u_{1, k}} E (z_{k}^{T} {\bar{Q}}_{11} z_{k} + u_{k}^{T} {\bar{Q}}_{12} u_{k} + J_{1 (k + 1)} | z_{k}) \\ = min_{u_{1, k}} E (z_{k}^{T} {\bar{Q}}_{11} z_{k} + u_{k}^{T} {\bar{Q}}_{12} u_{k} + E [({\bar{A}}_{1} z_{k} + {\bar{B}}_{1} u_{1, k})^{T} \\ P_{k + 1} ({\bar{A}}_{1} z_{k} + {\bar{B}}_{1} u_{1, k})] | z_{k}) \end{matrix}

(13)

Minimizing equation (13) with $u_{1, k}$ presented in equation (7), one has

\begin{matrix} J_{1}^{k} = E {z_{k}^{T} ({\bar{Q}}_{11} + K_{1, k}^{T} {\bar{Q}}_{12} K_{1, k} + E ([({\bar{A}}_{1} z_{k} + {\bar{B}}_{1} u_{1, k})^{T} \\ P_{1 (k + 1)} ({\bar{A}}_{1} z_{k} + {\bar{B}}_{1} u_{1, k})] | z_{k})) z_{k}} \\ = E (z_{k}^{T} P_{1 k} z_{k}), \end{matrix}

where

\begin{matrix} P_{1 k} = E [(\bar{A} - \sum_{i = 1}^{2} {\bar{B}}_{i} K_{i, k}^{T}) P_{1 (k + 1)} (\bar{A} - \sum_{i = 1}^{2} {\bar{B}}_{i} K_{i, k})] \\ + K_{1, k} {\bar{Q}}_{12} K_{1, k} + {\bar{Q}}_{11} . \end{matrix}

Note that $P_{1 k}$ and ${\bar{Q}}_{11}$ are symmetric, then the representation of the solution of Bellman equation (10) is the same as equation (11). We have proved that $u_{1, k}^{*} = - K_{1, k} z_{k}$ with $K_{1, k}$ being given in equation (8). $u_{1, k}$ minimizes $J_{1}^{N} (u_{1}, u_{2}^{*})$ and $J_{1}^{N} (u_{1}^{*}, u_{2}^{*}) = z_{0}^{T} P_{10} z_{0}$ . Then, following the same procedures, one can prove that $u_{2, k}^{*} = - K_{2, k} z_{k}$ and $J_{2}^{T} (u_{1}^{*}, u_{2}^{*}) = z_{0}^{T} P_{20} z_{0}$ . Thus, it is derived that the finite horizon game theoretic feedback gains (equation (7)) can minimize criterion (5) for system (3).

Moreover, under control law (7), both $z_{k}^{T} P_{1 k} z_{k}$ and $z_{k}^{T} P_{2 k} z_{k}$ can be chosen as a Lyapunov–Krasovskii functional candidate. One chooses the Lyapunov–Krasovskii functional candidate as $z_{k} P_{1 k} z_{k}$ . Since $P_{1 k} (k = 0, 1, \dots, N - 1)$ is positive definite, then $V_{k}$ is positive definite with nonzero $z_{k}$ . The difference can be calculated as

\begin{matrix} E (V_{k + 1} | z_{k}, \dots, z_{0}) - V_{k} \\ = E (z_{k + 1}^{T} P_{1 (k + 1)} z_{k + 1} | z_{k}, \dots, z_{0}) - z_{k}^{T} P_{1} z_{k} \\ = z_{k}^{T} (E [{(\bar{A} - \sum_{p = 1}^{2} {\bar{B}}_{p} K_{p, k})}^{T} P_{1 (k + 1)} (\bar{A} - \sum_{p = 1}^{2} {\bar{B}}_{p} K_{p, k})] - P_{1 k}) z_{k} \\ = - z_{k}^{T} (K_{1, k} {\bar{Q}}_{12} K_{1, k} + {\bar{Q}}_{1}) z_{k} \\ = - z_{k}^{T} N_{k} z_{k} . \end{matrix}

Then,

\begin{matrix} E (V_{k + 1} | z_{k}, \dots, z_{0}) = V_{k} - z_{k}^{T} N_{k} z_{k} \leq (1 - \frac{λ_{min (N_{k})}}{λ_{max (P_{1 k})}}) V_{k} \\ < (1 - \frac{μ}{σ}) V_{k} = γ V_{k} \end{matrix}

(14)

where $0 < μ < λ_{min (N_{k})}$ , $σ > λ_{max (P_{1 k})}$ , $μ < σ$ , $0 < γ < 1$ .

The conditional expectations have the following smoothing property [Hu and Zhu(2003)]:

E (V_{k + 1} | z_{k - 2}, \dots, z_{0}) = E [E (V_{k + 1} | z_{k - 1}, \dots, z_{0}) | z_{k - 2}, \dots, z_{0}] .

(15)

Similar to equation (14), one obtains

E (V_{k} | z_{k - 2}, \dots, z_{0}) = γ E (V_{k - 1} | z_{k - 2}, \dots, z_{0}) .

(16)

Substituting equation (15) into conditional expectation (14) and applying the smoothing property, one has

E (V_{k} | z_{k - 2}, \dots, z_{0}) = γ^{2} V_{k - 2} .

(17)

Repeating the above process, one obtains

E (V_{k}) < γ^{k} V_{0}, E (z_{k}^{T} P_{1 k} z_{k}) < γ^{k} V_{0}

(18)

E (z_{k}^{T} z_{k}) < \frac{γ^{k}}{λ_{min} (P_{1 k})} z_{0}^{T} P_{0} z_{0} \leq \frac{λ_{max} (P_{10})}{λ_{min} (P_{1 k})} = β γ^{k} z_{0}^{T} z_{0}

(19)

where $β = λ_{max} (P_{10}) / λ_{min} (P_{1 k}) > 0$ , and $E (x_{k}^{T} x_{k}) \leq z_{k}^{T} z_{k}$ . Since $u_{- 1} = \dots = u_{- h} = 0$ , one has

E ∥ x_{k} ∥^{2} = E (x_{k}^{T} x_{k}) < β γ^{k} z_{0}^{T} z_{0} = β γ^{k} x_{0}^{T} x_{0} = β γ^{k} ∥ x_{0} ∥^{2},

(20)

E (∥ x_{k}^{2} ∥ x_{0}, t_{0}) < β γ^{k} ∥ x_{0} ∥^{2},

(21)

where $β > 0$ , $0 < γ < 1$ . Therefore, it is derived that system (3) is exponentially mean-square stable from Definition 4. This completes the proof.□

Infinite horizon game theoretic LFC

From equation (5), one can see that only the finite horizon case is considered in Theorem 5. In this subsection, infinite horizon game theoretic LFC will be considered.

Different from the finite horizon cost function (5), the infinite horizon cost function is written as:

J_{i}^{\infty} = E {\sum_{k = 0}^{\infty} [z_{k}^{T} Q_{i 1} z_{k} + u_{k}^{T} Q_{i 2} u_{k}]}, i = 1, 2,

(22)

which yields

J_{i}^{\infty} = E {\sum_{k = 0}^{\infty} [z_{k}^{T} {\bar{Q}}_{i 1} z_{k} + u_{k}^{T} {\bar{Q}}_{i 2} u_{k}]}, i = 1, 2,

(23)

where $Q_{i 1}$ , $Q_{i 2}$ , ${\bar{Q}}_{i 1}$ and ${\bar{Q}}_{i 2}$ are the same as the corresponding items in equations (5) and (6).

Based on equation (23), the following theorem can be derived readily.

Theorem 6. The infinite horizon game theoretic LFC gain matrices $K_{1, k}$ and $K_{2, k}$ which minimize criterion (23) for system (3) are given as

u_{i, k} = - K_{i, k} z_{k}, i = 1, 2,

(24)

where

{\begin{matrix} K_{i, k} = {[{\bar{Q}}_{i 2} + E ({\bar{B}}_{i}^{T} P_{i} {\bar{B}}_{i})]}^{- 1} E ({\bar{B}}_{i} P_{i} {\bar{A}}_{i}) \\ P_{i} = lim_{N \to \infty} ξ_{* i} θ \\ {\bar{A}}_{1} = \bar{A} - {\bar{B}}_{2} K_{2, k} \\ {\bar{A}}_{2} = \bar{A} - {\bar{B}}_{1} K_{1, k} \end{matrix}

Moreover, with designed control law (24), system (3) is exponentially mean-square stable.

Proof. If there exist a group of solutions $(P_{1}, K_{1}, P_{2}, K_{2})$ for the above coupled equations, one can construct $u_{2, k}^{*}$ as $u_{2, k}^{*} = - K_{2, k} z_{k}$ , and by substituting $u_{2, k}$ into system (3), where $K_{2, k}$ is a constant matrix, one has

z_{k + 1} = {\bar{A}}_{1} z_{k} + {\bar{B}}_{1} u_{1, k},

(25)

where

{\bar{A}}_{1} = \bar{A} - {\bar{B}}_{2} K_{2, k} .

(26)

From Lemma 3, it is known that there exists $P_{1} = lim_{N \to + \infty} ξ_{*}^{N} θ$ . Under the control law $u_{1, k} = - K_{1, k} z_{k}$ , the finite horizon quadratic performance index $J_{1}^{N} (z_{0}, X)$ with ${\bar{Q}}_{10} = X$ can be written as

\begin{matrix} J_{1}^{N} (z_{0}, θ) = E {\sum_{k = 0}^{N - 1} [z_{k}^{T} {\bar{Q}}_{11} z_{k} + {\bar{u}}_{1, k}^{T} {\bar{Q}}_{12} {\bar{u}}_{1, k}]} \\ = z_{0}^{T} {\sum_{k = 0}^{N - 1} ζ_{K}^{k} ({\bar{Q}}_{11} + K_{1, k}^{T} {\bar{Q}}_{12} K_{1, k})} z_{0} \\ = z_{0}^{T} ξ_{*} θ z_{0} \end{matrix}

(27)

\begin{matrix} J_{1}^{N} (z_{0}, P_{1}) = E {z_{N}^{T} P_{1} z_{N} + \sum_{k = 0}^{N - 1} [z_{k}^{T} {\bar{Q}}_{11} z_{k} + {\bar{u}}_{1, k}^{T} {\bar{Q}}_{12} {\bar{u}}_{1, k}]} \\ = z_{0}^{T} {ζ_{K}^{N} P_{1} + \sum_{k = 0}^{N - 1} ζ_{K}^{k} ({\bar{Q}}_{11} + K_{1, k}^{T} {\bar{Q}}_{12} K_{1, k})} z_{0} \\ = z_{0}^{T} ξ_{*} P_{1} z_{0} . \end{matrix}

(28)

It is clear that $ξ_{*} θ \leq ξ_{*} P_{1}$ and $z_{0}^{T} ξ_{*} θ z_{0} \leq z_{0}^{T} ξ_{*} P_{1} z_{0}$ . Then $J_{1}^{N} (z_{0}, θ) \leq J_{1}^{N} (z_{0}, P_{1}) = z_{0}^{T} ξ_{*} P_{1} z_{0} = z_{0}^{T} P_{1} z_{0}$ .

Let $N \to + \infty$ , one has

lim_{N \to + \infty} ξ_{*}^{N} θ = J_{1 *}^{\infty} (z_{0}) \leq lim_{N \to + \infty} ξ_{*}^{N} P_{1} = z_{0}^{T} P_{1} z_{0} .

(29)

It can be known from equation (29) that $J_{1 *}^{\infty} (u_{1}^{*}, u_{2}^{*}) = z_{0}^{T} P_{1} z_{0}$ . Obviously, $K_{1}$ is a constant matrix.

Following the same procedures, one can prove that $u_{2, k}^{*} = - K_{2, k} z_{k}$ and $J_{2 *}^{\infty} (u_{1}^{*}, u_{2}^{*}) = z_{0}^{T} P_{2} z_{0}$ . The rest of the proof is similar to that of Theorem 5; here it is omitted. This completes the proof.□

From Theorems 5 and 6, the game theoretic-based optimal LFC gains for system (3) can be obtained for the finite and infinite horizon cases, respectively. However, Theorems 5 and 6 cannot be directly applied in practical engineering since the coupled items are included in equations (7) and (24). In the following subsection, we will provide an algorithm to make the LFC gains solvable.

An algorithm finding the optimal LFC gains

The controller gains $K_{1}$ and $K_{2}$ are coupled together in the solution of Riccati equations (7) and (24). Therefore, in this subsection an algorithm is provided to obtain the desired game theoretic optimal control gains.

Algorithm 1 is provided to calculate the desired game theoretic optimal controller gains $K_{1}$ and $K_{2}$ .

Algorithm 1 Algorithm for calculating game theoretic optimal control gains $K_{1}$ and $K_{2}$
1: Give the initial $δ$ , $K_{1} = 0$ , $P_{1} = 0$ , $K_{2} = 0$ , $P_{2} = 0$ .
2: With the given $K_{2}$ and $P_{2}$ , the $K_{1}$ and $P_{1}$ can be calculated by Theorem 6, set $er r_{1} = \| Δ P_{1} \|$ .
3: With the given $K_{1}$ and $P_{1}$ , the $K_{2}$ and $P_{2}$ can be calculated by Theorem 6, set $er r_{2} = \| Δ P_{2} \|$ .
4: If $er r_{1} + er r_{2} > δ$ , go to Step 2; otherwise, go to step 5.
5: Output the LFC gains $K_{1}, K_{2}$ .

Notice that although a two-area power system is considered in this work, one can readily derive the extended results to multi-area power systems following the main ideas presented in this paper. The corresponding results are omitted here for brevity.

A case study

In this section, a case study is carried out based on the two-area LFC power system under consideration of network-induced delays. The parameters of this system are listed in Table 1.

Table 1.

Parameters of LFC scheme.

Parameters	$T_{gi}$	$T_{ti}$	$K_{pi}$	$T_{pi}$	$T_{12}$	$r_{i}$
Values	0.08	0.3	12	20	0.2545	2.4

Without loss of generality, the quadratic performance matrices $Q_{11}$ , $Q_{12}$ , $Q_{21}$ , and $Q_{22}$ can be chosen as follows:

Q_{11} = Q_{21} = I_{9}, Q_{12} = 5, Q_{22} = 10 .

The forecast of $Δ {\bar{P}}_{d} (t)$ is described as that: $Δ {\bar{P}}_{d 1}$ stays zero throughout the control period, while a step perturbation $Δ {\bar{P}}_{d 2} = 0.1 p . u$ occurs in the second control area starting at the time $t = 0$ . It is assumed that network-induced delays vary stochastically in $[0, T]$ . The sampling period $T = 0.1$ s.

To demonstrate the merits of the game theoretic optimal controllers, the following two LFC schemes are implemented for this system.

Scheme 7. (Game theoretic optimal controllers) We obtain the following parameters of optimal controllers from equation (24) by adopting Algorithm 1:

\begin{matrix} K_{1} = [0.1047, 0.0239, 0.0067, 0.0582, 0.0080, \\ 0.0020, - 0.2322, 0.4263, - 0.0050, 0.0176, - 0.0003] \\ K_{2} = [- 0.0168, - 0.0035, - 0.0010, 0.0463, 0.0094, \\ 0.0026, 0.0221, - 0.0197, 0.2591, - 0.0010, 0.0108] \end{matrix}

Scheme 8. (PI controller) The two areas provide $Δ P_{ci}$ by identical PI controllers. Using the typical values in LFC systems (Jiang et al. 2012), one can set the proportional gains $K_{P 1} = K_{P 2} = 1$ , the integral gains $K_{I 1} = K_{I 2} = 0.2$ , and the frequency bias factors $β_{1} = β_{2} = 0.45$ . The adjusting speed $u (t)$ is obtained for comparison by differentiating $Δ P_{ci}$ .

Consider the network-induced time delays in Figure 3 and the obtained control law. Figures 4 and 5 give the frequency deviations in areas 1 and 2, respectively, where a step perturbation $Δ {\bar{P}}_{d 2} = 0.1 p . u$ is introduced in area 2. From Figures 4 and 5, one can see that scheme 7 can provide better frequency deviations than scheme 8 in both area 1 and area 2. As observed from Figure 6, scheme 7 provides much better Tie-line power deviations than scheme 8. Table 2 shows more details of the payoffs under different schemes. In fact, scheme 7 can guarantee better payoffs than scheme 8 in both areas 1 and 2. This statement is verified by Table 2.

Table 2.

Payoffs of different schemes

Performance	Scheme 1	Scheme 2
$J_{Area 1}$	0.0296	0.0532
$J_{Area 2}$	0.0309	0.0604

Conclusion

The game theoretic optimal LFC scheme for the two-area power system with network-induced delays has been investigated in this paper. A two-area power system LFC scheme has been designed by using stochastic differential games. The designed controllers can provide expected optimal performance for each control area. Since the Nash equilibrium has been achieved for each area of the two-area power system, all areas will obey the control scheduling of this optimal scheme. Simulation results have illustrated that the game theoretic-based optimal load frequency control can provide better performance than the traditional PI controller with network-induced delays.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the National Science Foundation of China (Grant no. 61673255); the Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning, China; the ‘333 Project’ Research Foundation of Jiangsu Province, China (Grant no. BRA2015358); the ‘Six Talent Peaks Project’ of Jiangsu Province, China (Grant no. DZXX-025); the ‘Qing Lan Project’ of Jiangsu Province, China; and the Natural Science Foundation of Jiangsu Province, China (Grant no. BK20161361).

ORCID iD

Yu-Long Wang

References

Astrom

(2012) Introduction to Stochastic Control Theory. Chelmsford, MA: Courier Corporation.

Bahgaat

El-Sayed

Hassan

MAM

, et al. (2014) Load frequency control in power system via improving PID controller based on particle swarm optimization and ANFIS techniques. International Journal of System Dynamics Applications 3(3): 1–24.

Chen

Wang

, et al. (2015) Cooperative control of power system load and frequency by using differential games. IEEE Transactions on Control Systems Technology 23(3): 882–897.

Dotta

Silva

ASE

Decker

(2009) Wide-area measurements-based two-level control design considering signal transmission delay. IEEE Transactions on Power Systems 24(1): 208–216.

Dou

Liu

(2013) Multi-agent based hierarchical hybrid control for smart microgrid. IEEE Transactions on Smart Grid 4(2): 770–778.

Duan

Zhang

Jiang

, et al. (2017) Structure-exploiting delay-dependent stability analysis applied to power system load frequency control. IEEE Transactions on Power Systems 32(6): 4528–4540.

Fang

Misra

Xue

, et al. (2012) Smart grid: the new and improved power grid – a survey. Communications Surveys Tutorials 14(4): 944–980.

Fosha

Elgerd

(1970) The megawatt-frequency control problem: a new approach via optimal control theory. IEEE Transactions on Power Apparatus and Systems 89(4): 563–577.

Gupta

Chow

(2010) Networked control system: overview and research trends. IEEE Transactions on Industrial Electronics 57(7): 2527–2535.

10.

Zhu

(2003) Stochastic optimal control and analysis of stability of networked control systems with long delay. Automatica 39(11): 1877–84.

11.

Huang

Wang

Zhang

, et al. (2017) Data-driven modelling and fuzzy multiple-model predictive control of oxygen content in coal-fired power plant. Transactions of the Institute of Measurement and Control 39(11): 1631–1642.

12.

Islam

Hannan

Shareef

, et al. (2017) An application of backtracking search algorithm in designing power system stabilizers for large multi-machine system. IEEE Transactions on Power Systems 237(10): 175–184.

13.

Jiang

Yao

, et al. (2012) Delay-dependent stability for load frequency control with constant and time-varying delays. IEEE Transactions on Power Systems 27(2): 932–941.

14.

Zhang

Liu

, et al. (2017) Distributed model predictive load frequency control of the multi-area power system after deregulation. IEEE Transactions on Industrial Electronics 64(6): 5129–5139.

15.

Wang

, et al. (2013) Decentralized sliding mode load frequency control for multi-area power systems. IEEE Transactions on Power Systems 28(4): 4301–4309.

16.

Mohsenian-Rad

Wong

VWS

Jatskevich

(2010) Autonomous demand-side management based on game-theoretic energy consumption scheduling for the future smart grid. IEEE Transactions on Smart Grid 1(3): 320–331.

17.

Pal

Negi

(2017) Robust output feedback control of 2D discrete systems with actuator saturation and time-varying delay. Transactions of the Institute of Measurement and Control 39(11): 1673–1695.

18.

Peng

Zhang

(2016) Delay-distribution-dependent load frequency control of power systems with probabilistic interval delays. IEEE Transactions on Power Systems 31(4): 3309–3317.

19.

Peng

Xie

(2017a) Observer-based non-PDC control for networked T–S fuzzy systems with an event-triggered communication. IEEE Transactions on Cybernetics 47(8): 2279–2287.

20.

Peng

Tian

Zhang

, et al. (2017b) Decentralized event-triggering communication scheme for large-scale systems under network environments. Information Sciences 380: 132–144.

21.

Peng

Zhang

Yan

(2018) Adaptive event-triggering H_∞ load frequency control for network-based power systems. IEEE Transactions on Industrial Electronics 65(2): 1685–1694.

22.

Pham

Trinh

Hien

(2016) Load frequency control of power systems with electric vehicles and diverse transmission links using distributed functional observers. IEEE Transactions on Smart Grid 7(1): 238–252.

23.

Saad

Han

Poor

, et al. (2012) Game-theoretic methods for the smart grid: an overview of microgrid systems, demand-side management, and smart grid communications. IEEE Signal Processing Magazine 29(5): 86–105.

24.

Shayeghi

Shayanfar

Jalili

(2009) Load frequency control strategies: a state-of-the-art survey for the researcher. Energy Conversion and Management 50(2): 344–353.

25.

Snodgrass

(1990) Cost aspects of AGC, inadvertent energy and time error. IEEE Transactions on Power Systems 5(1): 111–118.

26.

Wang

Y-L

Han

Q-L

(2018) Network-based modelling and dynamic output feedback controlr unmanned marine vehicles. Automatica 91: 43–53.

27.

Wang

Y-L

Han

Q-L

Fei

, et al. (2018) Network-based T–S fuzzy dynamic positioning controller design for unmanned marine vehicles. IEEE Transactions on Cybernetics. DOI: 10.1109/TCYB.2018.2829730.

28.

Wang

Y-L

Lim

C-C

Shi

(2017) Adaptively adjusted event-triggering mechanism on fault detection for networked control systems. IEEE Transactions on Cybernetics 47(8): 2299–2311.

29.

Wang

Y-L

Shi

Lim

C-C

, et al. (2016a) Event-triggered fault detection filter design for a continuous-time networked control system. IEEE Transactions on Cybernetics 46(12): 3414–3426.

30.

Wang

Y-L

Wang

T-B

Han

Q-L

(2016b) Fault detection filter design for data reconstruction-based continuous-time networked control systems. Information Sciences 328: 577–594.

31.

Weaver

Krein

(2009) Game-theoretic control of small-scale power systems. IEEE Transactions on Power Delivery 24(3): 1560–1567.

32.

Chen

Lou

(2011) A coordinated frequency control method based on differential game theory. Automation of Electric Power Systems 35(20): 41–46.

33.

Zhang

Jiang

, et al. (2013) Delay-dependent robust load frequency control for time delay power systems. IEEE Transactions on Power Systems 28(3): 2192–2201.

34.

Zhang

Shi

Wang

, et al. (2017b) Analysis and synthesis of networked control systems: a survey of recent advances and challenges. ISA Transactions 66: 376–392.

35.

Zhang

(2017) Online identification of power system equivalent inertia constant. IEEE Transactions on Industrial Electronics 64(10): 8098–8107.

36.

Zhang

X-M

Han

Q-L

Zhang

B-L

(2017a) An overview and deep investigation on sampled-data-based event-triggered control and filtering for networked systems. IEEE Transactions on Industrial Informatics 13(1): 4–16.

37.

Zhu

Liu

(2004) Infinite time stochastic optimal control of networked control systems with long delay on analysis and design of model-based fuzzy control systems. IEEE Transactions on Fuzzy Systems 21(3): 321–326.