Stability analysis of discrete-time time-varying stochastic systems with infinite Markov switching

Abstract

This paper studies the stability of discrete-time time-varying stochastic systems with infinite Markov switching. First, the concepts of strongly exponentially stable in mean square and exponentially stable in mean square with conditions are introduced, and their equivalent conditions are given by operator theory and stochastic analysis. Second, we introduce the Lyapunov equation related to strongly exponentially stable in mean square. Third, as an application of the proposed Lyapunov stability criterion, the relationship between the internal stability and the $l^{2}$ input–output state stability is derived for the infinite Markov jump system disturbed by finite-energy random disturbance. Finally, an example is given to verify its correctness and validity.

Keywords

Strongly exponentially stable in mean square infinite Markov switching input–output state stability time-varying systems Lyapunov equation

Introduction

In the past decade, the stability problem of discrete-time linear stochastic systems has developed widely and achieved fruitful results (Fang and Loparo, 2002; Fragoso and Costa, 2005). It involves stability analysis (Xiong, et al., 2013; Zhang et al., 2019), $H_{\infty}$ output feedback control (Pan et al., 2018), optimal control (Aberkane and Dragan, 2020), robust stabilization (Dragan et al., 2010; Xiang and Wang, 2011), estimation and filter design (Liu and Hou, 2020), and so on.

Stochastic systems with Markov switching have quickly become a hot topic of discussion among scholars. In 2006, Mao and Yuan (2006) systematically introduced stochastic differential equations with Markov switching. So far, the research results on finite Markov switching stochastic systems have been mature. For example, Ma et al. (2011) mainly by means of the operator theory introduced random bounded real lemma to study the infinite horizon $H_{2} / H_{\infty}$ control problem of discrete-time time-varying linear stochastic systems with finite Markov switching and multiplicative noise. Dragan and Morozan (2006a) studied the problem of exponential stability of the zero-state equilibrium of discrete time-varying linear equations described by a sequence of linear positive operators acting on a finite-dimensional ordered Hilbert space. Dragan and Morozan (2006b) studied the exponentially stable in mean square of a class of discrete-time time-varying linear stochastic systems with finite Markov switching and independent random disturbance and introduced four different definitions of exponentially stable in mean square, pointing out that they are not equivalent in the time-varying case. Also Zhou et al. (2016) discussed the stochastic Nash differential games for discrete-time finite Markov switching linear systems with noise-dependent states and applied the results to corresponding $H_{2} / H_{\infty}$ robust control problems. Ligang et al. (2021) designed a second-order sliding mode controller based on the linear sliding mode surface and applied it to the nonlinear affine system after standard transformation, and the controller ensured the convergence of the closed-loop system in finite time. Furthermore, the maximum convergence domain of sliding mode trajectory under state and actuator constraints was analyzed by geometric analysis method, and the convergence of system states under different conditions was discussed.

In recent years, the infinite Markov switching system has gradually attracted extensive attention of scholars. Dragan and Morozan (2008) and Ungureanu and Dragan (2013) discussed the exponential stability, stochastic detectability, and weak-detectability of zero equilibrium states of discrete-time time-varying linear equations described by sequences of linear positive bounded operators acting on ordered Hilbert spaces or on ordered Banach spaces. The original literature (Dragan and Morozan, 2008) studied the exponential stability of infinite Markov switched systems under the time-invariant cases. In the revised draft, we have further described the time-varying cases. Hou and Ma (2015a, 2015b) studied the discrete-time time-invariant stochastic systems with infinite Markov switching disturbed by finite energy with random disturbance and obtained a sufficient condition for $l^{2}$ input–output state stability. Hou and Ma (2015a) also discussed some equivalent conditions for the exponential stability of infinite Markov switching systems under time-invariant conditions, such as the spectral radius of the corresponding operator less than 1, Lyapunov stability theorem, and positive bounded solutions of affine equations.

It is well known that the stability of discrete-time time-varying stochastic systems with infinite Markov switching, multiplicative noises, and external input disturbances is a great challenge. The main difficult of this work is that strongly exponentially stable in mean square (SESMS) and exponentially stable in mean square with conditions (ESMS-C) are equivalent under time-invariant, but in the time-varying case, it does not hold. Another obvious difficulty is that causal and anti-causal Lyapunov-type operators associated with infinite Markov switching random systems no longer have adjoint relations.

Therefore, it is one of the main tasks of this paper to find the equivalent conditions of SESMS and ESMS-C for discrete-time time-varying stochastic systems with infinite Markov switching.

In this paper, we focus on exponential stability of discrete-time time-varying stochastic systems with infinite Markov jump system and multiplicative noises. The main contributions of this work are tripartite:

Under the case of time-varying, using operator theory, stochastic analysis, Lyapunov equation, linear matrix inequality, and other methods explores the relationship between the exponential stability of the discrete-time time-varying stochastic system with infinite Markov switching and the anti-causal operator associated with the system.

Considering that the control output of the system is also affected by multiplicative noise, when the system is disturbed by finite energy, the state stability of the time-varying system and the energy level of the system output are further discussed.

MATLAB is used to simulate the multiplicative noise, the jump trajectory of Markov chain, and the state trajectory of the system under consideration.

The results of this paper may include those of Hou and Ma (2015a). Compared with time-invariant systems, time-varying stochastic systems with infinite Markov switching are more complex and challenging, and their results are more general.

The outline of this paper is organized as follows. The exponentially stable in mean square and $l^{2}$ input–output stable of discrete-time time-varying stochastic systems with infinite Markov switching, multiplicative noise, and external disturbance are studied. Section “Preliminaries” provides some technical preliminaries and basic concepts, the concepts of SESMS and ESMS-C are defined using linear positive bounded operators in Banach spaces. Section “Major results” contains the main results. In section 3.1, we obtain the relationship between the anti-causal evolution operator and the conditional expectation. In section 3.1, deals with the SESMS and ESMS-C are equivalent to each other under certain conditions. In section 3.2, the concept of $l^{2}$ input–output state stability is described, and sufficient conditions for $l^{2}$ input–output state stability are given and proved by means of random analysis, operator theory, Lyapunov function, and other methods. In section “The numerical example,” the correctness and validity of sufficient conditions for $l^{2}$ input–output state stability are verified by examples and MATLAB simulation.

Preliminaries

Symbol description

Let $R^{n}$ represents the n-dimensional Euclidean space; let $R^{m \times n}$ represents the linear space of $m \times n$ real matrix; ‖‖ denotes the Euclidean norm of $R^{n}$ or the operator norm of $R^{m \times n}$ ; let $S_{n}$ be the set of $n \times n$ symmetric matrices; $A^{T}$ denotes the transpose of matrix (vector) A; $A > 0 (A \geq 0)$ represents A is positive (semi-positive); $D = {1, 2, \dots}, Z_{+} = {0, 1, 2, \dots}$ , $I = (I_{n}, I_{n}, \dots)$ , $I_{n}$ is the $n \times n$ identity matrix; $I$ represents the identity matrix sequence. $AN *$ is equivalent to $AN A^{T}$ , $A *$ is equivalent to $A A^{T}$ , and * denotes the symmetric terms; $a . s .$ represents the almost sure convergence; >> represent the uniformly positive; $σ [A]$ is the smallest $σ$ -algebra generated by A; $Ω$ represent the sample space; $P (A)$ denotes the probability of A; $P (A | B)$ denotes the probability of event B occurs under the condition of A; $E [A]$ represent the mathematical expectation of event A; $E [\cdot | A]$ represents the conditional expectation with the conditional event A; $r (■)$ is the spectral radius of an operator; let $H_{u, v}$ be the $σ$ -algebra generated by ${η_{s}, ω (t), 0 \leq s \leq u, 0 \leq t \leq v}$ ; $r (■)$ is the spectral radius of an operator; let ${\tilde{H}}_{u}$ be the $σ$ -algebra generated by ${η_{s}, ω (t), 0 \leq s \leq u, 0 \leq t \leq u - 1}$ .

Model description

In a complete probability space $(Ω, H, P)$ , a discrete-time stochastic system with infinite Markov switching, multiplicative noise, and external perturbations is given as following

{\begin{array}{l} x (t + 1) = [A_{0} (t, η_{t}) x (t) + B_{0} (t, η_{t}) v (t)] \\ + \sum_{k = 1}^{m} ω_{k} (t) [A_{k} (t, η_{t}) x (t) + B_{k} (t, η_{t}) v (t)] \\ z (t) = [C_{0} (t, η_{t}) x (t) + D_{0} (t, η_{t}) v (t)] \\ + \sum_{k = 1}^{m} [C_{k} (t, η_{t}) x (t) + D_{k} (t, η_{t}) v (t)] ω_{k} (t), t \in Z_{+} \end{array}

(1)

where $x (t) \in R^{n}, v (t) \in R^{n_{v}}, z (t) \in R^{n_{z}}$ represent the system state, external disturbance input, and control output, respectively. ${ω (t)}_{t \geq 0}$ is a sequence of independent random vectors or multiplicative noise, $ω (t) = (ω_{1} (t), \dots, ω_{k} (t), \dots, ω_{m} (t))$ , where ${η_{t}}_{t \in Z_{+}}$ and ${ω (t)}_{t \geq 0}$ are independent.

For further discussion, we make the following assumptions.

Assumption 1. (i) for all $t \in Z_{+}$ , $E [ω (t)] = 0,$ $E [| | ω (t) | |^{2}] < \infty$ , $E [ω (t) ω^{T} (t)] = I_{m}$ ; (ii) $ℱ_{t} = σ [ω (0), \dots, ω (t)]$ , $G_{t} = σ [η (0), \dots, η (t)]$ , for all $t > 0$ , $F_{t}$ and $G_{t}$ are independent of each other, $H_{u, v} = σ [η_{s}, ω (t), 0 \leq s \leq u, 0 \leq t \leq v]$ . If $u \geq 1$ , ${\tilde{H}}_{u} = σ [η_{s}, ω (t), 0 \leq s \leq u, 0 \leq t \leq u - 1]$ , for all $(u, v) \in Z_{+}$ .

The random variable ${η_{t}}_{t \geq 0}$ in system (1) is Markov Chain, its state space is a countable set D, and its switching depends on

P {η_{t + 1} = j | G_{t}} = p_{t} (η_{t}, j) . j \in D, t \geq 0

(2)

Let $P_{t} = {p_{t} (i, j)}_{i, j \in D}, p_{t} (i, j)$ is a scalar on the right side of equation (2); thus, a stochastic matrix sequence with infinite rows and columns can be obtained, and it has properties

{\begin{matrix} p_{t} (i, j) \geq 0 \\ \sum_{j = 1}^{\infty} p_{t} (i, j) = 1 \end{matrix}

(3)

We define $π_{t} (j) = P {η_{t} = j}, π_{t} = (π_{t} (1), π_{t} (2), \dots)$ is distribution of the random variable ${η_{t}}_{t \geq 0}$ , and satisfy

{\begin{matrix} π_{t} (j) \geq 0, \forall j \in D \\ \sum_{j = 1}^{\infty} π_{t} (j) = 1 \end{matrix}

(4)

Assumption 2. (i) For all $t \geq 0, P_{t}$ is non-degenerate stochastic matrix, i.e., for every $j \in D$ , there exists $i \in D$ , such that $p_{t} (i, j) > 0$ ; (ii) for all $i \geq 1$ , there is $π_{0} (i) = P {η_{0} = i} > 0$ , where $π_{0}$ is the initial distribution of the random variable ${η_{t}}_{t \geq 0}$ .

$l^{2} (0, \infty; R^{m})$ denotes the space of the $R^{m}$ value stochastic process ${y (t, ω) : Z_{+} \times Ω \to R^{m}}$ , $y (t, ω)$ is ${\tilde{H}}_{t}$ measurable and $\sum_{t = 0}^{\infty} E (‖ y_{t} ‖^{2}) < \infty$ . Therefore, $l^{2} (0, \infty; R^{m})$ is a real Hilbert space whose norm is induced by the usual inner product: $‖ y_{t} ‖_{l^{2} (0, \infty; R^{m})}^{2} = (\sum_{t = 0}^{\infty} E (‖ y_{t} ‖^{2}))^{\frac{1}{2}} < \infty$ .

Let $H_{1}^{m \times n}$ is the set ${H | H = (H (1), H (2), \dots), H (i) \in R^{m \times n}}$ , where $\sum_{i = 1}^{\infty} | | H (i) | | < \infty$ , it is easy to get $H_{1}^{m \times n}$ is a Banach space of norm $‖ H ‖_{1} = \sum_{i = 1}^{\infty} ‖ H (i) ‖$ . Similarly, the Banach space $H_{\infty}^{m \times n}$ is defined, and its norm $‖ H ‖_{\infty} = sup_{i \in D} ‖ H (i) ‖$ . When $m = n$ , $H_{1}^{n \times m} (H_{\infty}^{n \times m})$ can be abbreviated as $H_{1}^{n} (H_{\infty}^{n})$ . If $H (i) \in S_{n}, H (i) \geq 0, i \in D$ , $H_{1}^{n} (H_{\infty}^{n})$ is replaced by $H_{1}^{n +} (H_{\infty}^{n +})$ .

For $X, Y \in_{1}^{n +},$ $X \leq Y$ means for all $i \in D, X (i) \leq Y (i)$ and has the property $‖ X ‖_{1} \leq ‖ Y ‖_{1}$ , In addition, $B (H_{1}^{m \times n})$ represent the linear space of all bounded operators form $H_{1}^{m \times n}$ to $H_{1}^{m \times n}$ . $Γ \in B (H_{1}^{m \times n})$ ,and its induced norm is denoted by $| | Γ | |_{H_{1}^{m \times n}}$ .

Assumption 3. All parameters of system (1) have finite norm, $‖ A (t) ‖_{\infty} = sup_{i \in D} ‖ A (t, i) ‖ < \infty$ .

Next, we define the operator $L_{t} : H_{1}^{n} \to H_{1}^{n};$ $T_{t} (E_{t});$ $H_{\infty}^{n} \to H_{\infty}^{n}$ , $U \in H_{1}^{n}, V \in H_{\infty}^{n}$ as following

\begin{matrix} L_{t} U = (L_{t} U (1), L_{t} U (2), \dots) \\ E_{t} V = (E_{t} V (1), E_{t} V (2), \dots) \\ T_{t} V = (T_{t} V (1), T_{t} V (2), \dots) \end{matrix}

\begin{matrix} L_{t} U (i) = \sum_{k = 0}^{m} \sum_{j = 1}^{\infty} p_{t} (j, i) A_{k} (t, j) U (j) A_{k} {(t, j)}^{T} \\ E_{t} V (i) = \sum_{j = 1}^{\infty} p_{t} (i, j) V (j) \\ T_{t} V (i) = \sum_{k = 0}^{m} A_{k} {(t, i)}^{T} E_{t} V (i) A_{k} (t, i) \end{matrix}

(5)

Remark 1. It is easy to show that $L$ , $T$ , and $E$ are all linear positive operator. For infinite Markov jump systems, $T$ is not the adjoint operator of $L$ , as is different from the case of finite Markov jump system. In fact, $T$ and $L$ are defined on two different spaces.

And then, we define linear operators as following

T^{c} (t, s) = {\begin{matrix} L_{t - 1} L_{t - 2} \dots L_{s}, t > s \geq 0 \\ I, t = s \end{matrix}

(6)

T^{a} (t, s) = {\begin{matrix} T_{s} T_{s + 1} \dots T_{t - 1}, t > s \geq 0 \\ I, t = s \end{matrix}

(7)

Remark 2. $L_{t}, T_{t}, E_{t} \geq 0, L_{t} \in B (H_{1}^{n}); T_{t}, E_{t} \in B (H_{\infty}^{n})$ .

Remark 3. $T^{c} (t, s)$ and $T^{a} (t, s)$ are linear evolution operators generated by ${L_{t}}_{t \geq 0}$ and ${T_{t}}_{t \geq 0}$ , respectively, which also can be called causal operators and anti-causal operators in Zhou et al. (2016).

Next, we further introduce some basic concepts of infinite Markov switching systems to prepare for the study of $l^{2}$ input and output stability in this paper. Consider the discrete-time infinite Markov switching system $(A, P)$ as follows

x (t + 1) = [A_{0} (t, η_{t}) + \sum_{k = 1}^{m} A_{k} (t, η_{t}) ω_{k} (t)] x (t), t \in Z_{+}

(8)

Let $A (t) = A_{0} (t, η_{t}) + \sum_{k = 1}^{m} A_{k} (t, η_{t}) ω_{k} (t)$ , and define $Φ (t, s)$ as follows

Φ (t, s) = {\begin{matrix} A (t - 1) A (t - 2) \dots A (s), t \geq s + 1 \\ I_{n}, t = s \end{matrix}

(9)

$Φ (t, s)$ represent the fundamental solution matrix of system $(A, P)$ , for all $t \geq s > 0$ , the solution of systems $(A, P)$ satisfies: $x (t) = Φ (t, s) x (s)$ .

Definition 1. The zero-state equilibrium of discrete-time linear infinite Markov jump system

x (t + 1) = [A_{0} (t, η_{t}) + \sum_{k = 1}^{m} A_{k} (t, η_{t}) ω_{k} (t)] x (t), t \in Z_{+}

or $(A, P)$ is called stochastically stable if any $(x_{0}, η_{0}) \in R^{n} \times D$ , $\sum_{t = 0}^{\infty} E ({‖ x_{t} ‖}^{2}) < \infty$ .

Definition 2. We say that the zero-state equilibrium of discrete-time linear infinite Markov jump system $(A, P)$ is SESMS if there exist $β > 0, q \in (0, 1)$ such that $| | T^{c} (t, s) | | \leq β q^{(t - s)}$ for all $t \geq s \geq 0$ , $T (t, s)$ being the linear evolution operator on $S_{n}^{N}$ defined by the corresponding sequence of Lyapunov operator ${L_{t}}_{t \geq 0}$ .

Definition 3. The zero-state equilibrium of discrete-time linear infinite Markov jump system

x (t + 1) = [A_{0} (t, η_{t}) + \sum_{k = 1}^{m} A_{k} (t, η_{t}) ω_{k} (t)] x (t), t \in Z_{+}

or $(A, P)$ is called:

(i) Asymptotically mean square stable (AMSS) if for any $(x_{0}, η_{0}) \in R^{n} \times D$ , $lim_{t \to \infty} E (‖ x_{t} ‖^{2}) = 0$ .

(ii) ESMS-C if there exist $β \geq 1, q \in (0, 1)$ such that $E [‖ Φ (t, 0) x_{0} ‖^{2} | η_{0} = i] \leq β q^{t} ‖ x_{0} ‖^{2}$ , for all $t \in Z_{+},$ $(x_{0}, i) \in R^{n} \times D_{0}$ , and all Markov chains $({η_{t}}_{t \geq 0,} {P_{t}}_{t \geq 0}, D)$ satisfying the Assumption 1. Here, $D_{0} = {i | π_{0} (i) > 0}$ .

Major results

It is well known that the SESMS of the zero equilibrium state of time-invariant stochastic systems $(A, P)$ with finite Markov switching is equivalent to ESMS-C, but the SESMS of the zero equilibrium state of time-invariant stochastic systems $(A, P)$ with finite Markov switching is a sufficient condition for ESMS-C of zero equilibrium state of system $(A, P)$ . Dragan et al. (2010) used the periodicity to solve the equivalence problem between SESMS and ESMS-C of discrete-time time-varying stochastic systems with finite Markov switching. However, when the state space of Markov Chain is an uncountable set, the relationship between system stability changes greatly. Therefore, in order to further discuss the equivalent conditions of ESMS-C and SESMS of the system $(A, P)$ and the stability of the system, we first give Theorem 1 under the condition of Assumption 1 and obtain the relationship between the anti-causal evolution operator and the conditional expectation. Second, based on Theorem 1, under Assumptions 2 and 3, we can deduce that ESMS-C is equivalent to SESMS and give some equivalent properties of SESMS to prepare for Theorem 3. Finally, sufficient conditions for the $l^{2}$ input–output state stability are introduced and proved by operator theory and stochastic analysis, i.e., Theorem 3.

Theorem 1. If $t \geq 0, I \in D_{0}$ satisfy the following equation

[T^{a} (t, 0) V] (i) = E [Φ^{T} (t, 0) V (η_{t}) Φ (t, 0) | η_{0} = i]

where $V = (V (1), V (2), \dots) \in H_{\infty}^{n}$ .

Proof. First, define the operator $ϒ (t, 0) : H_{\infty}^{n} \to H_{\infty}^{n}$ , $t \geq 0$ , $ϒ (t, 0) V = (ϒ (t, 0) V (1), ϒ (t, 0) V (2), \dots)$

ϒ (t, 0) V (i) = {\begin{matrix} E [Φ^{T} (t, 0) V (η_{t}) Φ (t, 0) | η_{0} = i], i \in D_{0} \\ [T^{a} (t, 0) V] (i), i \in D / D_{0} \end{matrix}

(10)

Here $Φ (t + 1, 0) = A (t) Φ (t, 0)$ , $A (t)$ , and $Φ (t, 0)$ are all $H_{t}$ -measurable. In addition, $Φ (t, 0)$ is ${\tilde{H}}_{t}$ -measurable.

Considering the $σ [η_{0}] \subset {\tilde{H}}_{t} \subset H_{t}$ , because $A (t)$ and $Φ (t, 0)$ are all $H_{t}$ -measurable (where $H_{t}$ is the $σ$ -algebra introduced in Appendix A). According to the property of the conditional expectation in Xiang and Wang (2011), we can get equation as following

\begin{array}{l} E [Φ^{T} (t, 0) A^{T} (t) V (η_{t + 1}) A (t) Φ (t, 0) | η_{0} = i] \\ = \sum_{j = 1}^{\infty} E [Φ^{T} (t, 0) A^{T} (t) V (j) A (t) Φ (t, 0) 1_{{η_{t + 1} = j}} | η_{0} = i] \\ = \sum_{j = 1}^{\infty} E [E [Φ^{T} (t, 0) A^{T} (t) V (j) A (t) Φ (t, 0) 1_{{η_{t + 1} = j}} | {\tilde{ℋ}}_{t}] | η_{0} = i] \end{array}

(11)

Equation (12) can be deduced from part of the right end of equation (11)

\begin{matrix} E [Φ^{T} (t, 0) A^{T} (t) V (j) A (t) Φ (t, 0) 1_{{η_{t + 1} = j}} | {\tilde{H}}_{t}] \\ = E [E [Φ^{T} (t, 0) A^{T} (t) V (j) A (t) Φ (t, 0) 1_{{η_{t + 1} = j}} | H_{t}] | {\tilde{H}}_{t}] \\ = Φ^{T} (t, 0) E [A^{T} (t) p_{t} (η_{t}, j) V (j) A (t) | {\tilde{H}}_{t}] Φ (t, 0) \end{matrix}

(12)

Under the conditions of Assumption 1, we have that the random vector $ω (t)$ is independent of $\tilde{H}$ , this shows that: $E [ω (t) | {\tilde{H}}_{t}] = 0$ , $E [ω (t) ω^{T} (t) | {\tilde{H}}_{t}] = I_{m}$ .

At this point, let us we plug the $A (t) = A_{0} (t, η_{t}) + \sum_{k = 1}^{m} A_{k} (t, η_{t}) ω_{k} (t)$ and formula (12) into equation (11), then equation (13) can be obtained

\begin{matrix} E [Φ^{T} (t, 0) A^{T} (t) V (η_{t + 1}) A (t) Φ (t, 0) | η_{0}] \\ = E [Φ^{T} (t, 0) [\sum_{k = 0}^{m} A_{k}^{T} (t, η_{t}) \sum_{j = 1}^{\infty} p_{t} (η_{t}, j) V (j) A_{k} (t, η_{t})] * | η_{0}] \end{matrix}

(13)

That is, according to the definition of linear operator $T_{t}, E_{t}$ , equation (14) can be obtained as following

\begin{matrix} E [Φ^{T} (t, 0) A^{T} (t) V (η_{t + 1}) A (t) Φ (t, 0) | η_{0}] \\ = E [Φ^{T} (t, 0) T_{t} (V (η_{t})) Φ (t, 0) | η_{0}] \end{matrix}

(14)

Therefore, by means of the definition of $ϒ (t, 0) : H_{\infty}^{n} \to H_{\infty}^{n}, t \geq 0$ , and $i \in D_{0}$ , equation (15) holds

ϒ (t + 1, 0) V (i) = ϒ (t, 0) T_{t} V (i)

(15)

If $i \in D / D_{0}$ , we can get equation (16)

\begin{matrix} ϒ (t + 1, 0) V (i) = [T^{a} (t + 1, 0) V] (i) \\ = [T^{a} (t, 0) T_{t} V] (i) = ϒ (t, 0) T_{t} V (i) \end{matrix}

(16)

Therefore, for all $i \in D, t \geq 0$ , the following equation (17) can be obtained

ϒ (t + 1, 0) V (i) = ϒ (t, 0) T_{t} V (i)

(17)

So, $ϒ (t + 1, 0) = ϒ (t, 0) T_{t}$ . In summary, $ϒ (t, 0)$ and $T^{a} (t, 0)$ both satisfy the same discrete-time linear equation, as shown in equation (17). The uniqueness of solutions of initial value problem $ϒ (0, 0) V = T^{a} (0, 0) V = V$ , means if $t \geq 0, ϒ (t, 0) V = T^{a} (t, 0) V$ . So, under Assumption 1, Theorem 1 is proved.

Remark 4. Theorem 1 mainly uses operator theory and stochastic analysis to prove the equivalence relation between the anti-causal evolution operator and conditional expectation for stochastic systems with infinite Markov switching.

Furthermore, the Lyapunov equation for the exponential stability of the system $(A, P)$ is given.

Theorem 2. Under Assumptions 1–3, the following properties are equivalent:

(i) The zero equilibrium state of the system $(A, P)$ is SESMS.

(ii) There is a bounded sequence ${X_{t}}_{t \geq 0} \in H_{\infty}^{n}, X_{t} > 0$ satisfy

X (t, i) = \sum_{k = 0}^{m} A_{k}^{T} (t, i) E_{t} (X (t, i)) A_{k} (t, i) + I_{n}, i \in D

(iii) There is a bounded sequence ${Y_{t}}_{t \geq 0} \in H_{\infty}^{n}, Y_{t} >> 0, α > 0$ satisfy

\sum_{k = 0}^{m} A_{k}^{T} (t, i) E_{t} (Y (t, i)) A_{k} (t, i) - Y (t, i) \leq α I_{n}, i \in D

Proof. Let $x (t)$ is the state of system $(A, P)$ with respect to initial value problem $(x_{0}, i) \in R^{n} \times D_{0}$ . In order to prove the equivalence of this theorem, we first prove that SESMS is equivalent to ESMS-C.

According to the definition of the SESMS, ${L_{t}}_{t \geq 0}$ generates an exponentially stable evolution, then exist $β > 0, q \in (0, 1)$ , such that: $‖ T^{c} (t, 0) ‖ \leq β q^{t}$ . From the definition of positive bounded operator $L_{t},$ $T_{t}$ can be calculated as following

‖ T^{a} (t, 0) ‖ \leq ‖ T^{c} (t, 0) ‖ \leq β q^{t}

Using Theorem 1, the following equation can be deduced

E [{‖ Φ^{T} (t, 0) x_{0} ‖}^{2} | η_{0} = i] = x_{0}^{T} [T^{a} (t, 0) I] (i) x_{0}

Therefore, system $(A, P)$ is ESMS-C. On the contrary, if $(A, P)$ is the ESMS-C, there are independent random variables ${ω (t)}_{t \geq 0}$ and Markov chain $({η_{t}}_{t \geq 0,} {P_{t}}_{t \geq 0}, D)$ satisfy Assumption 1 and $P {η_{0} = i} > 0$ , that

E [{‖ Φ^{T} (t, 0) x_{0} ‖}^{2} | η_{0} = i] \leq β q^{t} {‖ x_{0} ‖}^{2}

where $β > 1, q \in (0, 1)$ , from Assumption 2, we know that if $P {η_{0} = i} > 0$ , then there exist j, $π_{t} (j) > 0$ , according to the Theorem 1 can obtain the following inequality

‖ T^{a} (t, 0) I ‖_{\infty} \leq β q^{t}

That is, $‖ T^{a} (t, 0) ‖ = ‖ T^{a} (t, 0) I ‖_{\infty} \leq β q^{t}$ , so $T_{t}$ generate an exponentially stable evolution, according to Theorem 7 in Ligang et al. (2021) this means that ${L_{t}}_{t \geq 0}$ generate an exponentially stable evolution, i.e., the zero-state equilibrium of $(A, P)$ is SESMS.

So, under Assumptions $1 - 3$ , the SESMS of system $(A, P)$ is equivalent to ESMS-C.

According to Zhou et al. (2016), we can obtain that ESMS-C is equivalent to $(ii) - (iii)$ . The equivalence of this theorem is proved.

Remark 5. SESMS requires only that the evolution operator generated by the corresponding Lyapunov type operator of the system obeys exponential decay, while ESMS-C requires not only that the mean square conditional expectation of the solution is exponential decay but also that the initial distribution probability of the Markov switching is greater than 0. In the case of time-invariant, these conditions can be equivalent, but in the case of time-invariant, the ESMS-C can be deduced from the SESMS of the system.

We further study the $l^{2}$ input–output state stability of for system (1) in the next section.

In system (1), external input perturbation $v = {v (t)}_{t \geq 0}$ is ${\tilde{H}}_{t}$ measurable stochastic process of $l^{2} (0, \infty; R^{nv})$ . For all the $v \in l^{2} (0, \infty; R^{n_{v}})$ , its corresponding norm is: $‖ v ‖_{l^{2} (0, \infty; R^{nv})}^{2} = (\sum_{t = 0}^{\infty} E (‖ v_{t} ‖^{2}))^{\frac{1}{2}}$ .

Let $x (t, x_{0}, v)$ be the solution of system (1) concerning external input disturbance $v = {v (t)}_{t \geq 0}$ and satisfy initial value condition $x (0) = x_{0}$ . Let $z (t, x_{0}, v)$ is the corresponding control output of system (1) with respect to $x (t, x_{0}, v)$ and $v = {v (t)}_{t \geq 0}$ .

Definition 4. System (1) is sad $l^{2}$ input–output state stability, if for any $(x_{0}, η_{0}) \in R^{n} \times D_{0}$ , such that $x (t, x_{0}, v) \in l^{2} (0, \infty; R^{n})$ and $z (t, x_{0}, v) \in l^{2} (0, \infty; R^{n_{z}})$ whenever $v \in l^{2} (0, \infty; R^{n_{v}})$ .

Theorem 3. If Assumptions 1–3 hold, the zero equilibrium state of system $(A, P)$ is SESMS and $v \in l^{2} (0, \infty; R^{n_{v}})$ , then system (1) satisfy

\sum_{t = 0}^{\infty} E [{‖ x (t, x_{0}, v) ‖}^{2}] < \infty

(18)

\sum_{t = 0}^{\infty} E [{‖ z (t, x_{0}, v) ‖}^{2}] < \infty

(19)

and system (1) can be called as $l^{2}$ input–output state stability.

Proof. If the zero equilibrium state of system $(A, P)$ is SESMS, then Theorem 2 shows that there is a bounded sequence ${X_{t}}_{t \in Z_{+}} \in H_{\infty}^{n +}$ , $i \in D$ , $X (t) = (X (t, 1), X (t, 2), \dots)$ satisfy

X (t, i) = \sum_{k = 0}^{m} A_{k}^{T} (t, i) E_{t} (X (t + 1, i)) A_{k} (t, i) + I_{n}

(20)

I_{n} \leq X (t, i) \leq c I_{n}

(21)

where $c \geq 1$ , and it is independent of variables t and i. Let $x (t) = x (t, x_{0}, v)$ , as described above, then build a Lyapunov function: $V (t, x, i) = x^{T} X (t, i) x$ . Without loss of generality, let state space of the infinite Markov chain satisfy $D_{0} = D$ , for any $(x_{0}, η_{0}) \in R^{n} \times D_{0}$ , $v \in l^{2} (0, \infty; R^{n_{v}})$ .

Because $V (t + 1, x (t + 1), j)$ is $H_{t}$ measurable, according to Assumption 1 and Property 1.1 in Dragan et al. (2010), the following equation can be obtained

\begin{array}{l} E [V (t + 1, x (t + 1), η_{t + 1}) | ℋ_{t}] \\ = \sum_{j = 1}^{\infty} E [V (t + 1, x (t + 1), j) 1_{{η_{t + 1} = j}} | ℋ_{t}] \\ = \sum_{j = 1}^{\infty} V (t + 1, x (t + 1), j) p_{t} (η_{t}, j) \end{array}

(22)

Consider that ${\tilde{H}}_{t} \subset H_{t}$ , $v (t), x (t), η_{t}$ are all ${\tilde{H}}_{t}$ measurable, and that ${ω (t)}_{t \in Z_{+}}$ and ${η_{t}}_{t \in Z_{+}}$ are independent of each other. According to the definition of operator $E_{t}$ of formula (5) and the property of the conditional expectation in Mao and Yuan (2006), the following equation can be deduced

\begin{matrix} E [V (t + 1, x (t + 1), η_{t + 1}) | {\tilde{H}}_{t}] \\ = \sum_{j = 1}^{\infty} p_{t} (η_{t}, j) E [V (t + 1, x (t + 1), j) | {\tilde{H}}_{t}] \\ = \sum_{k = 0}^{m} {[A_{k} (t, η_{t}) x (t) + B_{k} (t, η_{t}) v (t)]}^{T} E_{t} (X (t + 1, η_{t})) * \end{matrix}

(23)

Considering $σ [η_{0}] \subset {\tilde{H}}_{t}$ , based on equation (23), we can deduce the equation as following

\begin{matrix} E [V (t + 1, x (t + 1), η_{t + 1}) | η_{0}] \\ = E [E [V (t + 1, x (t + 1), η_{t + 1}) | {\tilde{H}}_{t}] | η_{0}] \\ = E [\sum_{k = 0}^{m} {[A_{k} (t, η_{t}) x (t) + B_{k} (t, η_{t}) v (t)]}^{T} E_{t} (X (t + 1, η_{t})) * | η_{0}] \end{matrix}

(24)

According to equation (24) and SESMS of system $(A, P)$ , we can obtain

\begin{matrix} E [V (t + 1, x (t + 1), η_{t + 1}) | η_{0}] - E [V (t, x (t), η_{t}) | η_{0}] \\ = - E [{‖ x (t) ‖}^{2} | η_{0}] + \sum_{k = 0}^{m} \\ E [{(B_{k} (t, η_{t}) v (t))}^{T} E_{t} (X (t + 1, η_{t})) * | η_{0}] \\ + 2 \sum_{k = 0}^{m} E [{(A_{k} (t, η_{t}) x (t))}^{T} E_{t} (X (t + 1, η_{t})) (B_{k} (t, η_{t}) v (t)) | η_{0}] \end{matrix}

(25)

From the above equation (25), we can write the following equation

\begin{matrix} E [V (t + 1, x (t + 1), η_{t + 1}) | η_{0}] - E [V (t, x (t), η_{t}) | η_{0}] \\ = - \frac{1}{2} E [{‖ x (t) ‖}^{2} | η_{0}] + \sum_{k = 0}^{m} E [{(B_{k} (t, η_{t}) v (t))}^{T} M (t, η_{t}) * | η_{0}] \\ - \frac{1}{2} E [{‖ \frac{1}{m + 1} x (t) - 2 \sqrt{m + 1} A_{k}^{T} (t, η_{t}) E_{t} (X (t + 1, η_{t})) (B_{k} (t, η_{t}) v (t)) ‖}^{2} | η_{0}] \end{matrix}

(26)

where $M (t, η_{t}) = E_{t} (X (t + 1, η_{t})) + 2 (m + 1) E_{t} (X (t + 1, η_{t}))$ $A_{k} (t, η_{t}) A_{k}^{T} (t, η_{t}) E_{t} (X (t + 1, η_{t}))$ , $β > 0$ , under Assumption 3, exists $β > 0$ , such that equation (27) is true

‖ B_{k} {(t, η_{t})}^{T} M (t, η_{t}) B_{k} (t, η_{t}) ‖_{\infty} \leq β

(27)

Substituting equation (27) back into equation (26), we get the following inequality

\begin{matrix} E [V (t + 1, x (t + 1), η_{t + 1}) | η_{0}] - E [V (t, x (t), η_{t}) | η_{0}] \\ \leq - \frac{1}{2} E [{‖ x (t) ‖}^{2} | η_{0}] + β (m + 1) E [{‖ v (t) ‖}^{2} | η_{0}] \end{matrix}

(28)

Using equation (21), the following inequality can be obtained

\begin{matrix} - \frac{1}{2} E [{‖ x (t) ‖}^{2} | η_{0}] \leq - \frac{1}{2 c} E [x^{T} (t) X (t, η_{t}) x (t) | η_{0}] \\ = - \frac{1}{2 c} E [V (t, x (t), η_{t}) | η_{0}] \end{matrix}

(29)

Substituting equation (29) into equation (28), the following equation holds

\begin{array}{l} E [V (t + 1, x (t + 1), η_{t + 1}) | η_{0}] \\ \leq μ E [V (t, x (t), η_{t}) | η_{0}] + β (m + 1) E [{‖ v (t) ‖}^{2} | η_{0}] \end{array}

(30)

where $μ = 1 - 1 / 2 c, 0 < μ < 1$ . The above equation (30) is calculated and inequality (31) is obtained

\begin{matrix} E [V (t, x (t), η_{t}) | η_{0}] \\ < μ^{t} E [V (0, x (0), η_{0}) | η_{0}] + β (m + 1) \sum_{s = 0}^{t - 1} μ^{t - 1 - s} E [{‖ v (s) ‖}^{2} | η_{0}] \end{matrix}

(31)

By summing equation (31), we can get

\begin{matrix} \sum_{t = 1}^{τ} E [V (t, x (t), η_{t}) | η_{0}] \\ < \frac{1}{1 - μ} {E [x_{0}^{T} X (0, η_{0}) x_{0} | η_{0}] + β (m + 1) \sum_{s = 0}^{τ - 1} E [{‖ v (s) ‖}^{2} | η_{0}]} \end{matrix}

(32)

And using the formula (21), if $τ \to \infty$ , the following inequality is obtained

\begin{matrix} \sum_{t = 1}^{\infty} E [{‖ x (t) ‖}^{2} | η_{0}] < \sum_{t = 1}^{\infty} E [V (t, x (t), η_{t}) | η_{0}] \\ \leq \frac{1}{1 - μ} {E [x_{0}^{T} X (0, η_{0}) x_{0} | η_{0}] + β (m + 1) \sum_{t = 0}^{\infty} E [{‖ v (t) ‖}^{2} | η_{0}]} \end{matrix}

(33)

For any $(x_{0}, η_{0}) \in R^{n} \times D_{0}$ , as $v \in l^{2} (0, \infty; R^{nv})$ satisfy $\sum_{t = 0}^{\infty} E [‖ x (t, x_{0}, v) ‖^{2}] < \infty$ .

Next, the output of system (1) is calculated, let $z (t) = z (t, x_{0}, v)$ and satisfy

\begin{matrix} z (t) = [C_{0} (t, η_{t}) x (t) + D_{0} (t, η_{t}) v (t)] \\ + \sum_{k = 1}^{m} [C_{k} (t, η_{t}) x (t) + D_{k} (t, η_{t}) v (t)] ω_{k} (t) \end{matrix}

Through calculation, we can get

\begin{matrix} E [{‖ z (t) ‖}^{2} | η_{0}] \\ = \sum_{k = 0}^{m} E {[{(C_{k} (t, η_{t}) x (t))}^{T} * + {(D_{k} (t, η_{t}) v (t))}^{T} *] | η_{0}} \\ + \sum_{k = 0}^{m} E [2 {(C_{k} (t, η_{t}) x (t))}^{T} (D_{k} (t, η_{t}) v (t)) | η_{0}] \end{matrix}

(34)

Equation (34) also can be written as follows

\begin{array}{l} E [{‖ z (t) ‖}^{2} | η_{0}] \\ = \sum_{k = 0}^{m} E {[\frac{3}{2} {‖ C_{k} (t, η_{t}) x (t) ‖}^{2} + 3 {‖ D_{k} (t, η_{t}) v (t) ‖}^{2}] | η_{0}} \\ - \sum_{k = 0}^{m} E [\frac{1}{2} ‖ C_{k} (t, η_{t}) x (t) - 2 D_{k} (t, η_{t}) v (t) ‖ | η_{0}] \\ \leq \sum_{k = 0}^{m} E [(\frac{3}{2} {‖ C_{k} (t, η_{t}) ‖}^{2} {‖ x (t) ‖}^{2} + 3 {‖ D_{k} (t, η_{t}) ‖}^{2} {‖ v (t) ‖}^{2}) | η_{0}] \end{array}

(35)

Therefore, under Assumption 3, there exist $λ_{1} > 0$ and $λ_{2} > 0$ , the formula (36) is true

‖ C_{k} (t, η_{t}) ‖^{2} \leq λ_{1}, ‖ D_{k} (t, η_{t}) ‖^{2} \leq λ_{2}

(36)

Using equation (36), the inequality (37) can be deduced as follows

\begin{matrix} E [{‖ z (t) ‖}^{2} | η_{0}] \leq \frac{3}{2} λ_{1} (m + 1) E [{‖ x (t) ‖}^{2} | η_{0}] \\ + 3 λ_{2} (m + 1) E [{‖ v (t) ‖}^{2} | η_{0}] \end{matrix}

(37)

For any $(x_{0}, η_{0}) \in R^{n} \times D_{0}$ , if $v \in l^{2} (0, \infty; R^{n_{v}})$ satisfy: $\sum_{t = 0}^{\infty} E [‖ z (t, x_{0}, v) ‖^{2}] < \infty$ . Therefore, under Assumptions 1–3, for any $(x_{0}, η_{0}) \in R^{n} \times D_{0}$ , $v \in l^{2} (0, \infty; R^{n_{v}})$ satisfy $x (t, x_{0}, v) \in l^{2} (0, \infty; R^{n})$ and $z (t, x_{0}, v) \in l^{2} (0, \infty; R^{n_{z}})$ , equation (38) as follows holds

\sum_{t = 0}^{\infty} E [{‖ x (t, x_{0}, v) ‖}^{2}] < \infty, \sum_{t = 0}^{\infty} E [{‖ z (t, x_{0}, v) ‖}^{2}] < \infty

(38)

Therefore, inequalities (18) and (19) are satisfied, then system (1) is $l^{2}$ input–output state stability, i.e., the proof of Theorem 3 is completed.

Remark 6. If system (1) is $l^{2}$ input–output state stability, then system $(A, P)$ is stochastic stable, which can be proved by perturbation of the external input $v (t) \equiv 0$ .

The numerical example

Example 1. Consider the system $(A, P)$ in the special case where the corresponding transition probability matrix of the Markov chain satisfies: if $j = 1$ , $p (i, j) = 1$ ; if $j \neq 1$ , $p (i, j) = 0$ ; Therefore, if it does not satisfy the proceeding conditions, then the transition probability matrix $P_{t}$ is called non-degenerate, i.e., Assumption 2 is not satisfied. The corresponding parameter matrix of system $(A, P)$ satisfies: $A_{0} (t, 1) = 0, A_{0} (t, i) = ti I_{n}, i \neq 1$ ; for $1 \leq k \leq m, i \in D, t \geq 0$ , there are $A_{k} (t, i) = 0$ . At this time, $lim_{t \to \infty} | | A (t) | |_{\infty} = \infty$ , i.e., Assumption 3 is not satisfied.

Then, for all $t \geq 1, j \neq 1$ , we can get $P {η_{t} = j} = 0$ , i.e., $η_{t} = 1 a . s .$ , then $Φ (t, 0) = 0 a . s .$ ; therefore, in this special case, the zero equilibrium state of system $(A, P)$ is ESMS-C.

According to the definition of evolution operator $T^{a} (t, s)$ and operator ${T_{t}}_{t \geq 0}$ , it can be calculated as following

[T^{a} (t + 1, t) I] (3) = T_{t} I (3) = \sum_{k = 0}^{m} A_{k} (t, 3)^{T} E_{t} I (3) A_{k} (t, 3) = 9 t^{2} I_{n}

Then, satisfy ${‖ T^{a} (t + 1, t) ‖}_{\infty} \geq t^{2}, \lim_{t \to \infty} {‖ T^{a} (t + 1, t) ‖}_{\infty} = \infty$ , so the zero equilibrium state of system $(A, P)$ is not SESMS.

In conclusion, in the absence of Assumptions 2 and 3, the Asymptotically Exponentially Stable in the Mean Square with Condition of system $(A, P)$ is not equivalent to SESMS.

Example 2. Under Assumptions 1–3, considering the special case of system (1) and make the corresponding transition probability matrix of Markov chain ${η_{t}}_{t \in Z_{+}}$ satisfy: if $j = i + 1$ , then $p (i, j) = 1$ ; if $j \neq i + 1$ , then $p (i, j) = 0$ . Let $P (η_{0} = 1) = 1 (i . e . D_{0} = {1})$ , $n = m = 1$ ; $B_{0} (t, η_{t}) = B_{1} (t, η_{t}) = 1$ , and corresponding parameters: $A_{0} (t, η_{t}) = A_{1} (t, η_{t}) = η_{t} (t + 2) / \sqrt{3} (η_{t} + 1) (t + 1)$ .

It can be obtained from the definition of evolution operator $T^{c} (t, s)$ and operator ${L_{t}}_{t \geq 0}$

| | T^{c} (t, 0) | | \leq {(\frac{2}{3})}^{t}

Then, the zero equilibrium state of system $(A, P)$ is SESMS.

Combined with the above parameters and shown in Figure 1, the switching trajectory of Markov chain ${η_{t}}_{t \geq 0}$ and the change trajectory of ${ω (t)}_{t \geq 0}$ are simulated by MATLAB, which shows that the infinite Markov switching is a step mode that grows over time. However, Figure 2 proves that the trajectory of state $x (t)$ of system $(A, P)$ decays to 0 as $t \to \infty$ . And $y (t) = \sum_{k = 0}^{t} E [| | x (k {) | |}^{2}]$ finally approaches to a constant value.

Meanwhile, for the external input disturbance $v (t) = 6 e^{- t} \cos 10 t$ in system (1), the initial value condition is $(x_{0}, η_{0}) = (6, 1)$ . According to Theorem 3, the formula can be obtained as follows

\begin{matrix} \sum_{t = 0}^{\infty} E [{‖ x (t, x_{0}, v) ‖}^{2}] < \infty \\ \sum_{t = 0}^{\infty} E [{‖ z (t, x_{0}, v) ‖}^{2}] < \infty \end{matrix}

Therefore, inequalities (18) and (19) can obtain that the discrete-time stochastic system (1) with infinite Markov switching is $l^{2}$ input–output state stability. It can be seen from Figure 3 that if the external input disturbance in system (1) is finite, the state $x (t)$ of the stochastic system (1) infinite Markov switching eventually decays to 0 with the increasing of time when the zero equilibrium state of system $(A, P)$ is SESMS. Correspondingly, $y (t) = \sum_{k = 0}^{t} E [| | x (k {) | |}^{2}]$ is also approach to a constant value. Therefore, Figures 1 and 3 prove the correctness and validity of sufficient conditions for $l^{2}$ input–output state stability described in Theorem 3.

Conclusion

In this paper, we study the ESMS and the $l^{2}$ input–output state stability for discrete-time time-varying stochastic systems with infinite Markov switching, multiplicative noise, and external disturbance. First, the concepts of SESMS and ESMS-C are given based on positive bounded operators in Banach spaces, and the equivalence between ESMS-C and SESMS are proved under bounded non-degenerate transition probability matrix and parameter matrix. Second, the relationship between the internal stability of the system and the $l^{2}$ input–output state stability is given under finite-energy external random perturbation. Finally, an example and simulation are used to verify its correctness and effectiveness.

The exponential stability and $l^{2}$ input–output state stability of discrete-time time-varying infinite Markov jump systems with multiplicative noises also can be more clearly used to study the infinite state Markov switching problem, extend the time-invariant case to the time-varying case, and then to the related control problems, and can also be used to study the related problems of the infinite time-varying semi-Markov jump.

Footnotes

Appendix A

Remark 7. It is easy to verify the following:

Definition 6. A Markov chain is a triple $({η_{t}}_{t \geq 0,} {P_{t}}_{t \geq 0}, D)$ , where for each $t \geq 0, η_{t} : Ω \to D$ is a random variable, ${P_{t}}_{t \geq 0}$ is a sequence of stochastic matrices $P_{t} \in R^{N \times N}$ with the property: $P {η_{t + 1} = j | G_{t}} = p_{t} (η_{t}, j)$ a.s. for all $t \geq 0$ , and $j \in D$ , where $G_{t} = σ [η (0), \dots, η (t)]$ , $t \geq 0$ .

${P_{t}}_{t \geq 0}$ is called the sequence of transition probability matrices, and $j \in D$ is the set of the states of Markov chain.

If the sequence P is constant, i.e., $P_{t} = P$ for all $t \geq 0$ , then $({η_{t}}_{t \geq 0,} {P_{t}}_{t \geq 0}, D)$ is called an homogeneous Markov chain.

Corollary 1. Under the assumptions $H_{1}$ and $H_{2}$ , we have

P {η_{t + 1} = j | H_{t}} = p_{t} (η_{t}, j) a . s .

for all $t \geq 0$ , $j \in D$ .

Definition 7. We say that the sequence $L = {L_{k}}_{k \geq k_{o}}$ defines a positive evolution if for all $k \geq l \geq k_{0}$ , the causal linear evolution operator $T (kl)^{c} \geq 0$ .

Definition 8. We say that the zero-state equilibrium of system (1) is ESMS if there exist $β \geq 1, q \in (0, 1)$ such that for any sequence of independent random vectors ${ω (t)}_{t \geq 0}$ and for any Markov chain $({η_{t}}_{t \geq 0,} {P_{t}}_{t \geq 0}, D)$ satisfying $H_{1}$ and $H_{2}$ , we have $E [{| Φ (t, s) x_{0} |}^{2}] \leq β q^{t - s} {| x_{0} |}^{0}$ for all $t \geq s \geq 0, x_{0} \in R^{n}$ .

Lemma 1. Under Assumptions 1–3, the following properties are equivalent:

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the Natural Science Foundation of Chongqing (NO: cstc2019jcyj-msxmX0240).

ORCID iD

Hongxia Zhao

References

Aberkane

Dragan

(2020) On the existence of the stabilizing solution of a class of periodic stochastic Riccati equations. IEEE Transactions on Automatic Control 65: 1288–1294.

Dragan

Morozan

(2006a) Exponential stability for discrete time linear equations defined by positive operators. Integral Equations and Operator Theory 54: 465–493.

Dragan

Morozan

(2006b) Mean square exponential stability for some stochastic linear discrete time systems. European Journal of Control 12: 373–395.

Dragan

Morozan

(2008) Discrete time linear equations defined by positive operators on ordered Hilbert spaces. Revue Roumaine Des Mathematiques Pures Et Appliquees 53: 131–166.

Dragan

Morozan

Stoica

, et al. (2010) Mathematical Methods in Robust Control of Discrete-Time Linear Stochastic Systems. New York: Springer, 2010.

Fang

Loparo

(2002) Stochastic stability of jump linear systems. IEEE Transactions on Automatic Control 47(7): 1204–1208.

Fragoso

Costa

OLV

(2005) A unified approach for stochastic and mean square stability of continuous-time linear systems with Markovian jumping parameters and additive disturbances. SIAM Journal on Control and Optimization 44: 1165–1191.

Hou

(2015a) Exponential stability for discrete-time infinite Markov jump systems. IEEE Transactions on Automatic Control 61: 4241–4246.

Hou

(2015b) Stability analysis of discrete-time stochastic systems with infinite Markov jump parameter. In: Proceedings of the American Control Conference, Chicago, IL, 1–3 July2015, pp. 4192–4197. New York: IEEE.

10.

Ligang

Wang

Gao

, et al. (2021) Constrained sliding mode control for MIMO nonlinear systems. Scientia Sinica Informationis 51(3): 430–448.

11.

Liu

Hou

(2020) Robust H2/ H∞ fuzzy filtering for nonlinear stochastic systems with infinite Markov jump. Journal of Systems Science and Complexity 33: 1023–1039.

12.

Zhang

Hou

, et al. (2011) Infinite horizon H2/H∞ control for discrete-time time-varying Markov jump systems with multiplicative noise. IFAC Proceedings Volumes 44: 4626–4631.

13.

Mao

Yuan

(2006) Stochastic Differential Equations with Markovian Switching. Singapore: World Scientific Publishing.

14.

Pan

Wang

(2018) H∞ output feedback controls for singular Markovian systems. In: 2018 37th Chinese Control Conference (CCC), Wuhan, China, 25–27 July2018, pp. 1545–1550. New York: IEEE.

15.

Ungureanu

Dragan

(2013) Stability of discrete-time positive linear operators on ordered Banach spaces and applications. Journal of Difference Equations and Applications 19: 952–980.

16.

Xiang

Wang

(2011) Robust stabilization of discrete-time switched non-linear systems with time delay under asynchronous switching. Transactions of the Institute of Measurement and Control 33(5): 591–609.

17.

Xiong

Lam

Shu

, et al. (2013) Stability analysis of continuous-time switched systems with a random switching signal. IEEE Transactions on Automatic Control 59(1): 180–186.

18.

Zhang

Liu

Zheng

, et al. (2019) Exponential stability for Markovian neutral stochastic systems with general transition probabilities and time-varying delay. Transactions of the Institute of Measurement and Control 41(2): 350–365.

19.

Zhou

Zhang

Zhu

, et al. (2016) Stochastic Nash Games and H2/ H∞ control for Discrete Markov switching systems. Control Engineering of China 2016: 6.