A modified iterative algorithm for the (Hermitian) reflexive solution of the generalized Sylvester matrix equation

Abstract

Recently, Ramadan et al. have focused on the following matrix equation:

$A_{1} V + A_{2} \bar{V} + B_{1} W + B_{2} \bar{W} = E_{1} V F_{1} + E_{2} \bar{V} F_{2} + C$

and propounded two gradient-based iterative algorithms for solving the above matrix equation over reflexive and Hermitian reflexive matrices, respectively. In this paper, we develop two new iterative algorithms based on a two-dimensional projection technique for solving the mentioned matrix equation over reflexive and Hermitian reflexive matrices. The performance of our proposed algorithms is collated with the gradient-based iterative algorithms. It is both theoretically and experimentally demonstrated that the approaches handled surpass the offered algorithms in the earlier referred work in solving the mentioned matrix equation over reflexive and Hermitian reflexive matrices. In addition, it is briefly discussed that a one-dimensional projection technique can accelerate the speed of convergence of the gradient-based iterative algorithm for solving general coupled Sylvester matrix equations over reflexive matrices without assuming the restriction of the existence of a unique solution.

Keywords

Matrix equation iterative algorithm reflexive matrix Hermitian reflexive matrix

Introduction

In this paper, the ensuing notation is used. We utilize $tr (A), A^{T}, A^{H}$ and $\bar{A}$ to denote the trace, the transpose, the conjugate transpose and the conjugate of the matrix $A$ , respectively. The inner product of $X, Y \in C^{m \times n}$ is defined by $〈 X, Y 〉 = Re [tr (Y^{H} X)]$ . Here we would like to comment that this inner product was used by Wu et al. (2011c) and its induced matrix norm is specified to be $∥ X ∥^{2} = Re [tr (X^{H} X)]$ . For an arbitrary square matrix $Z$ , $det (Z)$ stands for the determinant of $Z$ . Moreover, $C^{m \times n}$ represents the set of all $m \times n$ complex matrices. The set of all Hermitian orthogonal matrices, also known as reflection matrices, in $C^{n \times n}$ is represented by $ℍ O ℂ^{n \times n}$ , in other words, $P \in ℍ O ℂ^{n \times n}$ if and only if $P = P^{H} = P^{- 1}$ .

Definition 1.1. (Chen, 1998) Consider a given arbitrary matrix $P \in ℍ O ℂ^{n \times n}$ . The matrix $A \in C^{n \times n}$ is called a reflexive matrix with respect to $P$ , if $A = PAP$ . The set of all $n \times n$ reflexive matrices is denoted by $C_{r}^{n \times n} (P) .$

Definition 1.2. (Chen, 1998) Consider an arbitrary given matrix $P \in ℍ O ℂ^{n \times n}$ . The matrix $A \in C^{n \times n}$ is called a Hermitian reflexive matrix if $A = A^{H} = PAP$ . The set of all $n \times n$ Hermitian reflexive matrices is denoted by ${HC}_{r}^{n \times n} (P) .$

Matrix equations play a cardinal role in numerous areas, such as control theory, system theory, stability theory and some other fields of pure and applied mathematics. The problem of determining solutions to various kinds of matrix equations has been investigated widely in the literature, for example, see Beik and Salkuyeh (2011), Bouhamidi and Jbilou (2008), Chang and Wang (1993), Ding and Chen (2006), Ding et al. (2008), Huang et al. (2008), Jbilou and Riquet (2006), Li et al. (2008), Song et al. (2011), Wu et al. (2011c), Zhang (2011) and Zhou and Duan (2008) and the references therein. Before stating the main contributions of this paper, we give a brief survey on some of the recently presented works on the topic of solving matrix equations which are related to the subject of this paper.

In the literature the solution of some kinds of matrix equations has been studied from a theoretical point of view and explicit forms of their solutions have been derived. For instance, Jiang and Wei (2003) have investigated the solutions of complex matrix equations $X - AXB = C$ and $X - A \bar{X} B = C$ and derived explicit solutions to the equations by the method of characteristic polynomials and a method of real representation of a complex matrix, respectively. In Wu et al. (2011a), the conjugate product and the Sylvester-conjugate sum have been introduced and their prosperities have been established. Afterwards, Wu et al. exploited the obtained results to propose a unified approach for solving a general class of Sylvester-polynomial-conjugate matrix equations, which incorporate the Yakubovich-conjugate matrix equation as a special case. Recently, Wu et al. (2012) have obtained several explicit parametric solutions to the generalized Sylvester-conjugate matrix equation.

Gradient-based iterative algorithms are a common approach for solving matrix equations; for more details see Ding et al. (2010, 2011), Han et al. (2010), Li et al. (2010), Liu et al. (2012), Wang and Liao (2012), Wu et al. (2011b), Xie et al. (2009, 2010) and Zhang et al. (2011) and the references therein. More precisely, Beik et al. (2014) have developed a gradient-based algorithm for solving a general class of the coupled matrix equations over (anti-)reflexive matrices. Ding and Chen (2005a,b,c,d,e) have presented various iterative methods based on the hierarchical identification principle to solve different kinds of matrix equations. Ding et al. (2008) have utilized the gradient-based methods for solving the matrix equations $AXB = F$ and $AXB + CXD = F$ . In Dehghan and Hajarian (2011), the authors have handled two gradient-based algorithms for solving the following the matrix equation:

\sum_{i = 1}^{p} A_{i} X B_{i} + \sum_{j = 1}^{q} C_{j} Y D_{j} = F

over reflexive and anti-reflexive matrices. In Dehghan and Hajarian (2012), the idea of the gradient-based iterative method has been employed to construct two iterative algorithms for computing the generalized bisymmetric and skew-symmetric solutions of the matrix equation

\sum_{i = 1}^{l} A_{i} Y B_{i} = C

Moreover, Song et al. (2011) have considered the following coupled Sylvester-transpose matrix equations:

\sum_{η = 1}^{p} (A_{i η} X_{η} B_{i η} + C_{i η} X_{η}^{T} D_{i η}) = F_{i}, i = 1, 2, \dots, N

where $A_{i η} \in ℝ^{m_{i} \times l_{η}}$ , $B_{i η} \in ℝ^{n_{η} \times p_{i}}$ , $C_{i η} \in ℝ^{m_{i} \times n_{η}}$ , $D_{i η} \in ℝ^{l_{η} \times p_{i}}$ , $F_{i} \in ℝ^{m_{i} \times p_{i}}$ , $i = 1, \dots, N$ , $η = 1, \dots, p$ , are given matrices and $X_{η} \in ℝ^{l_{η} \times n_{η}}, η \in I [1, p]$ are the matrices to be determined.

In Hajarian and Dehghan (2013), two gradient-based iterative methods have been presented by extending the Jacobi and Gauss–Seidel iterations for solving the generalized Sylvester-conjugate matrix equation

A_{1} X B_{1} + A_{2} \bar{X} B_{2} + C_{1} Y D_{1} + C_{2} \bar{Y} D_{2} = E

over reflexive and Hermitian reflexive matrices. The algorithms handled rely on a fixed parameter $μ$ whose optimum value has not been obtained. In the case where the problems have a unique solution, a sufficient condition has been established under which the proposed algorithms are convergent for any initial reflexive and Hermitian reflexive matrices. Nevertheless, in a manner similar to that described in the next section, the idea of projection methods can be utilized to select the fixed parameter $μ$ in a progressive way in order to ameliorate the speed of convergence of the authors’ algorithms. Meanwhile, it turns out that we only need to suppose that the mentioned matrix equation is consistent.

More recently, Hajarian (2013) has proposed a gradient-based algorithm to solve

{\begin{matrix} A_{1} X_{1} B_{1} + C_{1} X_{2} D_{1} = E_{1} \\ A_{2} X_{1} B_{2} + C_{2} X_{2} D_{2} = E_{2} \end{matrix}

over the generalized centro-symmetric matrix pair $(X_{1}, X_{2})$ . It has been assumed that the mentioned coupled matrix equations have a unique centro-symmetric matrix pair and a gradient-based algorithm has been offered. The algorithm relies on a fixed parameter denoted by $μ$ and the following sufficient condition has been derived under which the proposed algorithm converges:

0 < μ < \frac{2}{{‖ A_{1} B_{1} ‖}^{2} + {‖ C_{1} D_{1} ‖}^{2} + {‖ A_{2} B_{2} ‖}^{2} + {‖ C_{2} D_{2} ‖}^{2}}

Nevertheless, the way of choosing the optimum value of $μ$ has not been discussed.

In Wu et al. (2010), a gradient-based method is developed for solving the coupled Sylvester-conjugate matrix equation with a unique solution. Recently, Ramadan et al. (2014) have offered a relaxed gradient-based algorithm for solving the extended Sylvester-conjugate matrix equation $AXB + C \bar{X} D = F$ by considering a relaxation parameter. Under the assumption that the considered matrix equations have a unique solution, a sufficient condition has been established for the convergence of the presented algorithms. However, the optimum values of the fixed parameters in the proposed algorithms have not been derived. Here we would like to comment that the rate of convergence of the proposed algorithms in Wu et al. (2010) and Ramadan et al. (2014) can be improved by exploiting the strategy in Section 2.2 of this paper without setting the restriction of the existence of a unique solution.

Consider the generalized Sylvester matrix equation

A_{1} V + A_{2} \bar{V} + B_{1} W + B_{2} \bar{W} = E_{1} V F_{1} + E_{2} \bar{V} F_{2} + C

(1)

where $A_{i}, B_{i}, E_{i}, C \in C^{m \times n}$ , $F_{i} \in C^{n \times n}$ are given matrices for $i = 1, 2$ and $V, W \in C^{n \times n}$ are unknown matrices. In Ramadan et al. (2013), Ramadan et al. have assumed that equation (1) has a unique (Hermitian) reflexive solution pair. More precisely, two iterative algorithms have been offered for solving the following two main problems.

Problem 1. Presume that (1) has a unique reflexive solution pair $[V^{*}, W^{*}]$ such that $V^{*} \in ℂ_{r}^{n \times n} (P)$ and $W^{*} \in ℂ_{r}^{n \times n} (Q)$ where $P$ and $Q$ are given reflection matrices. Find the unique reflexive solution pair of (1).

Problem 2. Presume that (1) has a unique Hermitian reflexive solution pair $[V^{*}, W^{*}]$ such that $V^{*} \in {HC}_{r}^{n \times n} (P)$ and $W^{*} \in {HC}_{r}^{n \times n} (Q)$ where $P$ and $Q$ are given reflection matrices. Find the unique Hermitian reflexive solution pair of (1).

The two subsequent algorithms have been suggested for solving Problems 1 and 2; for further details see Ramadan et al. (2013).

Algorithm 1. (Ramadan et al., 2013) Algorithm for Problem 1.

Choose a tolerance $ε$ and an arbitrary initial Gauss pair $[V (1), W (1)]$ such that $V (1) \in C_{r}^{n \times n} (P)$ and $W (1) \in C_{r}^{n \times n} (Q)$ .

Set $k = 1$ . Calculate

\begin{matrix} R (1) = C - A_{1} V (1) - A_{2} \bar{V (1)} - B_{1} W (1) - B_{2} \bar{W (1)} \\ + E_{1} V (1) F_{1} + E_{2} \bar{V} (1) F_{2} \end{matrix}

Compute

\begin{matrix} V (k + 1) = V (k) + \frac{μ}{8} [A_{1}^{H} R (k) + {\bar{A}}_{2}^{H} \bar{R (k)} \\ - E_{1}^{H} R (k) F_{1}^{H} - {\bar{E}}_{2}^{H} \bar{R (k)} {\bar{F}}_{2}^{H} \\ + {PA}_{1}^{H} R (k) P + P {\bar{A}}_{2}^{H} \bar{R (k)} P \\ - {PE}_{1}^{H} R (k) F_{1}^{H} P - P {\bar{E}}_{2}^{H} \bar{R (k)} {\bar{F}}_{2}^{H} P] \end{matrix}

(2)

\begin{matrix} W (k + 1) = W (k) + \frac{μ}{4} [B_{1}^{H} R (k) + {\bar{B}}_{2}^{H} \bar{R (k)} \\ + {QB}_{1}^{H} R (k) Q + Q {\bar{B}}_{2}^{H} \bar{R (k)} Q] \end{matrix}

(3)

Calculate

\begin{matrix} R (k + 1) = C - A_{1} V (k + 1) - A_{2} \bar{V (k + 1)} \\ - B_{1} W (k + 1) - B_{2} \bar{W (k + 1)} + E_{1} V (k + 1) F_{1} \\ + E_{2} \bar{V (k + 1)} F_{2} \end{matrix}

If $∥ R (k + 1) ∥ < ϵ$ Stop; else set $k = k + 1$ and go to Step 3.

Algorithm 2. (Ramadan et al., 2013) Algorithm for solving Problem 2.

Choose a tolerance $ε$ and an arbitrary initial Gauss pair $[V (1), W (1)]$ such that $V (1) \in {HC}_{r}^{n \times n} (P)$ and $W (1) \in {HC}_{r}^{n \times n} (Q)$ .

Set $k = 1$ . Calculate

\begin{array}{l} R (1) = C - A_{1} V (1) - A_{2} \bar{V (1)} - B_{1} W (1) \\ - B_{2} \bar{W (1)} + E_{1} V (1) F_{1} + E_{2} \bar{V} (1) F_{2} \end{array}

Compute

\begin{matrix} V (k + 1) & = V (k) + \frac{μ}{16} [A_{1}^{H} R (k) + {\bar{A}}_{2}^{H} \bar{R (k)} - E_{1}^{H} R (k) F_{1}^{H} \\ - {\bar{E}}_{2}^{H} \bar{R (k)} {\bar{F}}_{2}^{H} \\ + R^{H} (k) A_{1} + {\bar{R (k)}}^{H} {\bar{A}}_{2} - F_{1} R^{H} (k) E_{1} \\ - {\bar{F}}_{2} {\bar{R (k)}}^{H} {\bar{E}}_{2} \\ + {PA}_{1}^{H} R (k) P + P {\bar{A}}_{2}^{H} \bar{R (k)} P - {PE}_{1}^{H} R (k) F_{1}^{H} P \\ - P {\bar{E}}_{2}^{H} \bar{R (k)} {\bar{F}}_{2}^{H} P \\ + R^{H} (k) A_{1} P + P {\bar{R (k)}}^{H} {\bar{A}}_{2} P - P F_{1} R^{H} (k) E_{1} P \\ - P {\bar{F}}_{2} {\bar{R (k)}}^{H} {\bar{E}}_{2} P] \end{matrix}

(4)

\begin{matrix} W (k + 1) = W (k) + \frac{μ}{8} [B_{1}^{H} R (k) + {\bar{B}}_{2}^{H} \bar{R (k)} + R^{H} (k) B_{1} \\ + {\bar{R (k)}}^{H} {\bar{B}}_{2} \\ + {QB}_{1}^{H} R (k) Q + Q {\bar{B}}_{2}^{H} \bar{R (k)} Q + Q R^{H} (k) B_{1} Q \\ + Q {\bar{R (k)}}^{H} {\bar{B}}_{2} Q] \end{matrix}

(4)

Calculate

\begin{matrix} R (k + 1) = C - A_{1} V (k + 1) - A_{2} \bar{V (k + 1)} \\ - B_{1} W (k + 1) - B_{2} \bar{W (k + 1)} \\ + E_{1} V (k + 1) F_{1} + E_{2} \bar{V (k + 1)} F_{2} \end{matrix}

If $∥ R (k + 1) ∥ < ϵ$ Stop; else set $k = k + 1$ and go to Step 3.

As seen the parameter $μ$ in Algorithms 1 and 2 is a given parameter which remains fixed during all of the iteration steps. It has been proved that if $0 < μ < 1 / H$ where

\begin{array}{l} H = ∥ A_{1} ∥^{2} + ∥ A_{2} ∥^{2} + ∥ E_{1} ∥^{2} ∥ F_{1} ∥^{2} + ∥ E_{2} ∥^{2} ∥ F_{2} ∥^{2} \\ + ∥ B_{1} ∥^{2} + ∥ B_{2} ∥^{2} \end{array}

then Algorithm 1 (Algorithm 2) converges to the unique (Hermitian) reflexive solution of (1) for an arbitrary (Hermitian) reflexive initial guess $[V (1), W (1)]$ ; see Ramadan et al. (2013).

Main contribution

In Ramadan et al. (2013), the authors have not discussed the way of choosing optimal parameter $μ$ in their proposed algorithms whereas the reported numerical experiments illustrate that the algorithms converge too slowly. To overcome this drawback, we first consider the proposed algorithms as special cases of general algorithms which depend on two parameters. Afterwards, we tender modified versions of the algorithms. As a matter of fact, a two-dimensional projection technique is applied. In the new algorithms, the parameters are selected in a progressive manner such that at each iterate the new approximate solutions satisfy an optimality condition. We would like to comment here that our adopted technique for accelerating the speed of convergence of the algorithms given by Ramadan et al. (2013) can also be used for improving the speed of convergence of the gradient-based iterative algorithms proposed in the literature. In fact, as known, the gradient-based algorithms generally rely on a fixed parameter denoted by $μ$ . Hence we may use a special case of our earlier idea to improve the rate of convergence of the gradient-based algorithms. That is, we can apply a one-dimensional projection technique to determine the parameter in a progressive way such that the norm of the residual matrix corresponding to the new approximate solution is minimized over the set of some possible approximate solutions which incorporates the approximate solution obtained by the gradient-based algorithm. So far, the convergence of the gradient-based algorithms has been studied under the hypothesis that the main problem has a unique solution except the work presented by Salkuyeh and Beik (2013) which can be also modified by employing a projection technique. To the best of our knowledge, hitherto, when the gradient-based algorithm is examined for solving different kinds of (coupled) matrix equations over reflexive, anti-reflexive, centro-symmetric, bisymmetric and skew-symmetric matrices, it has been always assumed that the mentioned problem has a unique solution; for more details see Beik et al. (2014), Dehghan and Hajarian (2011, 2012, 2013) and Hajarian (2013). In Section 2.2, we briefly study the application of a projection technique to improve the convergence of the gradient-based iterative algorithm for resolving the coupled Sylvester matrix equations over reflexive matrices. As seen in Section 2.1, using a one-dimensional projection technique, we may omit the restriction of the existence of a unique solution and derive an algorithm which converges faster than the gradient-based algorithm.

The outline of this paper is organized as follows. The second section consists of two subsections. In Section 2.1, we present the modified versions of Algorithms 1 and 2. More precisely, we prove that our modifications lead to iterative algorithms with a convergence speed superior to those offered by Ramadan et al. (2013). In Section 2.2, we exploit the special case of our idea in Section 2.1 to improve the convergence rate of the gradient-based iterative algorithm for solving the consistent coupled Sylvester matrix equations without the restriction of the existence of a unique solution. In Section 3, we examine our presented approaches experimentally to confirm the established theoretical results. Finally, the paper is ended with a brief conclusion in Section 4.

Main results

In this section, we first apply a two-dimensional projection technique to ameliorate the speed of convergence of Algorithms 1 and 2 (Ramadan et al., 2013) which is the subject of Section 2.1. Next, in Section 2.2, we utilize a one-dimensional projection technique to improve the rate of convergence of the gradient-based iterative algorithm for solving the general coupled Sylvester matrix equations over reflexive matrices without setting the restriction that the considered problem has a unique solution. As the results in Section 2.2 can be proved with strategies similar to those used in Section 2.1, we omit the details.

On a two-dimensional projection technique

For simplicity, we exploit the two linear operators

\begin{matrix} A : C^{n \times n} \to C^{m \times n} \\ V \to A (V) : = A_{1} V + A_{2} \bar{V} - E_{1} V F_{1} - E_{2} \bar{V} F_{2} \end{matrix}

and

\begin{matrix} B : C^{n \times n} \to C^{m \times n} \\ W \to B (W) : = B_{1} W + B_{2} \bar{W} \end{matrix}

By using the linear operators $A$ and $B$ , we may rewrite (1) as follows:

A (V) + B (W) = C

(6)

The conjugate transposes of the linear operators $A$ and $B$ are represented by $A^{*}$ and $B^{*}$ and defined by $A^{*} (Z) = A_{1}^{H} Z + A_{2}^{H} \bar{Z} - E_{1}^{H} {ZF}_{1}^{H} - {\bar{E}}_{2}^{H} \bar{Z} {\bar{F}}_{2}^{H}$ and $B^{*} (Z) = B_{1}^{H} Z + {\bar{B}}_{2}^{H} \bar{Z}$ for $Z \in ℂ^{m \times n}$ .

Suppose that (6) has a (Hermitian) reflexive solution pair. The following proposition returns that the necessary and sufficient condition for (6) to have a unique (Hermitian) reflexive solution pair is that the corresponding homogenous matrix equation has a unique (Hermitian) reflexive solution pair $[0, 0]$ . The proof is straightforward, hence we discard it.

Proposition 2.1. The matrix equation $A (V) + B (W) = C$ has a unique (Hermitian) reflexive solution pair $[V^{*}, W^{*}]$ if and only if the following homogenous matrix equation,

A (V) + B (W) = 0

has a unique (Hermitian) reflexive solution pair $[V^{*}, W^{*}] = [0, 0]$ .

Now we establish the following useful proposition.

Proposition 2.2. Suppose that (6) has a unique (Hermitian) reflexive solution pair $[V^{*}, W^{*}]$ . For an arbitrary (Hermitian) reflexive pair of matrices $[Z, Y] \neq [0, 0]$ , the following matrix is nonsingular:

Ψ = (\begin{matrix} 〈 A (Z), A (Z) 〉 & 〈 A (Z), B (Y) 〉 \\ 〈 B (Y), A (Z) 〉 & 〈 B (Y), B (Y) 〉 \end{matrix})

(7)

Proof. It is obvious that

\begin{matrix} det [(\begin{matrix} 〈 A (Z), A (Z) 〉 & 〈 A (Z), B (Y) 〉 \\ 〈 B (Y), A (Z) 〉 & 〈 B (Y), B (Y) 〉 \end{matrix})] \\ = 〈 A (Z), A (Z) 〉 〈 B (Y), B (Y) 〉 - {〈 A (Z), B (Y) 〉}^{2} \\ = ∥ A (Z) ∥^{2} ∥ B (Y) ∥^{2} - {〈 A (Z), B (Y) 〉}^{2} \geq 0 \end{matrix}

(8)

Now, it is sufficient to show that the above inequality strictly holds. Presume, to the contrary, that equality holds. Note that the inequality in (8) follows from the Cauchy–Schwarz inequality and it reduces to equality if and only if there exists a real scalar $γ \neq 0$ such that $A (Z) = γ B (Y)$ , or equivalently,

A (Z) + B (- γ Y) = 0

In view of Proposition 2.1, the above relation implies that $Z = Y = 0$ which is contrary to our assumption that $[Z, Y] \neq [0, 0]$ .□

Let us assume that $k$ steps of Algorithm 1 (Algorithm 2) have been performed. Suppose that $[V (k), W (k)]$ is the $k$ th approximate solution pair with the corresponding residual matrix $R (k)$ , that is, $R (k) = C - A_{1} V (k) - A_{2} \bar{V (k)} - B_{1} W (k) - B_{2} \bar{W (k)} + E_{1} V (k) F_{1} + E_{2} \bar{V} (k) F_{2}$ .

Throughout the current subsection, we use:

The sets $S_{V}$ and $S_{W}$ , associated with the $k$ th approximate solution computed by Algorithm 1, specified as follows:

S_{V} = {V (k) + α [\frac{A^{*} (R (k)) + P A^{*} (R (k)) P}{2}] | α \in ℝ}

and

S_{W} = {W (k) + β [\frac{B^{*} (R (k)) + Q B^{*} (R (k)) Q}{2}] | β \in ℝ}

The sets ${\tilde{S}}_{V}$ and ${\tilde{S}}_{W}$ , associated with the $k$ th approximate solution computed by Algorithm 2, specified as follows:

{\tilde{S}}_{V} = {V (k) + \tilde{α} [\frac{A^{*} (R (k)) + P A^{*} (R (k)) P + {(A^{*} (R (k)) + P A^{*} (R (k)) P)}^{H}}{4}] | \tilde{α} \in ℝ}

and

{\tilde{S}}_{W} = {W (k) + \tilde{β} [\frac{B^{*} (R (k)) + Q B^{*} (R (k)) Q + {(B^{*} (R (k)) + Q B^{*} (R (k)) Q)}^{H}}{4}] | \tilde{β} \in ℝ}

It can be easily seen that for the $(k + 1)$ th approximate solution pair $[V (k + 1), W (k + 1)]$ obtained by Algorithm 1 (Algorithm 2), we have

\begin{array}{l} V (k + 1) \in S_{V} and W (k + 1) \in S_{W} \\ V (k + 1) \in {\tilde{S}}_{V} and W (k + 1) \in {\tilde{S}}_{W} \end{array}

To see this, it is sufficient to set $α = μ / 4$ and $β = μ / 2$ ( $\tilde{α} = μ / 4$ and $\tilde{β} = μ / 2$ ).

In what follows, for simplicity we set

P_{V} = \frac{A^{*} (R (k)) + P A^{*} (R (k)) P}{2}, P_{W} = \frac{B^{*} (R (k)) + Q B^{*} (R (k)) Q}{2}

{\tilde{P}}_{V} = \frac{A^{*} (R (k)) + P A^{*} (R (k)) P + {(A^{*} (R (k)) + P A^{*} (R (k)) P)}^{H}}{4}

and

{\tilde{P}}_{W} = \frac{B^{*} (R (k)) + Q B^{*} (R (k)) Q + {(B^{*} (R (k)) + Q B^{*} (R (k)) Q)}^{H}}{4}

Now we present the following two practical propositions. As a matter of fact, the established results reveal that $P_{V} = P_{W} = 0$ iff $〈 R (k), A (P_{V}) 〉 = 〈 R (k), B (P_{W}) 〉 = 0$ and similarly ${\tilde{P}}_{V} = {\tilde{P}}_{W} = 0$ iff $〈 R (k), A ({\tilde{P}}_{V}) 〉 = 〈 R (k), B ({\tilde{P}}_{W}) 〉 = 0$ .

Proposition 2.3. Let the matrices $P_{V}$ and $P_{W}$ be defined as before. Then

〈 R (k), A (P_{V}) 〉 = 〈 P_{V}, P_{V} 〉 and 〈 R (k), B (P_{W}) 〉 = 〈 P_{W}, P_{W} 〉

Proof. Invoking the fact that $P_{V}$ is a reflexive matrix and by straightforward computations, we deduce that

\begin{matrix} 〈 R (k), A (P_{V}) 〉 & = 〈 A^{*} (R (k)), P_{V} 〉 \\ = 〈 A^{*} (R (k)), \frac{P_{V} + P P_{V} P}{2} 〉 \\ = 〈 \frac{A^{*} (R (k)) + P A^{*} (R (k)) P}{2}, P_{V} 〉 \\ = 〈 P_{V}, P_{V} 〉 \end{matrix}

□

The second relation can be also derived with an analogous approach.

In a manner similar to that applied in the proof of the above proposition, we may establish the following proposition.

Proposition 2.4. Let the matrices ${\tilde{P}}_{V}$ and ${\tilde{P}}_{W}$ be defined as before. Then

〈 R (k), A ({\tilde{P}}_{V}) 〉 = 〈 {\tilde{P}}_{V}, {\tilde{P}}_{V} 〉 and 〈 R (k), B ({\tilde{P}}_{W}) 〉 = 〈 {\tilde{P}}_{W}, {\tilde{P}}_{W} 〉

In the following, we present two useful propositions. It turns out that $P_{V} = P_{W} = 0$ ( ${\tilde{P}}_{V} = {\tilde{P}}_{W} = 0$ ) iff $R (k) = 0$ . That is, $P_{V} = P_{W} = 0$ ( ${\tilde{P}}_{V} = {\tilde{P}}_{W} = 0$ ) iff the $k$ th approximate solution is the exact solution.

Proposition 2.5. Suppose that the matrix equation (6) is consistent over reflexive matrices and $[V^{*}, W^{*}]$ is an arbitrary reflexive solution pair of (6). Then

〈 P_{V}, V^{*} - V (k) 〉 + 〈 P_{W}, W^{*} - W (k) 〉 = 〈 R (k), R (k) 〉

Proof. As $V^{*} - V (k)$ and $W^{*} - W (k)$ are reflexive matrices, we derive that

\begin{matrix} 〈 P_{V}, V^{*} - V (k) 〉 + 〈 P_{W}, W^{*} - W (k) 〉 \\ = 〈 A^{*} (R (k)), V^{*} - V (k) 〉 + 〈 B^{*} (R (k)), W^{*} - W (k) 〉 \\ = 〈 R (k), A (V^{*} - V (k)) 〉 + 〈 R (k), B (W^{*} - W (k)) 〉 \\ = 〈 R (k), R (k) 〉 \end{matrix}

The proof of the next proposition is similar to the preceding proposition.

Proposition 2.6. Suppose that the matrix equation (6) is consistent over Hermitian reflexive matrices and $[V^{*}, W^{*}]$ is an arbitrary Hermitian reflexive solution pair of (6). Then

〈 {\tilde{P}}_{V}, V^{*} - V (k) 〉 + 〈 {\tilde{P}}_{W}, W^{*} - W (k) 〉 = 〈 R (k), R (k) 〉

Our goal is to determine the parameters $α^{*}$ and $β^{*}$ ( ${\tilde{α}}^{*}$ and ${\tilde{β}}^{*}$ ) such that the residual matrix $R (k + 1)$ is minimized. That is, if $V^{*} (k + 1) \in S_{V}$ and $W^{*} (k + 1) \in S_{W}$ ( ${\tilde{V}}^{*} (k + 1) \in {\tilde{S}}_{V}$ and ${\tilde{W}}^{*} (k + 1) \in {\tilde{S}}_{W}$ ) are obtained related to $α^{*}, β^{*}$ ( ${\tilde{α}}^{*}, {\tilde{β}}^{*}$ ) and with the associated residual matrix $R^{*} (k + 1)$ , then

∥ R^{*} (k + 1) ∥ = \min_{S_{k + 1}} ∥ R (k + 1) ∥

(9)

where $S_{k + 1} = {[V, W] | V \in S_{V} and W \in S_{W}}$ ( $S_{k + 1} = {[V, W] | V \in {\tilde{S}}_{V} and W \in {\tilde{S}}_{W}}$ ) and $R (k + 1)$ is the residual matrix corresponding to an arbitrary pair $[V (k + 1), W (k + 1)] \in S_{k + 1}$ .

To this end, we apply a two-dimensional projection approach. More precisely, we establish that it is sufficient to determine the parameters $α^{*}$ and $β^{*}$ such that

〈 R^{*} (k + 1), A (P_{V}) 〉 = 〈 R^{*} (k + 1), ℬ (P_{W}) 〉 = 0

(10)

and the parameters ${\tilde{α}}^{*}$ and ${\tilde{β}}^{*}$ such that

〈 R^{*} (k + 1), A ({\tilde{P}}_{V}) 〉 = 〈 R^{*} (k + 1), B ({\tilde{P}}_{W}) 〉 = 0

(11)

It is not difficult to verify that the orthogonality conditions (10) and (11) respectively are equivalent to saying that the parameters $α^{*}$ and $β^{*}$ are the solution to the following linear system of equations:

(\begin{matrix} 〈 A (P_{V}), A (P_{V}) 〉 & 〈 ℬ (P_{W}), A (P_{V}) 〉 \\ 〈 A (P_{V}), ℬ (P_{W}) 〉 & 〈 ℬ (P_{W}), ℬ (P_{W}) 〉 \end{matrix}) (\begin{matrix} α^{*} \\ β^{*} \end{matrix}) = (\begin{matrix} 〈 R (k), A (P_{V}) 〉 \\ 〈 R (k), ℬ (P_{W}) 〉 \end{matrix})

(12)

and the parameters ${\tilde{α}}^{*}$ and ${\tilde{β}}^{*}$ are the solution to the sequent linear system of equations

(\begin{matrix} 〈 A ({\tilde{P}}_{V}), A ({\tilde{P}}_{V}) 〉 & 〈 B ({\tilde{P}}_{W}), A ({\tilde{P}}_{V}) 〉 \\ 〈 A ({\tilde{P}}_{V}), B ({\tilde{P}}_{W}) 〉 & 〈 B ({\tilde{P}}_{W}), B ({\tilde{P}}_{W}) 〉 \end{matrix}) (\begin{matrix} {\tilde{α}}^{*} \\ {\tilde{β}}^{*} \end{matrix}) = (\begin{matrix} 〈 R (k), A ({\tilde{P}}_{V}) 〉 \\ 〈 R (k), B ({\tilde{P}}_{W}) 〉 \end{matrix})

(13)

As seen, if both of $P_{V}, P_{W}$ ( ${\tilde{P}}_{V}, {\tilde{P}}_{W}$ ) become zero then the $k$ th approximate solution is in fact the exact solution. Hence, in the next two theorems, without loss of generality we may assume that $[P_{V}, P_{W}] \neq [0, 0]$ and $[{\tilde{P}}_{V}, {\tilde{P}}_{W}] \neq [0, 0]$ and therefore Proposition 2.2 implies that the coefficient matrices of linear systems (12) and (13) are nonsingular. The following two theorems reveal the superior prosperities of the approximate solutions obtained after applying the projection technique in comparison with those computed via the algorithms presented by Ramadan et al. (2013). The following two theorems demonstrate that if, at each iterate, we use the offered projection technique instead of using fixed parameters then the residual matrix corresponding to the new approximation satisfies an optimality property. Meanwhile, in Ramadan et al. (2013) there is no suggestion for choosing the optimum value of the fixed parameter $μ$ , and only a sufficient condition for $μ$ is given.

Theorem 2.7. Suppose that (6) has a unique reflexive solution pair. If the coefficient matrix of the linear system (12) is nonsingular, then

∥ R^{*} (k + 1) ∥ \leq ∥ R (k + 1) ∥

where

\begin{matrix} R (k + 1) = R (k) - α A (P_{V}) - β B (P_{W}), \\ R^{*} (k + 1) = R (k) - α^{*} A (P_{V}) - β^{*} B (P_{W}) \end{matrix}

in which $α$ and $β$ are arbitrary real parameters.

Proof. By (10), it is not difficult to see that

\begin{matrix} R (k + 1) = R (k) - (α - α^{*} + α^{*}) A (P_{V}) - (β - β^{*} + β^{*}) B (P_{W}) \\ = R (k) - α^{*} A (P_{V}) - β^{*} B (P_{W}) - (α - α^{*}) A (P_{V}) \\ - (β - β^{*}) B (P_{W}) \\ = R^{*} (k + 1) - (α - α^{*}) A (P_{V}) - (β - β^{*}) B (P_{W}) \end{matrix}

Therefore, we may conclude that

\begin{matrix} 〈 R (k + 1), R (k + 1) 〉 = 〈 R^{*} (k + 1), R^{*} (k + 1) 〉 \\ + {(α - α^{*})}^{2} ∥ A (P_{V}) ∥^{2} \\ + {(β - β^{*})}^{2} ∥ B (P_{W}) ∥^{2} \\ + 2 (α - α^{*}) (β - β^{*}) 〈 A (P_{V}), B (P_{W}) 〉 \\ \geq 〈 R^{*} (k + 1), R^{*} (k + 1) 〉 \\ + 2 | (α - α^{*}) (β - β^{*}) | ∥ A (P_{V}) ∥ ∥ B (P_{W}) ∥ \\ + 2 (α - α^{*}) (β - β^{*}) 〈 A (P_{V}), B (P_{W}) 〉 \end{matrix}

By the Cauchy–Schwarz inequality, we have

| 〈 A (P_{V}), B (P_{W}) 〉 | \leq ∥ A (P_{V}) ∥ ∥ B (P_{W}) ∥

(14)

Thence, it can be deduced that

〈 R^{*} (k + 1), R^{*} (k + 1) 〉 \leq 〈 R (k + 1), R (k + 1) 〉

which completes the proof.□

With an approach entirely similar to that exploited in the proof of the previous theorem, we may establish the following theorem.

Theorem 2.8 Suppose that (6) has a unique Hermitian reflexive solution pair. If the coefficient matrix of the linear system (13) is nonsingular, then

∥ R^{*} (k + 1) ∥ \leq ∥ R (k + 1) ∥

where

\begin{matrix} R (k + 1) = R (k) - \tilde{α} A ({\tilde{P}}_{V}) - \tilde{β} B (P_{W}), \\ R^{*} (k + 1) = R (k) - {\tilde{α}}^{*} A ({\tilde{P}}_{V}) - {\tilde{β}}^{*} B ({\tilde{P}}_{W}) \end{matrix}

in which $\tilde{α}$ and $\tilde{β}$ are arbitrary real parameters.

Remark 2.9. In the hypothesis of Theorem 2.7 (Theorem 2.8), it has been assumed that the coefficient matrix of the linear system (12) ((13)) is nonsingular. We would like to comment here that if the coefficient matrix is singular, then from the proof of Proposition 2.2, it is not difficult to see that

P_{V} = P_{W} = 0 ({\tilde{P}}_{V} = {\tilde{P}}_{W} = 0)

(15)

In view of Proposition 2.5 (Proposition 2.6), the above relation implies that $R (k) = 0$ which shows that the solution pair $[V (k), W (k)]$ (the current approximation) is the exact (Hermitian) reflexive solution of (1).

Now, we may summarize our elaborated results for solving Problems 1 and 2 in the next two algorithms.

Algorithm 3. Proposed approach for solving Problem 1.

Choose a tolerance $ϵ$ and an arbitrary initial Gauss pair $[V (1), W (1)]$ such that $V (1) \in C_{r}^{n \times n} (P)$ and $W (1) \in C_{r}^{n \times n} (Q)$ .

Set $k = 1$ . Calculate

R (1) = C - A (V (1)) - B (W (1))

Compute

\begin{matrix} P_{V} = \frac{A^{*} (R (k)) + P A^{*} (R (k)) P}{2}, \\ P_{W} = \frac{B^{*} (R (k)) + Q B^{*} (R (k)) Q}{2} \end{matrix}

Determine $α^{*}$ and $β^{*}$ as the solution to the linear system (12) and set

\begin{matrix} V (k + 1) = V (k) + α^{*} P_{V} W (k + 1) = W (k) + β^{*} P_{W} \end{matrix}

Calculate

R (k + 1) = C - A (V (k + 1)) - B (W (k + 1))

If $∥ R (k + 1) ∥ < ϵ$ Stop; else set $k = k + 1$ and go to Step 3.

Algorithm 4. The proposed approach for solving Problem 2.

Choose a tolerance $ϵ$ and an arbitrary initial Gauss pair $[V (1), W (1)]$ such that $V (1) \in {HC}_{r}^{n \times n} (P)$ and $W (1) \in {HC}_{r}^{n \times n} (Q)$ .

Set $k = 1$ . Calculate

R (1) = C - A (V (1)) - B (W (1))

Compute

{\tilde{P}}_{V} = \frac{A^{*} (R (k)) + P A^{*} (R (k)) P + {(A^{*} (R (k)) + P A^{*} (R (k)) P)}^{H}}{4}

{\tilde{P}}_{W} = \frac{B^{*} (R (k)) + Q B^{*} (R (k)) Q + {(B^{*} (R (k)) + Q B^{*} (R (k)) Q)}^{H}}{4}

Determine ${\tilde{α}}^{*}$ and ${\tilde{β}}^{*}$ as the solution to the linear system (13) and set

\begin{matrix} V (k + 1) = V (k) + {\tilde{α}}^{*} {\tilde{P}}_{V} W (k + 1) = W (k) + {\tilde{β}}^{*} {\tilde{P}}_{W} \end{matrix}

Calculate

R (k + 1) = C - A (V (k + 1)) - B (W (k + 1))

If $∥ R (k + 1) ∥ < ϵ$ Stop; else set $k = k + 1$ and go to Step 3.

The following proposition demonstrates that Algorithm 3 converges to the solution of Problem 1 for an arbitrary given initial guess $[V (1), W (1)]$ where $V (1) \in C_{r}^{n \times n} (P)$ and $W (1) \in C_{r}^{n \times n} (Q)$ . For simplicity we may set $V (1) = W (1) = 0$ .

Proposition 2.10. Suppose that (6) has a unique reflexive solution pair. Then, Algorithm 3 converges to the solution of (6).

Proof. Assume that $k$ steps of Algorithm 3 have been performed. We point out that $V (k) \in S_{V}$ and $W (k) \in S_{W}$ for $α = β = 0$ , hence Theorem 2.7 implies that $∥ R (k + 1) ∥ \leq ∥ R (k) ∥$ . Consequently, at each step of Algorithm 3, the norm of the residual decreases which shows that the sequence $∥ R (k) ∥$ converges as $k \to \infty$ . Now, it is sufficient to prove that $∥ R (k) ∥ \to 0$ as $k \to \infty$ . Assume, to the contrary, that $∥ R (k) ∥ \to ℓ$ as $k \to \infty$ and $ℓ \neq 0$ . Hence $[P_{V}, P_{W}] \neq [0, 0]$ , and straightforward computations show that

〈 R (k + 1), R (k + 1) 〉 = 〈 R (k), R (k) 〉 - y^{T} U^{- 1} y

(16)

where $y^{T} = (〈 R (k), A (P_{V}) 〉, 〈 R (k), B (P_{W}) 〉)$ and

U = (\begin{matrix} 〈 A (P_{V}), A (P_{V}) 〉 & 〈 B (P_{W}), A (P_{V}) 〉 \\ 〈 A (P_{V}), B (P_{W}) 〉 & 〈 B (P_{W}), B (P_{W}) 〉 \end{matrix})

From (16), it is deduced that if $k$ eventually becomes large, we have $y^{T} U^{- 1} y = 0$ . On the other hand, from the proof of Proposition 2.2, it is revealed that $U$ and consequently $U^{- 1}$ are symmetric positive-definite matrices. As a result, $y^{T} U^{- 1} y = 0$ implies that $y = 0$ and hence

〈 R (k), A (P_{V}) 〉 = 0, 〈 R (k), B (P_{W}) 〉 = 0

In view of Proposition 2.3, the above relations demonstrate that $P_{V} = P_{W} = 0$ . Now, Proposition 2.5 implies that $R (k) = 0$ which is contrary to the assumption that $∥ R (k) ∥ \neq 0$ .□

The following proposition shows that for an arbitrary given initial guess $[V (1), W (1)]$ where $V (1) \in {HC}_{r}^{n \times n} (P)$ and $W (1) \in {HC}_{r}^{n \times n} (Q)$ (for simplicity we may set $V (1) = W (1) = 0$ ), Algorithm 4 converges to the solution of Problem 2. The proposition can be established with a strategy similar to that exploited in the proof of the previous proposition.

Proposition 2.11. Suppose that (6) has a unique Hermitian reflexive solution pair. Then, Algorithm 4 converges to the solution of (6).

In Dehghan and Hajarian (2013), the following coupled matrix equations have been considered:

A_{j} X + X B_{j} + C_{j} X^{T} + X^{T} D_{j} = E_{j}, j = 1, 2, \dots, l

(17)

where $A_{j}, B_{j}, C_{j}, D_{j}, E_{j}$ are given $n \times n$ real matrices. The authors have assumed that the above coupled matrix equations have a unique solution. The following iterative algorithm has been proposed:

\begin{matrix} X (k + 1) = X (k) + \frac{β}{8} {\sum_{i = 1}^{l} A_{i}^{T} R_{i} (k) + R_{i} (k) B_{i}^{T} \\ + R_{i} {(k)}^{T} C_{i} + D_{i} R_{i} {(k)}^{T}} + \frac{β α}{2} {\sum_{i = 1}^{l} R_{i} (k) + R_{i} {(k)}^{T}} \end{matrix}

where $R_{i} (k) = E_{i} - (A_{j} X (k) + X (k) B_{j} + C_{j} X {(k)}^{T} + X {(k)}^{T} D_{j}) .$ Dehghan and Hajarian (2013) have established a sufficient condition for fixed parameters $α$ and $β$ under which their proposed algorithm converges to the exact unique solution of (17). However, the established condition, in general, is not easy to check and the optimum values for $α$ and $β$ have not been obtained. We would like to point out here that the idea of the two-dimensional projection technique employed in this work can be used for improving the speed of convergence of Dehghan and Hajarian’s algorithm.

Application of a one-dimensional projection technique

This subsection is devoted to investigating the effect of a one-dimensional projection technique on improving the speed of convergence of the gradient-based algorithms proposed in the literature. For the basic concepts of one-dimensional projection techniques for solving linear system $Ax = b$ , we refer the reader to Chapter 5 of Saad (2003).

Recently, Zhou et al. (2009a,b, 2010) have examined the gradient-based algorithm for solving different kinds of (coupled) matrix equations. The propounded algorithms depend on a fixed parameter represented by $μ$ . In each of the works by Zhou et al. referred to earlier, first it has been assumed that the mentioned problem has a unique solution. Then necessary and sufficient conditions for the parameter $μ$ have been given under which the presented algorithm is convergent. In addition, the optimum value for the fixed parameter $μ$ has been derived in these works.

More recently, Salkuyeh and Beik (2013) have considered the following general coupled Sylvester matrix equations:

\sum_{j = 1}^{q} A_{ij} X_{j} B_{ij} = C_{i}, i = 1, \dots, p

(18)

where $A_{ij} \in ℝ^{r_{i} \times n_{j}}, B_{ij} \in ℝ^{m_{j} \times k_{i}}$ and $C_{i} \in R^{r_{i} \times k_{i}}$ are given matrices and $X_{j} \in ℝ^{n_{j} \times m_{j}}$ are the unknown matrices for $j = 1, \dots, q$ . The authors have shown that the restriction of the existence of a unique solution can be ignored when the convergence of the gradient-based algorithm is studied for solving (18). A necessary and sufficient condition for the fixed parameter $μ$ of the algorithm is established under which the algorithm converges to a solution of (18). Also, the optimum value for the fixed parameter $μ$ has been determined. However, it may become too expensive to compute the optimum value of $μ$ in general situations.

So far, whenever the convergence of the gradient-based algorithm is investigated to solve different kinds of (coupled) matrix equations over reflexive matrices, it has been assumed that the considered (coupled) matrix equations have a unique reflexive solution (group). Moreover, in this type of the works, just a sufficient condition for the fixed parameter is proved under which the algorithm is convergent, and its optimum value has not been derived; for further details see Beik et al. (2014), Dehghan and Hajarian (2011) and Hajarian (2013) and the references therein.

Note that every $n \times m$ matrix is a reflexive matrix with respect to the identity matrix. In what follows we consider (18) over complex number field. Now let us consider the following problem.

Problem 3. Presume that (18) is consistent over reflexive matrices. Find a group of matrices $(X_{1}^{*}, X_{2}^{*}, \dots, X_{q}^{*})$ which satisfies (18), such that $X_{j}^{*} = P_{j} X_{j} Q_{j}$ , where $P_{j} \in HO C^{n_{j} \times n_{j}}$ and $Q_{j} \in ℍ O ℂ^{m_{j} \times m_{j}}$ are given reflection matrices.

In this subsection we consider an alternative inner product and its corresponding induced norm. Suppose that $Y = (Y_{1}, Y_{2}, \dots, Y_{q})$ and $Z = (Z_{1}, Z_{2}, \dots, Z_{q})$ where $Y_{j}, Z_{j} \in C^{n_{j} \times m_{j}}$ for $j = 1, 2, \dots, q$ . In a natural way the inner product $〈 Y, Z 〉$ is defined as follows:

〈 Y, Z 〉 = Re [\sum_{j = 1}^{q} tr (Y_{j}^{H} Z_{j})]

(19)

Therefore, the norm of $Y = (Y_{1}, Y_{2}, \dots, Y_{q})$ , $Y_{j} \in C^{n_{j} \times m_{j}}$ for $j \in I [1, q]$ , is defined by

‖ Y ‖^{2} = Re [\sum_{j = 1}^{q} tr (Y_{j}^{H} Y_{j})]

See Wu et al. (2011c) for more details.

As known, the gradient-based algorithm for finding the solution of Problem 3 can be considered in the following form (Beik et al., 2014):

X (k + 1) = X (k) + μ P_{r}, k = 1, 2, \dots,

(20)

where $X (k) = (X_{1} (k), X_{2} (k), \dots, X_{q} (k))$ , $P_{r} = (P_{r, 1}, P_{r, 2}, \dots, P_{r, q})$ , $X_{j} (k), P_{r, j} \in C^{n_{j} \times m_{j}}$ for $j = 1, 2, \dots, q$ and

\begin{array}{l} P_{r, j} = \frac{1}{2} (\sum_{i = 1}^{p} A_{i j}^{H} R_{i} (k) B_{i j}^{H} + P_{j} (\sum_{i = 1}^{p} A_{i j}^{H} R_{i} (k) B_{i j}^{H}) Q_{j}), \\ j = 1, 2, \dots, q \end{array}

in which

R_{i} (k) = C_{i} - \sum_{j = 1}^{q} A_{ij} X_{j} (k) B_{ij}, i = 1, 2, \dots, p

and $X (1) = (X_{1} (1), X_{2} (1), \dots, X_{q} (1))$ is an arbitrary given initial guess such that $X_{j} (1) = P_{j} X_{j} (1) Q_{j}$ for given reflection matrices $P_{j} \in HO C^{n_{j} \times n_{j}}$ and $Q_{j} \in HO C^{m_{j} \times m_{j}}$ . From (20), we may easily conclude that $X_{j} (k) = P_{j} X_{j} (k) Q_{j}$ and $P_{r, j} = P_{j} P_{r, j} Q_{j}$ for $j = 1, 2, \dots, q$ . To obtain the least-norm solution, without loss of generality, we may simply set $X (1) = 0$ ; see Salkuyeh and Beik (2013).

For simplicity, we define the linear operators $A (.)$ and $A^{*} (.)$ as follows:

\begin{matrix} A : C^{n_{1} \times n_{1}} \times \dots \times C^{n_{q} \times n_{q}} \mapsto C^{r_{1} \times k_{1}} \times \dots \times C^{r_{p} \times k_{p}}, \\ X \mapsto A (X) : = (A_{1} (X), \dots, A_{p} (X)) \end{matrix}

and

\begin{matrix} A^{*} : C^{r_{1} \times k_{1}} \times \dots \times C^{r_{p} \times k_{p}} \to C^{n_{1} \times n_{1}} \times \dots \times C^{n_{q} \times n_{q}}, \\ Y \mapsto A^{*} (Y) : = (A_{1}^{*} (Y), \dots, A_{q}^{*} (Y)) \end{matrix}

where $X = (X_{1}, X_{2}, \dots, X_{q})$ , $Y = (Y_{1}, Y_{2}, \dots, Y_{p})$ , $A_{i} (X) = \sum_{j = 1}^{q} A_{ij} X_{j} B_{ij}$ and $A_{j}^{*} (Y) = \sum_{i = 1}^{p} A_{i j}^{H} Y_{i} B_{i j}^{H}$ for $i = 1, 2, \dots, p$ and $j = 1, 2, \dots, q$ . It is not onerous to verify that

〈 A (X), Y 〉 = 〈 X, A^{*} (Y) 〉

Now let us rewrite (18) as follows:

A (X) = C

(21)

where $C = (C_{1}, C_{2}, \dots C_{p})$ .

In order to improve the convergence of the gradient-based algorithm, we may apply the idea utilized in Section 2.1 to derive the parameter $μ$ in a progressive way. That is, in (20), we determine $μ$ such that $R (k + 1) = C - A (X (k + 1))$ is minimized over $S_{k + 1} = {X | X (k) + \bar{α} P_{r} for \bar{α} > 0}$ . Evidently, $S_{k + 1}$ includes the $(k + 1)$ th approximate solution obtained by the gradient-based algorithm for $\bar{α} = μ$ . As a matter of fact, we aim to find $α$ such that the corresponding residual matrix to $X (k + 1) = X (k) + α P_{r}$ satisfies the following optimality property:

∥ R (k + 1) ∥ = min_{S_{k + 1}} ∥ \bar{R} (k + 1) ∥

(22)

where $\bar{R} (k + 1) = C - A (\bar{X} (k + 1)) = R (k) - \bar{α} A (P_{r})$ . In a way similar to that utilized in the proof of Theorem 2.7, we may verify that it is sufficient to obtain $α$ such that

〈 R (k + 1), A (P_{r}) 〉 = 0

(23)

or equivalently,

α = \frac{〈 R (k), A (P_{r}) 〉}{〈 A (P_{r}), A (P_{r}) 〉}, 〈 A (P_{r}), A (P_{r}) 〉 \neq 0

The next two propositions can be established in the same manner as that used in the proofs of Propositions 2.3 and 2.5, respectively.

Proposition 2.12. Presume that $P_{r}$ is defined as before. Then

〈 R (k), A (P_{r}) 〉 = 〈 P_{r}, P_{r} 〉

Note that in the following proposition we do not restrict ourself to the case where Problem 3 has a unique solution. It is just assumed that (21) is consistent over reflexive matrices.

Proposition 2.13. Suppose that the coupled matrix equations (21) are consistent over reflexive matrices. Assume that $\hat{X} = ({\hat{X}}_{1}, {\hat{X}}_{2}, \dots, {\hat{X}}_{q})$ is an arbitrary reflexive solution group of (21) such that $\hat{X} = ({\hat{X}}_{1}, {\hat{X}}_{2}, \dots, {\hat{X}}_{q})$ where $P_{j} \in HO C^{n_{j} \times n_{j}}$ and $Q_{j} \in HO C^{m_{j} \times m_{j}}$ are given reflection matrices. Then

〈 P_{r}, \hat{X} - X (k) 〉 = 〈 R (k), R (k) 〉

Remark 2.14. Note that Proposition 2.13 reveals that if $P_{r} = 0$ then $R (k) = 0$ , in other words, $X (k)$ satisfies (21). On the other hand, Proposition 2.12 shows that if $A (P_{r}) = 0$ then $P_{r} = 0$ . Therefore without loss of generality we may assume that $〈 A (P_{r}), A (P_{r}) 〉 \neq 0$ . Because if

〈 A (P_{r}), A (P_{r}) 〉 = 0

then $A (P_{r}) = 0$ , hence Proposition 2.12 implies that $P_{r} = 0$ . Thus, as a result of Proposition 2.13, $X (k)$ is the exact solution of Problem 3 and we do not need to continue the algorithm.

Consequently, we may present the modified version of the gradient-based algorithm for solving Problem 3 by introducing the following recursive formula:

X (k + 1) = X (k) + \frac{〈 R (k), A (P_{r}) 〉}{〈 A (P_{r}), A (P_{r}) 〉} P_{r}, k = 1, 2, \dots

(24)

where $X (1)$ is a given reflexive group of matrices, and for finding the least-norm solution of Problem 3, we may simply set $X (1) = 0$ . Note that we stop computing the approximate solutions using (24) when $∥ R (k) ∥ < ϵ$ where $ϵ$ is a given positive tolerance and $R (k) = C - A (X (k))$ .

Now we present the following proposition which shows that the recursive formula (20) converges to a solution of Problem 3 for an arbitrary given initial guess $X (1)$ where $X (1) = (X_{1} (1), X_{2} (1), \dots, X_{q} (1))$ and $X_{j} (1) = P_{j} X_{j} (1) Q_{j}$ for $j = 1, 2, \dots, q$ .

Proposition 2.15. Suppose that (21) is consistent over reflexive matrices. Then, Algorithm 3 converges to a solution of Problem 3.

Proof. Consider the recursive formula (24). By Remark 2.14, without loss of generality, we may assume that

〈 A (P_{r}), A (P_{r}) 〉 \neq 0

(25)

By (23) and some straightforward computations, we conclude that the residual matrices corresponding to the $k$ th and $(k + 1)$ th approximate solutions satisfy

\begin{matrix} 〈 R (k + 1), R (k + 1) 〉 = 〈 R (k), R (k + 1) 〉 \\ = 〈 R (k), R (k) 〉 - \frac{〈 R (k), A (P_{r}) 〉}{〈 A (P_{r}), A (P_{r}) 〉} 〈 R (k), A (P_{r}) 〉 \\ = 〈 R (k), R (k) 〉 [1 - \frac{{〈 R (k), A (P_{r}) 〉}^{2}}{〈 A (P_{r}), A (P_{r}) 〉 〈 R (k), R (k) 〉}] \end{matrix}

From the Cauchy–Schwarz inequality, we deduce that

1 - \frac{{〈 R (k), A (P_{r}) 〉}^{2}}{〈 A (P_{r}), A (P_{r}) 〉 〈 R (k), R (k) 〉} \leq 1

By (25), we conclude that the above inequality holds strictly. Thence we deduce that

∥ R (k + 1) ∥ < ∥ R (k) ∥

Consequently, $∥ R (k) ∥ \to 0$ as $k \to \infty$ , in other words, the recursive formula (24) converges to a solution of Problem 3.□

Numerical experiments

In this section, we present two numerical examples to demonstrate the efficiency of the proposed algorithms and the validity of our theoretical results. All the of the numeric experiments are performed using Mathematica 6 with a machine unit round-off precision of around $10^{- 16}$ . We examine our proposed methods for solving two examples given in Ramadan et al. (2013). The numerical results reported in Tables 1 and 2 and Figures 1 and 2 reveal that our offered approaches for solving Problems 1 and 2 surpass Algorithms 1 and 2 handled by Ramadan et al. (2013). As the following examples are testing samples, the exact solutions are available in advance. Hence, we may utilize the following stopping criterion in all of the numerical results:

Table 1.

Numerical comparison results for Example 3.1.

Approach	$f (k)$	CPU time (s)	Iteration
Algorithm 1 (Ramadan et al., 2013)	$0.999648 \times 10^{- 5}$	3.744	11,702
Algorithm 2 (Proposed method)	$0.994972 \times 10^{- 5}$	0.062	98

Table 2.

Numerical comparison results for Example 3.2.

Approach	$f (k)$	CPU time (s)	Iteration
Algorithm 2 (Ramadan et al., 2013)	$0.992351 \times 10^{- 5}$	0.327	876
Algorithm 4 (Proposed method )	$0.892386 \times 10^{- 5}$	0.046	66

Figure 1.

The iterative steps $k$ versus the relative error $f (k)$ for Example 3.1.

Figure 2.

The iterative steps $k$ versus the relative error $f (k)$ for Example 3.2.

f (k) = \sqrt{\frac{{‖ V (k) - V^{*} ‖}^{2} + {‖ W (k) - W^{*} ‖}^{2}}{{‖ V^{*} ‖}^{2} + {‖ W^{*} ‖}^{2}}} < 10^{- 5}

where $[V^{*}, W^{*}]$ is the unique solution pair of Problem 1 (Problem 2) corresponding to the first (second) example.

Example 3.1. (Ramadan et al., 2013) In this example we aim to compare the performance of our proposed method with Algorithm 1 for solving Problem 1. Consider the matrix equation (1) such that

\begin{matrix} A_{1} = (\begin{matrix} 1 + 2 i & 3 - i \\ - 5 + 2 i & 4 + 2 i \\ 3 + 4 i & 2 + 6 i \end{matrix}), A_{2} = (\begin{matrix} 3 - 7 i & 4 + 5 i \\ 2 + 3 i & - 3 + 9 i \\ 8 + i & 2 - i \end{matrix}) \\ E_{1} = (\begin{matrix} - 2 + 4 i & 5 + 6 i \\ 1 + 3 i & - 4 + 2 i \\ 2 - i & 1 + 3 i \end{matrix}), E_{2} = (\begin{matrix} - 5 + 3 i & 1 - 2 i \\ 2 + i & 4 + 7 i \\ 3 + 5 i & 7 + 2 i \end{matrix}) \\ B_{1} = (\begin{matrix} 3 + i & 5 \\ 4 & 0 \\ i & 3 i \end{matrix}), B_{2} = (\begin{matrix} 11 + 3 i & 7 - i \\ 3 + i & 6 \\ 0 & 0 \end{matrix}), \\ F_{1} = (\begin{matrix} 3 & 1 + i \\ 5 + 2 i & 7 \end{matrix}) \\ F_{2} = (\begin{matrix} 4 + i & 7 - i \\ 0 & 4 \end{matrix}), P = Q = (\begin{matrix} - 1 & 0 \\ 0 & 1 \end{matrix}) \in HO C^{2 \times 2} \end{matrix}

The right-hand side $C$ is chosen such that $[V^{*}, W^{*}]$ is the unique solution of (1) where

\begin{array}{l} V^{*} = (\begin{matrix} 2 i & 0 \\ 0 & 3 i \end{matrix}) \in C_{r}^{2 \times 2} (P), \\ W^{*} = (\begin{matrix} 1 + 3 i & 0 \\ 0 & 5 - i \end{matrix}) \in C_{r}^{2 \times 2} (Q) \end{array}

It can be seen that Algorithm 1 is convergent when $0 < μ < 0.9 \times 10^{- 4}$ ; for more details see Ramadan et al. (2013). The authors of the work referred to pointed out that the largest values of $μ$ speed up the convergence. Hence, applying $μ = 0.9 \times 10^{- 4}$ in Algorithm 1 supplies the superior convergence speed and we have set $μ = 0.9 \times 10^{- 4}$ in Algorithm 1. In Table 1, we compare the application of Algorithm 1 to that of Algorithm 3 for solving Problem 1. The reported results certify our established theoretical results and demonstrate the better performance of Algorithm 3. For further detail, the numerical results for Algorithms 1 and 3 are depicted in Figure 1.

Example 3.2 (Ramadan et al., 2013) In the current example our object is to combine the application of our proposed method with Algorithm 2 for solving Problem 2. Consider the matrix equation (1) such that

\begin{matrix} A_{1} = (\begin{matrix} 0.9268 & 0.3739 + 0.0099 i \\ - 0.3044 - 0.9326 i & 0 \\ 0 & 0.3721 i \end{matrix}), \\ A_{2} = (\begin{matrix} 1.4715 + 0.0228 i & 0.0202 \\ 0.7701 & 0.0286 + 0.7924 i \\ - 0.0035 i & 0.0297 i \end{matrix}) \\ E_{1} = (\begin{matrix} 2.0019 - 1.7423 i & - 3.5410 \\ - 3.6451 i & 0 \\ 1.3327 & 1.4006 - 3.6504 i \end{matrix}), \\ E_{2} = (\begin{matrix} 1.3327 & 1.1076 - 1.6036 i \\ 0 & 0 \\ - 0.3614 & 0.7727 i \end{matrix}) \\ B_{1} = (\begin{matrix} 0.1834 + 0.5337 i & 1.3937 i \\ 0.9278 & 0.3044 + 1.9388 i \\ 0.3151 i & 0 \end{matrix}), \\ B_{2} = (\begin{matrix} 0 & 0 \\ 1.4759 i & 1.5313 + 1.3327 i \\ 0.4273 + 1.0075 i & 2.5833 \end{matrix}) \\ F_{1} = (\begin{matrix} 1.2038 - 1.1174 i & 0.7727 i \\ 0.6687 & 0.3514 i \end{matrix}), \\ F_{2} = (\begin{matrix} 0.1542 & 0.3044 i \\ 0 & 0.3271 + 0.0036 i \end{matrix}) \\ P = Q = (\begin{matrix} - 1 & 0 \\ 0 & 1 \end{matrix}) \in HO C^{2 \times 2} \end{matrix}

The matrix $C$ is defined such that $[V^{*}, W^{*}]$ is the unique solution pair of (1) where

\begin{array}{l} V^{*} = (\begin{matrix} 1.9722 & 0 \\ 0 & 1.8344 \end{matrix}) \in H C_{r}^{2 \times 2} (P), \\ W^{*} = (\begin{matrix} 1.3327 & 0 \\ 0 & 2.7549 \end{matrix}) \in H C_{r}^{2 \times 2} (Q) \end{array}

For Example 3.2, Algorithm 2 is convergent if $0 < μ < 0.0115$ . In Ramadan et al. (2013), the best speed of convergence has been reached when $μ = 0.0115$ . Hence, we have run Algorithm 2 with $μ = 0.0115$ . In Table 2, we compare the execution of Algorithm 2 with Algorithm 4. The associated results are also displayed in Figure 2. As seen, the illustrated experimental results demonstrate the validity of our presented theoretical results.

Conclusion

We have mainly focused on the solution of two problems. The first problem (Problem 1) is concerned with computing the unique reflexive solution of the matrix equation $A_{1} V + A_{2} \bar{V} + B_{1} W + B_{2} \bar{W} = E_{1} V F_{1} + E_{2} \bar{V} F_{2} + C$ . The second problem (Problem 2) deals with estimating the unique Hermitian reflexive solution of the mentioned matrix equation. Recently, Ramadan et al. (2013) have suggested two gradient-based algorithms to solve Problems 1 and 2. We have offered a two-dimensional projection technique to modify the authors’ algorithms. Theoretical and numerical results have revealed that our proposed approaches outperform the presented algorithms in Ramadan et al. (2013). In addition, we have briefly discussed how using a one-dimensional projection approach can improve the speed of convergence of the gradient-based iterative algorithm for solving coupled Sylvester matrix equations over reflexive matrices, without setting the restriction that the mentioned problem must have a unique solution.

Footnotes

Acknowledgements

The author would like to express her sincere gratitude to the editor and three anonymous referees for their valuable suggestions and constructive comments which have ameliorated the quality of the paper.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

References

Beik

FPA

Salkuyeh

(2011) On the global Krylov subspace methods for solving general coupled matrix equations. Computer & Mathematics with Applications 62(12): 4605–4613.

Beik

FPA

Salkuyeh

Moghadam

(2014) Gradient-based iterative algorithm for solving the generalized coupled Sylvester-transpose and conjugate matrix equations over reflexive (anti-reflexive) matrices. Transactions of the Institute of Measurement and Control 36(1): 99–110.

Bouhamidi

Jbilou

(2008) A note on the numerical approximate solutions for generalized Sylvester matrix equations with applications. Applied Mathematics and Computation 206(2): 687–694.

Chang

Wang

(1993) The symmetric solution of the matrix equations

AX + YA = C

AX A^{T} + BY B^{T} = C

and

(A^{T} XA, B^{T} XB) = (C, D)

. Linear Algebra and its Applications 179: 171–189.

Chen

(1998) Generalized reflexive matrices: Special properties and applications. SIAM Journal on Matrix Analysis and Applications 19(1): 140–153.

Dehghan

Hajarian

(2011) Solving the generalized Sylvester matrix equation

\sum_{i = 1}^{p} A_{i} X B_{i} + \sum_{j = 1}^{q} C_{j} Y D_{j} = F

over reflexive and anti-reflexive matrices. International Journal of Control, Automation, and Systems 9(1): 118–124.

Dehghan

Hajarian

(2012) The generalised Sylvester matrix equations over the generalised bisymmetric and skew-symmetric matrices. International Journal of Systems Science 43(8): 1580–1590.

Dehghan

Hajarian

(2013) Construction of an iterative method for solving generalized coupled Sylvester matrix equations. Transactions of the Institute of Measurement and Control 35(8): 961–970.

Ding

Chen

(2005a) Hierarchical identification of lifted state-space models for general dual-rate systems. IEEE Transactions on Circuits and Systems I: Regular Papers 52(6): 1179–1187.

10.

Ding

Chen

(2005b) Hierarchical gradient-based identification of multivariable discrete-time systems. Automatica 41(2): 315–325.

11.

Ding

Chen

(2005c) Hierarchical least squares identification methods for multivariable systems. IEEE Transactions on Automatic Control 50(3): 397–402.

12.

Ding

Chen

(2005d) Gradient based iterative algorithms for solving a class of matrix equations. IEEE Transactions on Automatic Control 50(8): 1216–1221.

13.

Ding

Chen

(2005e) Iterative least squares solutions of coupled Sylvester matrix equations. Systems & Control Letters 54(2): 95–107.

14.

Ding

Chen

(2006) On iterative solutions of general coupled matrix equations. SIAM Journal on Control and Optimization 44(6): 2269–2284.

15.

Ding

Liu

Ding

(2008) Iterative solutions of the generalized Sylvester matrix equations by using the hierarchical identification principle. Applied Mathematics and Computation 197(1): 41–50.

16.

Ding

Liu

. (2011) Hierarchical least squares identification for linear SISO systems with dual-rate sampled-data. IEEE Transactions on Automatic Control 56(11): 2677–2683.

17.

Ding

Liu

Ding

(2010) Iterative solutions to matrix equations of the form

A_{i} X B_{i} = F_{i}

. Computer & Mathematics with Applications 59(11): 3500–3507.

18.

Hajarian

(2013) A gradient-based iterative algorithm for generalized coupled Sylvester matrix equations over generalized centro-symmetric matrices. Transactions of the Institute of Measurement and Control. Epub ahead of print 13August2013. DOI: 10.1177/0142331213497499.

19.

Hajarian

Dehghan

(2013) The reflexive and Hermitian reflexive solutions of the generalized Sylvester-conjugate matrix equation. Bulletin of the Belgian Mathematical Society – Simon Stevin 20(4): 577–653.

20.

Han

Xie

Ding

. (2010) Hierarchical least squares based iterative identification for multivariable systems with moving average noises. Mathematical and Computer Modelling 51(9–10): 1213–1220.

21.

Huang

Ying

Gua

(2008) An iterative method for skew-symmetric solution and the optimal approximate solution of the matrix equation

AXB = C

. Journal of Computational and Applied Mathematics 212(2): 231–244.

22.

Jbilou

Riquet

(2006) Projection methods for large Lyapunov matrix equations. Linear Algebra and its Applications 415(2–3): 344–358.

23.

Jiang

Wei

(2003) On solutions of the matrix equations

X - AXB = C

and

X - A \bar{X} B = C

.Linear Algebra and its Applications 367: 225–233.

24.

Zhang

(2008) The generalized anti-reflexive solutions for a class of matrix equations

(BX = C, XD = E)

. Computer & Mathematics with Applications 27(1): 31–46.

25.

Liu

Ding

Shi

(2012) Least squares estimation for a class of non-uniformly sampled systems based on the hierarchical identification principle. Circuits, Systems and Signal Processing 31(6): 1985–2000.

26.

Wang

Zhou

. (2010) Least squares solution with the minimum-norm to general matrix equations via iteration. Applied Mathematics and Computation 215(10): 3547–3562.

27.

Ramadan

El-Danaf

Bayoumi

AME

(2013) Two iterative algorithms for the reflexive and Hermitian reflexive solutions of the generalized Sylvester matrix equation. Journal of Vibration and Control. Epub ahead of print 8 July 2013. DOI: 10.1177/1077546312474859.

28.

Ramadan

El-Danaf

Bayoumi

AME

(2014) A relaxed gradient based algorithm for solving extended Sylvester-conjugate matrix equations. Asian Journal of Control 16(5): 1–8.

29.

Saad

(2003) Iterative Methods for Sparse Linear Systems. Philadelphia, PA: SIAM.

30.

Salkuyeh

Beik

FPA

(2013) On the gradient-based algorithm for solving the general coupled matrix equations. Transactions of the Institute of Measurement and Control. Epub ahead of print 18September2013. DOI: 10.1177/0142331213504429.

31.

Song

Chen

Zhao

(2011) Iterative solutions to coupled Sylvester-transpose matrix equations. Applied Mathematical Modelling 35(10): 4675–4683.

32.

Wang

Liao

(2012) The optimal convergence factor of the gradient based iterative algorithm for linear matrix equations. Filomat 26(3): 607–613.

33.

Feng

Liu

. (2011a) The complete solution to the Sylvester-polynomial-conjugate matrix equations. Mathematical and Computer Modelling 53(9–10): 2044–2056.

34.

Feng

Duan

. (2010) Iterative solutions to coupled Sylvester–conjugate matrix equations. Computers & Mathematics with Applications 60(1): 54–66.

35.

Duan

(2011b) Iterative algorithms for solving a class of complex conjugate and transpose matrix equations. Applied Mathematics and Computation 217(21): 8343–8353.

36.

Hou

(2011c) Finite iterative algorithms for extended Sylvester-conjugate matrix equations. Mathematical and Computer Modelling 54(9–10): 2363–2384.

37.

Zhang

Liu

(2012) On closed-form solutions to the generalized Sylvester-conjugate matrix equation. Applied Mathematics and Computation 218(19): 9730–9741.

38.

Xie

Ding

(2009) Gradient based iterative solutions for general linear matrix equations. Computer & Mathematics with Applications 58(7): 1441–1448.

39.

Xie

Liu

Yang

(2010) Gradient based and least squares based iterative algorithms for matrix equations

AXB + C X^{T} D = F

. Applied Mathematics and Computation 217(5): 2191–2199.

40.

Zhang

(2011) A note on the iterative solutions of general coupled matrix equation. Applied Mathematics and Computation 217(22): 9380–9386.

41.

Zhang

Ding

Liu

(2011) Hierarchical gradient based iterative parameter estimation algorithm for multivariable output error moving average systems. Computer & Mathematics with Applications 61(3): 672–682.

42.

Zhou

Duan

(2008) On the generalized Sylvester mapping and matrix equation. Systems & Control Letters 57(3): 200–208.

43.

Zhou

Duan

(2009a) Gradient based iterative algorithm for solving coupled matrix equations. Systems & Control Letters 58(5): 227–333.

44.

Zhou

Lam

Duan

(2010) Gradient-based maximal convergence rate iterative method for solving linear matrix equations. International Journal of Computer Mathematics 87(3): 515–527.

45.

Zhou

Duan

. (2009b) Weighted least squares solutions to general coupled Sylvester matrix equations. Journal of Computational and Applied Mathematics 224(2): 759–776.