Modeling and image quality enhancement for dynamic compressive imaging system

Abstract

Traditional single-pixel imaging systems are aimed mainly at relatively static or slowly changing targets. When there is relative motion between the imaging system and the target, sizable deviations between the measurement values and the real values can occur and result in poor image quality of the reconstructed target. To solve this problem, a novel dynamic compressive imaging system is proposed. In this system, a single-column digital micro-mirror device is used to modulate the target image, and the compressive measurement values are obtained for each column of the image. Based on analysis of the measurement values, a new recovery model of dynamic compressive imaging is given. Differing from traditional reconstruction results, the measurement values of any column of vectors in the target image can be used to reconstruct the vectors of two adjacent columns at the same time. Contingent upon characteristics of the results, a method of image quality enhancement based on an overlapping average algorithm is proposed. Simulation experiments and analysis show that the proposed dynamic compressive imaging can effectively reconstruct the target image; and that when the moving speed of the system changes within a certain range, the system reconstructs a better original image. The system overcomes the impact of dynamically changing speeds, and affords significantly better performance than traditional compressive imaging.

Keywords

Compressive sensing dynamic compressive imaging image quality enhancement overlapping average algorithm image motion

Introduction

Compressive imaging (CI),^1–3 a key sub-field of compressive sensing theory research, obtains high resolution images using only a small number of sensors. Because of this advantage, compressive sensing theory has been widely applied in medical imaging,^4,5 image processing,⁶ remote sensing imaging,⁷ wireless communication⁸ and wireless sensor networks.⁹

At present, the most typical compressive imaging system described in the research literature is a single pixel camera.^10–13 The camera employs a large-scale array digital micro-mirror device (DMD) and a single sensor to realize the compressive sampling of the scene. However, this mode requires the foreground target to be in a static state or changing only slightly during the process of compressive sampling, otherwise the reconstructed image quality is blurred or distorted. Research interest is now focusing on the motion problems associated with foreground targets and compressive imaging systems.^14–17 Tong Q. et al. reconstructed an image in the infrared (IR) rosette scanning system by analyzing the relationship between the target image and the scene in each frame.¹⁵ Jiao S. et al. solved the problem of dynamic compressive sampling using prior knowledge of the target motion type and the ultrafast structured illumination.¹⁶ Yun Li. proposed a parallel complementary compressive sensing imaging system based on a block model¹⁷ and analyzed various characteristics of dynamic compressive sampling performance.

In this paper, a dynamic compressive imaging system based on push-sweep mode for the space remote sensing is proposed. Different from existing dynamic imaging systems, our system employs a single-column DMD to obtain the compressive measurement values of a foreground image. A method of image enhancement is then proposed that combines the measurement values with characteristics of the reconstructed image. Experimental results verify the feasibility of this compressive imaging system.

The paper consists of the following parts: the next section introduces compressive sensing theory; the Dynamic Compressive imaging section describes our novel dynamic compressive imaging system and performance analysis in detail; the Image Enhancement based on overlapping average algorithm section details the method of image quality enhancement and results of the experimental data analysis; the Conclusion section summarizes the work of this paper.

Theory of compressive sensing

Consider a sparse signal $x$ whose length is N, which can be expressed as a linear combination of sparse bases $Ω \in R^{N \times N}$ :

x = Ω θ

(1)

If there are only $K (K ≪ N)$ nonzero terms in vector $θ \in R^{N}$ , the signal $θ$ is a sparse representation of $x$ in domain $Ω$ .^1,18 For sparse signal reconstruction, when we know the measurement values $y \in R^{M}$ and measurement matrix $Φ \in R^{M \times N} (M ≪ N)$ , the coefficient vector $θ$ can be estimated using the minimum $l_{1}$ -norm, that is:

\hat{θ} = \arg \min {‖ θ ‖}_{1} s . t . y = ΦΩθ Ω θ

(2)

With the coefficient vector $\hat{θ}$ , we can reconstruct the original signal $x = Ω \hat{θ}$ .

Taking into consideration the influence of errors, equation (2) can be rewritten as

\hat{θ} = \arg \min {‖ θ ‖}_{1} s . t . {‖ y- Φ Ω θ ‖}_{2} \leq ε

(3)

where:

ε

is the permissible error.^19,20

When we obtain the compressive measurement values of a static target image, all measurement values contain the same original information. However, when the foreground target is dynamic, each measurement value contains mixed information relative to the target image.

Dynamic compressive imaging

Composition of the imaging system

A schematic diagram of the dynamic compressive imaging system is shown in Figure 1. The system is based on single-column DMD, and uses push-sweep mode to capture the measurement value of the foreground image. The compressive sampling part is composed of an imaging lens, a single-column DMD, a collecting lens, and a detector. The terminal recovers the original image using a reconstruction algorithm.

Figure 1.

Schematic diagram of the dynamic compressive imaging system.

Dynamic compressive imaging recovery model

For clarity, the foreground image is $X = [x_{1}, x_{2}, \dots, x_{N}] \in R^{N \times N}$ , the measurement matrix is $Φ \in R^{M \times N} (M ≪ N)$ . We use the measurement matrix $Φ$ to obtain the measurement values $y_{i} \in R^{M}$ of the i-th column vector $x_{i} = {[\begin{matrix} x_{1, i} & \dots & x_{N, i} \end{matrix}]}^{T} \in R^{N}$ of the image. According to the theory of compressive sensing, the measurement values $y_{i}$ can be written as:

y_{i} = [\begin{matrix} y_{1, i} \\ ⋮ \\ y_{M, i} \end{matrix}] = Φ x_{i} = [\begin{matrix} ϕ_{11} & \dots & ϕ_{1 N} \\ ⋮ & ⋱ & ⋮ \\ ϕ_{M 1} & \dots & ϕ_{M N} \end{matrix}] [\begin{matrix} x_{1, i} \\ ⋮ \\ x_{N, i} \end{matrix}] = [\begin{matrix} φ_{1} \\ ⋮ \\ φ_{M} \end{matrix}] x_{i}

(4)

where:

φ_{m} = [\begin{matrix} ϕ_{m 1} & ϕ_{m 2} & \dots & ϕ_{m N} \end{matrix}] \in R^{N} (1 \leq m \leq M)

Because the system adopts push-sweep mode to obtain the measurement values, there is a certain deviation between the current measurement value and the previous one. The dynamic sampling process is shown in Figure 2. To solve the dynamic problem, we need to remodel the recovery model.

Figure 2.

The black area is the foreground image and the red area is the single-column DMD.

Assuming that the size of the DMD micro-mirror is $a \times a$ , the average speed of the foreground image moving on the DMD is $v$ , and the sampling frequency of the system is $f$ . For each sampling, the relative moving distance of the foreground image on the DMD is:

△ s = \frac{v}{f}

(5)

For convenience of expression, we define the relative motion ratio p as:

p = \frac{△ s}{a}

(6)

Assuming that the two adjacent vectors of the foreground image are $x_{1} \in R^{N}$ and $x_{2} \in R^{N}$ , when obtaining the measurement vector $y_{1} = {[\begin{matrix} y_{1, 1} & \dots & y_{M, 1} \end{matrix}]}^{T} \in R^{M}$ of the vector $x_{1}$ , according to equation (4), the first value $y_{1, 1}$ can be expressed as:

y_{1, 1} = φ_{1} \cdot x_{1}

(7)

When obtaining the M-th measurement value, due to the relative moving distance $(M - 1) \cdot △ s$ between DMD and the first column of pixels (i.e., $x_{1}$ ), the measurement value $y_{M, 1}$ should be:

\begin{matrix} y_{M, 1} = φ_{M} \cdot [1 - (M - 1) \cdot p] \cdot x_{1} + φ_{M} \cdot (M - 1) \cdot p \cdot x_{2} \\ = [\begin{matrix} 1 - (M - 1) \cdot p & (M - 1) \cdot p \end{matrix}] [\begin{matrix} φ_{M} & 0 \\ 0 & φ_{M} \end{matrix}] [\begin{matrix} x_{1} \\ x_{2} \end{matrix}] \end{matrix}

(8)

As can be seen from equation (8), the measurement value $y_{M, 1}$ contains the next column of vector $x_{2}$ information. After gathering M measurement values, we get the measurement vector:

y_{1} = [\begin{matrix} y_{1, 1} \\ ⋮ \\ y_{M, 1} \end{matrix}] = Ψ [\begin{matrix} x_{1} \\ x_{2} \end{matrix}]

(9)

where,

Ψ \in R^{M \times 2N}

Ψ = [\begin{matrix} [\begin{matrix} 1 & 0 \end{matrix}] [\begin{matrix} φ_{1} & 0 \\ 0 & φ_{1} \end{matrix}] \\ ⋮ \\ [\begin{matrix} 1-(M -1) \cdot p & (M -1) \cdot p \end{matrix}] [\begin{matrix} φ_{M} & 0 \\ 0 & φ_{M} \end{matrix}] \end{matrix}]

(10)

Therefore, it can be deduced that the measurement vector $y_{i}$ of the i-th column of the foreground image can be expressed as:

y_{i} = Ψ [\begin{matrix} x_{i} \\ x_{i + 1} \end{matrix}]

(11)

Combining equations (2) and (11), we can get a novel recovery model of the dynamic compressive imaging system:

{\begin{cases} \hat{θ} = \arg \min {‖ θ ‖}_{1} s . t . y_{i} = Ψ Ω θ \\ y_{i} = Ψ [\begin{matrix} x_{i} \\ x_{i + 1} \end{matrix}] \end{cases}

(12)

where:

Ψ

is a new measurement matrix.

It can be seen from equation (12) that the original image can be recovered by using a traditional reconstruction algorithm (such as Orthogonal Matching Pursuit),^21–23 and that only one reconstruction is needed to obtain the two adjacent vectors $x_{i}$ and $x_{i+ 1}$ of the image. If we combine all the vectors in order, we get two original images, as shown in Figure 3.

Figure 3.

The order diagram of reconstructed vectors.

Figures 4 to 6 show this result and compare it with the result of a traditional CI recovery model. From Figures 4 and 5, we can see that the image effect of the upper part of the reconstructed image which is reconstructed by dynamic CI worsens with the increase of the relative motion ratio p, while the image effect of the lower part is just the opposite. The main reason for this is that as the relative motion ratio p increases, the information in the image vector $x_{1}$ contained in the measurement value decreases, so the image effect of the upper part will degrade. However, for the lower part, the information in the vector $x_{2}$ will increase, and the image effect will improve. The image which is reconstructed via a traditional CI recovery model also degrades with increasing relative motion ratio p. Figure 6 further verifies this conclusion through the Peak Signal-to-Noise Ratio (PSNR) of the reconstructed image.

Figure 4.

The reconstructed images of Lena: (a) Original image:128 × 128. (b1) M = 100, p = 0. (b2) M = 100, p = 0.003. (b3) M = 100, p = 0.006. (b4) M = 100, p = 0.009. (c1) M = 100, p = 0. (c2) M = 100, p = 0.003. (c3) M = 100, p = 0.006. (c4) M = 100, p = 0.009.

Figure 5.

The reconstructed images of Remote: (a) Original image:256 × 256. (b1) M = 200, p = 0. (b2) M = 200, p = 0.0025. (b3) M = 200, p = 0.0035. (b4) M = 200, p = 0.0045. (c1) M = 200, p = 0. (c2) M = 200, p = 0.0025. (c3) M = 200, p = 0.0035. (c4) M = 200, p = 0.0045.

Figure 6.

Relationship between PSNR and relative motion ratio p: (a) M = 100. (b) M = 200.

In addition, it can be seen from Figure 6 that when the relative motion ratio p is small, the image quality of the reconstructed upper part is almost the same as that of the traditional CI image; but when the relative motion ratio p is large, the image quality of the reconstructed lower part is significantly better than that of the traditional CI image. Therefore, for dynamic CI, regardless of how the value of relative motion ratio p is taken, an original image of goodquality can always be reconstructed. This overcomes the influence of the dynamic change of the system’s speed and reduces the problem of the changing speed. Its performance is obviously better than that of the traditional CI.

Image enhancement based on overlapping average algorithm

To make full use of the reconstructed image information and get a higher quality image, we enhance the quality of the upper and lower parts of the reconstructed image using an overlapping average algorithm, as shown in Figure 7.

Figure 7.

Flow chart of image enhancement.

Let us suppose that the upper part $g_{1} (m, n)$ and the lower part $g_{2} (m, n)$ of the reconstructed image are:

{\begin{matrix} g_{1} (m, n) = f_{1} (m, n) + e_{1} (m, n) \\ g_{2} (m, n) = f_{2} (m, n) + e_{2} (m, n) \end{matrix}

(13)

where:

f_{i} (m, n), i = 1, 2

is the original image and

e_{i} (m, n), i = 1, 2

is random white noise. After overlapping and averaging equation (13), the output image is:

\begin{matrix} g (m, n) = \frac{1}{2} \sum_{i = 1}^{2} g_{i} (m, n) \\ = \frac{1}{2} \sum_{i = 1}^{2} f_{i} (m, n) + \frac{1}{2} \sum_{i = 1}^{2} e_{i} (m, n) \end{matrix}

(14)

According to the overlapping average algorithm, to maximize the signal-to-noise ratio (SNR) of output image, the images $f_{1} (m, n)$ and $f_{2} (m, n)$ should satisfy the following relationship:

f_{1} (m, n) = f_{2} (m, n)

(15)

From the analysis in the previous section, we know that the values of the parameters M and p in equation (8) directly affect the quality of the images g₁ðm; nþ and g₂ðm; nþ. Three scenarios are discussed in the following:

If $1 - (M - 1) \cdot p > (M - 1) \cdot p$ ; then,

p < \frac{1}{2 (M - 1)}

(16)

The proportion of the vector $x_{1}$ in the measurement value $y_{M, 1}$ is relatively large, the influence of the vector x₂ on the image g₁ðm; nþ is relatively small;

2. If $1 - (M - 1) \cdot p < (M - 1) \cdot p$ ; then,

p > \frac{1}{2 (M - 1)}

(17)

The proportion of the vector x2 in the measurement value yM;1 is relatively large, the influence of the vector x₂ on the image g₁ðm; nþ is relatively large;

3. If $1 - (M - 1) \cdot p = (M - 1) \cdot p$ ; then,

p = \frac{1}{2 (M - 1)}

(18)

The proportion of the vectors $x_{1}$ and $x_{2}$ in the measurement value $y_{M, 1}$ are the same, so so the influence is equivalent. At this point, the SNR of the output image $g (m, n)$ is greatest, and the output image can be described as:

\begin{array}{l} g (m, n) = f_{1} (m, n) + \frac{1}{2} \sum_{i = 1}^{2} e_{i} (m, n) \\ = f_{2} (m, n) + \frac{1}{2} \sum_{i = 1}^{2} e_{i} (m, n) \end{array}

(19)

To verify the image enhancement effect, the paper uses the PSNR to compare the performance of the image before and after the enhancing process. Results are shown in Figure 8. It can be seen from Figure 8 that the closer the PSNRs of the upper and lower parts of the reconstructed image are, the more obvious the enhancement effects of the output image quality will be; and conversely, the worse the enhancement effect will be. When, the parameters M and p satisfy equation (18), the PSNR of the output image is the greatest.

Figure 8.

Performance before and after image enhancement (Lena:128 × 128): (a) M = 80. (b) M = 100.

Under the premise of satisfying equation (18), we further analyze the influence of parameter M on the quality of the output image $g (m, n)$ and compare performance with the results of the traditional CI. The results are shown in Table 1. As can be seen from Table 1, with the number of measurements M increasing, the PSNR of each reconstructed image is improved, but the image quality after enhancement is clearly better than that of traditional CI.

Table 1.

Performance comparison between the image enhancement method and traditional CI.

Image	Resolution	Measure	Imaging methods	M = 70	M = 80	M = 90	M = 100
Lena	128 × 128	PSNR(dB)	Traditional CI	22.1825	20.0491	24.8143	25.4453
			The upper part	22.1893	24.0679	24.8311	25.5303
			The lower part	20.4737	21.4550	21.8954	22.1830
			Image enhancement	25.3742	26.2651	27.9289	28.8529
Image	Resolution	Measure	Imaging methods	M = 160	M = 180	M = 200	M = 220
Remote	256 × 256	PSNR(dB)	Traditional CI	18.8979	19.5876	20.5542	21.1541
			The upper part	18.8937	19.5853	20.5340	21.1844
			The lower part	17.5699	17.9898	18.4699	18.8026
			Image enhancement	20.2353	20.9546	21.8160	22.4708

Conclusion

Aiming at the problems of traditional CI, this paper proposes a dynamic imaging system based on push-sweep mode. Unlike traditional imaging systems, the system can reconstruct two adjacent column vectors of the foreground image at one time, and produce two foreground images simultaneously.

Simulation results show that when the dynamic moving speed of the system changes within a certain range, a good quality original image can always be obtained, which effectively overcomes the impact of the dynamic change of the moving speed.

To make full use of the reconstructed image information and improve the output image quality, this paper proposes an image enhancement method based on an overlapping average algorithm. Experimental results show that the method is effective.

Footnotes

Acknowledgements

I thank JoAnn Brooks, Editor, from Liwen Bianji, Edanz Editing China, for editing the English text of a draft of this manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is supported by the Key Projects of Natural Science Research in Universities in Anhui (no. KJ2017A531), the Teaching Demonstration Course of Communication Principle in Anhui (no. 2049) and Key Discipline Construction Project of Hefei University (no. 2018xk03).

References

Donoho

DL.

Compressed sensing. IEEE Trans Inform Theory 2006; 52: 1289–1306.

Haupt

Nowak

Compressive sampling for signal detection. In: IEEE international conference on acoustics. USA: IEEE, 2007.

Candes

Wakin

MB.

An introduction to compressive sampling. IEEE Signal Process Mag 2008; 25: 21–30.

Konar

Vajuvalli

Rao

, et al. Accelerated dynamic contrast enhanced MRI based on region of interest compressed sensing. Magn Reson Imaging 2019; 67: 18–23.

Huang

Wang

Zhu

Compressed sensing MRI reconstruction with multiple sparsity constraints on radial sampling. Math Probl Eng 2019; 2019: 1–14.

Shimobaba

Endo

Nishitsuji

, et al. Computational ghost imaging using deep learning. Opt Commun 2018; 413: 147–151.

Fowler

JE.

Compressive pushbroom and whiskbroom sensing for hyperspectral remote-sensing imaging. In: International conference on image processing, Paris, France, 2014, pp.684–688.

Zhu

Liu

, et al. Impulse radio UWB signal detection based on compressed sensing. Commun Netw 2013; 5: 98–102.

Lindberg

Amat

AGI

Wymeersch

Compressed sensing in wireless sensor networks without explicit position information. IEEE Trans Signal Inform Process Networks 2017; 3: 404–415.

10.

Duarte

Davenport

Takhar

, et al. Single-Pixel imaging via compressive sampling. IEEE Signal Process Mag 2008; 25: 83–91.

11.

Chan

Charan

Takhar

, et al. A single-pixel terahertz imaging system based on compressed sensing. Appl Phys Lett 2008; 93: 1–3.

12.

Sun

Edgar

Bowman

, et al. 3D computational imaging with single-pixel detectors. Science 2013; 340: 844–847.

13.

You

, et al. Spotlight SAR sparse sampling and imaging method, based on compressive sensing. Sci China (Inform Sci) 2012; 55: 1816–1829.

14.

Abolbashari

Magalhaes

Araujo

, et al. High dynamic range compressive imaging: a programmable imaging system. Opt Eng 2012; 51: 071407-1–071407-8.

15.

Tong

Jiang

Wang

, et al. Image reconstruction of dynamic infrared single-pixel imaging system. Opt Commun 2018; 410: 35–39.

16.

Jiao

Sun

Gao

, et al. Motion estimation and quality enhancement for a single image in dynamic single-pixel imaging. Opt Express 2019; 27: 12841–12854.

17.

Y-H

Wang

X-D

Wang

, et al. Modeling and image motion analysis of parallel complementary compressive sensing imaging system. Opt Commun 2018; 423: 100–110.

18.

Kabashima

Wadayama

Tanaka

, et al. A typical reconstruction limit of compressed sensing based on Lp-norm minimization. arXiv: Inform Theory 2009; 2009: L09003.

19.

Candes

Romberg

Tao

, et al. Stable signal recovery from incomplete and inaccurate measurements. Commun Pure Appl Math 2006; 59: 1207–1223.

20.

Gill

Wang

Molnar

The in-crowd algorithm for fast basis pursuit denoising. IEEE Trans Signal Process 2011; 59: 4595–4605.

21.

Liu

Song

Zhuang

, et al. Sensing signal recovery via creditability-estimation based matching pursuit. IEICE Trans Fundam 2015; 98: 1234–1243.

22.

Tropp

Gilbert

AC.

Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans Inform Theory 2007; 53: 4655–4666.

23.

Tropp

Gilbert

Strauss

, et al. Algorithms for simultaneous sparse approximation: part I: greedy pursuit. Signal Process 2006; 86: 572–588.