The Correction of Recovered Spectral Images in a Hadamard Transform Spectral Imager Based on a Digital Micro-Mirror Device

Abstract

The Hadamard transform spectral imager based on a digital micromirror device (DMD) is a novel type of spectral imager developed in recent years. This paper describes the designing scheme of the Hadamard encoding mask and analyzes the encoding process of the detector pixels. Generally the Hadamard encoding mask constructed by DMD cannot completely encode the dispersed spectrum of all the pixels according to the Hadamard matrix; therefore, the spectral images recovered by inverse Hadamard transform inevitably have errors. A correction method for the recovered spectral images is proposed. The experimental results show that this method improves the quality of the recovered spectral images.

Keywords

Hadamard transform Digital micro-mirror device (DMD)Encoding mask Spectral imager Correction method.

INTRODUCTION

The Hadamard transformation is a known type of optical multiplex technique that distinguishes itself by the enhancement of the signal-to-noise ratio (SNR) for the reconstructed signals.¹ Interest in Hadamard transform spectroscopy has been limited due to the inability to perform accurate high-order masking in a reasonable amount of time. However, this technique has progressed in the past decade with improvements in encoding mask technology.^2,–6 More recent advances in digital micro-mirror array (DMA) technology and the commercial availability of the Texas Instruments Digital Micro-mirror Device™ (DMD™) in particular have promoted the development of Hadamard transform techniques once again, and many scholars have done a lot of research work in utilizing the DMA as a spatial light modulator (SLM) for generating a stationary Hadamard encoding mask.^7,–16 The Hadamard transform spectral imager (HTSI) based on DMD does not require macroscopic moving parts to provide spectral multiplexing capability, so it can be built in a robust, small-volume, and lightweight package.¹ However, the Hadamard encoding mask generated by DMD usually cannot guarantee the dispersed spectrum of every pixel of the input image to be encoded perfectly based on the Hadamard matrix. This paper describes this phenomenon and proposes a correction method for the recovered spectral images.

INSTRUMENT CONCEPT

The principles of operation of our HTSI based on DMD are illustrated in Fig. 1. The instrument consists of two gratings and a DMD placed in the intermediate focal plane between them. The first grating disperses the image into spectral components and images them onto the DMD. The DMD spectrally encodes the image by modulating the intensity of selected spectral bands in the dispersed image. The encoded image is then spectrally recombined by the second grating and imaged onto a photodetector.

Fig. 1.

The schematic diagram of the ray path for our engineering HTSI.

Spectral encoding is achieved by transmission and rejection of particular bands. This operation is accomplished with the programmable digitally controlled DMD. The DMD is programmed to generate a dynamic series of spatial patterns that perform the transform functions. Spectral encoding of this sort can be seen as equivalent to applying a Hadamard transform to the spectrally resolved image.⁷ The encoded images acquired from the detector are decoded by the inverse Hadamard transform (IHT) to provide the spectrum for each individual pixel.

HADAMARD ENCODING MASK AND THE EXISTENT PROBLEM

Design of the Hadamard Encoding Mask. In our HTSI, the DMD is controlled by a program to generate the encoding mask according to the row vector of the cyclic S matrix. Figure 2 is a pattern of the encoding mask constructed according to the 7-order cyclic S matrix. The 7-order Hadamard encoding mask encodes circularly according to the column vectors of the 7-order cyclic S matrix in time for each DMD micro-mirror and it encodes circularly with the width of l/7 of the encoding channel in space. Therefore, the encoding mask looks like a series of bars in white and black, the white bar denoting the “1” state of the micro-mirrors reflecting the light onto the detector and the black bar denoting the “0” state of the micro-mirrors reflecting the light to the space except the detector.

Fig. 2.

Hadamard encoding mask constructed based on the 7-order cyclic S matrix.

Calculating the width of the each encoding channel is the key to constructing the encoding mask. Now we discuss how to determine the column numbers of micro-mirrors for each encoding channel.

Assuming the response range of the HTSI is λ₁ * λ₂ and the corresponding geometrical width to the dispersed spectrum on the mask of every pixel of the input image is l, to encode the dispersed spectrum of every pixel based on the Hadamard matrix, we ensure that the width of l is completely divided into seven even elements by the encoding channels. Therefore, the width of each encoding channel is designed to be l/7. Then we can calculate the column numbers of the micro-mirrors corresponding to the width l/7 for each encoding channel and construct the Hadamard encoding mask by the programmed setting of the DMD micro-mirrors.

Inaccurate Encoding. There is still an intractable problem to be solved in the encoding process of the dispersed spectrum. For example, as can be seen from Fig. 3, a and b are the dispersed spectra of two different pixels of the input images. For a, the dispersed spectrum just covers the width of seven encoding channels of the encoding mask, and obviously its encoding process follows the Hadamard transform. For b, the dispersed spectrum occupies eight encoding channels and is encoded into eight spectral elements, which is an inaccurate encoding process (Fig. 4); obviously if we recover it by IHT directly, there will be large errors in the seven obtained spectral elements.

Fig. 3.

The dispersed spectrum of two pixels in different positions.

Fig. 4.

The encoding and re-imaging process for the dispersed spectrum of a pixel encoded into 8 elements.

THE IMPROVED DESIGN OF THE HADAMARD ENCODING MASK

Cyclic Encoding of the Detector Pixels. In fact, the encoding process of the detector pixels changes circularly in space. In Fig. 5, the green solid lines are used to distinguish different encoding regions on the detector; all the pixels in the same region encode based on the same matrix, while different regions encode based on different S matrices.

Fig. 5.

The distribution of the cyclic encoding region for the detector pixels. The detector pixels in different regions encode based on different S matrices.

Six-Order Hadamard Encoding Mask. The encoding process of the dispersed spectrum in Fig. 3 can be studied by analyzing the encoding process of the detector pixels instead. We extract a row of pixels in a cyclic encoding region as samples to study their encoding process (Fig. 6). Each row of pixels in a cyclic encoding region includes 10 pixels. To ensure that the dispersed spectrum of every pixel is encoded completely into seven elements, the width of the channel of the encoding mask should be designed to be l/6, then almost all the dispersed spectrum of the pixels will be encoded into seven elements and therefore can be recovered by IHT. However, the seven encoded spectral elements are uneven.

Fig. 6.

The encoded dispersed spectrum based on 6-order encoding mask. All the dispersed spectrum are encoded into 7 spectral elements.

As can be seen from Fig. 6, although the dispersed spectrum of the pixels p₁ * p₁₀ are encoded into seven elements, the spectral elements with the same serial number have different wavelength ranges. There is spectral offset between the spectral elements with the same serial number. As a result, the spectral images recovered by IHT cannot accurately reflect the spectral information of a specified wavelength range.

If we let the HTSI operate based on a higher-order S-matrix, the spectral images with high spectral resolution can be obtained by IHT. Then we can merge several hyperspectral images with the adjacent wavelength range into one spectral image to obtain the multispectral images with smaller spectral offset. However, this approach is rough and theoretical errors still exist.

CORRECTION OF THE RECOVERED SPECTRAL IMAGES

The width of each encoding channel in our HTSI is designed to be l/7. The process of encoding a row of detector pixels in a cyclic encoding region is shown in Fig. 7. We extract the pixel p₆ in this region as a sample to study. Its dispersed spectrum is encoded into eight spectral elements. The first spectral element and the eighth spectral element always have the same encoding state (“0” or “1”). These two spectral elements will be merged into the first spectral element in spectrum recovery. Therefore, we still obtain seven recovered spectral elements by IHT for p₆. Obviously it is unreasonable and the recovered spectral images need correction.

Fig. 7.

The encoding and the decoding process of the dispersed spectrum based on the 7-order encoding mask.

Figure 8 shows the corresponding relationship between the eight actual encoded spectral elements and the ideal seven spectral elements of the pixel p₆.

Fig. 8.

The corresponding relationship between the actual dispersed spectrum encoded into 8 elements and the ideal 7 spectral elements.

If we use y₁, y₂, y₃, y₄, y₅, y₆, and y₇ to denote the actual recovered spectral elements, and x₁, x₂, x₃, x₄, x₅, x₆, and x₇ for the ideal seven spectral elements, then the following equation is established based on Fig. 8.

{\begin{array}{l} y_{1} = k x_{1} + (1 - k) x_{7} \\ y_{2} = k x_{2} + (1 - k) x_{1} \\ y_{3} = k x_{3} + (1 - k) x_{2} \\ y_{4} = k x_{4} + (1 - k) x_{3} \\ y_{5} = k x_{5} + (1 - k) x_{4} \\ y_{6} = k x_{6} + (1 - k) x_{5} \\ y_{7} = k x_{7} + (1 - k) x_{6} \end{array}

(1)

where k is a correcting coefficient of less than 1, written in matrix form:

\begin{matrix} [\begin{matrix} y_{1} \\ y_{2} \\ y_{3} \\ y_{4} \\ y_{5} \\ y_{6} \\ y_{7} \end{matrix}] & = & [\begin{matrix} k & 0 & 0 & 0 & 0 & 0 & 1 - k \\ 1 - k & k & 0 & 0 & 0 & 0 & 0 \\ 0 & 1 - k & k & 0 & 0 & 0 & 0 \\ 0 & 0 & 1 - k & k & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 - k & k & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 - k & k & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 - k & k \end{matrix}] \\ \times [\begin{matrix} x_{1} \\ x_{2} \\ x_{3} \\ x_{4} \\ x_{5} \\ x_{6} \\ x_{7} \end{matrix}] \end{matrix}

(2)

Now we discuss how to obtain the correction coefficient for the detector pixels in one of the cyclic encoding regions.

If the ten pixels p₁ * p₁₀ lie in a small region with well-proportioned hue on the panchromatic image, then they should have the same spectrum. We can take these ten pixels as the training samples to obtain the correction coefficients.

Figure 7 shows that the pixel p₁ and the pixel p₁₀ are at the edges of a cyclic encoding region. Their encoding errors are very small and they are almost encoded into seven spectral elements. Therefore, we can decode these two pixels by IHT directly to obtain its seven spectral elements with small errors. To reduce the errors further, we average the two recovered spectral vectors of these two pixels as the ideal spectrum of the ten pixels.

If we substitute the gray values of the obtained seven ideal spectral elements into the column vector [x₁ x₂ x₃ x₄ x₅ x₆ x₇]^T of Eq. 2 and substitute the gray values of the actual recovered spectral elements of p₂ * p₉ obtained by IHT into the column vector [y₁ y₂ y₃ y₄ y₅ y₆ y₇]^T, respectively, then all the correcting coefficients for p₂ * p₉ can be solved approximately using the least square method with ease.

Because the encoding process of the detector pixels changes circularly in space, to obtain the correction coefficients of all the pixels, we only need to evaluate the correction coefficients of the detector pixels in one of the cyclic encoding regions rather than one by one. The recovered spectral images can be corrected through Eq. 2 after the correction coefficients are obtained.

EXPERIMENT

First we use our HTSI to record seven encoded images of a butterfly model. The panchromatic image and the encoded images are shown in Fig. 9. From the region with the well-proportioned hue on the panchromatic images, we extract ten pixels in a cyclic encoding region as the samples to train the correction coefficients.

Fig. 9.

The panchromatic image and the seven encoded images of the butterfly model.

The encoded gray values of the ten pixels in the training region are shown in Table I. The gray values of the ten pixels recovered directly by IHT are shown in Table II.

TABLE I.

The encoded gray values of the ten pixels in the training region.

	The encoded gray values
Pixel	Based on S₁	Based on S2	Based on S3	Based on S4	Based on S5	Based on S6	Based on S7
p ₁	83	124	146	189	153	171	114
p ₂	88	125	150	187	157	167	114
p ₃	90	127	153	184	158	164	112
p ₄	95	129	158	180	160	157	108
p ₅	98	132	162	177	162	152	105
p ₆	103	134	167	173	164	146	101
p ₇	108	137	173	169	166	139	99
p ₈	115	141	181	182	169	128	91
p ₉	120	144	186	158	172	121	87
p ₁₀	122	146	191	157	175	120	85

TABLE II.

The gray values of the ten pixels recovered by IHT in the training region.

	The recovered gray values
Pixel	First spectral element	Second spectral element	Third spectral element	Fourth spectral element	Fifth spectral element	Sixth spectral element	Seventh spectral element
p ₁	8	10	12	36	53	56	70
p ₂	13	10	13	35	52	57	68
p ₃	17	10	13	33	51	57	67
p ₄	24	9	12	30	49	56	66
p ₅	30	9	12	28	48	56	65
p ₆	37	9	12	25	46	56	63
p ₇	45	8	11	22	44	55	62
p ₈	57	8	11	18	40	55	59
p ₉	64	9	13	26	43	54	64
p ₁₀	6	10	14	38	55	58	68

The ideal spectral vector is obtained by averaging the recovered spectral elements of p₁ and p₁₀ in Table II with ease. We substitute this ideal spectral vector [7 10 13 37 54 57 69]^T into the column vector [x₁ x₂ x₃ x₄ x₅ x₆ x₇]^T of Eq. 2 and substitute the recovered spectral vectors of p₂ * p₉ in Table II into the column vector [y₁ y₂ y₃ y₄ y₅ y₆ y₇]^T, respectively. The correcting coefficients of the pixel p₂ * p₉ can be solved approximately by the least squares method. They are 0.90, 0.84, 0.72, 0.63, 0.52, 0.39, 0.20, and 0.08.

Now we use these obtained coefficients to correct another group of recovered spectral images recorded by our HTSI.

In Fig. 10, M and N are two testing pixels. Their coefficients are evaluated with the above-mentioned method beforehand. The correction coefficient of M is 0.39, and the correction coefficient of N is 0.84. We extract the spectrograms of the two testing points with the spectrometer FieldSpec® HandHeld (Analytical Spectral Devices, U.S.) and show them in Fig. 10. The spectral response range of our HTSI is 500 nm * 710 nm, so the wavelength range of the seven spectral elements is 550 nm * 530 nm, 530 nm * 560 nm, 560 nm * 590 nm, 590 nm * 620 nm, 620 nm * 650 nm, 650 nm * 680 nm, and 680 nm * 710 nm, respectively. For M and N, we integrate the intensity of the seven spectral elements among their wavelength range and obtain seven discrete values. The seven discrete values are connected to form spectral curves as the benchmark and are also shown in Fig. 10.

Fig. 10.

The spectrum of the two testing points extracted by radiation meter and the spectral curves obtained after integration.

We decode the seven spectral elements of M and N by IHT directly and use the dashed line to denote the obtained spectral curves in Fig. 11. Substituting the spectral vectors obtained by IHT in the previous step and the correction coefficients of the two testing points into Eq. 2, respectively, we can obtain the corrected spectral vectors, and the corrected curves are denoted by the solid lines in Fig. 11.

Fig. 11.

The spectrograms of the two testing points before correction and after correction.

As can be seen from Fig. 11, the solid curves are more similar to the benchmark curves in Fig. 10 than the dashed curves. The first data points in the spectrogram before revision are outliers. This is because the seventh spectral elements of the two testing pixels have higher intensity than the others and contribute to the first spectral elements by IHT. However, the outliers return to the vicinity of the normal values after being corrected.

Spectral angle match (SAM) is an algorithm to identify and classify the targets by calculating and comparing the angle between the spectral vector of the target and the spectral vector of the sample. By calculating, the spectral angles between the benchmark vectors and the spectral vectors before correction of M and N are α_M = 12.8° and α_N = 21.1°; however, the spectral angles between the benchmark vectors and the spectral vectors after correction of M and N are β_M = 1.2° and β_N = 1.8°. Obviously the spectral angles between the benchmark vectors and the spectral vectors after correction are smaller than the angles between the benchmark vectors and the spectral vectors before correction. Experimental results show that the correction method proposed in this paper is effective to improve the quality of the recovered spectral images.

CONCLUSION

In DMD-based HTSI, the Hadamard encoding mask cannot guarantee the dispersed spectrum of all the pixels to be encoded perfectly based on the Hadamard matrix; therefore, the recovered spectral images need to be corrected. The mathematical model of correction is deduced and a practical correcting method for the recovered spectral images is proposed. Experimental results reveal that the quality of the recovered spectral images after correction is improved.

References

Harwit

Sloane

N.J.A.

Hadamard Transform Optics. New York: Academic Press. 1979.

Fateley

W.G.

Hammaker

R.M.

Paukstelis

J.V.

Wright

S.L.

Orr

E.A.

Mortensen

A.N.

Latas

K.J.

“Application of a two-dimensional hadamard encoding mask for the imaging of thin-layer chromatography plates by laser-induced fluorescence or surface-enhanced Raman scattering and for use with a photoacoustic detector to generate three-dimensional photoacoustic images”. Appl. Spectrosc. 1993. 47(9): 1464–1470.

Hanley

Q.S.

Jovin

T.M.

“Highly Multiplexed Optically Sectioned Spectroscopic Imaging in a Programmable Array Microscope”. Appl. Spectrosc. 2001. 55(9): 1115–1123.

Hanley

Q.S.

Verveer

P.J.

Jovin

T.M.

“Spectral Imaging in a Progammable Array Microscope by Hadamard Transform Fluorescence Spectroscopy”. Appl. Spectrosc. 1999. 53(1): 1–10.

Hammaker

R.M.

Mortensen

A.N.

Orr

E.A.

Bellamy

M.K.

“Multidimensional Hadamard Transform Spectrometry”. J. Parallel Distributed Comput. 1995. 247(2).

Bellamy

M.K.

Mortensen

A.N.

Hammaker

R.M.

Fateley

W.G.

“Chemical Mapping in the Mid- and Near-IR Spectral Regions by Hadamard Transform/FT-IR Spectrometry”. Appl. Spectrosc. 1997. 51(4): 477–486.

Hanley

Q.S.

Verveer

P.J.

Jovin

T.M.

“Optical Sectioning Fluorescence Spectroscopy in a Programmable Array Microscope”. Appl. Spectrosc. 1998. 52(6): 783–789.

Hanley

Q.S.

Lidke

K.A.

Heintzmann

Arndt-Jovin

D.J.

Jovin

T.M.

“Fluorescence lifetime imaging in an optically sectioning programmable array microscope (PAM)”. Cytometry Part A. 2005. 67A(2): 112–118.

Fulwyler

Hanley

Q.S.

Schnetter

Young

I.T.

Jares-Erijman

E.A.

Arndt-Jovin

D.J.

Jovin

T.M.

“Selective photoreactions in a programmable array microscope (PAM): Photoinitiated polymerization, photodecaging, and photochromic conversion”. Cytometry Part A. 2005. 67A(2): 68–75.

10.

Vujkovic-Cvijin

Goldstein

Fox

M.J.

Higbee

S.D.

Becker

L.S.

Ooi

T.K.

“Adaptive Spectral Imager for Space-Based Sensing”. Proc. SPIE. 2006. 6206: 62060X.

11.

Gehm

M.E.

Kinast

“Adaptive spectroscopy: towards adaptive spectral imaging”. Proc. SPIE. 2008. 6978: 69780I.

12.

B.L.

Feng

D.Z.

Fan

X.H.

Qian

Q.M.

“Analysis and study of the interlaced encoding pixels in Hadamard transform spectral imager based on DMD”. Opt. Lasers Eng. 2012. 50(3): 458–464.

13.

DeVerse

R.A.

Hammaker

R.M.

Fateley

W.G.

“Hadamard transform Raman imagery with a digital micro-mirror array”. Vib. Spectrosc. 1999. 19(2): 177–186.

14.

Fateley

W.G.

Hammaker

R.M.

DeVerse

R.A.

Coifman

R.R.

Geshwind

F.B.

“The other spectroscopy: demonstration of a new de-dispersion imaging spectrograph”. Vib. Spectrosc. 2002. 29(1-2): 163–170.

15.

Goldstein

Vujkovic-Cvijin

Fox

Gregor

Jamine

Cline

Adler-Golden

, “DMD-based adaptive spectral imagers for hyperspectral imagery and direct detection of spectral signatures”. Proc. SPIE. 2009. 7210: 721008.

16.

Love

S.P.

“Programmable matched filter and Hadamard transform hyperspectral imagers based on micro-mirror arrays”. Proc. SPIE. 2009. 7210: 721007.