Design of a freeform lens for structured light illumination by deconvolution algorithm and feedback modification

Abstract

In this paper, we propose a method combining a deconvolution algorithm and a feedback modification to design a freeform lens for structured light illumination. The deconvolution algorithm can alleviate the blur effect caused by the extended light source and the feedback modification can further improve the quality of the structured light by iteration. A lens for a linear sinusoidal pattern is designed by the proposed method. Simulation results show the contrast is 89%, compared to 57% achieved by the conventional design method.

1. Introduction

Nowadays, structured light illumination systems have been widely used in fluorescence microscopy for higher resolution^1,2 and 3D shape measurement for contactless and precise measurements.^3,4 Most of the systems generate the structured light by a grating or spatial light modulator,^5–7 but these are complicated and expensive. As light-emitting diodes (LEDs) become widely used in common life and industry and freeform optics are widely used in non-imaging optics^8–12 and imaging optics^13–16 an LED with a freeform optical element has become an alternative way to achieve structured illumination with less cost and difficulty.

Most freeform optics design methods for LEDs are based on an ideal point source. However, the actual extended sources will blur the patterns, and decrease the contrast. In this paper, we consider the actual pattern generated by an extended source as the convolution of the pattern generated by a point source and the point spread function (PSF) of the freeform optical element. Then we introduce the deconvolution algorithm to calculate a new pattern generated by the point source that can eliminate the blur effect. Finally, a feedback modification method is used to improve to quality of the structured light. Using the methods proposed in this paper, a freeform lens is designed. It can generate a linear sinusoidal pattern on the target plane, and the contrast is 88%, demonstrating the suitability of a freeform optical element for designing structured light illumination systems.

2. Design method

2.1. The grids division of the light source

In this paper, our goal is to design a freeform lens to generate a linear sinusoidal illumination pattern on the target plane. The light source is an LED chip with a size of 1 mm × 1 mm. The linear sinusoidal pattern means the illuminance distributes sinusoidally in one direction, but uniformly in the vertical direction. Considering the characteristic of the illumination pattern and matching the rectangular grids on the target plane, the grids of the light source are divided in the $(u, v)$ coordinate system,⁸ which is shown in Figure 1. The light source O is the origin, OP is an arbitrary ray emitted from the source, u is the angle between the ray and x axis, v is the angle between the component on the yOz plane of the ray and the z axis.

Figure 1.

The $(u, v)$ coordinate system

The relationship between $(u, v)$ and $(x, y, z)$ in the Cartesian coordinate system is:

\begin{matrix} {\begin{matrix} x = r cos u \\ y = r sin u sin v \\ z = r sin u cos v \end{matrix} \end{matrix}

(1)

where

r^{2} = x^{2} + y^{2} + z^{2}

, and the unit solid angle is:

d Ω = sin u d u d v

(2)

The LED source usually can be approximated as a Lambertian emitter, so the luminous intensity of the light source is:

I (u, v) = I_{0} sin u cos v

(3)

where I₀ is the luminous intensity along the z axis. Combining equations (2) and (3), the luminous flux of the light source can be described as:

\begin{matrix} Φ = \int I (u, v) d Ω = \int \int I_{0} {sin}^{2} u cos v d u d v \\ = I_{0} \int_{u min}^{u max} {sin}^{2} u d u \int_{v min}^{v max} cos v d v \end{matrix}

(4)

where umin, umax, vmin and vmax are the integral limits of u, v, respectively.

Equation (4) is the formula to calculate the luminous flux in an arbitrary solid angle, which can help establish the mapping relationship between the light source and the target plane later.

2.2. Deconvolution method

While designing a freeform optical element for uniform illumination, the actual LED can be assumed as an ideal point light source if its size is small enough compared to the optical element. However, when designing a structured light illumination system which requires high contrast, the blur effect of the extended source has to be considered. If the extended source is considered as a bunch of point sources during the design, the actual illumination pattern can be described as the convolution of the desired sinusoidal illumination pattern and the PSF of the optical element.^3,9,16 So, the ideal sinusoidal pattern will deteriorate and decrease the contrast of the fringes.

The convolution of the designed sinusoidal illumination pattern and the PSF can be described as:

E (y) = E cos (y) \otimes PSF

(5)

Because in this design, the illuminances distribute sinusoidally only along the y direction, so, we only consider the blur effect on y direction. In equation (5), $E cos (y)$ is the desired sinusoidal distribution, PSF is the PSF of the lens, ⊗ represents the operation of convolution, and $E (y)$ is the actual illuminance distribution along the y direction.

So, in order to eliminate the blur effect, the designed illuminance distribution along y direction should be:

E' (y) = E cos (y) \otimes^{- 1} PSF

(6)

where

E' (y)

is the designed illuminance distribution along y direction, which will help establish the mapping relationship, and

\otimes^{- 1}

represents the operation of deconvolution, The PSF can be approximated as a rectangular window with a width of A⁹:

A = \frac{H_{3} - H_{2}}{H_{1} + (H_{2} - H_{1}) / n} a

(7)

where n is the refractive index of the lens, a is the width of the LED chip, H₁, H₂, H₃ are the distances from the source to the bottom of the lens, to the top of the lens and to the target plane, respectively. Knowing

E cos (y)

and PSF, the

E' (y)

can be easily calculated by using the Richardson–Lucy algorithm,^17–19 which is applied in image processing to alleviate the blur effect and improve the quality of the image. The

E cos (y)

, PSF, the

E' (y)

are shown in Figure 2, since the distribution of illuminance along y direction is periodic, only the centre period of the

E cos (y)

and

E' (y)

is plotted.

Figure 2.

(a) The sinusoidal pattern (b) the rectangular PSF, and (c) the target pattern of the point source

2.3. Calculating the mapping relationship and constructing the freeform lens

After obtaining $E' (y)$ , and with the help of the energy conservation law, which can be described as:

\int \int I (u, v) d Ω = \int \int E' (x, y) d x d y

(8)

The mapping relationship can be calculated by using equation (4) and the numerical integration of the illuminance on the target plane. The mapping relationship is showed in Figure 3. In this design case, x on the target plane only corresponds to u, and y only corresponds to v.

Figure 3.

Mapping the relationship between source and target plane

Then, after the mapping relationship between $(u, v)$ and $(x, y)$ is obtained, the next process is the calculation of discrete points on the freeform surface. Using Snell’s Law, we can calculate the normal vector of an arbitrary point P on the surface, then we can obtain the tangent plane of the surface at point P. The intersection of the tangent plane and next incident ray is the next point on the surface. So, from a given initial point, all the discrete points’ coordinates of the freeform surface can be calculated. More details of this construction algorithm are described specifically in Luo et al.¹⁰ Then, we import the coordinates of the points into a 3D modelling software such as SolidWorks. By using the software, we can obtain the model of the freeform lens.

2.4. Feedback modification

Through the steps from Section 2.1 to 2.3, a freeform lens is designed and constructed. However, because of the calculation error during the surface construction,¹¹ the error of the deconvolution algorithm and numerical integration, the approximation of the PSF, and the energy loss when the rays passes through the lens surfaces, the simulation result may have a large deviation from the ideal one. So, we introduce a feedback modification method^10,12 to adjust the mapping relationship obtained in Section 2.3.

Considering the characteristics of the illuminance pattern and the relationship, the x and v are modified by the simulation data, respectively. Let $Δ x_{m}$ , $Δ v_{n}$ be the increment of x and v:

\begin{matrix} {\begin{matrix} Δ x_{m} = x_{m + 1} - x_{m}, m = 1, 2 \dots M \\ Δ v_{n} = v_{n + 1} - v_{n}, n = 1, 2 \dots N \end{matrix} \end{matrix}

(9)

where M is the number of grids along the x direction, and N is the number of grids along the y direction. Then, the feedback coefficients are defined as:

α_{m} = \frac{{Ex}_{m}^{i}}{{Ex}_{m}^{0}}, i = 1, 2, 3 \dots

(10)

β_{n} = \frac{{Ey}_{n}^{i}}{{Ey}_{n}^{0}}, {Ey}_{n}^{0} \neq 0, i = 1, 2, 3 \dots

(11)

where

{Ex}_{m}^{0}

and

{Ey}_{n}^{0}

are the desired illuminance at the grid,

{Ex}_{m}^{i}

and

{Ey}_{n}^{i}

are the illuminance at the grid after ith simulation which are exported from the optical simulation software. Then, the modified

Δ x_{m}

and

Δ v_{n}

are:

Δ x_{m}^{i + 1} = Δ x_{m}^{i} \cdot α_{m}

(12)

\begin{matrix} {\begin{matrix} Δ v_{n}^{i + 1} = Δ v_{n}^{i} / β_{n}, {Ey}_{n}^{0} \neq 0 \\ Δ v_{n}^{i + 1} = 0, {Ey}_{n}^{0} = 0 \end{matrix} \end{matrix}

(13)

where

Δ x_{m}^{i + 1}

Δ v_{n}^{i + 1}

are the modified values, and the modified x and v are also obtained. Then, the mapping relationship between

(u, v)

and

(x, y)

is modified, and a new freeform lens can be constructed, and if its simulation results is not satisfactory enough, the feedback iteration can continue until the performance is good enough.

3. Design example

The example is to generate a linear sinusoidal pattern on the target plane, whose period is 25 mm, the width of the LED chip is $a = 1 mm$ . The distances from the source to the bottom of the lens, the top of the lens and the target plane are: H₁ = 20 mm, H₂ = 40 mm and H₃ = 450 mm, respectively. The lens material is chosen as PMMA, the refractive index is $n = 1.49386$ . The size of illumination area is $800 mm \times 800 mm$ . The lens has a bottom flat surface and a freeform surface, so, the lens cannot receive the luminous flux in the whole semi sphere, so we make $u min = π / 6, u max = π / 2$ , and, $v min = 0, v max = π / 3$ , to make sure the lens can receive most of the luminous flux emitted from the source.

Using the design method we propose in Section 2, a freeform lens is designed and iteratively modified 15 times. The top view side, side view and the 3D model of the optimized lens are showed in Figure 4, and the basic sizes of the lens are marked.

Figure 4.

(a) Top view, (b) side view and (c) a 3D model of the freeform lens

We also designed a lens for structured light illumination without using the deconvolution method and feedback modification for comparison. The simulation result is showed in Figure 5. To evaluate the quality of the structured light, we define the contrast as:

contrast = \frac{{\bar{E}}_{max} - {\bar{E}}_{min}}{{\bar{E}}_{max} + {\bar{E}}_{min}}

(14)

where

{\bar{E}}_{max}

and

{\bar{E}}_{min}

are the average of the maximum illuminance of bright fringes and the minimum of dark fringes, respectively. The contrast of the structured light in Figure 5 is only 57%, calculated by equation (14). After using the deconvolution method and feedback modification, the contrasts are 82% and 89.3%, respectively. The simulation result of the final lens is showed in Figure 6.

Figure 5.

Simulation result of the lens without using the deconvolution method and feedback modification

Figure 6.

Simulation result of the lens designed using the deconvolution method and feedback modification

From the simulation data of the final lens, ${\bar{E}}_{max} = 255.9 lux$ and ${\bar{E}}_{min} = 14.4 lux$ , the relative root mean square (RMS) between the simulated profile and the ideal sinusoidal profile is 16%, and the uniformity of the illuminance along the x direction is about 94.5%. These data show that the proposed method is able to design a freeform lens for precise structured light illumination with an extended light source.

In addition, we should discuss the positional tolerance between the LED and the lens, which will help in the assembly of the system. Firstly, we discuss how the position error along the x axis ( $Δ x$ ) affects the quality of the structured illumination. Figure 7(a) shows the change of contrast and RMS with $Δ x$ , and Figure 7(b) shows the simulation result when $Δ x = 1 mm$ . From the simulation, it is obvious that $Δ x$ has little effect on the quality of the structured illumination. Secondly, the effect of the positional error along the y axis ( $Δ y$ ) is discussed. Figure 8(a) shows the change of contrast with $Δ y$ , the contrast deteriorates sharply as $Δ y$ increases; Figure 8(b) shows the simulation result when $Δ y = 0.4 mm$ . So, as the simulation shows, it is critical to control the positional error along the y axis. Finally, the position error along the z axis ( $Δ z$ ) is discussed. Figure 9(a) shows the change of contrast with $Δ z$ . Although all the values of contrast are larger than 0.75, the variation of contrast is not stable. Figure 9(b) and (c), which are the simulation results when $Δ z = 1 mm$ and $Δ z = - 1 mm$ , respectively, show that the periods of the fringes are different from the ideal value of 25 mm. So the position error along z axis needs to be controlled carefully as well.

Figure 7.

(a) The change of contrast and RMS with $Δ x$ , (b) the simulation result when $Δ x = 1 mm$

Figure 8.

(a) The change of contrast with $Δ y$ , (b) the simulation result when $Δ y = 0.4 mm$

Figure 9.

(a) The change of contrast with $Δ z$ , (b) the simulation result when $Δ z = 1 mm$ and (c) $Δ z = - 1 mm$

4. Conclusion

In this paper, we develop a method which combines a deconvolution algorithm and feedback modification. This method can achieve an accurate linear sinusoidal structured light illumination, and it can be used to design structured light in other periodic patterns as well, for example, a triangle fringe pattern. To realize that, we should replace the sinusoidal distribution of illumination $E cos (y)$ in equation (6) with another distribution function which represents a different periodic pattern, then, by following the steps described in this paper, we can obtain a lens that generates a structured illumination in another pattern.

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Key Research and Development Program of China (2016YFF0101400), Natural Science Foundation of Guangdong Province (2016A030313473, 2015A030310278), the Key Technologies R&D Program of Guangdong Province (2016B090918057), the Key Technologies R&D Program of Guangzhou City (201704020038, 201704020182), and supported by State Key Laboratory of Pulp and Paper Engineering (No. 201537).

References

Cogswell

. Doubling the lateral resolution of wide-field fluorescence microscopy using structured illumination. Proceedings of the SPIE 2000; 3919: 141–150.

Fedosseev

Belyaev

Frohn

Stemmer

. Structured light illumination for extended resolution in fluorescence microscopy. Optics and Lasers in Engineering 2005; 43: 403–414.

Zwick

Heist

Steinkopf

Huber

Krause

. 3D phase-shifting fringe projection system on the basis of a tailored free-form mirror. Applied Optics 2013; 52: 3134–3146.

Geng

. Structured-light 3D surface imaging: a tutorial. Advances in Optics and Photonics 2011; 3: 128–160.

Bloom DM, Leone MA. Structured light system, US Patent 20110169915, 2010.

Neil

Juskaitis

Wilson

. Method of obtaining optical sectioning by using structured light in a conventional microscope. Optics Letters 1997; 22: 1905–1907.

Kner

Chun

Griffis

Winoto

Gustafsson

. Super-resolution video microscopy of live cells by structured illumination. Nature Methods 2009; 6: 339–342.

Zhang

Liu

. Freeform lenses for illumination of specific shapes in (u,v) coordinate system. Optical Review 2016; 23: 77–83.

Liang

Feng

. Deconvolution method in designing freeform lens array for structured light illumination. Applied Optics 2015; 54: 1114–1117.

10.

Luo

Feng

Han

. Design of compact and smooth free-form optical system with uniform illuminance for LED source. Optics Express 2010; 18: 9055–9063.

11.

Wang

Qian

Luo

. Discontinuous free-form lens design for prescribed irradiance. Applied Optics 2007; 46: 3716–3723.

12.

Situ

Han

Luo

. Combined feedback method for designing a free-form optical system with complicated illumination patterns for an extended LED source. Optics Express 2011; 19(Suppl 5): A1022–A1030.

13.

Yang

Zhu

Jin

. Direct design of freeform surfaces and freeform imaging systems with a point-by-point three-dimensional construction-iteration method. Optics Express 2015; 23: 10233–10246.

14.

Hou

Zhu

Yang

Jin

. Construction method through forward and reverse ray tracing for a design of ultra-wide linear field-of-view off-axis freeform imaging systems. Journal of Optics 2015; 17: 055603–055603.

15.

Zhu

Yang

Jin

. Design method of surface contour for a freeform lens with wide linear field-of-view. Optics Express 2013; 21(22): 26080–26092.

16.

Riser

Cassarly

. Analysis of single lens arrays using convolution. Optical Engineering 2001; 40: 805–813.

17.

Richardson

. Bayesian-based iterative method of image restoration. Journal of the Optical Society of America 1972; 62: 55–59.

18.

Lucy

. An iterative technique for the rectification of observed distributions. Astronomical Journal 1974; 79: 745–745.

19.

Ingaramo

York

Hoogendoorn

Postma

Shroff

Patterson

. Richardson-Lucy deconvolution as a general tool for combining images with complementary strengths. Chemphyschem 2014; 15(4): 794–800.