Abstract
Under condition of uneven illumination, directly recognizing the document image by OCR would lead to a bad effect, which is a difficult point in document recognition. To solve this problem above, a Retinex-based illumination correction method was proposed. Firstly, the images were supposed to be divided into blocks, which were processed respectively by Otsu binarization. The light conditions can be got through the feedback results of processing, which helps to have uneven illumination image blocks achieved. Then images can be enhanced by the improved Retinex method, which extends dynamic range of image block gray value and enhances details of image block. Finally, images can be corrected by sauvola binarization processing. Experimental results show that the algorithm can eliminate the impact of uneven illumination. Compared with the common local binarization and enhancement methods, the OCR recognition rates are significantly increased. At the same time, the combination of global and local methods promotes the applicable scope of being in condition of samples and different illumination.
Introduction
With the development of digital image technology and the popularization of camera equipment, document images have become an important medium for people to transmit information, and the acquired document information usually needs to be processed by computer before they can be better utilized. After document images are converted into document files, which are provided to various groups or organizations, such as some visually impaired people who want to listen to newspapers and letters, the sorting of parcels and etc. Among many document recognition methods, OCR (Optical Character Recognition) is one of the most efficient methods to extract text information from document images. However, due to the impact of the shooting environment and the differences of images themselves to light absorption and reflection [1], the document images are uneven in quality, which affects the OCR recognition rate badly. Among them, the effect of uneven illumination is more prominent, which can cause a large area of document recognition error or false recognition as a picture. Of course, uneven illumination in the field of identification is a common problem, such as the identification of some two-dimensional code [2] etc.
At present, the commonly used method is to improve the local binarization of document image directly [3]. Of course, for a variety of specific environmental samples, scholars have also proposed a model of image reconstruction [4, 5], to eliminate the impact of light, to achieve good results, but its lack of universality of other samples. So, some studies are very concerned about the adaptability of the algorithm. In the literature 6, the algorithm of edge detection is proposed, which has good universality. But these local algorithms are relatively slow, the literature 7 also presents a block adaptive document image binarization method based on the judgment of the light. But the accuracy needs to be improved, and the improved local binarization processing dark image will generate a lot of noise.
Image enhancement algorithm [8] can reduce the influences of local shadow or bright light for the document image to a minimum as far as possible. Therefore, it can achieve a good result by appropriately combining it with local binarization method. This paper combines the document image improved Retinex [9] enhancement algorithm with Sauvola [10, 11] binarization method to deal with uneven illumination of the image. Comparing with other single binarization methods or enhancement algorithms, it can reduce the broken pen and noise cases. At the same time, this paper put forward a feedback type judgment method based on Otsu algorithm to the judgment for image illumination condition, and it make a more accurate judgment comparing with some methods which are simply judged by counting some image characteristics. Considering the general document image resolution is rather large and using localized processing will take much time, this paper adapt the method to block the image processing firstly. Combining Ostu [12] binarization with Retinex or Sauvola algorithm to deal with images separately by judging the degree of the uneven illumination.
Basic theory
Retinex (Reintal-Cortex) theory which is a kind of image enhancement theory based on color invariant theory. It was presented by the Land and others in the 1970s. The theory is based on scientific experiments and scientific analysis. Its formula is:
The essence of the Retinex theory is removing the effect of the light weight to the original image and acquiring the reflection component of the object, so the original appearance of the object itself is available.
The signal-scale Retinex (single-scale Retinex, SSR) which was proposed by Jobson and others and the multi-scale Retinex (multi-scale Retinex, MSR) [13] which was developed on the basis of signal-scale Retinex. SSR algorithm can be represented as:
In the formula, c is the surrounding scale; k is the normalization constant and the surrounding Eq. (3) should satisfy the Eq. (4).
From the formula it can be clearly seen that the surrounding scale c determines the performance of the SSR to a great extent. Therefore Jobson et al proposed an MSR algorithm based on the following Eq. (5):
In the formula, n represents the number of scales, and it is generally divided into three scales, which are high-scale, medium-scale and low-scale;
The advantage of multi-scale is that it can obtain the reflected image information obtained by multiple surrounding scales c, and then through the weight accumulation, it can obtains a more balanced image of the real object.
Analysis and judgment of illumination uneven image
In the processing of non-uniform light image, many methods are directly for the original image of some image features, such as gray-scale pixel frequency, statistics, and then a large number of document samples tested to determine whether the document image block is a non-uniform light experience value. This statistical data directly against the original image limits its applicability. For some lighting conditions change faster images are often judged to be not accurate enough.
Most of the document images in this paper are pure document images, it does not contain a large number of complex types of pictures, and finally the prospects are only plain text and some simple types of line images. Through the research on the processing effect and performance of various binarization, it is found that Otsu algorithm can balance the processing speed and effect for some text blocks with uniform illumination. The basic idea of Otsu algorithm is that based on gray histogram of image. A segmentation threshold that maximizes the variance between classes, and then binarizes the image with this threshold as the boundary. However, the Otsu algorithm is not good enough for some unevenly illuminated image blocks.
Comparison of Otsu method.
Three maps in the Fig. 1, Fig. 1b is a direct result of a global Otsu algorithm. From a visual point of view, the result will appear a large number of black blocks, and the black block area is basically consistent with the shadow area of the original map, reflecting the Otsu algorithm sensitivity to changes in lighting conditions. Figure 1c is the result of the processing that the original image is processed by Otsu algorithm after divided into uniform image block (4 * 4 blocks).From the overall effect of the treated figure, it can properly reduce the impact of uneven illumination and reduce the generation of black blocks to a certain extent. At the same time, it can be clearly observed from the processing effect of the block that the region block affected by the illumination unevenness will still mistake a small amount of background as text, resulting in a partial black block. The Otsu algorithm is used to judge the difference between light uniformity and non-uniform light block.
Otsu binarization is performed for each bin and the black pixels (foreground) in the resulting image block are counted as a proportion of the bin. The number of black pixels is denoted by blackCount, width and height represent the image block width, high, then the black pixel ratio is:
Each image block obtained from the blackScale is shown in Table 1 above.
blackScale value of image block
For some light uneven image block, after Otsu binarization processing it will be a large area of shadows, which greatly improve blackScale. As shown in Fig. 3, the text is relatively dense relatively uniform light block (the third row of the second column), which reached only 0.093, and as shown in Fig. 3, the relatively low degree of light unevenness of the image block (the third row of the fourth column) ratio reached 0.160. From the experimental data in Table 1, it can be found that there is a clear disparity in the ratio between the light unevenness and the light unevenness, so that the light uniformity block and the uneven light block can be distinguished by a critical value at the middle.
blackScale
blackScale


After having counted a large number of image blocks of the sample statistics and analyzed the text space and text width, the critical value of scaleThreshold is 0.135, which can achieve good results. However, in order to further improve the accuracy of the threshold, this paper uses interactive way, so that customers can selectively according to the visual effect to judge the effect of treatment, and feedback to scaleThreshold. The minimum black pixel ratio of the uneven illumination is recorded as scaleMin, and the maximum black pixel ratio of the uniform illumination block is recorded as scaleMax, and the scaleThreshold is continuously updated according to the Eq. (7).
Because the text image is different, the appropriate threshold will be targeted to change, each text image needs a specific scaleThreshold value of thresholdSp. The upper limit of the image uniform block blackScale is 0.150 which recorded as hScale; the lower limit of image uneven block’s blackScale is 0.100 which recorded as lScale. Only those text is extremely dense light uniform block or uneven light intensity of the image block whose blackScale is only possible between 0.100 and 0.150.
In this case, it is possible to count the number of image blocks in each segment by blackScale. The number of image blocks of each segment is represented as follows:
When the blackScale is lower than 0.100, the number of image blocks in this segment is iCountL; When the blackScale is between 0.100 and 0.150, the number of image blocks in this segment is iCountM; When the blackScale is more than 0.150, the number of image blocks in this segment is iCountH.
When a document image is processed by Otsu, the statistical iCountM is zero, the direct use of scaleThreshold value. So it is not necessary to redefine the specific critical value of the graph. If it is not zero, it is necessary to change the segmentation threshold by the Eq. (8).
Taking thresholdSp as the boundary between the uniform light block and the non-uniform light block can give an accurate feedback of the illumination condition of the image block. When the scaleRatio is bigger than thresholdSp, then this block is divided into uneven light block; when the scaleRatio is not bigger than thresholdSp, then this block is divided into light uniform block.
For the non-uniform illumination block, the position coordinates of the block are preserved, and then the corresponding original image block is further processed based on the Retinex algorithm.
After having analyzed the classical SSR algorithm and MSR algorithm and considered the trade-off between the time and effect, this paper adopts the SSR algorithm to improve the text image. At the same time, taking into account that the influences of the light to picture is the brightness of the information of the image, transform the sample of non-uniform block of color space to YUV [14]. And then only for its Y component single-scale SSR enhancement, so as not to affect its sample color information.
General MSR application in de-fog [15, 16] is relatively more, this type of study of the original image gray-scale distribution is not very fixed, relatively scattered, so it’s necessary to select multiple scales to ensure that the gray level of the treatment effect is consistent. For the text image, the gray-scale generally concentrated at both ends, compared to other images, the text boundary transition is more obvious, and in addition to the title of the text strokes outside the width of the general will not exceed 5 pixels, so we do not need to choose the too large range of scale to carry on SSR enhancement, and we want to get the extreme binary image finally, the gray distribution is single, so we do not need to spend too much time considering the inconsistency of sample processing effect, This would be counterproductive. As shown in Fig. 5 for the original SSR processing effect.
SSR. Improved SSR.

Although it largely weakened the impact of light conditions, from a visual point of view. Gray level of the text and background are very close, so it will be difficult to distinguish the difference. Therefore, it is necessary to select a region that can pull the low gray scale as far as possible, so we can achieve balance between performance and speed. Thus in this paper, according to the above analysis of image strokes, select the scale range of 3 to 5 range. For text images, this paper improves the surround function of SSR:
In this formula,
In this formula,
In this formula,
As can be seen from Fig. 5, the improved SSR algorithm makes it possible to distinguish the background from the literal. It can be well observed that uneven illumination block which Otsu did not handle well (Obtained from the judgment according to the blackScale) during post-processing algorithm improved SSR, background and text can be well visually separate areas. The extent of the impact of light is also a significant decline, although the text has a slightly blurred, this level does not affect the OCR on its recognition.
Of course, for some of the text image of which the text is relatively small, in many blocks, there may still be uneven illumination in the text space, and the block does not need to use SSR for processing. Therefore, it is possible to determine whether or not it is a blank space by counting the number of gradation levels of the gradation histogram of the block image block (i.e., the character portion). If so, it is binarized directly and all set to white.
Then, the block is re-converted back to the RGB color space, and the blocks are respectively grayed out, followed by local binarization.
The typical local binarization algorithms contain Niblack algorithm [17] and Sauvola algorithm. Although the Niblack algorithm can segment the characters with low gray level, only a few local information of the pixels are taken into account and a lot of noise will be generated. The Sauvola algorithm overcomes the shortcomings of the Niblack algorithm, which makes the threshold value smaller for the background with the higher gray value, so as to filter out a certain amount of noise in the background under the same standard deviation. The improved SSR image is processed first and then processed by the Sauvola algorithm. The results show that the method almost does not produce noise for the common illumination uneven sample.
Final binarization image. Local image.

As can be seen from Fig. 7, this algorithm eliminates the influence of all lighting. The use of binary image representation makes the text and the background clearly separated. At the same time, from the local image of Fig. 7 we can also see that it basicly does not produce noise which is because of the segmentation is not clear, and the structure of the text is also very full, rarely broken pen writing situation. And for some of the lighting conditions change faster and overall dim image, it can achieve satisfactory results.
The difficulty of image processing for illuminated nonuniform text lies in the judgment of the uneven illumination area and the adaptability of the correction method. Since the gray histogram of the illumination unevenness image does not have two distinct peaks, it is not possible to deal with it using a global binarization of a single threshold. However, only using the local binarization processing will cost much time and produce a large number of noise. And it will affect the final OCR recognition. In fact, for the block itself to reduce the amount of information in the image at the same time, to a certain extent, reduce the image block of light on the text of the degree of impact. Otsu processing is used to distinguish these relatively uniform illuminations from non-uniformly illuminated blocks, and Otsu further enhances the illumination of the unevenly illuminated blocks as reflected in the binary image, and it can be clearly judged that it is different from the uniform light block. The improved SSR and Sauvola binarization are used for the non-uniform light, and the ideal result is obtained.
First, the gray-scale image is divided into N * N (N in 2 to 8) image blocks. For the general text image, most of its text area will be in the middle of the image, the image border area will have a appropriate blank area. If the N value is too large, it will cause too little foreground information in the segmentation block, and with the increase of the block, block effect and processing time will increase; If the N value is too small, uniform illumination and uneven brightness regions can become difficult to be separated, thus less than ideal results. After experiment, we can get the best effect and performance and the text and the text are balanced when the N is 4.
Then, performed the Otsu binarization on the image blocks, and judge the result of the binarization image. If it is determined that the light is not homogeneous, the block is subjected to improved SSR enhancement and Sauvola processing. Figure 8 is the basic flow chart of the correction method in this paper.
The basic flow chart of the correction method.
As described above, the detailed steps of the calibration method are as follows:
The image is divided into 4 * 4 sub-image blocks. Traverse the image block and use the formula:
to gray the image block. Use Otsu binarization to perform on the image blocks and count the ratio of black pixel ratio. The blackScale
And obtain the original image of the block to count the pixel frequency sum of which the gray level is greater than 90. If sum The enhancement function is performed by using the surround function as the single-scale SSR Eq. (2) of the Eq. (10) and proceeds to step 6. Use Sauvola algorithm for binary segmentation as a result of the output. Combine the results obtained in 3 and enter the final binary image.
The environment of test experimental
This paper is compiled and debugged in Microsoft Visual Studio 2005 and implemented in C
Results comparison and analysis
The experimental results are compared with the local Niblack, Sauvola algorithm and the sub block algorithm of literature 7. In this paper, the experimental visual effect of the algorithm can refer to Figs 7 and 7. Figure 9 for the table type and simple graphics type through the effect of the various algorithms to deal with.
From the visual effects of each method detailed view of local processing point of view, Niblack algorithm can extract the text information, text prospect does not occur broken pen, etc. But a lot of noise will appear at the boundary of the text, causing difficulty in OCR recognition; Sauvola algorithm will still generate some noise; for the algorithm in the literature 7, in some lighting conditions which change fast or relatively dark areas will produce a certain amount of noise. For this algorithm, it will not have noise in the background, text messages can be properly extracted, at the same time, some of their borders, arrows, images and other information can also be reserved. So OCR recognition can easily judge it as a picture and don’t make it mistakes with the wrong characters.
As Fig. 10 the results can clearly see the effect. For the algorithm in the literature 7, the noise was more obvious, and greatly influence the identification of OCR.
Performance analysis of algorithm
Performance analysis of algorithm
Processing results of each algorithm.
Processing result.
Compared with experiment result of Table 2, Niblack algorithm will produce a lot of noise around the word for uneven brightness images of text, and the recognition rate is also increased relatively little. Because Sauvola algorithm eliminates a lot of noise, there is a certain improvement of recognition rate but it takes too much time. The algorithm of literature 6 can get good recognition rates for most text, but in the light changes rapidly or dark images as mentioned above will appear a certain amount of noise, resulting in recognition rates significantly decrease. In this paper, the algorithm can get a high recognition rate in the time consuming of acceptable range, and it can also improve the application range of the sample. Of course, as can be seen from the experimental data, plain text has a higher recognition rate compared to the tables and sample of simple graphics type. The main reason is the latter type of graphic text area generally is irregular and some of the lines closed to the text also affect text segmentation.
For the uneven illumination caused by these changes in illumination, shadows of external objects, paper twist, shooting from different angles mentioned above, the experimental results show that this method can get a good binary image for all kinds of common text types, and it can be used to solve the problem of the illumination. It can achieve balance in the amount of noise, text clarity, and get a good visual effects and recognition rate.
Based on the Otsu algorithm, this paper proposes a method which has wide applicability and high recognition rate to correct the uneven text image based on the Retinex theory. Compared with the single global or local threshold value of two segmentation methods, the overall treatment effect will be greatly improved, it can achieve good balance between noise and clarity of text. And corresponding to some improvements and to do a modest enhancement of the algorithm, as described in the literature 7, there is a great improvement in the scope of application, for the dark image will have a good effect.
At present, a good and satisfactory correction effect can be obtained by using the conventional light nonuniform text image taken by the instrument. Treatment of dark text image will be greatly improved, but for some light source is not stable enough, resulting in some text image processing will be a certain amount of noise illumination uneven transition region is not very smooth, the effect is to be further improved.
Footnotes
Acknowledgments
This work was supported by the National Natural Science Foundation of China under the Grants 61371142 and a special fund of School (No. XN060).
