International Journal of Scientific & Engineering Research, Volume 4, Issue 9, September-2013 1973

ISSN 2229-5518

Single Image Super Resolution Image Reconstruction with

Morphological Regularization Parameters

Mohanthi Kakarla,M. Sreedhar Reddy,N. Lakshmi Narayan

Abstract— Multiscale morphological operators are studied extensively in the literature for image processing and feature extraction purposes. This paper model a nonlinear regularization method based on multiscale morphology for edge-preserving super resolution (SR) image reconstruction.SR image reconstruction is formulated as a deblurring problem and then the inverse problem is solved using Bregman iterations. The proposed algorithm can suppress inherent noise generated during low-resolution image formation as well as during SR image estimation efﬁciently. Experimental results show the effectiveness of the proposed regularization and reconstruction method for SR image.

Keywords — Bregman iteration, deblurring, morphologic regularization, operator splitting, subgradients.

1. INTRODUCTION

THE basic goal is to develop an algorithm to enhance the spatial resolution of images captured by an image sensor with a ﬁxed resolution. This process is called the super resolution(SR) method and it has remained an active research topic for the last two decades. A number of fundamental assumptions are made about image formation and quality, it lead to different SR algorithms. These assumptions include the type of motion, the type of blurring, and also the type of noise. It is also important whether to produce the very best HR image possible or an acceptable HR image as quickly as possible.SR algorithms may vary depending on whether only a single low-resolution (LR) image is available (single frame SR) or multiple LR images are available (multiframe SR). SR image reconstruction algorithms work either: 1) in the frequency domain or 2) in the spatial domain.This paper focus only on spatial domain approach for multiframe SR image reconstruction.

This paper is based on the regularization framework, where the HR image is estimated based on some prior knowledge about the image (e.g., degree of smoothness) in the form of regularization. Bayesian maximum a posteriori (MAP) estimation based methods use prior information in the form of a prior probability density on the HR image and provide a rigorous theoretical framework. MAP based joint formulation is proposed that judiciously combine motion estimation, segmentation, and SR together.The probability based MAP approach is equivalent to the concept of regularization.
-----------------------------------------------------------

• Author name is Mohanthi K currently pursuing masters degree program in Systems and Signal Processing in JNTU University, India,

918790239148. mohanangel419@gmail.com

• Co-Author name is Sreedhar reddy M currently nt Professor working as

an Assistr engineering inMRCET,JNTU University, India,

919441592391. maale.sreedharreddy@gmail.com
The ﬁrst successful edge preserving regularization method for denoising and deblurring is the total variance (TV) (L1 norm) method.Another interesting algorithm, proposed by Farsiu et al., employs bilateral total variation (BTV) regularization. To achieve further improvement, Li et al. used a locally adaptive BTV (LABTV) operator for the regularization.
All these regularization terms for SR image reconstruction lead to a stable solution, their performance depends on optimization technique as well as regularization term. For example, with gradient descent optimization technique, the LABTV regularization outperforms BTV, which gives better result than TV regularization. On the other hand, based on TV regularization, Marquina and Osher obtained superior result by employing Bregman iteration. So we envisage that even better results would be obtained by combining Bregman iteration and a more sophisticated regularization method that can suppress noise in LR images and ringing artifacts occurred during capturing the details of the HR image.
A new regularization method based on multiscale morphologic ﬁlters are proposed,which are nonlinear in nature. Morphological operators and ﬁlters are well-known tools that can extract structures from images. Since proposed morphologic regularization term uses nondifferentiable max and min operators, developed an algorithm based on Bregman iterations and the forward–backward operator splitting using subgradients. The results produced by the proposed regularization are less affected by aforementioned noise evolved during the iterative process.

2. PROBLEM FORMULATION

The observed images of a scene are usually degraded by blurring due to atmospheric turbulence and inappropriate camera settings. The LR images are further degraded because of down sampling by a factor determined by the intrinsic camera parameters. The

International Journal of Scientific & Engineering Research, Volume 4, Issue 9, September-2013 1974

ISSN 2229-5518

relationship between the LR images and the HR image
can be formulated as
Yk = DFHk X + ek , k =1,2,…K ( 1)
where Yk , X, and ek represent lexicographically ordered column vectors of the kth LR image of size M, HR image of size N and additive noise, respectively. F K is a geometric warp matrix and H K is the blurring matrix of size N×N incorporating camera lens/CCD blurring as well as atmospheric blurring. D is the downsampling matrix of size M × N and k is the index of the LR images. Assuming that the LR images are taken under the same environmental condition and using same sensor, H K becomes the same for all k and may be denoted simply by H. the LR images are related to the HR image as
Yk = DFk HX + ek , k =1,2,…K ( 2)
Since under assumption, D and H are the same for all LR images, avoid downsampling and then upsampling at each iteration of iterative reconstruction algorithm by merging the upsampled and shifted-back LR images Yk together. After applying upsampling and reverse shifting, Yk will be aligned with HR image X. Suppose Yk denotes the upsampled and reverse-shifted kth LR image obtained through reverse effect of DFK of (2). That means Yk = F -
1 D T Y K , where D T is the upsampling operator matrix of
size N × M and is an N × N matrix that shifts back (reverse effect of F K ) the image. The equation from (2) as,

F k - 1 D T Y k = F k - 1 DT DF k H X + F - D T e k , k = 1, 2,..., K

i.e., Y k = R k H X + ÿ, k =l ,2,...,K
where R k = F k - 1 D T DF k captures the contribution of pixels of the blurred image HX from which the LR image Yk is generated. In this formulation, purely integervalued translational shifts are chosen with respect to the HR grid. F K has only 0 and 1 entries. Since D is just a downsample matrix applied on F K , R K has only 0 and 1 entries. For a more general formulation, if incorporate rotational shifts as well in our motion model and/or if use bilinear interpolation for noninteger translational shift, R K has real entries in the range [0, 1] and it is called as weight matrix. Assume a purely integer-valued translational shift and R K is called an index matrix.
Then the relation between the HR and LR images can be
rewritten as
Y= R HX + e (3) Regularization has used in conjunction with
iterative methods for the restoration of noisy degraded images in order to solve an ill-posed problem and prevent over-fitting. Then the SR image reconstruction can simply be formulated as
X = a r g min { Ɣ (X) : ║ RHX− Y║2 2 < η }
Where η is a scalar constant depending on the noise
variance in the LR images.

3. MORPHOLOGIC REGULARIZATION

Let B be a disk of unit size with origin at its center and S B be a disk structuring element (SE) of size s. Then the morphological dilation D S (X) of an image X of size M X N at scale S is defined as

where ( S B) ( I ) is a set of pixels covered under SE S B translated to th -th pixel X I . Similarly, the morphological erosion E S (X) at scale S is defined as
Morphological opening O S (X ) and closing C S (X )

by SE S B are defined as follows:
In multiscale morphological image analysis ,the difference between the sth scale closing and opening extracts noise particles and image artifacts in scale S and may be used for denoising purposes.

4. SUBGRADIENT METHODS AND

BREGMAN ITERATION

Bregman iteration is used in the field of computer vision for finding the optimal value of energy functions in the form of a constrained convex functional. In constrained and unconstrained problems , the “fixed point continuation” (FPC) method is used to solve the unconstrained problem by performing gradient descent steps iteratively. The linearized Bregman algorithm is derived by combining the FPC and Bregman iteration to solve the constrained problem in a more efficient way. An algorithm is developed on Bregman iteration and the proposed morphologic regularization for the SR image reconstruction problem.
4.1.Bregman Iteration
The proposed penalized splitting approach and corresponds to an algorithm whose structure is characterized by two-level iteration. There is an outer loop, which progressively diminishes the penalization parameter λ in order to obtain the convergence to the global minimum, and an inner loop, which iteratively,

International Journal of Scientific & Engineering Research, Volume 4, Issue 9, September-2013 1975

ISSN 2229-5518

using the two-step approach, minimizes the penalization
function for the given value of λ .
The general scheme of the bound constrained algorithm
is the following.

Algorithm:

Given F(∙), y, Ts , β>0 γ>0, 0<r<1, Toll ≥0, λmin and λ0
such that 0< λmin≤ λ0.
Set k=0, ụ0,0 =0 and λ0,0 = λ0
. Step A-1: Start with the outer iterations while (λ k,0 > λ min and ║Ts ụk,0− y║2 > Toll) Step B-0: Start with the inner iterations
i = 0;

Step B-1: Updating Step:

υ k,i = ụk,i+ β TS T(y-TS ụk,i )

Constrained Nonlinear filtering Step:

ụk,i+1 =arg minụεC {1/2 λk,i β║ụ− υ k,i ║2 2 +F(ụ)}

Convergence Test:

If F(ụk,i+1 ) – F(ụk,i )/F(ụk,i+1 ) ≥ γ λ k,i
i=i+1
λk,i = λk,i-1 go to Step B-1 otherwise go to Step A-2.
Step A-2: Outer Iteration Updating
k=k+1
λk,0 = r ∙ λk-1,i+1 ụk,0 = ụk-1,i+1 endwhile
Terminate with ụk,0 as an approximation of x
The automatic stopping criterion of the outer loop depends upon which problem we are considering. If we want to recover an exactly sparse gradient image from noisy-free acquisitions the parameter Toll can be set to 0, and with min λ of the order of the machine precision, we should obtain a numerically exact reconstruction. On the other hand, if we deal with compressible images or noisy data the stopping rule is governed by Toll. Note that there is considerable freedom in the proposed algorithm. In fact, both the orthogonal transform T and the sparsity inducing norm F(.),F(u) represents the total variation of the image and Ts is the linear parameter .

5. EXPERIMENTAL RESULTS

The performance of the proposed method is good,compared to existing SR reconstruction methods. The proposed algorithm is referred to as “Breg + Morph”, this is compared with the existing technique “Breg + TV”.
A typical 512× 512 gray-level HR image is chosen
. Some LR images are generated from this HR image and later reconstructed an HR image from these generated LR images as shown in fig.1.(a)-(b).The plots of proposed
method i.e “Breg+Morph” and existing technique
“Breg+TV” are shown in fig.1.(c)-(d).

The peak signal-to-noise ratio (PSNR) and SSIM is computed, as the quantitative measures of quality of reconstructed HR image with respect to the original HR image. The blurring is chosen as a 5 × 5 Gaussian smoothing kernel with scale parameter σ = 2.5 and the matrix H is formed accordingly. The downsampling factoris chosen to be 5 and the matrix D is constructed. An interesting portion is marked on resultant HR image and is displayed on the top-right/left corner after zooming it for careful study of the visual quality. The model parameters are chosen as γ = 1, μ = 0.5 and β is chosen as the reciprocal of the noise variance. The algorithm is implemented using MATLAB 7.6 and run on a regular Desktop with a 3-GHz Intel quad-core processor and 8 GB of RAM. Hence, the proposed method can achieve better quality with less number of iterations.
(a)Original HR image (b) Synthesized LR image

(c)Breg+TV (d) Breg+Morph
Fig.1. Results of the various SR image reconstruction methods with a small amount of noise (σ = 2). (a) Original HR image of a chart. (b) One of the degraded LR images. (c)–(d) SR reconstructed image using the Bregman iteration method with TV and morphologic regularization, respectively.
The proposed SR reconstruction gives comparable results with the existing methods in terms of both PSNR and SSIM. A more systematic study of performance of different algorithms for different amounts of noise and the blurring parameter is conducted. In Fig.3.(a)-(b), the average PSNR and SSIM are ploted for proposed and Breg+TV methods. In this,

International Journal of Scientific & Engineering Research, Volume 4, Issue 9, September-2013 1976

ISSN 2229-5518

different amounts of Gaussian noise (standard deviation σ = 0 to 10) is added to LR images and applied various SR reconstruction algorithms.This is done on a set of images, and then the average PSNR and average SSIM are plotted.

(a)
(b)

Fig.2. Analysis of the performance of SR image reconstruction algorithms applied on different gray images and then average quantitative measures are plotted. (a)–(b) PSNR and SSIM of SR algorithms for noisy LR images with additive Gaussian noise.
(b)
Fig. 3. Comparison of reconstruction qualities of Breg+TV and proposed methods versus the number of iterations for the experiment in Fig. 2. (a) Illustration of how residue of data fidelity term approaches the threshold value to terminate the Algorithm 1.(b) PSNR up to 100 iterations for different algorithms as indicated in (a).
In Fig. 3(a), we plot how Breg+TV and Breg+Morph algorithms approach the terminatingcondition. Table I shows the number of iterations, and fig. 1 shows the corresponding numerical time comparison of the results. We can observe that the morphologic regularization yields SR reconstructed image of better quality compared to other regularization methods with less number of iterations.in fig. 3(b) we plot the objective function of Breg + TV and Breg + Morph up to the number of iterations as given in Table I. This figure shows that, after initial irregularities, the values of the objective function show an overall a decrease. In fig. 3(c), we plot improvement in PSNR of the estimated image X(n) versus n for different SR reconstruction methods. These two plots, i.e., fig. 3(a) and (c), together showthe reconstruction qualities of different methods with number of iterations. Hence, with a chosen bound on residue as the stopping criterion for SR algorithms, the proposed method can achieve better quality with less number of iterations.
The proposed method extended by computing Blocking effect, Homogeneity and ISNR. The discontinuity between the adjacent blocks of image is called blocking effect, here the discontinuity is calculated. The similarity between the pixels is calculated by homogeneity. Improvement in signal to noise ratio is calculated by ISNR,it is given by

(a)
Where j,i are total number of pixels in horizontal and
vertical dimensions of image. f(i,j),y(i,j) and g(i,j) are represents original, degraded and restored images.
The below table gives the comparision of number of iterations and the process time for different existing methods.so,from the table it is cleared that the proposed

International Journal of Scientific & Engineering Research, Volume 4, Issue 9, September-2013 1977

ISSN 2229-5518

method is better than other methods because the number
of iterations are less for proposed method.
TABLE I
TIME COMPARISON OF DIFFERENT METHODS

METH OD

Grd

Grd+

BTV

Grd+

LABTV

Breg

+TV

Breg+ Morp h

ITERAT

-IONS

221

211

308

TIME

(s)

19.7

32.47

290.48

6.83

12.45

CONCLUSION

This paper presented an edge-preserving SR image reconstruction problem as a deblurring problem with a new robust morphologic regularization method. Then put forward two major contributions. First, proposed a morphologic regularization function based on multiscale opening and closing, which could remove noise efficiently while preserving edge information. Next, employed Bregman iteration method to solve the inverse problem for SR reconstruction with the proposed morphologic regularization. It is known that multiscale morphological filtering can reduce noise efficiently, so a successfully regularization method is used based on multiscale morphology. The experimental results shows that it works quite well, in fact better than existing methods. Nonlinearity of the regularization function is handled in a linear fashion during optimization by means of the subgradient and proximal map concept. The morphologic regularization method proposed here was tested only on SR reconstruction problem, this method is extended by computing Blocking effect, Homogeneity and ISNR.
sampled measured images,” IEEE Trans. Image Process.,
vol. 6, no. 12, pp. 1646–1658, Dec. 1997.
[4] H. Shen, L. Zhang, B. Huang, and P. Li, “A MAP approach for joint motion estimation segmentation and super resolution,” IEEE Trans. Image Process., vol. 16, no.
2, pp. 479–490, Feb. 2007.
[5] W. T. Freeman and T. R. Jones, “Example-based super resolution,” IEEE Comput. Graphics Appl., vol. 22, no. 2, pp. 56–65, Mar. 2002.

[6] J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process., vol. 19, no. 11, pp.2861–2873, Nov. 2010.

[7] K. I. Kim and Y. Kwon, “Single-image super-
resolution using sparse regression and natural image prior,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no.
6, pp. 1127–1133, Jun. 2010.
[8] W. Dong, L. Zhang, G. Shi, and X. Wu, “Image deblurring and superresolution by adaptive sparse domain selection and adaptive regularization,” IEEE Trans. Image Process., vol. 20, no. 7, pp. 533–549, Jul. 2011. [9] M. Protter, M. Elad, H. Takeda, and P. Milanfar, “Generalizing the nonlocal-means to super-resolution reconstruction,” IEEE Trans. Image Process., vol. 18, no. 1, pp. 36–51, Jan. 2009.
[10] H. Takeda, P. Milanfar, M. Protter, and M. Elad,
“Super-resolution without explicit subpixel motion
estimation,” IEEE Trans. Image Process., vol. 18, no. 9, pp.
1958–1975, Sep. 2009.

ACKNOWLEDGMENT

The authors would like to thank Associate Editor B. Wohlberg for his constructive criticism and valuable suggestions to upgrade this paper to its present form.

REFERENCES

[1] S. Lertrattanapanich and N. K. Bose, “High resolution image formation from low resolution frames using Delaunay triangulation,” IEEE Trans. Image Process., vol.
11, no. 12, pp. 1427–1441, Dec. 2002.
[2] A. J. Patti and Y. Altunbasak, “Artifact reduction for set theoretic super resolution image reconstruction with edge adaptive constraints and higher-order interpolants,” IEEE Trans. Image Process., vol. 10, no. 1, pp. 179–186, Jan.
2001.
[3] M. Elad and A. Feuer, “Restoration of a single super- resolution image from several blurred, noisy and under-