International Journal of Scientific & Engineering Research, Volume 5, Issue 12, December-2014 44
ISSN 2229-5518
Different Image Registration Methods – An Overview
Ms. Ritu Singh Phogat, Mr. Hardik Dhamecha, Dr. Manoj Pandya , Mr. Bharat Chaudhary , Dr. Madhukar Potdar
Abstract The main aim of this paper is to provide a review of classic image registration methods. Registration is a fundamental stage in 3D and 2D reconstruction process, which is used to match two or more images that can be taken in different moments, from different sensors, or different viewpoints. Two images are aligned into a common coordinate system in order to monitor the subtle changes between them. Registration algorithms compute transformations to set correspondence between the two images. Previous surveys of image registration divide the field algorithmically, focussing on how registration is accomplished. This approach provides a good basis for classifying different techniques, however it does little to illuminate the conditions important to the problem of registration. Understanding the image registration problem space and the conditions under which specific algorithms perform best allows for further automation of image registration, and makes the field more accessible to non-experts.
Index Terms— Automatic Image Registration, Feature Detection, Matching, Mapping Function, Resampling, Transformation
—————————— ——————————
In image processing, for example, when combining the information content of image, we are interested in the relationship between two or more images. The analysis of this relationship usually be easily controlled or dealt with once a correspondence is set up between the images. The task of setting up this correspondence is called image registration. Image registration is basically of two types:
1. Image to image registration: Here two or more images
are aligned to integrate or fuse corresponding pixels that
represent the same objects.
2. Image to map registration: Here input image is warped to match the map information of a base image, while retaining the original spatial resolution. Image registration is the process of calculating spatial transforms which align a set of images to a common observational frame of reference, often one of the images in the set. Registration is a key step in any image analysis or understanding task where different sources of data must be combined. It is a critical component of computational photography [1] remote sensing [2], [3], automatic manufacturing processes, and medical image processing [4], [5]. More recently it has been used to create navigable models of a scene from a database of photographs [6], to remove unwanted objects from overlapping images, etc. During registration process two situations become evident. First, it is impossible to
————————————————
• Ritu Singh Phogat, Masters Student in Electronics And Communication, MEFGI College, Rajkot.
• Hardik Dhamecha, Asst.Professor in EC Department, MEFGI College,Rajkot
• Bharat Chaudhary and Manoj Pandya,(project scientist), Dr. Madhukar
Potdar, project diretor in BISAG, Gandhinagar.
determine which of the coordinate system points of one image matches with the corresponding points of another
image, this is known as matching problem, and is also the most time consuming stage during the execution of algorithm. Second, there is need of transformation in the three dimensional information of one of the images regarding its coordinate system and its relationship to the image that was chosen as its reference. In simple words, image registration process is an automatic or manual procedure which tries to find corresponding points between two images and align them spatially to minimize a desired error, that is, a consistent distance measure between two images. There is a rapid development in image acquisition devices during last few decades and also growing amount and diversity of images that are obtained has invoked the research on automatic image registration. The intention of our article is to cover relevant approaches introduced and in this way map the current development of registration techniques. Methods published before 1992 that became classic or that introduced key ideas, which are still in use, are included as well to retain the continuity and to give complete view of image registration research. We want to summarize main approaches and point out interesting parts of the registration methods instead of going into the details of particular algorithms or describe some results of comparative experiments.
Past image registration surveys have provided a methodological taxonomy for understanding the different algorithms used to solve various registration problem. The work of Barbara Zitova and Jan Flusser [7] divides the field into two groups which are area and feature based methods. Area based methods are preferably applied when in images prominent details are absent and distinctive information is provided by grey levels / colours rather by local shapes and structure. Feature based matching methods are applied when there is more local structural information carried by image intensities. These methods make use of image features that are derived from feature extraction algorithm.
IJSER © 2014 http://www.ijser.org
International Journal of Scientific & Engineering Research, Volume 5, Issue 12, December-2014 45
ISSN 2229-5518
Point of sharp variations such as edges, corners, contours, surfaces, point of intersection etc, what carries valuable information about images are used for matching. Their model includes four steps of image registration, which are: Feature detection, Feature matching, Mapping function design, and Image transformation and resampling. They provide a model of variation according to the manner in which the images were acquired which include different viewpoints, different times / conditions, different sensors, and finally scene to model registration. Within their survey they do not use this mapping directly; however in many cases they discuss the type of problem each method has been designed to solve, allowing a similar mapping of methodology from situation. The four basic steps of image registration are as follows:
• Feature detection: The Salient and distinctive
objects (Closed boundary regions, Edges,
Contours, Line intersections, Corners, etc) in both reference image and sensed image are detected automatically or manually by a domain expert.
• Feature matching: The correspondence between the features in the reference and sensed image is established. Matching method is based on image content or on the symbolic description of control point-set.
• Transform model estimation: The parameters and type of the so-called mapping functions, aligning the sensed image with the reference image, are estimated.
• Image resampling and transformation:
By means of the mapping functions the sensed image is transformed.
FIG 1: STEPS IN IMAGE REGISTRATION PROCESS
In this method cross-correlation statistical approach is used for registration. It is often used for template matching or
pattern recognition in which the location and orientation of a template or pattern is found in picture. Cross correlation is a similarity measure or match metric. For template T and image I, where T is small compared to I, the two dimensional normalized cross-correlation function measures the similarity for each translation. If template matches the image, then the peak of cross correlation will be at C (i, j). Since local image intensity would influence the measure so the cross correlation should be normalized.
The flatness of the similarity measure maxima (due to the self-similarity of the images) and high computational complexity are the major drawbacks of correlation methods. By pre-processing or by using the edge or vector correlation the maximum can be sharpened effectively.
• Computing features in the images.
•Control points in reference image are corresponded with feature points in data image.
• Spatial mapping.
Control points for point matching play an important role in
this method. Control points may be corners, line of intersections, points of locally maximum curvature on contour lines, centre of windows having locally maximum curvature and centre of gravity of closed-boundary regions.
Feature based registration methods overcomes the limitation of correlation method and are often in use, particularly because of their easy hardware implementation, which makes them useful for real-time applications. Disadvantage of this method refers to the remarkableness of the window content. There are high chances that a window containing a smooth area without any crucial details will be matched incorrectly with other smooth areas in the reference image due to its non-saliency.
IJSER © 2014 http://www.ijser.org
International Journal of Scientific & Engineering Research, Volume 5, Issue 12, December-2014 46
ISSN 2229-5518
The features for registration should be preferably detected in distinctive parts of the image. Windows may not have this property, whose selection is often not based on their content evaluation.
For matching image feature point’s high statistical features are used in this method. For extracting regions of interest from the image, colour image segmentation is used. For example, for a given set of colour mean is calculated which is denoted as “m”. For segmentation process each RGB pixel in an image is classified as having a colour in specified range or not. Also the Euclidean distance is required to measure similarity. Let ‘z’ be an arbitrary point in RGB space and ‘T’ be the threshold. Euclidean distance between ‘z’ and ‘m’ is given by as shown below:
D (z, m) =║z-m║= [(z-m)T(z-m)]1/2
The locus of points is the sphere of radius T and point that is contained within, or on the surface of the sphere satisfy specified colour criterion. Coding these two sets of points in the image with black and white produces a binary, segmented image. Noise is removed after segmentation process using Gaussian filter. Thresholds blurred the image and then obtain the contour of an image.
Feature based methods do not use the gray values for matching and hence overcomes the limitations of spatial methods. Feature based method filter out the redundant information. Accuracy of this method is more but the limitation is that it is manual and slow.
Registration of multispectral / multisensory images is a challenging area. In general such images have different gray level characteristics and simple techniques such as those based on area correlation cannot be applied directly. This section is an attempt to solve this difficult problem by employing a basic concept from information theory. According to Manjusha Deshmukh and Udhav Bhosle [11], it is found that entropy of image does not change even if histogram has changed. Even after randomly shuffling pixels of image, entropy of image remains same. Also they concluded that natural images contain less uncertainty. In this type of images pixel intensity values depend on neighbouring pixels. In other words, in natural image the value of pixel is likely to be very close to some of its neighbours. Hence this dependency reduces the total entropy [10].
It is observed that proposed method yields a more accurate registration than any other registration method. But this
method has its own limitations. When images are of low resolution, when images contain little information, or when the region of overlap is small then mutual information result in mis-registration .It has one more limitation of speed , when time is an important constraint one cannot use this method . Although it has some limitations entropy and mutual information are best approaches for multimodal image registration.
It is observed that in frequency based method accuracy is more than correlation method but less as compared to other methods. But if we extract image features and then apply Fourier method accuracy increases. In frequency domain it should be noted that some form of interpolation must be used.
John Holland and his colleagues formally introduced genetic algorithms (GAs) in paper [12] as mentioned in reference. GA’s are based on the natural concept of evolution, suggesting that diversity helps to ensure a population survival under changing environmental conditions. Genetic algorithm (GA) is used to match the corresponding features and fit the satellite image on the vector map by optimizing the transformation accuracy on checked and control points. GA’s have intrinsic parallelism and are iterative procedures. They maintain a population of P candidate solutions encoded in the form of chromosome string of some alphabets, typically the binary symbols "0" and "1". The initial population can be selected heuristically or randomly. A fitness function, representing the quality of each candidate solution, is evaluated for each generation. Fit candidates, those with fitness value higher than certain threshold, will be selected for the reproduction in the next generation. The selected candidates are combined using a
IJSER © 2014 http://www.ijser.org
International Journal of Scientific & Engineering Research, Volume 5, Issue 12, December-2014 47
ISSN 2229-5518
genetic reproduction operation called “crossover”. The crossover operator exchanges portions of the chromosome to produce better candidates with higher fitness in the next generation. It is necessary to perturb the string of the chromosome to guarantee that the chances of searching a particular subspace of the problem space is never zero and for this task mutation operator is then applied. This prevents the algorithm from becoming trapped in local optima [13], [14]. The whole population is evaluated again in the next generation. The process continues until a termination criterion is reached. The termination criteria may be finding an acceptable approximate solution, reaching a specific number of generations, or convergence of the solution.
FIG 2: CONCEPT OF GENETIC ALGORITHM
According to F. Samadzadegan and M. Hoseini [15] automatic registration using genetic algorithm has proved to be very efficient and reliable for satellite imageries based on digital vector maps. The implemented methodology has the following characteristics:
• Utilization of a multi-resolution representation of information and mathematical models.
• Employing a Genetic algorithm for conjugate feature identification and modelling.
4-D manifolds in a Riemannian space (referred to as embedded maps). Registration is performed as a surface evolution matching one embedded map to another using a diffusion process. The approach differs from those existing in that it takes an a priori estimation of image regions where topological changes are present, for example lesions, and generates a dense vector field representing both the shape and intensity changes necessary to match the images. The algorithm outputs both a diffeomorphic deformation field and an intensity displacement which corrects the intensity difference caused by topological changes. Multiple sets of experiments are conducted on magnetic resonance imaging (MRI) with lesions from OASIS and ADNI datasets. These images are registered to either a brain template or images of healthy individuals. An exemplar case registering a template to an MRI with tumour is also given. The resulting deformation fields were compared with those obtained using diffeomorphic demons, where topological changes are not modelled. These sets of experiments demonstrate the efficacy of our proposed REM method for registration of brain MRI with severe topological differences.
Maintz [8] provides insight into the use of registration in medical imaging, providing important methods and variations relevant to that field. The taxonomy divides both algorithmically and based on the modality of the data, again providing a similar mapping. Pluim et al. [9] also survey medical imaging, focusing on Mutual-Information- Based registration techniques. Their taxonomy classifies algorithms into two main categories: methodological aspects and aspects of application. E. Parmehr, C.Fraser, C. Zhang, J. Leach[19] has presented a new hybrid intensity- based approach that utilizes both statistical and functional relationships between images, particularly in the case of registering aerial images and 3D point clouds. An important advantage of LiDAR systems is direct acquisition of 3D coordinates of ground objects. However, LiDAR data can have difficulty revealing information along break lines.
In this paper we have categorized different methods/techniques for image registration, their advantages and also how they are different from other methods has been included. When we register the images with local geometric distortions, nonlinear we come across two basic problems. One of them is how to match control points while the second problem is how to choose mapping
IJSER © 2014 http://www.ijser.org
International Journal of Scientific & Engineering Research, Volume 5, Issue 12, December-2014 48
ISSN 2229-5518
functions for registration as sometimes it is difficult to discriminate the image deformation and real changes
occurred in the scene. An automatic matching method
cannot be used for matching of control points (first problem) as the deformation between images is arbitrary (Zitova and Flusser, 2003). In the future, we need to develop an expert image registration method from a combination of various approaches which takes care of the type of the given task and provides an appropriate solution. In pixel based method cross correlation is used as similarity measure. It is observed that in natural images like buildings or scenery, correlation method shows match at multiple points. The feature based method makes use of features like point of intersection, edges, corners, centers of contours etc. for matching sample template with reference image. But this method is manual and hence time consuming. The method combining image features with correlation method have many advantageous properties of both feature-based and intensity based. It overcomes the limitation of intensity based method. Contour based methods do not use the gray values for matching and hence overcomes the limitations of spatial methods. Feature based method filter out the redundant information. Accuracy of this method is more but the limitation is that it is manual and slow. In frequency based method accuracy is more than correlation method but less as compared to other methods. But if we extract image features and then apply Fourier method accuracy increases. In frequency domain it should be noted that some form of interpolation must be used. These are some of the conclusions about methods used for registration of images which are in same spectral band. Image registration is difficult when images are obtained through different sensor types. It is observed that method based on information theory yields a more accurate registration than any other registration method. But this method has its own limitations. When images are of low resolution, when images contain little information, or when the region of overlap is small then mutual information result in mis-registration. It has one more limitation of speed, when time is an important constraint one cannot use this method. Although it has some limitations entropy and mutual information are best approaches for multimodal image registration. Polynomial transformation was mentioned as a major tool for transforming all tie points from image space to object space. The proposed algorithm was extensively tested on different brain MR image datasets carrying minor to severe pathology. The registration results are compared with those obtained from diffeomorphic demons, which demonstrate the effectiveness of our proposed algorithm in terms of converging to correct registration in the presence of lesion.
The authors wish to thank T.P. Singh, Director, Bhaskaracharya Institute of Space Application and Geo- Informatics for his valuable support and inputs in the research work.
[1] A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, and M. Cohen, “Interactive digital photomontage,” in SIGGRAPH ’04: ACM SIGGRAPH 2004 Papers, New York, NY, USA, 2004, pp. 294–302.
[2] T. M. Lillesand and R. W. Kiefer, Remote Sensing and Image
Interpretation, 6th ed. Wiley, 2007
[3] J. B. Campbell, Introduction to remote sensing, 4th ed. Guildford
Press, 2008.
[4] J. Maintz and M. Viergever, “A survey of medical image registration”, Medical Image Analysis, vol. 2, no. 1, pp. 1–36, 1998
[5] J. Pluim, J. Maintz, and M. Viergever, “Mutual-informationbased registration of medical images: a survey”, Medical Imaging, IEEE Transactions on, vol. 22, no. 8, pp. 986–1004, Aug. 2003. 313
[6] N. Snavely, R. Garg, S. M. Seitz, and R. Szeliski, “Finding paths through the world’s photos,” ACM Transactions on Graphics (Proceedings of SIGGRAPH 2008), vol. 27, no. 3, pp. 11–21, 2008
[7] B. Zitova and J. Flusser, “Image registration methods: a survey,” Image and Vision Computing, vol. 21, pp. 977–1000, 2003.
[8] J. Maintz and M. Viergever, “A survey of medical image registration,” Medical Image Analysis, vol. 2, no. 1, pp. 1–36, 1998
[9] J. Pluim, J. Maintz, and M. Viergever, “Mutual-informationbased registration of medical images: a survey,” Medical Imaging, IEEE Transactions on, vol. 22, no. 8, pp. 986–1004, Aug. 2003.
[10] Mert Rory Sabuncu, “Entropy-Based image registration, Phd thesis.
[11]Manjusha Deshmukh and Udhav Bhosle,”A Survey Of Image Registration”, International Journal of Image Processing (IJIP), Volume (5) : Issue (3) , 2011
[12] Holland, J., 1975. Adaptation of Natural and Artificial Systems, The University of Michigan Press, Ann Arbor.
[13] Goldberg, D.E., 1989. Genetic Algorithms in Search, Optimization
& Machine Learning, Addison-Wesley Longman
[14] Husband, P., 1990. Genetic Algorithms in Optimisation and Adaptation, Advances in Parallel Algorithms Kronsjo and Shumsheruddin ed., pp. 227-276, 1990
[15] Farhad Samadzadegan, Sara Saeedi, Mohammad Hoseini,” Automatic Image to Map Registration Based on Genetic Algorithm”, Proceedings of the 2nd WSEAS International Conference on Remote Sensing, Tenerife, Canary Islands, Spain, December 16-18, 2006
[16] Xiaoxing Li , Xiaojing Long, Paul Laurienti, Christopher Wyatt, and ALZHEIMER’S DISEASE NEUROIMAGING INITIATIVE,” Registration of Images With Varying Topology Using Embedded Maps”, IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 31, NO. 3, MARCH 2012
[17] S. Kullback, Information theory and statistics. Dover Publications, Inc., 1968.
IJSER © 2014 http://www.ijser.org
International Journal of Scientific & Engineering Research, Volume 5, Issue 12, December-2014 49
ISSN 2229-5518
[18] T.M. Cover and J.A. Thomas, Elements of information theory, John
Wiley & Sons, Inc., 1991
[19] Ebadat G. Parmehr, Clive S. Fraser, Chunsun Zhang, Joseph
Leach, “Automatic Registration of Aerial Images with 3D LiDAR Data
Using a Hybrid Intensity-Based Method”, 978-1-4673-2181-
5/12/$31.00 ©2012 IEEE
IJSER © 2014 http://www.ijser.org