International Journal of Scientific & Engineering Research, Volume 6, Issue 4, April-2015 1300

ISSN 2229-5518

Recognizing Surgically Altered Face Images

Using SIFT and LDA

Sajna S

Dr. Shyama Das Asitha B Raj sajna5252@gmail.com

IHRD College of Engineering, Chengannur

Abstract— we are currently witnessing a strong worldwide popular in plastic surgery. Plastic surgery is the process of reconstructing and repairing the part of the body by transfer of tissues. People take plastic surgery to correct feature defects and improve attractiveness. This paper focused on recognizing before and after plastic surgery face images. Nonlinear variation in plastic surgery is difficult task to recognize pre and post-surgery images. Linear Discriminant Analysis is proposed to match before and after plastic surgery images. The algorithm generates multiple face granules at three levels of granularity. Here SIFT is used as the feature extractor for extracting discriminating information from granules which uni fi ed i n LDA. The proposed algorithm yields high identification accuracy and less time complexity as compared to existing algorithms.

Index Terms— Scale Invariant Feature Transform (SIFT), Linear Discriminant Analysis (LDA)

——————————  ——————————

1 INTRODUCTION

LASTIC surgery is the process of repairing the part of the body by transfer of tissues to correct the feature anoma- lies. With reduction in cost and time the popularity of
plastic surgery is increasing.Plastic Surgery procedures are beneficial for a patient who is suffering from several kinds of disorders caused due to excessive structural growth of facial features or skin tissues .These procedures amend the facial features and skin texture thereby providing a makeover in the appearance of face [1].Even the widespread acceptability in the society encourages individuals to undergo plastic surgery for cosmetic reasons. According to the statistics provided by the American Society for Aesthetic Plastic Surgery for year
2010 [2], there is about 9% increase in the total number of cos- metic surgery procedures, with over 500,000 surgical proce- dures performed on face. Fig. 1 shows the effect of plastic sur- gery on facial appearances, variations in facial appearance, texture, and structural geometry.
There are two types of plastic surgeries available as ana- lyzed by Singh et al. [3]. They are local surgery and global surgery. Local plastic surgery deals with correcting jaw and teeth, nose structures of the face, as well as chin, forehead, eyelids [3]. Global plastic surgery deals with completely changing the facial structure which is known as full face lift. Such type of non-linear variations introduced by these plastic surgeries remains a great challenge for face recognition algo- rithms to deal with. As popularity of such plastic surgeries is increasing in today’s world due to affordable cost, yet there is no face recognition algorithm to address these variations effi- ciently [3].

Fig. 1. Illustrating the variations in facial appearance, texture, and structural geometry caused due to plastic surgery (images taken from internet).
Global plastic surgery completely transforms the face and is recommended in cases where functional damage is to be cured such as patients with fatal burns or trauma. In these kind of surgeries, facial appearance, skin texture, and feature shapes vary drastically thus making it arduous for any face recogni- tion system to recognize pre- and post-surgery faces [3]. Rhyt- idectomy (full face-lift) is used to treat patients with severe burns on face and neck. It can also be used to reverse the effect of aging and get a younger look, thus modifying the appear- ance and texture of the whole face[1]. Analogous to rhytidec- tomy, skin peeling procedures such as laser resurfacing and chemical peel alter the texture information thus affecting the performance of face recognition algorithms [1][3]. These pro- cedures are used to treat wrinkles, stretch marks, acne, and other skin damages caused due to aging and sunburns [1]. In local plastic surgery small variation between pre and post- surgery images features of the face remains similar to the orig- inal images. Singh et al. [3] analysed several types of local and global plastic surgery procedures and their effect on different face recognition algorithms. They have experimentally shown that the nonlinear variations introduced by surgical proce-

International Journal of Scientific & Engineering Research, Volume 6, Issue 4, April-2015 1301

ISSN 2229-5518

dures are difficult to address with current face recognition algorithms. De Marsico et al. [4] developed an approach to integrate information derived from local regions to match pre- and post-surgery face images. Recently, Aggarwal et al. [5] proposed sparse representation approach on local facial frag- ments to match surgically altered face images. Though recent results suggest that the algorithms are improving towards addressing the challenge, there is a significant scope for fur- ther improvement.

Fig 2: Block Diagram for proposed approach
Fig 2 shows block diagram of proposed system. Proposed algorithm starts with non-disjoint face granules where each granule represents different information at different size and resolution [1]. Further, feature extractor, Scale Invariant Fea- ture Transform (SIFT) [7].SIFT will provide diverse information from the face granules. LDA (Linear Discriminant analysis) [21] is used for recognizing altered faces due to plastic surgery.

2 RELATED WORK

2.1 Sparse Representation

Face recognition algorithms either use facial information or extract features and process them in the presence of variations such as pose, expression, illumination and disguise. Aggarwal et al. [5] proposed sparse representation approach on local facial fragments to match surgically altered face images. Sparse representation approach consists of the following steps:
• Localization of face and primary facial features
• Generation of training matrix
• Sparse recognition

2.2 Component-Based Recognition System

The global approaches and a component-based approach [9][10] to face recognition and evaluate their robustness against pose changes have presented. The global method con- sists of face detector which extracts the face from an input im-
age and propagates it to a set of SVM classifiers that perform the face recognition.

2.3 Local Binary Patterns

The LBP [11] operator was originally designed for texture description. The operator assigns a label to every pixel of an image by thresholding the 3x3-neighborhood of each pixel with the center pixel value and considering the result as a bi- nary number. Then the histogram of the labels can be used as a texture descriptor.

3 PROPOSED SYSTEM

Face Recognition algorithms either uses algorithms of two kinds, the algorithm which deals with the holistic image and the other type is the algorithm which extract features and pro- cess them as parts [3]. Both the inner and outer facial regions in face recognition are used in producing diverse information as observed by Campbell et al [13]. The observations of Singh et al. [3] states that surgical procedure if done, leads to change in more than one facial region. With large variations caused in the appearance, texture and shape of different facial regions, it has been a great challenge for the face recognition algorithms to match gallery image with the probe image. Surgically al- tered face matching pre as well as post-surgery images can be implemented using three different modules. They are:
• Face Granulation
• Feature Extraction
• LDA

3.1 Face Granulation

Let M be the face image of size n X m. Face granules are generated in three levels of granularity[14][15]. The first level provides global information at multiple resolutions [1]. Inner and outer facial information are extracted at the second level [1]. At the third level, features are extracted from the local fa- cial regions [1].
1. First Level of Granularity: In this level of granularity the face images are divided in to six granules using Gaussian and la- placian operators [16]. The Gaussian operator generates a low pass filtered images by iteratively convolving each of the con- stituent images with a 2-D Gaussian kernel [16].Similarly, the Laplacian operator [16] generates a series of band-pass images [16]. The first level of granularity thus compensates for the variations in facial texture, such as face-lift, skin resurfacing etc. [1].
2. Second level of granularity: Campbell et al. [13] proposed hori- zontal and vertical granules are generated in the second level of granularity. The second level of granularity provides resili- ence to variations in inner and outer facial regions such as
chin, forehead, ears, and cheeks [13].

International Journal of Scientific & Engineering Research, Volume 6, Issue 4, April-2015 1302

ISSN 2229-5518

3. Third level of granularity: In the third level of granularity, 16 local facial regions are extracted using the golden ratio face template [17]. Each of these regions is a granule representing local information that provides unique features for addressing variations due to plastic surgery [1].

Fig, 3 Horizontal face granules from the second level of granu- larity

Fig, 4 Third level of granularity
Fig, 5 Vertical face granules from the second level of granulari- ty

3.2 Feature Extraction

Three levels of granularity which provides face granules with varying information which in turn is used to extract facial features [1]. Some granules contain fiducial features such as eyes, nose, and mouth while some granules predominantly contain skin regions such as forehead, cheeks, and outer facial
region [1].Therefore, different feature extractors are needed to encode the information from the granules. Here Scale Invariant Feature Transform (SIFT) [7] is used as the feature extractor, which is fast, discriminating, rotation invariant, and robust to changes in gray level intensities due to illumination [1].

3.1.1 Scale Invariant Feature Transform (SIFT)

SIFT [7] is a scale and rotation invariant descriptor that generates a compact representation of an image based on the magnitude, orientation, and spatial vicinity of image gradients [1]. Scale Invariant Feature Transform (SIFT) features extract- ed from images to help matching between different views of the object. SIFT, as proposed by Lowe [7], is a sparse de- scriptor that is computed around the detected interest points. However, SIFT can also be used in a dense manner where the descriptor is computed around predefined interest points [1]. SIFT descriptor is computed in a dense manner over a set of uniformly distributed non-overlapping local regions of size 32
X 32. SIFT descriptors computed for the sampled regions are
then concatenated to form the image signature [1]. SIFT fea- tures are extracted in four steps:
• Creating the difference of gaussian pyramid
The first step is to construct a Gaussian "scale space" from the input image by convolution of the original image with Gaussian functions. The difference of Gaussian (DoG) [7] is calculated as the difference between two filtered images.one is k scale to other [7].
• Extrema detection
This stage is to find the extrema points in the DoG pyra- mid. To detect the local maxima and minima of DoG each point is compared with the pixels of all its 26 neighbors [7]. If this value is the minimum or maximum this point is an extre- ma [7].
• Key points elimination
This step eliminates some points from the list of keypoints with low contrast or is poorly localized on an edge [7].
• Orientation assignment
This step is to assign a consistent magnitude and orienta- tion to the key points. Using magnitude and gradient orienta- tion histogram is formed within a region around the key point [7].
• Descriptor computation

International Journal of Scientific & Engineering Research, Volume 6, Issue 4, April-2015 1303

ISSN 2229-5518

A keypoint descriptor is created by first computing the gradient magnitude and orientation at each image sample point in a region around the keypoint location [7]. These sam- ples are then accumulated into orientation histograms sum- marizing the contents over 4x4 subregions, the contents over
4x4 subregions [7]. 4x4 descriptors computed from a 16x16 sample array.
The SIFT features are computed at the edges and they are invariant to image scaling, rotation, addition of noise. Due to their distinctiveness, this enables the correct match for key- points between faces [7]. Every feature is a vector of dimen- sion 128 distinctively identifying the neighborhood around the key point [7].

3.3 Linear Discriminant Analysis

Linear Discriminant Analysis (LDA) [21] has been a popu- lar method for extracting features that preserves class separa- bility. LDA [21] perform dimensionality reduction. The high- dimensional data is mapped into a lower dimensional space. Every face granule has diverse information, which if combined together can provide discriminating information for face recognition [1]. Moreover, psychological studies in face recog- nition [12] have also shown that some facial regions are more discriminating than others and hence, contribute more to- wards the recognition accuracy. Certain feature selection methods are to select features which in turn produce diverse information for improving the performance [1]. Sequential feature selection (SFS) [19] and sequential floating forward selection (SFFS) [19] are widely used feature selection methods that evaluate feature set by sequentially adding or removing the features at a time [1]. Definitive feature selection approach concatenates different and performs dimensionality reduction using PCA to yield the final feature set [1]. Genetic algorithm [1] is used for dimensionality reduction. Genetic algorithms often fail to maintain diversity among individual solutions (chromosomes) and cause the population to converge prema- turely [1]. Certain optimistic problems cannot be solved by the genetic algorithm and take huge amount of time to execu- tion.These existing feature selection techniques do not yield high performance for face recognition. To overcome this prob- lem, LDA [21] is used as the dimensionality reduction. LDA is less time complexity and high identification accuracy.
Surgically altered face recognition involve data with a large number of features [22–24]. Analysis of such data is chal- lenging due to the curse-of dimensionality [25, 26], which states that an enormous number of samples are required to perform accurate predictions on problems with a high dimen- sionality. Dimensionality reduction, which extracts a small number of features by removing irrelevant, redundant, and noisy information, can be an effective solution [27]. The com- monly used dimensionality reduction methods include super- vised approaches such as linear discriminant analysis (LDA) [21]. LDA computes an optimal transformation (projection) by
minimizing the within-class distance and maximizing the be- tween-class distance simultaneously, thus achieving maximum class discrimination [21]. It has been widely used in many fields of information processing, such as machine learning, data mining, information retrieval, and pattern recognition [21]. Each granules contain discriminating information is used to extract features using SIFT [7] extractor. SIFT [7] provide large number of feature in each granules. LDA [21] perform dimensionality reduction. The high-dimensional features are mapped into a lower dimensional space and chi-square dis- tance [1] is used to compare two SIFT descriptors [1]. This ap- proach produce high identification accuracy and less time complexity compared with existing algorithm. This approach involves searching of large spaces and finding sub optimal solutions.

4 EXPERIMENTAL RESULTS

In the proposed method the image M is divided into face granules, SIFT is used to extract facial information, from the face granule. Linear discriminant analysis perform dimension- ality reduction .The high dimensional features are mapped to lower dimensional features. Compare the pre and post- surgery image features. Proposed system provides better ac- curacy and less time complexity than the existing system. Fig 6 shows the comparison of existing system and proposed sys- tem, which is more accurate than existing system.

Fig 6. Comparison of existing system and proposed system

5 CONCLUSION

The proposed system presents an efficient face recognition algorithm with plastic surgery. First collect the input images from before and after plastic surgery, perform the face recogni- tion algorithm for both images. The image M is first divided into face granules using three different levels of granularity; we get non disjoint face granules. In the first level of granulari- ty, granules are generated by Gaussian and Laplacian opera- tors to extract information from image pyramids. The second level of granularity divides the image into horizontal and ver-

International Journal of Scientific & Engineering Research, Volume 6, Issue 4, April-2015 1304

ISSN 2229-5518

tical face granules. The third level of granularity extracts dis- criminating information from local facial regions. Further, SIFT extractor is used to extract feature information from face granules from different levels of granularity. LDA perform dimensionality reduction and encoding discriminatory infor- mation for each face granule. Chi-square distance is used to compare two SIFT descriptors. Thus proposed approach of matching post-surgery images with pre surgery images pro- vides high identification accuracy and less time complexity when comparing with the existing recognition algorithms.

REFERENCES

[1] Recognizing Surgically Altered Face Images Using Multiobjective Evolutionary Algorithm Himanshu S. Bhatt, Student Member, IEEE, Samarth Bharadwaj, Student Member, IEEE,Richa Singh, Member, IEEE, and Mayank Vatsa, Member,IEEE

[2] American Society for Aesthetic Plastic Surgery 2010 Statistics 2010 [Online]. Available: http://www.surgery.org/media/statistic

[3] R. Singh, M.Vatsa, H.S.Bhatt, S. Bharadwaj, A. Noore, and S.

Nooreyezdan, “Plastic Surgery: A new dimension to face

recognition“, IEEE Trans, Inf Forensics Security, vol 5. No 3. Pp, 441-

448. Sep 2010.

[4] M. De Marsico, M. Nappi, D. Riccio, and H. Wechsler, “Robust face recognition after plastic surgery using local region analysis,” in Proc. Int. Conf. Image Analysis and Recognition, 2011, vol. 6754, pp. 191–200.

[5] G. Aggarwal, S. Biswas, P. J. Flynn, and K. W. Bowyer, “A sparse representation approach to face matching across plastic surgery,” in Proc.Workshop on the Applications of Computer Vision, 2012, pp. 1–7.

[6] H. S. Bhatt, S. Bharadwaj, R. Singh, and M. Vatsa, “On matching sketches with digital face images,” in Proc. Int. Conf. Biometrics:Theory Applications and Systems, 2010, pp. 1–7.

[7] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,”Int. J. Comput. Vis., vol. 60, no. 2, pp. 91–110, 2004.

[8] F. Vafaee and P. C. Nelson, “A genetic algorithm that incorporates an adaptive mutation based on an evolutionary model”, in Proc. Int. Conf. Machine Learning and Applications, 2009, pp. 101–107.

[9] B.Heisele, P. Ho, J. Wu, and T. Poggio, “Face recognition: component based versus global approaches”, Computer Vision and Image Understanding, vol. 91, pp. 6–21, 2003.

[10] B. Weyrauch, B. Heisele, J. Huang, and V. Blanz, “Component based face recognition with 3d morphable models”, in Proceedings of International Conference on Computer Vision and Pattern Recognition Workshop,2004,pp.85–91.

[11] Timo Ojala, Matti Pietikainen, Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns, IEEE Trans. Pattern Analysis and Machine Intellegence; 971-987; 2002

[12] ] P. Sinha, B. Balas, Y. Ostrovsky, and R. Russell, “Face recognition by humans: Nineteen results all computer vision researchers should know about,” Proc. IEEE, vol. 94, no. 11, pp. 1948–1962, Nov. 2006.

[13] R. Campbell, m. Coleman, J. Walker,P. J. Benson, S. Wallace, J.

Michelotti, and S. Baron-Cohen, “When does the inner-face

advantage in familiar face recognition arise and why?”, Vis. Cognit, vol.6,no. 2, pp. 197-216, 1999.

[14] A. Bargiela andW. Pedrycz, Granular Computing: An Introduction.

Boston: Kluwer Academic, 2002.

[15] T. Y. Lin, Y. Y. Yao, and L. A. Zadeh, Data Mining, Rough Sets and

Granular Computing. New York: Physica-Verlag, 2002.

[16] P. J. Burt and E. H. Adelson, “Amultiresolution spline with application to image mosaics,” ACM Trans. Graphics, vol. 2, no. 4, pp.

217–236, 1983.

[17] K. Anderson and P. W. McOwan, “Robust real-time face tracker for cluttered environments,” Comput. Vis. Image Understand., vol. 95, pp.

184–200, 2004.

[18] C. J. C. Burges. A tuturial on support vector machines for pattern recognition. Data mining and knowledge discovery, (submitted),

1998.

[19] A. K. Jain, R. P. W. Duin, and J. Mao, “Statistical pattern recognition: A review,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 1, pp.4–

37, Apr. 2000.

[20] T. Ahonen, A. Hadid, and M. Pietikainen, “Face description with local binary patterns: Application to face recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 12, pp. 2037–2041, Dec. 2006.

[21] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, Eigenfaces vs. fisherfaces: recognition using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.

[22] A. K. Jain, P. Flynn, and A. A. Ross. Handbook of Biometrics.

Springer, 2007.

[23] A. K. Jain and S. Z. Li. Handbook of Face Recognition. Springer- Verlag, 2005.

[24] A. K. Jain, A. A. Ross, and S. Prabhakar. An introduction to biometric recognition. .IEEE Transactions on Circuits and Systems for Video Technology, 14(1):4–20, 2004.

[25] R. E. Bellman. Adaptive Control Processes: A Guided Tour. Princeton

Universsity Press, 1961.

[26] C. J. C. Burges. Geometric methods for feature extraction and dimensional reduction. In Oded Maimon and Lior Rokach, editors, Mining and Knowledge Discovery Handbook: A Complete Guide for Practitioners and Researchers,pages 59–92. Springer, 2005.

[27] S. Yan, D. Xu, B. Zhang, H.-J. Zhang, Q. Yang, and S. Lin. Graph embedding and extensions: A general framework for dimensionality reduction. IEEETransactions on Pattern Analysis and Machine Intelligence, 29(1):40–51, 2007.