International Journal of Scientific & Engineering Research, Volume 6, Issue 2, February-2015 744

ISSN 2229-5518

A Survey on Face Recognition of Identical Twins

Kanchan Patil, Prof. Sachin Bojewar

Abstract - Recent studies have shown that face recognition performance degrades considerably for images of identical twins. Human face matching capability is often taken into consideration as a bench-mark for assessing and improving automatic face recognition algorithms. Here, this paper will show human capability to distinguish between identical twins. If humans are able to distinguish between facial images of identical twins, it would suggest that humans are capable of identifying discriminating facial traits that can potentially be useful to develop algorithms for this very challenging problem. If humans viewing a pair of facial images can perceive if the image pairs belong to the same person or to a pair of identical twins. The paper consists of experiments results, which are conducted on 186 twin subjects, making it the largest such study in the literature to date. And observation will show that humans can perform the task significantly better if they are given enough time and tend to make more mistakes when images differ in imaging conditions. The paper analysis also suggests that humans look for facial marks like moles, scars, etc. to make their decision and do worse when presented with images lacking such marks. Experiments with automatic face recognition systems show that human observers outperform automatic matchers for this task

Index Terms— siblings, facial images, Gabor wavelets, biometrics, iris, identical twins, HSIRB

1 INTRODUCTION

——————————  ——————————
Recognition of facial images of identical twin siblings pos- es a considerable challenge for any face recognition algorithm because of the strong similarity between the face images. Re- cent research has showed that the performance of automatic face recognition technology deteriorates drastically when the images belong to identical twin siblings as compared to when they correspond to unrelated persons [1]. The degradation is shown to be far more drastic for face than for other biometrics such as iris and fingerprint. Humans are very good at identify- ing people from their images, and so human face recognition performance is often considered as a guideline for assessing face recognition algorithms [2]. To the best of our knowledge, no systematic human study has been performed that address- es the task of distinguishing between identical twins from their face images. Here, we perform experiments to determine if humans viewing a pair of facial images can perceive wheth- er the images belong to the same person or to a pair of identi- cal twin siblings. If humans are able to distinguish between facial images of identical twin siblings, it might mean that they are capable of observing discriminating traits which can po- tentially be used to improve the performance of face recogni- tion technology. In this investigation, human participants view pairs of facial images and respond according to their level of certainty whether they belong to the same person or to identical twins. First, we study the human performance when

————————————————

• Kanchan Patil is currently pursuing master’s degree program in Computer engineering in Mumbai University, India.

E-mail: kanchan.patil218@gmail.com

• Prof. Sachin Bojewar, Asst. Prof., Dept. of Computer Engineering, Vidya- lankar Institute of Technology, Mumbai University, India.

E-mail: sachin.bojewar@vit.edu.in
the participants view the images for a limited time of two se- conds, which has been shown to be sufficient for matching
images of unrelated persons [2]. We conduct another experi- ment to analyse whether humans can do better when the viewing time is increased. The variation in performance when the input images are taken under controlled indoor conditions or in an outdoor environment is also analysed as part of the second experiment. We also study which facial features are most useful for humans to correctly distinguish between iden- tical twins. The human performance is also compared against traditional and commercial automatic face matchers. The re- sults of this investigation can be used to improve the perfor- mance of existing face recognition algorithms so that they are more suited to handle the challenges posed by facial images of identical twin siblings.

2 RELATED WORK

Since identical twins cannot be distinguish by their DNA, there is increased interest in using different biometrics traits for distinguishing between identical twins. Modelling facial expressions as isometric of the facial surface. A hybrid feature by combining the traditional holistic facial appearance feature with a facial dynamics feature has also been shown to be suc- cessful in distinguishing between facial images of one pair of identical twins [3]. A face recognition system based on an op- tical recognition principle was also shown to be successful in distinguishing between identical twin siblings for a database of ten pair subjects [4]. Soft biometric facial marks have been used to differentiate identical twins on a dataset which con- tained facial images from five pairs of identical twins [5]. Since they were tested on very small number of twin pairs, the con- clusion may not be statistically significant. Sun et al. [1] con- ducted unimodal and multimodal matching experiments on fingerprint, face and iris biometrics collected from 66 pairs of identical twins. They focused on issue that it is much easier to

International Journal of Scientific & Engineering Research, Volume 6, Issue 2, February-2015 745

ISSN 2229-5518

distinguish between identical twin siblings using iris and fin- gerprint biometric compared to using facial images. There has been some work on distinguishing between identical twins based on other biometrics like palm, fingerprint, iris [6], and speaker identification [8].
Humans are naturally trained to recognize faces from birth
and there is strong evidence that suggests face recognition
activity in humans takes place in the fusiform face area of the cortex. Thus there has been a lot of interest in developing al- gorithms which replicate the human visual processing for face recognition. For example, biologically inspired features in the form of Gabor wavelets have been successfully used for rec- ognizing faces. It has been also seen that the performance of automatic algorithms can be considerably improved by fusing it with human performance. Though quite a few human stud- ies have been conducted in the past to study various aspects of general face matching problem [2], to the best of our knowledge, there has been no systematic study of human abil- ity to distinguish between identical twins from their face im- ages.

3 PREVIOUSLY OBTAINED DATASET AND RESULTS

In this section, we will discuss the dataset used and the re- sults obtained through that experimental setup, and these set- up explained as, the participants who took part in the study and the experiment protocol.

3.1 Dataset:

The twins data used in these setup was obtained from data collection sessions at the Twins Days Festival in
Twinsburg, Ohio in August 2009 [10]. The dataset consists of
186 subjects, of which 34 are male and the remaining 152 are
female. The twins participating in the data collection report their identity as identical twins. No DNA testing was per- formed to confirm the claims. All data collected at the festival followed a data collection protocol approved by the Human Subjects Institutional Review Board (HSIRB) at the University of Notre Dame.

3.2 Participants:

A total of 23 volunteers were appointed to participate in the recognition experiment. They did not undergo any prior prac- tice or training for the work. The volunteers were offered ten dollars for participation, and an additional five dollars if they correctly classified 80% or more of the twin image pairs. The experiment was approved by HSIRB at the University of Notre Dame.

3.3 Experiment Protocol:

In these experiment, the participants were given a brief verbal description of the study, and asked to read and sign a form. Then they were asked to start a computer program that pre- sented instructions along with a few sample trials. This was followed by 180 trials out of which 90 trials corresponded to match pairs while the other 90 corresponded to non-match
pairs. The match and non-match pairs were separated and presented in a random order to each participant. The images used in this experiment were captured in an indoor environ- ment with controlled lighting, frontal pose and neutral expres- sion. The images were cropped based on the eye locations so that only the face portion was visible. This confirmed that the responses of the participants were not affected by external factors like clothing, hair style, etc. In each attempt, the com- puter program shown a pair of facial images followed by a prompt for a decision on whether they correspond to the same person or to identical twin siblings. The images were shown for two seconds. Then the participants were asked the follow- ing question:
Are the two images of the same person or of identical twin
siblings?
They were required to select one out of five possible respons-
es.
1) Sure they are the same person
2) Think they are the same person
3) Don’t know
4) Think they are identical twin siblings

5) Sure they are identical twin siblings

Fig. 1. An example of pair of displayed images. Here the images are of the same person

Fig. 2. An example of a pair of displayed images. Here the images are of identical twin siblings.

International Journal of Scientific & Engineering Research, Volume 6, Issue 2, February-2015 746

ISSN 2229-5518

Fig. 1 and Fig. 2 show examples of face images shown. The two images in Fig. 1 belong to the same person whereas the two images in Fig. 2 belong to identical twin siblings. Fig. 1. An example of a pair of shown images. Here the images are of the same person.

4. EXPERIMENTAL RESULTS OBTAINED AND ANALYSIS

In this section, we describe in detail the results obtained
from the experiment described above.

4.1 Are humans able to distinguish between identical

twins?

To find the overall accuracy, we count the number of times each participant correctly classified the pair of images to be of the same person or of identical twin siblings. For example, if the two images are of the same person, we consider the re- sponses Sure they are the same person and Think they are the same person as correct responses. Similarly, if the two images are of identical twin siblings, we consider the responses Sure they are identical twin siblings and Think they are identical twin siblings as correct responses. Across the 23 participants, the maximum accuracy attained is 90.56% and the minimum accuracy is 60.56%. The average accuracy is 78.82%. We use a one-tailed t-test to evaluate the null hypothesis that humans did not perform better on this work than random guessing. Thus, this produced statistically significant evidence that the participants performed better than random. The Receiver Op- erating Characteristic (ROC) of the performance is shown in Fig. 3 (blue dotted curve). The ROC provides a complete pic- ture of human accuracy in distinguishing between identical twins at the assessed confidence threshold levels (1 through 5) and is drawn using the same procedure as in [2]. The verifica- tion rate or hit rate is computed as the proportion of matched pairs correctly judged to be of the same person. The false ac- ceptance rate or false alarm rate is calculated as the proportion of non-match pairs judged incorrectly to be of the same per- son.
Fig. 3. Human performance when the image pairs are viewed for limited time vs. unlimited time

4.2 Do humans perform better when they are certain?

As mentioned previously, the participants had the choice to respond according to their level of confidence. For example, if the participant felt that the two images belonged to the same person, they could choose either Sure they are the same person or Think they are the same person, depending on their confidence level. It shows that the confidence level varies significantly across the participants. On one hand, one of the participants was certain for 118 out of 180 trials, while on the other hand, three participants were not certain of any of their responses. The average number of certain responses across all participants was 60 out of 180 trials. Considering the trials for which the participants were certain about their response, the average accuracy of correct classification of the image pairs as belonging to the same person or to a pair of identical twins is
93.12%. Thus the performance is significantly better on the
subset of trials where the participants were certain about their response.

4.3 Self-learning

The participants who volunteered for the study did not undergo any prior training to classify images of identical twins and none of them had an identical twin sibling. Here it is analyze whether the participants can learn by themselves the subtle differences between the facial images of identical twins. The average improvement in the performance in the second half of the try as compared to the first half is 1.5%. Out of the 23 participants, 14 performed better in the second half, while only seven performed better in the first half. This improvement might mean that as the participants viewed more images of twins, they trained themselves and performed better in the second half of the trials. A one-tailed t-test shows that the difference is not statistically significant (p-value

International Journal of Scientific & Engineering Research, Volume 6, Issue 2, February-2015 747

ISSN 2229-5518

0.3065). In this study they mentioned, the participants did not receive any feedback after each visual stimuli whether their response was correct or not. Providing feedback on their response could have helped the participants learn better and thus perform better in the second half as compared to the first half.

4.4 Are males more easy to classify than females?

Several researchers have studied the effect of gender on the face recognition performance, and though individual studies find men or women easier to recognize, there is no consistent gender effect [10]. Here, researcher investigated if such gender effects are present when humans are asked to distinguish between identical twins. The number of male pairs in the twins data used for the experiment is considerably lower than that of female pairs. The total number of correct and incorrect responses for male pairs is 571 and 153 respectively with an accuracy of 78.87%. On the other side, the total number of correct and incorrect responses for female pairs is 2693 and
723 respectively with an accuracy of 78.84%. So there is no
significant difference in matching accuracy for male and
female pairs. Similar results have been reported for facial images of unrelated persons [10].

5. EXPERIMENT RESULTS FOR VIEWING IM- AGES FOR UNLIMITED TIME

As mentioned in above analysis that humans do not perform very well in distinguishing between images of identical twin siblings given just two seconds to view images. Studies have shown that for face images of unrelated persons taken under different illumination conditions, increasing the viewing time of the images beyond two seconds does not significantly increase the recognition performance of human participants [2]. So here author designrd another experiment to test if humans can do a better job in distinguishing between identical twins when given sufficient time. He also want to test how external imaging factors affect the human performance and understand which facial features are most important for humans to distinguish between identical twin siblings. The visual stimuli used for this experiment consisted of 100 pairs out of which 50 pairs were images captured under controlled indoor conditions while the remaining 50 pairs were captured in outdoor uncontrolled environment. For each subset of 50, 25 were match pairs (images of same person) and the other 25 were non-match pairs. The four subsets of image pairs were interspersed and presented in a random order to the participants and the order was different for each participant. In this experiment, the participants were given unlimited time to view the image pairs and make their decision [7]. As in the first experiment, each trial consisted of the computer program showing a pair of facial images with a prompt for a decision on whether they belong to the same
person or identical twin siblings. The participants were also asked which features helped them make the decision. They were given the following choices: Eyebrows, Eyes, Nose, Lips, Moles/Scars/Freckles, Skin color/Texture, Wrinkles, Facial hair or Make-up.

5.1 Do humans perform better when given unlimited time?

One of the goals of mentioned experiment is to explore if increasing the viewing time makes it easier for humans to distinguish between identical twins. Across the 25 participants, the maximum accuracy attained is 100% and the minimum accuracy is 78%. The mean recognition accuracy for the mentioned experiment is 92.88% and the median is 95%. Fig. 3 (red solid curve) shows the ROC obtained. For generating the ROC, author considered only the indoor image trials from the second experiment for fair comparison. As observed that increasing the viewing time significantly increases the matching accuracy. The significant improvement in performance with increase in viewing time that is observeed in executed experiment can be attributed to the fact that facial images of twins are very similar with only subtle differences which can be better perceived given sufficient time [7]. Authors interpretation is that the added time is used by the participants to consider local features in making their decision.

5.2 Do humans perform better on controlled image

pairs than on uncontrolled pairs?

Fig. 4 shows a pair of uncontrolled images used in the experiment. Although the images are of the same person, they
appear very different due to illumination effects.

Fig. 4. Example of a pair of images of the same person taken outdoors with uncontrolled illumination

International Journal of Scientific & Engineering Research, Volume 6, Issue 2, February-2015 748

ISSN 2229-5518

Fig. 5. Human performance for image pairs taken in controlled and uncontrolled settings.
As described earlier, out of the 100 image pairs used in the second experiment, 50 were controlled image pairs and the remaining 50 were uncontrolled image pairs with equal number of match and non-match pairs of each type. The mean accuracy obtained here for the controlled image pairs is
94.96% and that for the uncontrolled image pairs is 90.80%. Fig. 5 shows the ROCs corresponding to the controlled and uncontrolled pairs. It is observed that humans find it harder to match the uncontrolled image pairs as compared to the con- trolled pairs. Thus it performed a one-tailed t-test to evaluate the null hypothesis that the performance for the controlled image pairs and the uncontrolled image pairs come from dis- tributions with equal mean. The resulting p-value is 0.0035 [7]. Thus, we have statistically significant evidence that the partic- ipants did better on the controlled pairs than on the uncon- trolled pairs. So presence of external factors like illumination, etc. tend to make the already challenging problem of distin- guishing identical twin siblings even more difficult.

5.3 What are the types of features that humans consid- er important to distinguish between identical twins?

In this study, given a pair of facial images, the participants were asked to choose the facial features which helped them in making their decision. Fig. 6 shows which feature types were chosen as important for the correct as well as incorrect re- sponses. From the figure, it is evident that for the correct re- sponses, the most important feature type chosen is moles/scars/freckles and it is significantly more important than any of the other types. For the incorrect responses, none of the features seem significantly more important than the others. This observation suggests that humans are more likely to be incorrect in their decision if they cannot find moles/scars/freckles in the images.
Fig. 6. Useful feature for distinguishing image of identical

twin siblings.

6. TECHNIQUES MENTIONED FOR AUTOMATIC FACE MATCHING PERFORMANCE

Now we investigate the ability of automatic face matching
algorithms to distinguish between identical twins. We experi-

mented with two commercial matchers (Pittpatt [12] and Cog- nitec [13]) in addition to standard holistic face matching ap- proaches based on Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). The algorithms were tested for the same 100 pairs used in the second experiment. Both the traditional matchers and Pittpatt performed verypoorly in this task. This result is in agreement with recent research [1] that has shown that currently available face recognition algorithms perform poorly on facial images of identical twins. Only Cognitec matcher performed comparable to humans and Fig. 8 shows the ROC obtained. The average human performance (computed from all the 25 participants) is also shown for comparison. As can be seen from Fig. 8, human observers outperform the automatic matcher for almost the entire range of False Accept Rate (FAR).
Fig. 8. Comparision of human performance against a commercial face recognition engine

International Journal of Scientific & Engineering Research, Volume 6, Issue 2, February-2015 749

ISSN 2229-5518

Author further investigate if the automatic matcher and humans show similar behaviour with regard to difficulty in distinguishing between identical twins. If algorithms and hu- mans behave differently, human expertise can potentially be used to guide development of better computer algorithms. Author analysed the non-match pairs that were found to be most difficult (corresponding to the highest similarity scores) by the automatic algorithm. Two non-match image pairs which got the highest similarity scores (greater than 0.9995) in the automatic experiment. But humans did reasonably well in distinguishing between these identical twin pairs (accuracy of
88% and 100% respectively). This indicates that human ob-
servers were able to capture facial characteristics different from the automatic algorithm that helped them do well on these pairs [7]. Author noticed that these image pairs differ in moles/freckles distribution and author conjecture that this may be a reason for the good performance of the humans. Therefore, one potential way to incorporate human knowledge to improve machine performance is to robustly detect facial marks and use them for facial characterization in addition to existing feature set used by automatic algorithms.

7. CONCLUSION

In this paper we can conclude that there no any automated system or program or application is available that can 100% identify or find the match in twins. So with the reference of this paper I will look forward to implement a system for iden- tifying solution to find the exact match between the twins. All above experimental results is being taken from a study of Face Recognition of Identical Twins by Human.

[9] E. Meyers and L. Wolf, “Using biologically inspired features for face processing,” International Journal of Computer Vision, vol. 76, no. 1, pp.

93–104, January 2008.

[10] Y. M. Lui, D. Bolme, B. A. Draper, J. R. Beveridge, G. Givens, and P. Phillips, “A meta-analysis of face recognition covariates,” in IEEE Interna- tional Conf. On Biometrics: Theory, Applications And Systems, 2009, pp.

1–8.

[11] “Twins days festival official website,” http://www.twinsdays.org/. [12] “Pittsburgh pattern recognition,” http://www.pittpatt.com/.

[13] “Cognitec: The face recognition company,” http://www.cognitec- systems.de/.

REFERENCES

[1] Z. Sun, A. A. Paulino, J. Feng, Z. Chai, T. Tan, and A. K. Jain, “A study of multibiometric traits of identical twins,” in Proc. SPIE, Biometric Tech- nology for Human Identification VII, April 2010.

[2] A. J. O’Toole, P. J. Phillips, F. Jiang, J. Ayyad, N. Pnard, and H. Abdi, “Face recognition algorithms surpass humans matching faces across changes in illumination,” IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, vol. 29, no. 9, pp. 1642–1646, September 2007.

[3] N. Ye and T. Sim, “Combining facial appearance and dynamics for face recognition,” LNCS, vol. 5702, pp. 133–140, 2009.

[4] K. Kodate, R. Inaba, E. Watanabe, and T. Watanabe, “Facial recognition by a compact parallel optical correlator,” Measurement Science and Tech- nology, vol. 13, pp. 1756–1766, 2002.

[5] U. Park and A. Jain, “Face matching and retrieval using soft biomet- rics,” IEEE Transactions on Information Forensics and Security, vol. 5, no.

3, pp. 406–415, September 2010.

[6] K. Hollingsworth, K. Bowyer, and P. Flynn, “Similarity of iris texture between identical twins,” in IEEE Conf. on Computer Vision and Pattern Recognition Workshops, 2010, pp. 22–29.

[7] Soma Biswas, Kevin W. Bowyer and Patrik J. Flynn, Dept. of Computer Science and Engg., University of Notre Dame, A Study of Face Recogni- tion of Identical Twins by Humans.

[8] A. Ariyaeeiniaa, C. Morrison, A. Malegaonkara, and B. Black, “A test of the effectiveness of speaker verification for differentiating between identi- cal twins,” Science and Justice, vol. 48, no. 4, pp. 182–186, December 2008