International Journal of Scientific & Engineering Research, Volume 3, Issue 6, June-2012 1

ISSN 2229-5518

Development strategy of eye movement controlled rehabilitation aid using Electro - oculogram

Anwesha Banerjee, Shounak Datta, Amit Konar, D. N. Tibarewala

AbstractThis paper proposes a strategy to develop an eye movement controlled rehabilitation aid using Electro-oculogram (EOG) to help severely paralyzed persons. Here, acquisition of EOG is done with a designed circuit. From EOG, eye movements in left an d right directions are classified using radial basis function (RBF) artificial neural network (ANN). For classification wavelet coefficients are used as signal feature. This offline training of the neural network can be used afterwards to generate real time control signals for the implementation of the EOG controlled rehabilitation aids. The approach and challenges concerned with the same have also been discussed in th is paper.

Index Termsartificial neural network, classification, data acquisition, Electro-oculogram (EOG), human computer interface (HCI), rehabilitation aid, wavelet transform

1 INTRODUCTION

—————————— ——————————
N modern civilization, one of our main challenges is to present the physically challenged members of our society with the gift of movement — the chance to a better life. There are some diseases like Amyotrophic Lateral Sclerosis (ALS), sub cortical stroke, brain or spinal cord injury, cerebral palsy, muscular dystrophies, multiple sclerosis, Guillain-Barre syndrome, some rare cases of Parkinson disease, Multiple Sclerosis etc. which leads to a condition called locked-in state (LIS) when the patient’s peripheral and central motor system gets completely destroyed but sensory or cognitive functions remain active. These diseases impair the neural pathways that control muscles or impair the muscles themselves. The prob- lems occur due to disease, age or sometimes it is congenital. According to WHO, there are almost 650 million people or more who are solely physically challenged. Given the growth in life expectancy in the world (in the countries of the Organi- zation for Economic Cooperation and Development (OECD) it is expected that the proportion of older persons aged 60 years and older will have reached a ratio of 1 person in 3 by the year
2030), a large part of its population will experience functional problems.
The situation needs attention. An efficient alternative way to communicate without speech and hand movements is im- portant to increase the quality of life for patients suffering from neural diseases or other illnesses or congenital problem or age which destroys proper limb and facial muscular res- ponses.
Hence, the area of study related to the Human Computer Interface (HCI) is very important to help such severely para- lyzed patients [1]. The HCI systems use biopotentials (e.g. EEG, EMG, ECG, EOG etc.), i.e., electrical signals generated from human body to control external devices. This technology aims to increase or maintain their communication and control options for severely paralyzed patients.
Sometimes, people are largely paralyzed by massive brainstem lesions and cannot move their muscles but are able to control their eye movement. Electroencephalogram (EEG) or Electromyogram (EMG) needs complex signal processing and acquiring Electrocardiogram (ECG) 24×7 is not possible always. EOG system is fairly easy to construct using surface electrodes placed around the eye socket with good face access and easy to work in real time [2] which makes EOG a better option over other biopotentials.

In our experiment, a data acquisition system is designed for EOG and from the acquired EOG signal useful features are extracted using wavelet transform. The features obtained are then classified according to the direction of eye ball movement using artificial neural network. This classification is done of- fline to train the network. It can then be used to produce EOG based control signals for HCI by microcontroller program- ming.

EOG Acqui

DAQ design visual stimulus

————————————————

Anwesha Banerjee is currently pursuing masters degree program in biomedi- cal engineering in Jadavpur University, India. E-mail: anwe- sha.banerjee@ymail.com

Shounak Datta is currently pursuing masters degree program in electronics

and tele communication engineering in Jadavpur University, India. E-mail:

shounak.jaduniv@gmail.com

Prof Amit Konar is in the department of electronics and telecommunication

engineering in Jadavpur University, India. E-mail: konaramit@yahoo.co.in

Prof D.N.Tibarewala is with school of bioscience and engineering in Jadavpur

University, India. E-mail: biomed.ju@gmail.com

offline trai

feature extraction classification

control

control signal generation motor rotation

Fig. 1. Block diagram repesentation of the proposed scheme

IJSER © 2012

http://www.ijser.org

International Journal of Scientific & Engineering Research, Volume 3, Issue 6, June-2012 2

ISSN 2229-5518

2 ELECTRO-OCULOGRAPHIC SIGNAL

The corneo-retinal potential resulting from a dipole (eye ball), generated between the cornea and the retina is called Electro- oculogram (EOG). The potential is produced due to the movement of the eye ball and can be acquired noninvasively by placing electrodes in the surrounding region of the eye.

Amplitude

10° Right

Time

tation amplifier (implemented using IC AD620) having high input impedance and CMRR followed by a second order low pass filter with a cut off of 20Hz and a high pass filter of 0.1Hz cut off to eliminate unwanted data. For filter designing IC OP07s are used. Gain is applied in various stages. Amplifier has a gain of 200 and 10 gain is provided by the filters. Thus an overall gain of 2000 is reached. For biopotential signal ac- quisition, isolation is an important factor to be considered for the subject’s safety as well as for the safety of the instruments. Power isolation is provided by the use of a dual output hybrid DC-DC converter (MAU 108) and signal isolation is obtained by optically coupling the amplifier output signal with the next stage. To achieve this HCNR 200 is used.

30° Left

Amplitude

Time

Fig. 2. Change in EOG signal according to the movement of the eye balls

Signal amplitude remains in the range of few micro volts.
EOG signal has a particular pulse shape for eye ball move-
ment in either direction. Pulse duration of the signal is approx-
imately 200ms on average and the signal magnitude changes
from 5-20 micro volts for a degree of eye ball movement [3].
The amplitude of the EOG signal changes depending on the
angle through which the eyeball was moved. When eye ball is
moved to one side the voltage remains positive (or negative)
and returns to zero when looking straight. The pulse pro-
duced by leftward movement is nearly the same as produced by rightward movement in both amplitude and pulse dura- tion. The signal potential remains the same even with the eyes closed. One problem of EOG signal is that head or body
movement alters the DC level of the signal.
Tracking the progress of many ophthalmological diseases
such as retinitis pigmentosa and also neural diseases (e.g. Par-
kinson’s, Alzheimer’s’) [4] can be done by analyzing EOG.
EOG signal is used to detect and assess degenerative muscular

Fig. 3. Designed data acquisition system for EOG

3.2 VISUAL CUE

Five subjects (two males & three females) of the age group 23

± 2 years participated in the experiment. During the process the subjects were made to sit in a relaxing chair. Each partici- pant was asked to perform a visual task. The subjects were asked to move their eyeballs to the said direction according to the instructions. The participants were directed to look at cer- tain marked points on the screen while seating 1m away from the same. The points were fixed at specific distances measured for 15° movement of the eye ball with the transition from one point to another.
Subject


disorders like laziness of the eye in tracking moving objects. Characteristics of the signal reveal its potential to produce eye movement controlled rehabilitation aids for severely para- lyzed persons. EOG is also used for drowsiness detection [5] and cognitive process modeling [6].
-15°
+15°

3 EXPERIMENTAL PROCEDURE

3.1 ACQUISITION OF EOG

EOG signal is available in the frequency range of 0.1 to 20
Hz and the amplitude lies between 100-3500 micro volts. A minimum voltage gain of 2000 is needed to process the signal further [5].
The signal acquired from the electrodes is fed to an instrumen-

Fig. 4. Shown stimulus for 15° viewing angles at a distance of 1 meter

None of the subjects had been exposed to or were informed of the experimental hypothesis prior to this study.

IJSER © 2012

http://www.ijser.org

International Journal of Scientific & Engineering Research, Volume 3, Issue 6, June-2012 3

ISSN 2229-5518

3.3 OFFLINE TRAINING

1. Signal pre-processing
The acquired signal is filter using band pass filter with cut
off frequencies 1Hz and 15Hz. The filter is implemented in
MATLAB using filter builder toolbox.
2. Feature Extracion
Wavelet transform does not suffer from the time-frequency trade off inherent in Short Time Fourier Transform (STFT) and Fourier Transform (FT) as the multi-scale approximation inhe- rent in wavelet transform allows for effective localization of the signal with various characteristics. Thus for a non- stationary signal like EOG, it is an effective analysis tool.
A wavelet is a waveform of limited duration that has an av-
erage value of zero. These wavelets are obtained from a single prototype wavelet called the mother wavelet by dilations, con- tractions and shifting, which is the fundamental approach of
wavelet transformation [7]. The property of wavelet transfor-
Hidden layer has a variable number of neurons. Each neuron consists of a radial basis function centered on a point with the same dimensions as the predictor variables. The output layer has a weighted sum of outputs from the hidden layer to form the network outputs.

xi hi (x)

f(x)

wi

mation to discriminate both temporal and spatial domain pa- rameters make it an inevitable tool for feature extraction from EEG signals. The continuous wavelet transform (CWT) treats a function of time in constituent oscillations, localized in both time and frequency [8]. CWT is defined as follows:
Input
Layer
Hidden
Layer

Fig. 5. Structure of RBFNN

Output
Layer




The output of the network is expressed by the following eq- uation




The Gaussian activation function is given by (eq. 4)

Where * denotes complex conjugation, τ is referred to as the translation, giving the position in time, and s the scale parame- ter, which is inversely related to the frequency content. Ψ (τ) called the mother wavelet. The discrete wavelet transform (DWT) in turn, is the result of selecting scales and translations based on powers of two, [9] yielding a more efficient yet accu- rate analysis.
3. Classification
After completion of the feature extraction process, features were fed into the ANN classifier to identify the direction of eye movement [10]. In this study, radial basis function neural network classifier is used.
ANN attempts to recreate the computational mirror of the biological neural network. An ANN is a mathematical model consisting of a number of highly interconnected processing elements organized into layers, the geometry and functionality of which have been likened to that of the human brain [11].
A radial basis function network is an artificial neural net- work that uses radial basis functions as activation functions. It is a supervised learning technique. Radial basis function (RBF) networks typically have three layers: an input layer, a hidden layer with a non-linear RBF activation function and a linear output layer. One neuron in the input layer corresponds to each predictor variable. With respects to categorical variables, n-1 neurons are used where n is the number of categories.




Where r is the radius or standard deviation and c is the cen- ter or average taken from the input space.
The learning process is based on adjusting the parameters of the network to reproduce a set of input-output patterns. There are three types of parameters, viz. the weight w between the hidden nodes and the output nodes, the center c of each neu- tron of the hidden layer and the unit width r.
This type of NN is used for classification by converting the input space into a hyperdimensional space to be able to linear- ly separate input classes. During the training phase, multiple input feature vectors generated from the raw data and their corresponding outputs are used to supervise the network [12]. The corresponding coefficient matrix is generated to map the given input to a correct class.
The efficient approximation capability even for the data that is not linearly separable is one of the biggest advantages of this algorithm [13]. Moreover, RBF networks are able to learn with a limited amount of training data.

3.4 PROPOSED STRATEGY OF REAL TIME CONTROL

After training the neural network with sufficiently large

IJSER © 2012

http://www.ijser.org

International Journal of Scientific & Engineering Research, Volume 3, Issue 6, June-2012 4

ISSN 2229-5518

amount of labeled training data, the resulting trained network is ready to classify unlabeled test data into appropriate classes. The input (or input vector) can then be fed into the trained neural network resulting in an output (or output vector) which signifies the class into which the input was classified.
This phenomenon can be made use of to classify the input data obtained from the EOG, during online operation, into the class corresponding to the direction of eye movement, viz. movement towards the left side of the subject, movement to- wards the right side of the subject. The aim of this classifica- tion is to turn a motor in the appropriate direction correspond- ing to the direction of eye movement of the subject. The movement of the motor can be controlled by a microcontroller such as 8051, PIC, etc. A combination of such motors can therefore be used to enable the subject to effectively control the movement of the rehabilitation aid using EOG signal only.
The trained neural network characterized by the weights and biases of each of the neural nodes, can be implemented using programming on the microcontroller unit mentioned above. Hence, during online operation, the data from the EOG directly fed into the microcontroller via an A/D converter when input into the software equivalent of the trained neural network will result in the generation of an output which can in turn be used to rotate the motor interfaced with the microcon- troller in the right direction.
The pseudo code for the implementation of the process de-
scribed above has been given below. The neural network dealt
with in the following code has a single hidden layer.
out(2,j)=nonlinearity[net(2,j)];
for (k=0 and k=1) //1 and 0 are the 2 classes
net(3,k)=bias(3,k)+sumof[out(2,j)*weight3(j,k)];

//suming for all j

out(3,k)=nonlinearity[net(3,k)];
if (out(3,0)>out(3,1))
[ class=0;
return (class);
] elseif (out(3,1)>out(3,0))
[ class=1;
return (class);
] else
return err;
]
function TurnMotor(class) [ if (class==1)
turn motor right;
elseif (class==0)
turn motor left;
else
return err;
]
The following figure shows the proposed setup for online operation.
main
[ loop: initialise variables;
check for errors;
if (error)
terminate program;
call ReadInput;
store input;
call NeuralNet;
EOG
Circuit CH0
A D C

Vref

AGND

VCC

40

Motor Driving Circuit
store output;
call TurnMotor(class);
loop;
]
function ReadInput
[ if (ADC is ready)
read input;

DGND

Microcontrol- ler Unit

20

GND

if (error)
else
]

return err;
return input value;

Fig. 6. Implementation of Online motor rotation with eye ball mov e- ment using a microcontroller

Using the mechanism described above, the motor can be
function NeuralNet
[ for (all i) //i = number of input nodes
net(1,i)=bias(1,i)+input(i)*weight1(1,i);

//weight matrices stored in data memory

out(1,i)=nonlinearity[net(1,i)];
for (all j)//j = number of nodes in hidden layer
net(2,j)=bias(2,j)+sumof[out(1,i)*weight2(i,j)];

//suming for all i

made to follow the eye movements of the subject, as observed from EOG readings, during real time operation. Thus, a physi- cally challenged person will be able to achieve mobility using the rehabilitation aid by eye movements only.

4 RESULTS AND DISCUSSION

The EOG data is taken by placing electrodes in the neigh-

IJSER © 2012

http://www.ijser.org

International Journal of Scientific & Engineering Research, Volume 3, Issue 6, June-2012 5

ISSN 2229-5518


borhood region of the eye. EOG signal is acquired using Ag- Agcl disposable electrodes. The placement of the electrodes can be seen in the figure 7.

Horizontal

Electrodes

Ground

Electrode

Fig. 7. Placement of electrodes to acquire EOG

The raw EOG data is filtered in the frequency range of 1-
15Hz. The observed EOG signal from the above mentioned
stimulus is shown below.

Fig. 8. EOG signal for eye movement of 15° in left and right respectively

Fig. 9. Filtered form of the EOG signal shown in figure 9


In our analysis we have used ‘db’ mother wavelet for dis- crete wavelet transform. The coefficients obtained from the transform are used as features for classification.

Fig.10. Level 5 DWT coefficients of the EOG signal

During offline training of the neural network, high classifi- cation accuracy is observed. Performance of classification is calculated by confusion matrix

Fig. 11. Confusion matrix for RBFNN classifier to classify left-right eye movement direction from the EOG signal shown in fig 9

The offline programming is done in the MATLAB 2008 envi- ronment.
The problem of online implementation, as proposed in section III D, still requires further research. There are various challenges associated with the online implementation of the rehabilitation aid. Some of these problems are discussed be- low.
1. ADC resolution
The resolution of the A/D converter to be used for the on- line implementation is a question of primary importance. The performance of the classification and hence of the rehabilita- tion aid is in direct proportion to the resolution of the ADC. However, the resolution is also directly proportional to the memory consumption of the microcontroller unit. Therefore, a suitable ADC having optimum resolution must be chosen after considering all the concerned factors.
2. Memory Requirement
The software implementation of a complex neural network on a microcontroller unit may require a large amount of mem- ory. While most microcontrollers can be interfaced with exter- nal memory units, the question of data handling capability of the controllers also demands consideration. The limited data handling capabilities of a microcontroller may lead to com- plexities.
3. Synchronization
Synchronizing the real world signals generated by the sub- ject with the clock cycles of the microcontroller unit is essen- tial. If the signals are not synchronized, the microcontroller may generate erroneous signals resulting in false triggering of the rehabilitation aid. This will not only lead to inconvenience but may also result in accidents.

IJSER © 2012

http://www.ijser.org

International Journal of Scientific & Engineering Research, Volume 3, Issue 6, June-2012 6

ISSN 2229-5518

4. Selection of Optimum Triggering Criteria
As the subject will only be using eye movements to control the various functions of the rehabilitation aid, a large number of combinations of the two possible types of eye movements will be required for triggering all the different functions of the same. These are referred to as Triggering Criteria. The ques- tion of selecting the optimum set of triggering criteria, without compromising either the effectiveness of the procedure or the convenience of the subject, is a vital one. Observing the actions of trained subjects in a real world environment may aid in a research of this kind.
5. Traning of the user
The user of the EOG controlled rehabilitation aid must be trained according to the operations. Otherwise the aid may not be handled properly and it can increase the problem of the patient.

5 CONCLUSION

In this study, stimulated Electro-oculogram (EOG) signal is acquired with a designed DAQ, wavelet features are extracted and radial basis neural network is used for classifying left- right eye ball movement. Thus the training of the network is done. Once all the challenges have been overcome, the online implementation of the rehabilitation aid can be accomplished. Then, it can be further applied for eye movement controlled HCI.
For further investigation other features like power spectral density (PSD), Auto Regressive (AR) parameters [14] can be experimented with. Use of other features may result in im- proved classification accuracy.

REFERENCES

[1] F. Cincotti, D. Mattia, F. Aloise, et al., “Non-invasive brain computer interface system: towards its application as assistive technology,” Brain Research Bulletin, vol. 75, no. 6, pp. 796– 803, 2008.

[2] Barea, R., Boquete, L., Mazo, M., Lopez, E.,”System for Assisted Mobility Using Eye Movements Based on Electrooculography.” IEEE Transaction on Neual Systems and Rehabilitation Engineering. vol.

10, No. 4, pp. 209-218, December 2002.

[3] Roy Choudhury S, Venkataramanan S., Harshal B. Nemade, Sahambi J.S., “Design and Development of a Novel EOG Biopotential Amplifier”, IJBEM Vol. 7, No. 1, 2005.

[4] Liu C., Chaovalitwongs W., Pardalos P., Seref O., Xanthopoulos P., Sackellares J., Skidmore F., “Quantitative analysis on electrooculography(EOG) for neurodegenerative disease”.

[5] Rajan A., Shivakeshavan.R.G , Vijay Ramnath.J ,”Electrooculogram based Instrumentation and Control System (IC System) and its applications for Severely Paralysed Patients”, Intl. Conf. on Biomedical and Pharmaceutical Engineering ,2006.

[6] Kim M., Kim C. And Lee J., “Classifying Neuro-Biological Signals By Evolutionary Fuzzy Classifier Construction”, Sice Annual Conference In Sapporo, 2004.

[7] Bukhari W. M., Daud , W., Sudirman R.,” A Wavelet Approach on Energy Distribution of Eye Movement Potential towards Direction”, IEEE symposium on Industrial Electronics and Applications, October

2010.

[8] Walnut DF: An introduction to wavelet analysis, Boston, MA, USA: Birkhäuser; 2002.

[9] Guido Dornhege, Matthias Krauledat, Klaus-Robert Muller, and Benjamin Blankertz. General signal processing & Machine learning tools for BCI analysis.

[10] Barea R., Boquete L., Mazo M., López E., Bergasa L.M., “Wheelchair guidance strategies using EOG”, Journal of Intelligent and Robotic Systems, Vol.34, pp 279–299, 2002.

[11] Guven A, Kara S, “Classification of electro-occulogram using

artificial neural network”,Expert systems with applications,Vol. 31, pp. 199-205, 2006.

[12] P. J, Sandberg, “Universal approximation using radial basis functions network,” Neural Computation, vol. 3. pp. 246-257, 1991

[13] Cinar E. and Sahin F.,” EOG Controlled Mobile Robot Using Radial

Basis Function Networks”, IEEE, 2009.

[14] Bulling A, Roggen D, “Recognition of visual memeory recall processes using eye movement analysis”, UbiComp’11, September

2011.

IJSER © 2012

http://www.ijser.org