The research paper published by IJSER journal is about A Comparative Study of Usability Methods for Mobile Applications 1

ISSN 2229-5518

A Comparative Study of Usability Methods for

Mobile Applications

Bassfar Zaid, Rozinah Jamaludin, Bajaba Wafaa

Abstract: conducting sufficient usability test requires planning and attention to the evaluation details. In common, usability test met hods for software take into considerations, planning usability questions, selecting a representative sample and recruiting participant s, and preparing the test materials and actual test environment. Several issues were reported while choosing the suitable usability test method for mobile applications, especially for indicating the way for conducting the test. Therefore, this paper aims to demonstrate the most u sed testing methods for the evaluation purposes of mobile applications.

Index Terms: Usability testing, mobile services, evaluation methods, human computer interaction.

1 INTRODUCTION

——————————  ——————————
Usability testing generally consists of evaluating the vari- ous software applications based on the consideration of differ- ent terms relate to the application functionalities. However, the usability testing is usually concern at computer and mobile applications which involves usability of these applications in a development process. Usability tests methods for mobile ap- plications are frequently take place among definite number of users using a think aloud protocol. The process starts when participants in the usability test given a certain tasks to work on, and enable users to think aloud while navigating the mo- bile applications [1]. Such process helps to provide us with the require information relate to the user aspects and the effects of mobile applications interface on their way of thinking and acting along with the further comments.
Usability testing usually conducted among group of users, given the fact that involving large population into the testing would help to increase the opportunity to solve problems relate to the mobile applications. Such aspect can be found while iden- tifying the urgency of procedures associated to a problem [2]. The most urgent actions are needed when the problem prevents completion of the task. These actions were addressed and cate- gorized by different researchers based on the severity type of problems, which involves; high (failure while execution), me- dium (task almost can be executed) and low (slight problems) [1].
Nevertheless, usability testing of mobile applications is con- sidered to be as an up-and-coming area of research that asso- ciated to the human computer interaction (HCI). Irit in [3] ac- knowledged that, it is normally conventional while conducting usability test on mobile applications to face several challenges connected to the central actions. Establishing a new testing for mobile applications also demands the user to be aware of the current needs. Evaluating mobile functionalities relate to the field of mobile HCI is greatly determined by technology and focus that mainly aims to carry out solutions. As well, a few prior researchers were conducted to estimate the effects of usa- bility testing methods on the evaluation of mobile applications utilized within the field of mobile HCI [4]. Both Wynekoop and Conger have established the foundation of evaluating the mo- bile user interface based on HCI methods that consists of case- studies, field studies, action research, laboratory experiments,
survey research, applied research, basic research, and normative writings. Therefore, establishing these tests require considering the following actions:
i) Prepare the suitable environment to conduct test;
ii) Identify the number of user to be involved into the evaluation;
iii) Give tasks, so that users can decide based on the usa- bility test questions;
iv) Provide assistant and support;
Some researchers (i.e., [5-6]) pointed out the importance of
addressing the require questions relate to how mobile applica-
tions is constructive and what is the challenging that may found
while performing tasks through mobile devices under real use
conditions. Progressively, they acknowledged the effects of the
integration of incorporating setting methods; consist on case
and field implications along with the action research. Standard
evaluation settings were designed to assist a definite evaluation purposes based on the combination of other settings. This com- bination helps to offer reliable opportunities to evaluate mobile applications with real-world user cases.
Designing usability test methods are usually include the user friendliness, interface, navigation, understanding, overall reac- tion, usefulness, etc. these factors are not limited, due to the everyday updating in the current market. Duh et al. in [7] ex- plained the significant effects of these factors on reducing the mental and physical stress, improve user-device operability. Such process helps to determine the user needs in order to im- prove the mobile applications quality to fit their needs [1]. From the other hands, usability testing for mobile applications are also involves measuring the following aspects:

i) Performance: Measure the time and the number of

steps required for completion of basic tasks? (For ex-
ample, load time of application, exiting an application,
moving to next screen, etc.)

ii) Accuracy: Is the application giving required results

within a defined tolerance level?

iii) Memorability: How much does the user remember af-

terwards or after periods of non-use?

iv) Errors: How many errors do users make, how severe

are these errors, and how easily can they recover from
the errors?

IJSER © 2012

http://www.ijser.org

The research paper published by IJSER journal is about A Comparative Study of Usability Methods for Mobile Applications 2

ISSN 2229-5518

v) User satisfaction or Emotional response: How does the user feel about the tasks completed? Is the user confi- dent, stressed? Would the user recommend this system to other people?

2 LITERATURE REVIEW

Different evaluation methods for usability test are designed an updated from time to time to fit the current user perspec- tives [8]. Wireless usability and, particularly in mobile applica- tions, is determined in the justification of the settings and functionalities provided for users to do their tasks using con- servative interactive testing environments that may involve more than one group to carry the evaluation. Based on the current evolution in technology and applications, there are no a definite and appropriate principles to conduct the evaluation on applications runs through mobile platforms, where the li- mited and order representation is the essential feature to be measured. Therefore, this paper is conducted to categories the most used evaluation methods of usability testing. Identifying the usability testing methods would help to address the re- quire measurement criteria, which are being carried out these days in the mobile devices field.
A study was conducted by González et al. [9] used a forma- tive testing method for evaluating the proposed approach in which association rules and decision trees are utilized to ex- pand the current qualitative usability testing process in order to generate a usability diagnosis from a qualitative viewpoint. Usability problems patterns belonging to academic pages are assessed by processing 3450 records which store qualitative information collected by means of a heuristic evaluation. Fig- ure 1 presents the proposed modification on the qualitative usability testing process.

Fig1. Qualitative usability testing process

While Bernhaupt, et al.,[10] conducted their study to address the current challenges in using usability testing methods in evaluating mobile applications. They also introduced the dif- ferent usability evaluation methods (UEMs) for mobile appli- cations to the aim of enhancing and assuring easy to use user mobile interfaces and applications. They reported the impor-
tance of some evaluation methods such as classical’ methods in the field of mobile applications. The argument was mostly towards the external effects that broadened, varied, and changed the evaluation demands of testing usability for mo- bile application. The classical methods were seen to be as an efficient way to conduct evaluation along with the integration of some methodological variations for testing usability in the area of mobile applications.
Kjeldskov and Stage in [11] aknowledged that usability evalu- ation of systems for mobile is considerd to be an emerging area of new scholaers. They listed the most evaluation tech- niques for usability testing of mobile applications in laborato- ry settings. They also pointed out the importance of the se- lected techniques in simplifing the systematic data collection in a controlled environment and support the identification of usability problems. The proposed method includes different mption aspects of user actions while engaging in the evalua- tion. Two usability experiments were involved in their study to measure the effects of the selected techniques. Some simili- raties were revealed by the experiments, but none of them turned out to be completely comparable. Meanwhile, the some challenges were also reported during the experiments which categorized as cosmetic.

Fig2. Three different combinations of techniques applied in three test user groups.

Duh et al. [7] evaluated the effectiveness of conventional laboratory usability test methods for mobile applications. They determine the main differences between usability tests on mobile phones that taken place in laboratory and real life environment. They also pointed out that usability testing me- thods of various mobile applications and services is an emerg- ing area of research in the field of HCI.
Based on the aforementioned studies, the main evaluation methods use to observe the user opinion on the functionalities of mobile applications are functionality, reliability, usability, efficiency, maintainability and portability. Usability is the main measurement for mobile application to be understood, learned, used, and attractive by the user. The most commonly

IJSER © 2012

http://www.ijser.org

The research paper published by IJSER journal is about A Comparative Study of Usability Methods for Mobile Applications 3

ISSN 2229-5518

used methods are; heuristic evaluation, cognitive walk- throughs evaluation, conventional user test, laboratory testing, and field testing. Table 1 presents a comparison between the selected evaluation methods.
Table1. Comparison between the usability evaluation me- thods for mobile applications.

3 METHODS

In general, the purpose of research method is to provide a complete description of the way that this research was de- signed and developed. There are a number of stages involved in the production of research document. One of these is a good methodological approach using appropriate data collection techniques [18]. Practical considerations largely guide the choice of the organization used for this study.
This study used a comparative study method to find out the different sorts of usability evaluation methods for mobile ap- plication by conducting a comparative study involving differ- ent previous researches conducted in both field and laboratory environments. The adapted method addressed the wide used methods. As well, the data for this research was also obtained from multi sources that were extracted from the annual re- ports, journals, textbooks and other relevant publications re- lated to monitoring and evaluation software. Table 1 shows
the compression of the most used evaluation methods for mo- bile application in terms of author, object, and assign.

4 DISCUSSION

Usability is a measurement of evaluation of a definite attribute that related to the "ease of use and to learn". As mentioned earlier by different prior researches, whom clarified the impor- tance of usability evaluation estimations. In this paper, we have found that usability result for mobile applications are usually obtained through a qualitative usability testing, this assumption is supported by González et al. in [9] whom in- cluded a number of different methods focused on analyzing the interface of a particular system. Moreover, evaluating mo- bile applications is differing from one application to another in terms of the complexity level associated with context to pro- vide a common analysis. Many researchers found that analyz- ing the common occur problems in usability testing are chal- lenging. Identifying such problems can help to evaluate a new interface belonging to the usability tests. And as mentioned by Bernhaupt et al. in [10], usability testing methods are ar- gues for a combination of both field evaluation methods and traditional laboratory testing in the user-centered design and development process. Finally, we have found that an evalua- tion guidelines are required to establish an efficient tests as declared by Duh et al. [7] who justified the importance of con- ventional usability tests for mobile devices.

5 CONCLUSION

Usability evaluation is occupying a central part of software development based on the results extracted from quantitative and qualitative evaluations. This paper introduced the most widely used methods for conducting usability testing on mo- bile applications. The slandered evaluation criteria related with usability was addressed in this paper based on the pre- vious researches. The usability tests can be carried out in la- boratories or in real scenarios (field test). We have found that usability evaluation methods for mobile application differ from one application to another based on the level of complex- ity.

REFERENCES

[1] T. Kallio and A. Kaikkonen, "Usability testing of mobile applications: A comparison between laboratory and field testing," Journal of Usability Stu- dies, vol. 1, pp. 4-16, 2005.

[2] A. Betiol and W. de Abreu Cybis, "Usability testing of mobile devices: A comparison of three approaches," Human-Computer Interaction- INTERACT 2005, pp. 470-481, 2005.

[3] I. Irit, "Usability Evaluation of Multimodal Interfaces," Multimodal hu-

man computer interaction and pervasive services, vol. 2, p. 443, 2009.

[4] J. L. Wynekoop and S. A. Conger, "A review of computer aided soft- ware engineering research methods," The Information Systems Research Arena of the 90's: Challenges, Perceptions, and Alternative Approaches, vol. 1, pp. 129-154, 1990.

[5] S. Tamminen, et al., "Understanding mobile contexts," Personal and

Ubiquitous Computing, vol. 8, pp. 135-143, 2004.

[6] T. Bohnenberger, et al., "Location-aware shopping assistance: Evalua-

IJSER © 2012

http://www.ijser.org

The research paper published by IJSER journal is about A Comparative Study of Usability Methods for Mobile Applications 4

ISSN 2229-5518

tion of a decision-theoretic approach," Human Computer Interaction with

Mobile Devices, pp. 48-48, 2002.

[7] H. B. L. Duh, et al., "Usability evaluation for mobile device: a compari- son of laboratory and field tests," in Proceedings of the 8th conference on Human-computer interaction with mobile devices and services Helsinki, Fin- land, 2006, pp. 181-186.

[8] T. Magal-Royo, et al., "Evaluation methods on usability of m-learning

environments," interactive mobile technologies, vol. 1, p. 22, 2007.

[9] M. P. González, et al., "Enhancing usability testing through datamining techniques: A novel approach to detecting usability problem patterns for a context of use," Information and Software Technology, vol. 50, pp. 547-568,

2008.

[10] R. Bernhaupt, et al., "Usability evaluation methods for mobile applica- tions," Handbook of research on user interface design and evaluation for mobile technology, vol. 2, pp. 745-758, 2008.

[11] J. Kjeldskov and J. Stage, "New techniques for usability evaluation of mobile systems," International Journal of Human-Computer Studies, vol. 60, pp. 599-620, 2004.

[12] C. van Elzakker, et al., "Field-Based Usability Evaluation Methodology for Mobile Geo-Applications," Cartographic Journal, The, vol. 45, pp. 139-

149, 2008.

[13] J. Nielsen, "Heuristic evaluation," Usability inspection methods, vol. 24, p. 413, 1994.

[14] C. Lewis and C. Wharton, "Cognitive walkthroughs," Handbook of human-computer interaction, vol. 2, pp. 717-732, 1997.

[15] F. T. W. Au, et al., "Automated usability testing framework," in Pro-

ceedings of the ninth conference on Australasian user interface, Wollongong, Australia, 2008, pp. 55-64.

[16] J. Kjeldskov and C. Graham, "A review of mobile HCI research me-

thods," Human-computer interaction with mobile devices and services, pp. 317-

335, 2003.

[17] M. Hertzum, "User testing in industry: A case study of laboratory, workshop, and field tests," in Proceedings of 5th ERCIM workshop on "user interfaces for all", 1999, pp. 59-72.

[18] A. Aderinto, "A SURVEY OF THE REFORM PROGRAMME," Nige- ria's reform programme: issues and challenges, p. 66, 2007.

————————————————

Bassfar Zaid, Centre for Instructional Technology and Multimedia, Un- iversiti Sains Malaysia, Penang, Malaysia, zaidb12@hotmail.com

Rozinah Jamaludin, Centre for Instructional Technology and Multimedia,

Universiti Sains Malaysia, Penang, Malaysia, rozinah@usm.my

Bajaba Wafaa, Accounting Department, Imam Muhammad Ibn Saud Is-

lamic University, Saudi Arabia, wbajaba@hotmail.com

IJSER © 2012

http://www.ijser.org