International Journal of Scientific & Engineering Research, Volume 4, Issue 5, May-2013
ISSN 2229-5518
Assessment on Contents of the Learning
Management System
Zurinah Suradi, Nazatul Shima Abdul Rani
Abstract— Most assume that adult learners are not concerned with access to the LMS or even with using it in their learning and would prefer a more conventional mode of learning. However, at the Asia e University in Malaysia, most of the learners are adult learners who are actively engaged in utilizing learning materials available in the learning management system (LMS). A sample group of 145 adult learners responded to the study, demonstrating that the contents of the university’s learning management system are of quality, complete, relevant and current. In addition, the sample group was satisfied (e-satisfaction) and will keep using the LMS (e-retention) in future.
Index Terms— learning management system, contents, e-learning, quality information, information completeness, information relevance, information timeliness (current), e-satisfaction, e-retention.
—————————— ——————————
HE Learning management system (LMS) is widely used in e-learning higher education institutions including the Asia
e University, the Wawasan Open University, Unitar Inter- national University, and the Open University [1],[2],[3]. This is due to the communication interaction facilities provided by the system. In general, the LMS was developed using web- based technology that aims to provide three types of commu- nication interactions, namely 1) between students and aca- demic facilitators via internet communication tools like email, chat, forum, etc., 2) between students and the system, and 3) among peer students via email, chat, and so on [4]. A survey conducted by previous researchers showed that most universi- ty students preferred Moodle over other types of learning management systems [5], and which is what the Asia e Uni- versity, Malaysia uses as a means to deliver course materials to students. In parallel with the communication interaction facilities requirements, Moodle also allows interaction be- tween the course administrator, students, and course facilita- tors [6]. As such, the quality of the learning materials is an important aspect that needs to be examined by learning pro- viders, especially in private higher education institutions that opt for e-learning in delivering teaching materials to students. For the purposes of this paper, the content of the learning management systems or LMS will be evaluated based on its information accessibility, information completeness, informa- tion relevance, and information timeliness.
The advancement of technology and greater emphasis on e- learning has spurred the development of a learning manage- ment system that can cater to the learning styles of higher education institution students.
————————————————
• Zurinah Suradi is currently an Associate Professor in at the School of ICT at the Asia e University, Malaysia, PH-. E-mail: zurinah.suradi@aeu.edu.my
• Nazatul Shima Abdul Rani is currently a Senior Lecturer at the School of
Management at the Asia e University, Malaysia, PH-. E-mail: shi- ma.rani@aeu.edu.my
The reasons most cited for this development are the capacity to deliver training anytime and anywhere, as well as cost sav- ing (as there are no traveling, accommodation or tutor costs incurred). However, one of the main disadvantages of the e- learning experience is isolation and a lack of tutor guidance [7],[8]. Nevertheless, students have shown that they are will- ing to use the LMS provided it is user friendly and provides good support services like the IT Helpdesk and if, more im- portantly, it can fulfill the needs of on-demand content [9]. As such, the learning management system refers to systems that organize and provide students, teachers and administrators with access to online learning services [10]. Most learning management systems allows users to upload course informa- tion into the Web, including text, documents, files of many types including MS-Office, audio, images, grade books, as- sessments, hyperlinks, discussion boards and chat rooms [11],[12],[13]. In the online learning environment, the academ- ic facilitator that uses LMS for a given course can administer said course, upload learning materials, give online tests and feedback, and so on. As for the students’ side, the system al- lows them to upload documents, download learning material for learning purposes and interact with the academic facilita- tor through forums, chat and so on. The LMS is also able to provide on-demand content as needed by learners. In learn- ing management systems, the main characteristics of learning objects are reusability, accessibility, portability, interoperabili- ty and durability [14].
In the Hexagonal e-learning assessment model (HELAM), con- tent quality falls under technical issues, and the items under consideration are curriculum management, course flexibility, interactivity, practice exams, case studies, slides, content qual- ity, assignments, tutorials, and exams [10]. All these reflect that good content information quality is important to encour- age students to use the system. For the purposes of this study,
IJSER © 2013
International Journal of Scientific & Engineering Research, Volume 4, Issue 5, May-2013
ISSN 2229-5518
information quality is divided into four areas namely informa- tion accessibility, information completeness, information re- levance and information timeliness [10],[11].
Information accessibility has become crucial in an education setting where higher learning institutions use learning man- agement systems to deliver course materials and content to students. Most higher education institutions that utilize learn- ing management systems will usually use Blackboard, WebCT, Learning Space, Moodle and other platforms to pro- videWeb-based content [12]. A survey conducted by previous researchers indicated that a system that was unified and easy to access while allowing quick and easy location or retrieval of information was seen as the most desirable by most internet users [11],[15]. However, this depended on the accessibility of the system’s design [16]. The accessibility of a system’s design would allow users to perform retrieval activites with ease re- gardless of whether different machines or operating systems were used.
The content or information is expected to be skewed towards industry standards so that it can be reused and interchanged across the various learning platforms. Since the cost of devel- oping material for e-learning content is far greater than devel- oping material for traditional classroom instruction, this means that the cost will be reduced when more students are able to access the content. The content is also reusable and easily adapted to the learning environment [8], [11]. Further, information completeness is valuable to students as it will help them adhere to the right direction and not go astray in their studying.
LMS content should be clear, user-friendly, well-designed, and interactive [10],[17],[18],[19],[20],[21]. The information uploaded into the LMS must be relevant because it will be the key component in addressing the students’ study needs.
The contents of the LMS should be current, timely, up-to-date, and contemporary [10],[17],[18],[19]. This is an important cri- terion for the contents of the LMS since it will meet the right knowledge requirements at the right time for the students. Current and updated content will be of more useful to stu- dents then the outdated content. This implies that there is a sense of time sensitivity related to the contents in which stu- dents will use the learning materials in their study provided that the information is timely.
E-satisfaction is used to measure the overall satisfaction of the online learning experience over a period of time for e-learning
students. Students who perceived e-learning to be useful and val- uable are more likely to be satisfied with their learning expe- rience [22]. Student satisfaction varies depending on the objec- tives of the system provided [23], for example, the need for the LMS to provide the necessary information with up-to-date learn- ing resources, timely announcements, and other relevant learning materials needed to enhance the delivery of learning materials to e-learning students. However, if that which is to be presented or made available is lacking in quality, then the students’ e- satisfaction will consequently be lower. Studies have indicated that information quality (content) is also used in evaluating the e- satisfaction of an e-learning system [24], [25].
E-retention or retention is used to determine the loyalty of stu- dents or customers [26]. Most e-learning students in Malaysia and many other countries are balancing their studies with their careers, personal lives, and family commitments. However, if the e-learning students dare to take up the challenge to familiarize themselves with the new technology and benefit from the learn- ing experience, their e-satisfaction and e-retention will increase [27], [28], [29], [30].
Questionnaires were distributed to the Asia e University via an e-mail blast, and 145 people responded to it. The intended questionnaire is for all students that are enrolled for the Sep- tember 2012 semester. Our target was for 200 students to re- spond to the questionnaire, and we only collected 145 usable sets.
The items for each component were derived from past litera- ture, as shown below.
Table 1: Item Descriptions
Items –Contents – Information Accessibility | References |
1. The information in the PLS or LMS is easily retrievable. 2. The information in the PLS or LMS is easily accessible. 3. The information in the PLS or LMS is easily obtainable. 4. The information in the PLS or LMS is quickly accessible when needed. | [10],[17],[18],[19] |
Items – Contents – Information Completeness | References |
1. The information in the PLS or LMS includes all the necessary values. 2. The information in the PLS or LMS is complete. 3. The information in the PLS or LMS is suffi- ciently complete for our studies. 4. The information in the PLS or LMS covers the needs of our studies. 5. The information in the PLS or LMS has suffi- cient breadth and depth for our studies. | [10],[12],[17],[18] ,[19],[20] |
Items –Contents – Information Relevance | References |
IJSER © 2013
International Journal of Scientific & Engineering Research, Volume 4, Issue 5, May-2013
ISSN 2229-5518
Employment Unemployed Employed
Self-Employed
35 24.1
97 66.9
13 9.0
Total 145 100.0
Age Group
Below 25
26 – 30
31 - 35
36 - 40
41 – 45
46 - 50
Above 50
38 26.2
16 11.0
21 14.5
20 13.8
20 13.8
10 6.9
20 13.8
Total 145 100.0
Ethnic Group
Malay Chinese Indian Indonesian Others
28 19.3
36 24.8
43 29.7
37 25.5
1 0.7
Total 145 100.0
AeU Program
DBA Master Bachelor Diploma
7 4.8
90 62.1
43 29.7
5 3.4
Total 145 100.0
Number of Semesters at AeU
Year 1
Year 2
Year 3
More than 3 years
94 64.8
35 24.1
12 8.3
4 2.8
Total 145 100.0
Sector Type
Private Public Others
97 66.9
10 6.9
38 26.2
Total 145 100.0
Gender
Male
Female
54 37.2
91 62.8
145 students responded to the survey, and it is representative of the total population of the Asia e University students in
Total 145 100.00
Years of Working
Malaysia. Table 2 shown demographic profiles of the e- learning students responded to the survey. Only 24.1% of the respondents were unemployed and the rest were employed.
66.9% of the respondents worked in the private sector, 6.9 % in the public sector, and 26.2% in other sectors. The age group of
5 years and less
6 – 10
11 – 15
16 – 20
21 – 25
More than 25
61 52.14
26 22.22
13 11.10
6 5.12
6 5.12
5 4.3
the respondents showed that 26.2% were below 25 years old,
11% were in 26-30 years old, 49.6% were 31 - 50 years old, and
Total (missing 28) 117 100.0
AeU School
13.8% were above 50. As such, most of the respondents were above 30 years old. About 62.8% of the respondents were fe- male. 19.3% were Malay, 24.8% were Chinese, 29.7% were In-
School of Management
School of Education
School of ICT
School of Graduate Studies
67 46.2
43 29.7
33 22.8
2 1.4
dian, 25.5% were Indonesian and 0.7% was from other ethnic groups. In terms of work experience, 47.6% of the respondents have worked for more than 5 years. 62.1% of the respondents were pursuing a PhD, 46.2% of the respondents were from the School of Management and 64.8%. of the respondents were in their first year at AeU.
Table 2: Demographic Profiles
Item Frequency Percentage
Total 145 100.0
145 students agreed that the learning management system at the Asia e University is accessible because it is easily retrieva- ble, accessible, obtainable and quickly accessible when needed. The respondents rated the ease of accessibility of the information in the PLS or LMS as being quite high. This indi- cates that the learning materials are easily accessible to stu-
IJSER © 2013
International Journal of Scientific & Engineering Research, Volume 4, Issue 5, May-2013
ISSN 2229-5518
dents for their learning purposes and needs. Table 3: Descriptive Analysis on Accessibility
Items | Mean | Std. Deviation |
1. The information in the PLS or LMS is easily retrievable. 2. The information in the PLS or LMS is easily accessible. 3. The information in the PLS or LMS is easily obtainable. 4. The information in the PLS or LMS is quickly accessible when needed. | 3.8414 3.8483 3.8138 3.7517 | 0.84718 0.90777 0.88181 0.92444 |
145 students at the Asia e University agreed that the informa- tion in the learning management system is complete as the information includes all the necessary values, is sufficiently complete, covers the needs of their studies and has sufficient breadth and depth for their studies. On the other hand, stu- dents also perceived that the information in the PLS or LMS was less than complete. This implied that some improvement is needed to fulfill the information completeness criteria.
Table 4: Descriptive Analysis on Completeness
tion, students will be at ease with their studying and learning. Table 6: Descriptive Analysis on Timeliness
Items | Mean | Std. Deviation |
1. The information in the PLS or LMS is sufficiently current for my studies. 2. The information in the PLS or LMS is sufficiently timely for my studies. 3. The information in the PLS or LMS is sufficiently up-to-date for our studies. 4. The information in the PLS or LMS is contemporary for our studies. 5. The information in the PLS is old- fashioned for our studies. | 3.7103 3.5862 3.6414 3.5310 2.9172 | 0.84916 0.90972 0.91800 0.85029 1.11495 |
145 students responded to the survey agreeing that they are satisfied with the LMS services, have increased their frequency in visiting it, would recommend that others use the LMS for learning purposes, believe that the LMS provides complete information necessary for learning activities, and are overall satisfied with the LMS.
Table 7: Descriptive Analysis on e-satisfaction
Items | Mean | Std. Deviation |
1. I am satisfied with the services offered by the PLS or LMS. 2. I have increased my frequency of visit- ing the PLS or LMS. 3. I have recommended that others use the PLS or LMS for learning. 4. The PLS or LMS provides me with the complete information necessary for my learning activities. 5. Overall, I am satisfied with the PLS or LMS. | 3.7103 3.6621 3.6207 3.6552 3.7103 | 0.9422 0.8516 0.9134 0.8529 0.8573 |
The 145 students responding to the survey agreed that the information is relevant as it is useful, relevant, appropriate and applicable to their studies. Most students agreed that the learning materials uploaded to the LMS are relevant to their studies. This suggests that the information relevance criterion has been fulfilled.
Table 5: Descriptive Analysis on Relevance
Items | Mean | Std. Deviation |
1. The information in the PLS or LMS is useful for our studies. 2. The information in the PLS or LMS is relevant to our studies. 3. The information in the PLS or LMS is appropriate for our studies. 4. The information in the PLS or LMS is applicable to our study. | 3.7517 3.7724 3.8000 3.7103 | 0.88609 0.84775 0.81309 0.82426 |
The 145 students responding to the survey agreed that the information in the information management system is timely as it is sufficiently current, up-to-date, and contemporary. All
145 respondents disagreed that the information in the learning management system is old-fashioned. With timely informa-
The 145 respondents said that they would continue using the LMS, would recommend it to others, would encourage others to use the LMS, have positive opinions of the LMS and would still use the LMS even if they encountered problem while using it.
Table 8: Descriptive Analysis on e-retention
Items | Mean | Std. Deviation |
1. I intend to continue using the PLS or LMS. 2. I recommend the PLS or LMS to others. 3. I encourage others to use the PLS or LMS. 4. I have a positive opinion on the PLS or LMS. 5. I will continue using the PLS even if I have a problem using it. | 3.8759 3.8000 3.8000 3.7862 3.7931 | 0.8406 0.9021 0.8787 0.9588 0.8811 |
There are positive and direct correlations of the content in terms of accessibility, completeness, relevance and current- ness. There are high correlations of e-satisfaction with all the content or information quality of the LMS. However, for e- retention, it was shown that there was a moderate correlation
IJSER © 2013 http://www.ijser.org
International Journal of Scientific & Engineering Research, Volume 4, Issue 5, May-2013
ISSN 2229-5518
with accessibility, completeness and timeliness, but a high correlation with relevance. E-satisfaction and e-retention is highly correlated for this study. These results suggest that the information quality of the content of the LMS is positively re- lated to e-satisfaction and e-retention. This implies that con- tent information quality must be of a high standard so that the students will be satisfied and will continue to use the LMS. Table 9: Pearson Correlation Findings
This study presents an initial investigation on content infor- mation quality dimensions that affect e-satisfaction and e- retention. Students perceived that the content information quality dimensions: accessibility, completeness, relevance and
satisfctn e-
1 [1] Blueprint on Enculturation of Lifelong Learning 2011-2020, Ministry
retention
This study shows that the parameters of information content quality are positively related to satisfaction and retention. This implies that students will use a learning management system provided that the information content is complete, relevant, timely and accessible. The findings also implied that learning materials and resources uploaded to the LMS must be of high quality so that the students will return and use the LMS regu- larly to help them with their studies. Subject matter experts developing learning materials should be evaluated so that they would be better equipped to capture the students’ inter- est. Better and improved learning materials would lead better opportunities in enticing potential students to enroll with the university.
A similar study may be expanded to other universities in Ma- laysia to evaluate whether the students at other universities that have adopted e-learning also experience high e- satisfaction and e-retention. The institutions which have im- plemented the LMS should take the initiative to evaluate the content information quality uploaded into the system. Fur- thermore, an important avenue for future research would be to include system quality and service quality.
This research explores the relationship between content in- formation quality and e-satisfaction and e-retention. The re- search findings suggest that information quality is an impor- tant issue of consideration for higher learning institutions where students are solely dependent on the LMS for their stu- dies and learning. Broken links and errors found within the contents would lead to an abandonment of the use of the LMS by students. Furthermore, poor information quality provided by the institutions in question would result in a loss of poten- tial students. As such, it is important for higher learning insti- tutions to assess the information quality in their LMS.
of Higher Education, Perpustakaan Negara Malaysia, Univision Press
Sdn Bhd, 2011.
[2] M. R. A. Karim, and Y. Hashim, “The experience of the e-learning implementation at the Universiti Pendidikan Sultan Idris, Malaysia,” Malaysia Online Journal of International Technology, vol. 1, no 1, pp. 50-
59, 2004.
[3] Z. Suradi, and N. S. A. Rani, and N. M. Khan , “Factors Critical for
Learning Management System in On-line Distance Learning (ODL)”,
Management, vol. 3, no. 1, pp. 50-53, 2013. doi: 10.5923/j.mm.
20130301.10.
[4] Z. Yanga, S. Caib, Z. Zhouc, and N. Zhou, N. (2005). “Development
and validation of an instrument to measure user perceived service quality of information presenting Web portals.” Information & Man- agement, vol. 42, pp. 575–589.
[5] B. Stewart, D. Briton, M. Gismondi, B. Heller, D. Kennepohl, R.
McGreal, and C. Nelson, “Choosing MOODLE: An evaluation of learning management systems at Athabasca University,” J. of Dis- tance Education Technologies, vol. 5, no. 3, pp. 1-7, 2007.
[6] S. T. Pektas, and H. Demirkan, "Experiences with MOODLE as a
Communication Tool for Design Teamwork: A Users' Perspective"
METU Journal of the Faculty of Architecture, vol. 28 no. 2, 2011, pp.
227-241, 2011.
[7] S.S. Siaw, W.C. Chang, W.H. Hung, and H.M. Huang, “Attitudes Toward Search Engines as a Learning Tool: Approach of Liaw and Huang’s Research Model”, Computers in Human Behavior, vol. 22, no.
2, pp. 177-190, 2006.
[8] J. Watson, and G. Hardaker, “Steps towards personalized learner management system (LMS): SCORM implementation,” Campus-Wide Information Systems, vol. 22, no. 2, pp. 56-70, 2005.
[9] D. Weaver, C. Spratt, and C. Sid Nair, “Academic and student use of a
learning management system: Implications for quality”, Australasian
Journal of Educational Technology, vol. 24, no. 1, pp. 30-41, 2008.
[10] S. Ozhan, R. Koseler, and N. Baykal, “Evaluating learning manage- ment systems: Adoption of hexagonal e-learning assessment model in higher education,” Transforming Government: People, Process and Policy, vol. 3, no. 2, pp. 111-130, 2009.
[11] M. Abdellatief, A.B.M. Sultan, M.A. Jabar, and R. Abdullah, “A tech- nique for quality evaluation of e-learning from developers perspec- tive,” American Journal of Economics and Business Administration, Vol.
3, no. 1, pp. 157-164, 2011.
[12] A.M. Johnson, and S. Ruppert, “An evaluation of accessibility in
online learning management systems,” Library Hi-Tech, vol. 20, no. 4, pp. 441-451, 2002.
IJSER © 2013 http://www.ijser.org
International Journal of Scientific & Engineering Research, Volume 4, Issue 5, May-2013
ISSN 2229-5518
[13] A. Ioannou, and R. Hannafin, “Deficiencies of course management systems- Do students care?” The Quarterly Review of Distance Educa- tion, vol. 9, no. 4, pp. 415-425, 2008.
[14] P. Alves, and J. Uhomoibhi, “Issues of e-learning standards and iden- tity management for mobility and collaboration in higher education,” Campus-Wide Information Systems, vol. 27, no. 2, pp. 79-90, 2010.
[15] R. V. Small, S. Sutton, M. Miwa, C. Urfels, and M. Eisenberg, “Infor- mation seeking for instructional planning: an exploratory study,” J. of Research on Computing in Education, vol. 31, no. 2, pp. 204-219, 1998.
[16] (Earl, et al. 2008)
[17] H. Hussin, F. Bunyarit, and R. Hussein, “Instructional design and e-
learning: Examining learners’ perspective in Malaysian institutions of higher learning,” Campus-Wide Information Systems, vol. 26, no. 1, pp.
4-19, 2009.
[18] J.E. Klobas, and T.J. McGill, “The role of involvement in learning
management system success, “ J. of Computing High Education, vol. 22, pp. 114-134, 2010.
[19] A. Moisio, and R. Smeds, “E-learning: A service offering,” Knowledge and Process Management, Vol. 11, no. 4, pp. 252-260, Oct-Dec 2004.
[20] P. Tynjala, and P. Hakkinen, “E-Learning at work: theoretical under- pinnings and pedagogical challenges,” J. of Workplace Learning, vol. 17 no. 5/6, pp. 318 – 336, 2005.
[21] S. Iqbal, and I.A. Qureshi, “Learning Management Systems (LMS): Inside Matters,” Information Management and Business Review, vol. 3, no. 4, pp. 206-216, 2011.
[22] I. Adamson, and J. Shine, “Extending the New Technology Accep- tance Model to Measure the End User Information System Satisfac- tion in Mandatory Environment: A Bank’s Treasury”, Technology Analysis & Strategic Management, vol. 15, no. 4, pp. 441-455, 2003.
[23] C. S. K. Lai, and G. Pires, “Testing of a Model Evaluating e- Government Portal Acceptance and Satisfaction”, The Electronic Jour- nal Information System Evaluation, vol. 13, no. 1, pp. 35-46, 2010.
[24] J. C. Roca, C. M. Chiu, and F. J. Martı´nez, “ Understanding e- learning continuance intention: An extension of the Technology Ac- ceptance Model”, International Journal Human-Computer Studies, vol.
64, pp. 683–696, 2006.
[25] Jen-Her Wu, Tzyh-Lih Hsia, Yi-Wen Liao, Robert D. Tennyson, “What determinates student learning satisfaction in a blended e- learning system environment?”, PACIS 2008, access date: 3/12/12, http://www.pacis-net.org/file/2008/PACIS2008 _Camera- Ready_Paper_149.pdf.
[26] J. Womble, “E-learning: The Relationship Among Learner Satisfac- tion, Self-efficacy, and Usefulness.” The Business Review, Cambridge, vol. 10, no. 1, pp. 182-188, 2008.
[27] B.Yang, “Can Adult Learning Theory Provide a Foundation for Hu- man Resource Development?” Advances in Developing Human Re- sources, vol. 6, no. 2, pp. 129-145, 2004.
[28] M. Prensky, “Digital Native, Digital Natives, Digital Immigrants”,
On the Horizon, vol. 9, no. 5, pp. 1-6, 2001.
[29] Y.H. Chu, “The Effects of an Educational Program on Technological
Literacy: The Example of a Design College in Taiwan”, Education, vol.
126, no. 3, pp. 547-558, 2006.
[30] C. Isik, and S. Yilmaz, “E-learning in life-long education: A computa- tional approach to determining listening comprehension ability,” Education Information Technology, vol. 16, pp. 71-88, 2011.
IJSER © 2013 http://www.ijser.org