Joseph Fong Fu Lee Wang (Eds.) Blended Learning Workshop on Blended Learning 2007 Edinburgh, United Kingdom 15-17 August, 2007 Proceedings Organized by: The Hong Kong Web Society Sponsored by: City University of Hong Kong Preface The Workshop on Blended Learning 2007 provides an open forum for two kinds of professionals: Computer Scientists and Educators, to share their ideas and experiences on supplementing e-Learning with classroom learning. In the reality of today’s education, students can learn anywhere and at any time through Internet. Very often, students can learn new advanced computer technologies through self learning on the Internet, and universities can only provide basic knowledge in the classroom. As a result, there is a need to combine both learning methods to speed up the students' learning abilities. This workshop focuses on e-Learning as a supplementary learning to classroom learning. The era of e-Learning enables students learning any time and anywhere through Internet. It can reduce teachers’ administration work to a minimum by computerizing clerical work. It can also personalize each student’s need by adaptive learning. However, traditional classroom learning is needed for face-to-face lecturing and tutoring. Above all, the experimental work must be supervised and monitored for its progress. As a result, it is important to combine the benefits of both e-Learning and classroom learning into an enhanced teaching method, namely blended learning. Lecturers can then motivate students to learn both in breadth by getting important referenced course materials and practice online, and also in depth by detailed discussion with the lecturers in the face-to-face classroom. The issue is how to create a curriculum which can fit students’ personal needs. Education theory, peer-to-peer learning, and web-based technologies should be applied together in defining techniques and guidelines for the creation of the e-Learning contents and blended curricula. The Workshop on Blended Learning 2007 has record a total of 139 papers. The Workshop had the collaboration of an 18-member program committee composed of well-known blended learning experts and researchers from America, Asia, Australia and Europe. The wide coverage of program committee meant that the Workshop reach out to more participants and thus ensured good-quality paper submissions. The reviewing of papers was carried out by reviewers chosen among the Program Committee members and invited reviewers. The reviewing process was managed using the OpenConf Conference Management System developed by Zakon Group. Twenty-eight papers were finally selected. On behalf of the Program Committee of WBL 2007, we thank all authors for their submissions and camera-ready copies of papers, and all participants for their active participation in the workshop. We also acknowledge the sponsor, program committee members and individuals who gave their continuous help and support in making the workshop a success. We hope that the workshop will continue to grow. Joseph Fong (City University of Hong Kong) Fu Lee Wang (City University of Hong Kong) August 2007 Editors Joseph Fong Department of Computer Science City University of Hong Kong Kowloon Tong Hong Kong e-mail: csjfong@cityu.edu.hk Fu Lee Wang Department of Computer Science City University of Hong Kong Kowloon Tong Hong Kong e-mail: flwang@cityu.edu.hk Conference Organization The Workshop on Blended Learning 2007 (WBL 2007) was organized by the Hong Kong Web Society and sponsored by City University of Hong Kong. Workshop Co-chairs Joseph Fong, City University of Hong Kong Fu Lee Wang, City University of Hong Kong Program Committee Bressan, Stéphan (National University of Singapore, Singapore) Cheng, Yin Cheong (Hong Kong Institute of Education, Hong Kong) Corkindale, David (University of South Australia, Australia) Fong, Joseph (Hong Kong Web Society, Hong Kong) Hughes, John (National University of Ireland, Ireland) Karlapalem, Kamal (International Institute of Information Technology, India) Kwan, Reggie (Caritas Francis Hsu College, Hong Kong) Lau, Rynson (University of Durham, U.K.) Le, Gruenwald (University of Oklahoma, U.S.A.) Li, Qing (Hong Kong Web Society, Hong Kong) Lim, Cher Ping (Edith Cowan University, Australia) McLeod, Dennis (University of Southern California, U.S.A.) Meersman, Robert (Vrije Universiteit Brussel, Belgium) Orlowska, Maria (The University of Queensland, Australia) Sharman, Rex (English Unlimited, Hong Kong) Smith, Peter (Sunderland University, U.K.) Zhao, Wei (Texas A & M University, U.S.A.) Zhu, Zhiting (East China Normal University, China) Reviewers Alberts, Philip (Brunel University, U.K.) Barker, Philip (University of Teesside, U.K.) Bressan, Stéphan (National University of Singapore, Singapore) Chan, Fun Ting (The University of Hong Kong, Hong Kong) Fong, Joseph (Hong Kong Web Society, Hong Kong) Huang, Shi-Ming (National Chung Cheng University, Taiwan) Jones, Norah (University of Glamorgan, U.K.) Lau, Rynson (University of Durham, U.K.) Lim, Cher Ping (Edith Cowan University, Australia) McLeod, Dennis (University of Southern California, U.S.A.) Smith, Peter (Sunderland University, U.K.) Siu, Cheung Kong (The Hong Kong Institute of Education, Hong Kong) Wang, Fu Lee (City University of Hong Kong, Hong Kong) Zhu, Zhiting (East China Normal University, China) Table of Contents Keynote Paper Towards a Definition and Methodology for Blended Learning Won Kim Papers Towards Blended Learning of Computer Programming Supported by an Automated System M. Choy, S. Lam, C.K. Poon, F.L. Wang, Y.T. Yu, L. Yuen Web-based Logging of Classroom Teaching Activities for Blended Learning Joseph Fong Blended Learning for Programming Courses: A Case Study of Outcome Based Teaching & Learning Fu Lee Wang, Joseph Fong, Marian Choy 1 9 19 30 Blended Learning with Webs, Wikis and Weblogs Philip Barker 42 Blended Learning: Beyond Web Page Design for the Delivery of Content Philip P Alberts, Linda A Murray, Darren K Griffin, Julia E Stephenson 53 Discovery of Educational Objective on e-Learning Resource: A Competency Approach Shi-Ming Huang, Hsiang-Yuan Hsueh, Jing-Shiuan Hua Blended Learning: Experiences of Adult Learners in Hong Kong P. W. R. Lee, F. T. Chan 66 79 An Observational Study on Blended Learning for Japanese Language Studies in a Local University in Hong Kong Kenneth K.C. Lee, Melody P.M. Chong 88 Structured Blended Learning Implementation for an Open Learning Environment Jason K. Y. Chan, Ken C. K. Law 101 The Disruptive Effect of Technology: a University Case Study Norah Jones The Marriage of Rousseau and Blended Learning: An Investigation of 3 Higher Educational Institutions’ Praxis Esyin Chew, Norah Jones, David Turner 114 123 Developing an On-line Medical Curriculum Management Platform Jenny Fang, Francis Wong, Raymond Chu 136 Concepts of Blended Learning for Different Content Types Andreas Henrich, Stefanie Sieber 150 Blended Learning: Towards a Mix for SMEs – Stakeholders and their Priorities Sabine Moebs, Stephan Weibelzahl Supporting Self-Regulated Learning in a Blended Course Giuliana Dettori, Donatella Persico A Study on the Impact of the Use of an Automatic and Adaptive Free-text Assessment System during a University Course Diana Perez-Marin, Ismael Pascual-Nieto, Enrique Alfonseca, Eloy Anguiano, Pilar Rodriguez 162 174 186 A Wiki-Based Collaborative Learning Design and Its Effects in Secondary Math Studies Liming Zhang, Chan Lam Wong 196 Implementation of an E-Learning System – Learning Design and Learning Objects Management and Evaluation through Stardardization of Resources Hugo Rego, Tiago Moreira, Francisco José Garcia 206 Visualisation of Learners’ Contributions in Chat Conversations Stefan Trausan-Matu, Traian Rebedea, Alexandru Dragan, Catalin Alexandru 217 HECACEJ: B-Learning Tool for Static Content Creation in Joomla! Angel Mora-Bonilla, Domingo López-Rodríguez, Enrique Mérida-Casermeiro, Salvador Merino-Córdoba 227 MAAS – Mobile Administrative and Assessment System Apple W P Fok, Horace H S Ip, Chilli C K Chan 237 Large-scale Computer-Assisted Assessment in Computer Science Education: New Possibilities, New Questions Mario Amelung, Michael Piotrowski, Dietmar Rösner 257 Exploratory Research on an Affective e-Learning Model Liping Shen, Enrique Leon, Victor Callaghan, Ruimin Shen 267 Design and Development of Blended Learning through LMS K. P. Hewagamage, S.C. Premaratne, K.H.R.A Peiris 279 An Access Monitoring Tool Based on Cookies for Course Management Systems Raquel Hijón-Neira, Ignacio López-López, Ángel Velázquez-Iturbide, Francisco Domínguez-Mateos Learning News Writing Using Emergent Collaborative Writing Technology Wiki Will Wai-kit Ma, Allan Hoi-kau Yuen Description Patterns in Learning Design Authoring Systems Lei Xu, Li Zheng, Jing Liu, Yintao Liu, Fang Yang 292 303 315 M-Learning: A Pedagogical and Technological Model for Language Learning on Mobile Phones Yannick Jolliet 327 Author Index 340 Towards a Definition and Methodology for Blended Learning Won Kim School of Information and Communication Engineering Sungkyunkwan University, Suwon, S. Korea wonkim@skku.edu Abstract. Blended learning is known roughly as combining the traditional instructor-led classroom learning and technology-based elearning. Although there have been attempts to define the term blended learning, and to understand how to make blended learning work best, considerable further research and experimentation are needed. This paper offers a precise definition of blended learning and blended learning program, and proposes a methodology for making blended learning work best. Keywords: blended learning, blended learning methodology, elearning. 1. Introduction The recent rapid advances in information technology, including the Internet and the Web, have had significant impact on numerous aspects of the daily living of the mankind and the society of the industrialized parts of the world. One of the aspects is education. Technological advances and wide availability of personal computers, CDs, the Internet, the Web, broadband access to the Internet, etc. have resulted in e-learning (also known as distance learning or Web-based learning). Numerous corporations (and government branches) have adopted e-learning to train their employees and inform their business partners and customers (citizens). Universities (and schools of other types, including vocational schools, informal municipal classes for their citizens) have adopted e-learning to some extent to supplement or augment the traditional instructor-led classroom courses. People on their own have adopted e-learning as a vehicle for self-study on numerous subjects of interest for various purposes. The traditional instructor-led classroom learning is a proven and effective means of learning, with full opportunities for interaction between the instructor and students, the learning-inducing stress of exams and homework, Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 1-8, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 2 Won Kim and relationship forming among students, etc. However, the requirement for the students to be in the classroom on designated days and times makes it difficult for certain students. Further, the lack of equipment in the classroom may make it difficult for the instructor to teach certain topics effectively. The technology-enabled e-learning can help address such difficulties posed by the limitations of the traditional classroom learning. There is a variety of types of e-learning, ranging from degree-conferring cyber universities and vocational schools, certificate-conferring programs, corporate training programs using videos of instructors delivering instructions – overlaid with PowerPoint pages, to people watching CDs on their computers or browsing the Web for materials on specific topics of interest. However, pure e-learning has its own limitations. Depending on the types of e-learning and how the e-learning is designed, these may include the absence of an instructor, poorly created or inconsistent materials (course contents) – compounded by the absence of an instructor or tutor, the absence of a deadline pressure – which tends to lead to lower learning efficiency, etc. As both the traditional classroom learning and e-learning simultaneously offer strengths and suffer from limitations, it is only natural to combine the strengths of the two into blended learning. In this sense, it may be said that today the most natural form of learning is blended learning. The traditional classroom learning has required research, or at least thoughtful considerations on the part of the instructors for generations, to make it effective for learning. Similarly, research is still needed to realize the full potential of e-learning. Blended learning requires yet additional research to make the combination of the traditional classroom learning and e-learning as effective as possible. There have been several earlier attempts to define blended learning [1][2][3][4][5][6][7][8][9]910][11][12]. There have been efforts to define a methodology for designing blended learning programs [6][8][9][10]. In this paper, I will first give a precise definition of blended learning and blended learning program, and then, based on the definitions, propose a methodology for making a blended learning program work best. 2. A Precise Definition of Blended Learning As a first approximation, blended learning may be defined as a combination of classroom learning and e-learning. This definition can, and needs to, be made more precise, although I will use the term e-learning loosely to mean learning outside the traditional classroom using information technology for the delivery of the learning materials (i.e., course contents). The approach I will take is to first consider the different types of learning that are possible. Towards a Definition and Methodology for Blended Learning 3 There are at least three key dimensions to consider in classifying learning: physical class-based or virtual, formal or informal, and scheduled or selfpaced. Learning may be classified as physical class-based or virtual classbased. Physical class-based learning is the traditional classroom learning where the instructor and the students must physically be present in the classroom at the same time. Virtual class-based learning does not require the instructor and students to be present in the classroom. E-learning is necessarily virtual class-based. Learning may be classified as formal or informal. Formal learning is tied to earning a degree, certificate, or official credit. Informal learning is not. Learning may also be classified as scheduled or unscheduled. Scheduled learning proceeds on a fixed schedule, while there is no fixed schedule for informal self-paced learning. To be precise, there are two types of schedule: one is the schedule for the class (i.e., lectures), and another is the schedule for a course (i.e., start and end of a semester, and schedules for homework and exams). The traditional course in a university has both the class schedule and the course schedule. Self-paced learning does not come with a class schedule. However, one type of self-learning comes with a course schedule, and another type comes with no course schedule. If we are to go with the three dimensions of learning outlined above, there are eight possible combinations. However, upon a closer examination, the number of possible combinations is much larger than eight. First, it makes sense to refine the notion of schedule, from scheduled and self-paced, into class schedule, course schedule, and no schedule. Further, it is possible to introduce one additional dimension of learning, namely, availability of faceto-face interactions outside the class between the students and the instructor/tutor. For example, an independent study or an industry co-op course does not involve a lecture class; however, it requires face-to-face interactions outside the class between the students and the instructor/supervisor/tutor. These additional considerations will potentially result in 24 learning types. I will not list them here, but the following are some of the possible learning types. 1. Formal, class-scheduled and course-scheduled, physical class learning. This is the traditional learning. 2. Formal, course-scheduled, no physical class, but face-to-face interactionbased learning. Examples of this type of learning include dissertation research, independent studies and industry co-op courses offered by universities. 3. Formal, class-scheduled and course-scheduled, e-learning. This is distance learning, and is the e-learning equivalent of the traditional learning. Students do not need to be in the classroom, and can participate in the class from anywhere. However, the class is offered on a fixed schedule, just like the traditional class. 4 Won Kim 4. Formal, course-scheduled e-learning. This type of e-learning has been adopted by universities and corporations and government branches. 5. Informal, class-scheduled and course-scheduled, physical class learning: This is the traditional learning corresponding to auditing where students do not receive a formal credit. 6. Informal, unscheduled e-learning. This type of e-learning is adopted by people who would like to simply learn something about a topic on their own from anywhere at any time. I can now define blended learning precisely as “a combination of two or more of all possible learning types” discussed above. There is one important qualifier to this definition. At least one of the learning types must be a physical class-based type (regardless of whether there is a course-schedule; or whether it is formal; or whether there is face-to-face interaction outside the physical classroom), and at least one other learning type must be e-learning type (regardless of whether there is a class schedule or a course-schedule; or whether it is formal; or whether there is face-to-face interaction outside the physical classroom).This is to make sure blended learning remains a combination of some form of traditional learning and some form of elearning. Now that I have defined blended learning precisely, I will define blended learning program. A blended learning program is a coherently designed learning program that is applied to a range of learning activities, ranging from a topic or a course fragment, to a course, and even a curriculum. A blended course is a course that consists of some course fragments of traditional learning types and other course fragments of e-learning types. A blended curriculum is a set of courses, where some of the courses are blended, some are purely e-learning courses, and others are purely traditional courses. A blended learning program would generally refer to a number of courses, or an entire curriculum, although it is possible to talk about it in the context of a single course, a single topic, or even a course fragment (a part of a course, involving multiple topics), or even an entire institution (i.e., across multiple curricula). I am not sure what percentage of a course must be e-learning types before the course can reasonably be called a blended course. Similarly, I am not sure how many of the courses must be blended courses or purely e-learning courses, before a curriculum can reasonably be called a blended curriculum. It appears that for both the course and curriculum the percentage may be at least 33 in order to prevent unreasonable claims from some schools or corporations who only seek propaganda value while essentially offering only the traditional learning or only formal e-learning. Towards a Definition and Methodology for Blended Learning 3. 5 Methodology for Creating a Blended Learning Program Blended learning, and e-learning for that matter, does not just happen by simply injecting some e-learning elements into a traditional course, or some elearning courses into a traditional curriculum. There are several important issues in designing an effective blended learning program, both at the level of a course and a curriculum. Any methodology for creating a blended learning program must include three elements, as follows. 1. Setting the scope and objectives of the blended learning program. 2. An analysis of the relevant issues under the overall objectives set. 3. Addressing any problems identified in the analysis. 4. Measuring the results of the blended learning program. The scope of a blended learning program, as already discussed, would normally be an entire curriculum or at least a number of courses. Appropriate objectives must be selected and used to guide the design and implementation of a blended learning program. There are various potential objectives for implementing a blended learning program, as follows. 1. Increased learning effectiveness (for the students or employees), over either pure traditional classroom learning or pure e-learning. 2. Increased convenience (for the students or employees). In the case of corporations, blended the e-learning component of a blended learning program can make it easier for the employees when business trips or highpriority business meetings come up to prevent them from attending scheduled in-class training. 3. Enhanced image (for the school or the corporation). The progressive image may be projected both internally (to own students or employees) and externally (the general public, customers, the government, news media, the financial analysts, etc.). 4. Cost savings (for the school or the corporation). The cost savings may result from possibly reducing the number of instructors, and, in the case of corporations, having employees not use critical work hours for scheduled in-class training. 5. Classroom space savings (for the school or the corporation). The e-learning component of a blended learning program can help ease the classroom space needs by having students and employees learn more from outside the classrooms. The freed-up classroom space can potentially be used for other purposes. 6. Reduced traffic and parking congestion on the campus or the corporation. Key issues to be carefully analyzed follow. 6 Won Kim • Cost issue: To add the e-learning element into a traditional course or curriculum, equipment, hardware, networking and software are needed in general. There is the associated cost of installing, maintaining and upgrading them; possibly hiring and paying technical support staff; possibly adding administrative staff; training the staff and instructors; content development and upgrade, etc. • Management issue: A decision must be made as to how much of the work should be done with in-house staff or outsourced to service organizations. This issue is closely related to the cost issue, and includes the following considerations. 1. Purchase, installation, and management of the equipment, hardware, networking and software. 2. Selecting technologies to use. This has to do with setting and observing school-wide or corporation-wide standards on the equipment, hardware, networking, and software, so that there is overall consistency, relative ease of receiving services and volume discounts from the vendors, etc. 3. Hiring and training technical and administrative staff. • Selection of vendors: Once it is decided to outsource some of the work and people needs, vendors must be selected. There needs to be a set of criteria for selecting the vendors, including cost estimations provided, strategic importance they attach to the work being outsourced, their capabilities and limitations, their financial stability, reputations from their existing customers, etc. • Considerations of the students: In designing the e-learning component of a blended learning program, the students’ and employees’ ability to access the e-learning contents must be taken into account. The access ability includes the computing power of the PCs they will use, the types of peripherals that come with the PCs, operating systems and their versions that run on the PCs, web browser versions they will use, Internet access bandwidth, availability or accessibility to video conferencing, etc. Course contents should not include parts that a lot of students and employees (accessing from their homes) cannot view or listen with the computers and Internet bandwidth available to them. Further, the students’ and employees’ special needs and situations that may affect the scheduling of face-to-face meetings or exams, the reaction to the contents of the course materials, etc. should be taken into account. These include religion, culture, national origin, physical handicap, etc. In particular, course contents should not include statements, illustrations, examples, etc. that may offend the students and employees. • Considerations of the classroom space: Since the e-learning part of a blended learning program does not in general require the use of a Towards a Definition and Methodology for Blended Learning 7 classroom, the classroom can be put to good use. Good scheduling of a blended program that consists of a number of courses in the same course period can yield a fair number of free classrooms on a regular basis during the course period. If the classroom space is at a premium, the freed-up classrooms can be quite helpful. • Determination of the composition of learning types: To make a blended learning program work best, both at the curriculum and course level, the best composition of the learning types must be determined. The determination of the best composition must be done on the basis of the precise definition of blended learning discussed earlier. Further, it must be done by considering all of the issues discussed above. It is important that there be some pre-determined set of measures and a methodology for measuring the degree of success of the blended learning program. The measures would include actual costs incurred and actual benefits achieved. The benefits in turn would include all the objectives chosen for the blended learning program. 4. Concluding Remarks The emergence of e-learning, and the realization of the strengths and limitations of the traditional classroom learning and e-learning, have naturally led to the notion of blended learning. In this paper, I first provided a much more precise definition of blended learning and blended learning program than available in the literature. Then, based on the definition, I proposed a practical approach for developing a methodology for designing a blended learning program. Looking forward, in my view, just as large corporations have been the first to adopt e-learning to train their employees on a wide variety of subjects, such as company orientation, leadership, project management, six sigma, interpersonal skills, new products, etc., it will be the corporations that will combine e-learning with the traditional learning into blended learning. Further, municipal governments are likely to add e-learning components to the continuing education programs they run for their citizens on a wide variety of subjects, such as first-aid, the Web basics, etc. As such, I would like to urge academic researchers interested in advancing the methodology for designing blended learning programs to set their bearings to corporate training programs and municipal government’s continuing education programs, and the very wide and general nature of the subjects on which they conduct training and education. 8 Won Kim Acknowledgments. This research was supported by the Korean Ministry of Information and Communication under the ITRC IITA-2006-(C1090-06030046) grant. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. Bersin & Associates, “Blended Learning: What Works?”, May 2003, http://www.e-learningguru.com/wpapers/blended_bersin.doc Josh Bersin, “Blended Learning: Finding What Works”, http://www.clomedia.com/content/templates/clo_feature.asp?articleid=357&zon eid=30 Charles Dziuban, Joel Hartman, Patsy Moskal, “Blended Learning”, University of Central Florida, Educause Center for Applied Research, Research Bulletin, vol. 2004, number 7, March 30, 2004. http://ecampus.nl.edu/resources/webct/BlendedLearningUCFStudy.pdf Epic Group plc., “Blended Learning” Epic white paper, http://www.epic.co.uk/content/resources/white_papers/Epic_Whtp_blended.pdf Charles Graham, “Blended Learning Systems: Definition, Current Trends, and Future Directions”, in Handbook of Blended Learning: Global Perspectives, Local Designs, Pfeiffer Publishing, San Francisco, USA, 2006. http://www.uab.edu/it/instructional/technology/docs/blended_learning_systems. pdf Michael McGinnis, “Building a Successful Blended Learning Strategy”, http://www.ltimagazine.com/ltimagazine/article/articleDetail.jsp?id=167425 Martin Oliver and Keith Trigwell, “Can ‘Blended Learning’ Be Redeemed?”, E-Learning, vol. 2, no. 1, 2005. http://www.wwwords.co.uk/pdf/viewpdf.asp?j=elea&vol=2&issue=1&year=20 05&article=3_Oliver_ELEA_2_1_web&id=59.13.102.130 Allison Rossett, Felicia Douglis, and Rebecca Frazee, “Strategies for Building Blended Learning”, Learning Circuits, 2003, http://www.learningcircuits.org/2003/jul2003/rossett.htm Harvi Singh and Chris Reed, “A White Paper: Achieving Success with Blended Learning”, Centra Software, 2001. http://www.centra.com/download/whitepapers/blendedlearning.pdf Harvi Singh, “Building Effective Blended Learning Programs”, Educational Technology, Nov./Dec. 2003, pp. 51-54. http://www.bookstoread.com/framework/blended-learning.pdf Richard Voos, “Blended Learning – What Is It and Where Might It Take Us?”, http://sloan-c.org/publications/view/v2n1/blended1.htm Wikipedia, “Blended Learning”, http://en.wikipedia.org/wiki/Blended_learning Towards Blended Learning of Computer Programming Supported by an Automated System1 M. Choy, S. Lam, C.K. Poon2, F.L. Wang, Y.T. Yu and L. Yuen Department of Computer Science, City University of Hong Kong Abstract. Learning computer programming is known to be difficult for many beginners. With the primary aim to improving the practice of teaching and learning of computer programming, we have developed a web-based automated system, known as PASS, for use in our courses. PASS has provided capabilities with opportunities for new pedagogy and innovative strategies for both teaching and learning. This paper describes the design and functionalities of PASS, how it can be used to enhance students’ interest of learning and monitor their progress, and illustrates several approaches with which PASS can facilitate blended learning of good practices in computer programming. Keywords: Computer programming, automated system, learning. 1 Introduction Thanks to the booming development of the Internet, there is currently a trend towards the convergence between learning via on-campus classes and remote learning through the network. Blended learning, combining online and face-to-face instruction, has become one of the hot topics in both the academic and corporate circles [7]. It is believed that, in the long run, the word “blended” will eventually be dropped as it becomes the norm in pedagogy. Learning computer programming has been known to be difficult for many beginners [3]. Programming skill has to be acquired through lots of practice [4]. When teaching programming, the instructor typically gives many programming exercises and assignments to students, hoping that they will gain enough hands-on practice. However, assessment of programming exercises and assignments by manual inspection of code on paper is notoriously inefficient and error-prone [9]. Alternatively, the instructor may require students to perform demonstrations of the execution of their code, or to submit their program source code so that the instructor may execute the programs manually one by one. In this way, the correctness of the students’ programs can be more reliably evaluated, but still this method is labourintensive and time-consuming. More importantly, such a practice is educationally undesirable because students generally have to wait for a long time before they can receive feedback from the instructors to help them improve their work. 1 This work is supported in part by a Teaching Development Grant (project no. 6000118) from City University of Hong Kong. 2 Corresponding author. Phone: +852-27887157. Fax: +852-27888614. Email: ckpoon@cs.cityu.edu.hk . Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 9-18, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 10 M. Choy et al. With the primary aim to improving the practice of teaching and learning of computer programming, we have developed a web-based automated Programming Assignment aSsessment System, better known as PASS, for use in our courses [5]. The basic function of the system is to allow the student to submit their program code, automatically execute the program with predefined test cases (determined by the instructor beforehand), and return the execution results to the student. In this way, the student gets instant feedback from PASS as to whether his/her program is correct with respect to the selected test cases. When the submitted program is incorrect, PASS will also indicate to the student exactly at which position the actual output differs from the correct (expected) output. For assignment submissions, the instructor can execute students’ programs with a much larger set of hidden test cases in a single batch so that a more comprehensive assessment of the submissions can be performed efficiently. Afterwards, all the execution results can be released to the students by the instructor so that they know exactly which test cases their programs fail, if any. This kind of prompt feedback to students in the whole class was rarely possible before PASS was developed [6]. Since its first introduction a few years ago, PASS has proved to be a valuable and indispensable tool for both students and instructors [6, 11]. PASS has now undergone significant revisions, with enhanced capabilities that have created opportunities for new pedagogy and innovative strategies for both teaching and learning. This paper describes the design and functionalities of PASS, how it can be used to enhance students’ interest of learning and monitor their progress, and illustrates several approaches with which PASS can facilitate blended learning of good practices in computer programming. In the next section, the system functions and design of the current version of PASS will be described. Section 3 describes, with the aid of a running example, several approaches to blended learning using PASS, followed by concluding remarks in Section 4. 2 The Programming Assignment aSsessment System (PASS) PASS was first developed in our university in 2004 as an executable prototype for demonstration [5], but it was soon re-engineered to a working system for production use [11]. Although the first version of PASS contained the core functionalities which were already very helpful to students, it nevertheless carried its built-in design limitations that significantly hindered its enhancement. To facilitate its sustained use, we have recently substantially re-designed it with major revisions. In this section, we shall first present the core entities and their relations, and then some newly enhanced functionalities in the current version (version 3) of PASS. These functions have provided capabilities that enable new approaches to the teaching and learning of computer programming that are later described and illustrated in the next section. Towards Blended Learning of Computer Programming 11 2.1 System functions and design Core entities. Central to PASS are several core entities: problem, submission, test result, course and various roles of user. A problem refers to a programming exercise or assignment issued to students for practice or assessment. It includes a description (specification) of the program to be written, supporting files (such as additional documentation, library code or data files) and a set of test cases for assessing the correctness of program submissions. Each problem is associated with a release date (when the problem is publicised), a submission period (during which students may submit their programs), and a close date (after which submission is not allowed). A student may view a given problem, work on it to produce a program to solve the problem, and then submit the program to PASS for testing. A program plus any auxiliary files submitted to PASS is a submission. The result of executing a submitted program with preset test cases in PASS is called a test result, which includes the information of test runs, memory and time spent, and other related information. The creator of the problem, usually the instructor, may specify the compiler with which the submitted programs are compiled and executed. For practice problems, students can view all the test cases (including both test inputs and expected outputs) when their programs are tested in PASS. These test cases are thus said to be open. For assessment problems, the instructor may specify some test cases to be open, and leave the remaining (usually a larger number of) test cases to be hidden, which are not revealed to students before the submission period ends. The hidden test cases are only executed for the purpose of grading students’ submissions. They are designed to prevent students from tailoring their programs to test cases and to encourage students to ensure that their programs are generally correct. PASS can handle many courses at the same time. Apart from the system administrator who looks after the technical administration and support, there are four other roles of user in PASS, namely course leader, course developer, tutor and student. Course leader is overall in charge of a course, normally the course lecturer. A course leader has all rights to access and manage the records of all students and problems in his/her course. Students can view the problems in their course(s), submit programs and check the results and statistics of their program executions. Tutor and course developer are the roles of people who assist the course leader to manage the course with limited access rights. A tutor can access the details and performance of the students of his/her group(s). A course developer is responsible for designing the problems and test cases, but is not allowed to access the private records of students. Such a design enables the course leader to designate part of the work load of problem administration to course developers without worrying about the disclosure of privacy of students. The relationships among the core entities in PASS are summarised in Figure 1. 12 M. Choy et al. Fig. 1. Relationships among core entities in PASS. Submissions monitoring and analysis. PASS preserves all the submissions made by students and analyzes such information to facilitate the monitoring of the overall progress and performance of the course, selected groups of students, or each individual student. The computed statistics can be grouped by different attributes. For example, when grouped by students, the instructor can easily identify the students with outstanding performance, as well as those who consistently lag behind. On the other hand, when grouped by exercises or assignments, common difficulties or problems encountered by students can be revealed. The instructor may further inspect finer details of the problems by checking the effectiveness of each test case in detecting program faults. Temporal statistics are also available to view the usage and activity profiles of the students. All statistics are displayed in easy-to-read charts, and the instructor may focus attention to selected groups or problems. Selected statistics can be disclosed to students so that they can evaluate their own performance in relation to that of their peers. Towards Blended Learning of Computer Programming 13 Problem sharing in problem repository. In general, the problems created for a course are only accessible by the participants in the course. To facilitate collaboration among instructors, a mechanism is designed in PASS for sharing and reuse of problems. An instructor may choose to release his/her problems to a repository in PASS so that instructors in other courses can access and reuse them as appropriate. The instructor can retrieve a problem from the repository, amend it to suit his/her own course, and optionally release this new version to the repository. The original version of the problem will be retained intact to make sure that any amendment will not affect other PASS users who are making use of the original problem. 3 Approaches to Blended Learning using PASS The teaching and learning of computer programming is a challenging job. A welldesigned pedagogy can make the instruction most effective. However, the traditional mode of education imposes a lot of constraints to the design of pedagogy. Certain types of teaching and learning activities are extremely difficult to conduct within a standard classroom setting. Students often find it difficult to master those advanced programming paradigms without adequate practice. Blended learning combines different media and methods to maximise students’ learning. PASS enables us to blend our courses to render the teaching and learning of computer programming more effective. It is believed that effective instruction involves working the content to provide stepwise learning experiences with checks along the way to assure that each step is learned [10]. It is important to ensure that students are well-trained in the fundamentals to the extent that they can eventually consider some problems with high-level complexity. We pay extra care to design the teaching and learning activities to incorporate stepwise learning. Students taking computer programming course very often come with various backgrounds and ability levels. PASS allows us to design exercises with different levels of difficulty to fulfil the need of students with various backgrounds and ability levels. Also, we can create a series of test cases with different levels of difficulty for the same programming problem (see Figures 2 and 3). To illustrate the idea, we take the programming exercise of solving a quadratic equation as a running example (Figure 2). We may group the test cases into three levels of difficulty, namely, the beginner level, intermediate level and advanced level (Figure 3a). The equations which have two distinct real roots are considered relatively easier; and therefore we classify the corresponding test cases as at the beginner level (Figure 3b). The test cases which correspond to quadratic equations with one repeated root or two complex roots are classified as at the intermediate level (Figure 3c). The exceptional cases (such as those corresponding to the cases when the equations become linear or identities) are classified as at the advanced level (Figure 3d). PASS allows us to tell the students the level of difficulty of each exercise (Figure 3a). For the same problem, students can attempt the exercises based on their capability. 14 M. Choy et al. For example, the less talented students may design a simple program to solve the problem at the beginner level of difficulty (Figure 3b). If they submit their programs to attempt exercises at other levels of difficulty, they will fail in those test cases (Figure 2c-d). Instead, they must enhance their programs in order to solve the problem at the intermediate level of difficulty. However, the talented students may work directly to solve the problem at the intermediate level of difficulty. They can even try to challenge the exceptional test cases by submitting their programs to solve the problem at the advanced level of difficulty. Eventually, they should come up with a single program which can solve the problem up to a certain level of difficulty. This approach allows the students to regulate their own learning pace. Additionally, we may require students to study the given test cases for each exercise to figure out how we select the test cases so as to learn how to test their programs on their own. A Programming Exercise of Solving a Quadratic Equation Write a program to solve a quadratic equation. The general form of a quadratic equation is ax2 + bx + c = 0, where a, b, c are real numbers. When a ≠ 0, the solution of 2 the equation is given by the quadratic formula x = − b ± b − 4ac . 2a In this exercise, we assume that all the inputs are integers, and that the outputs are to be displayed in descending order. Test Cases at the Beginner Level of Difficulty Input a = 1, b = –5, c = 6 a = 2, b = –7, c = –15 a = 1, b = 6, c = 8 Expected Output x = 3, 2 x = 5, –1.5 x = –2, –4 Test Cases at the Intermediate Level of Difficulty Input a = 1, b = –2, c = 1 a = 1, b = 2, c = 5 a = 2, b = 12, c = 18 Expected Output x=1 x = –1+2i, –1–2i x = –3 Test Cases at the Advanced Level of Difficulty Input a = 0, b = 2, c = 4 a = 0, b = 0, c = 0 a = 0, b = 0, c = –4 Expected Output x = –2 x = any real numbers No solution Fig. 2. Description of a programming exercise at different levels of difficulty. Towards Blended Learning of Computer Programming (a) Programming Exercises at Different Levels of Difficulty (b) A Sample Run of Submission to an Exercise at the Beginner Level of Difficulty (c) A Sample Run of Submission to an Exercise at the Intermediate Level of Difficulty (d) A Sample Run of Submission to an Exercise at the Advanced Level of Difficulty Fig. 3. Exercises at different levels of difficulty and sample runs of submissions in PASS. 15 16 M. Choy et al. Furthermore, a programming course typically has a large class size. It is difficult to closely monitor individual student’s learning progress in the traditional teaching mode. The instructor does not have enough time to interact with all students in a class of hundreds of students within a few hours of lectures and tutorials each week. PASS has recorded all the program submission activities of students. These data provide very useful information of students’ performance. For example, we can easily know a student’s progress in the course based on the number of attempts to each exercise and the number of exercises completed. The summary of student performance supports fast decision making as well. We can identify the slow learner quickly and offer corresponding help. At the same time, we encourage the talented students to offer peer assistance to other students. Moreover, we find that the talented students put in additional efforts to study all related materials before they offer help. Besides, they can further enhance their understanding by explaining the programs to others. This approach improves the learning of both groups of students. As instructors of computer programming courses, we not only have to teach students how to write programs, but also need to develop our students with good programming practices, such as modularity, reusability and information hiding. With the support of PASS, we have incorporated and promoted these good programming practices in designing our teaching and learning activities. When we design a programming activity, we break the program into smaller modules in the form of functions and classes. The students are required to implement their individual modules in separate files. The files are then uploaded to PASS for testing by using the test driver which is pre-loaded to PASS by the instructor. This approach highlights the modularity of computer programs. The students are exposed to programs that are built from modules so that they learn the concept of modularity of program by implicit learning [2]. When developing an application, students will have to divide their solutions into modules as functions and classes. Moreover, we require the students to archive all the files developed in their activities. When designing a programming activity, we intentionally require the students to make use of some modules developed in previous activities by importing the corresponding files. For example, we may require students to develop a program to solve a quadratic inequality (Figure 4) based on the module developed earlier in the programming exercise of solving a quadratic equation (Figure 2). Similarly, as before, we create exercises at different levels of difficulty (Figure 3a). The least talented students can solve the inequality by using their simple programs that solve a quadratic equation, while the talented students can try some challenging test cases such as when the quadratic inequality has one solution or no solution. In this way, students will naturally acquire the concept of code reuse through their own experience of reusing the previously developed code, as concrete experience is important in the learning cycle [8]. On the other hand, we sometimes ask students to exchange files and develop their applications based on modules written by other students. For different exercises of the same programming problem, we sometimes provide different implementations of the same module as separate exercises in PASS. Students are required to submit their solutions to all exercises of the problem. In order to pass all the exercises, the students need to write the code purely based on the module interface. By doing these exercises in PASS, students will gradually recognise the importance of information hiding. Towards Blended Learning of Computer Programming 17 A Programming Exercise of Solving a Quadratic Inequality Write a program to solve a quadratic inequality based on the module you developed earlier in the programming exercise of solving a quadratic equation. In general, a quadratic inequality can be written in one of the following standard forms, where a, b, c are real numbers: ax2 + bx + c ≥ 0 ax2 + bx + c > 0 ax2 + bx + c ≤ 0 ax2 + bx + c < 0 Suppose that the equation ax2 + bx + c = 0 has two real roots x1 and x2 , where x1 < x2 . If a > 0 , the solution sets of the inequalities are, respectively, as follows. Inequality: ax2 + bx + c ≥ 0 . Solution: ( –∞, x1 ] ∪ [ x2 , +∞ ) Inequality: ax2 + bx + c > 0 . Solution: ( –∞, x1 ) ∪ ( x2 , +∞ ) Inequality: ax2 + bx + c ≤ 0 . Solution: [ x1 , x2 ] Inequality: ax2 + bx + c < 0 . Solution: ( x1 , x2 ) Fig. 4. A programming exercise based on a previously completed module. Finally, traditional pedagogy focuses on the development of small applications. Without the support of related technology, students usually develop small applications by writing the code solely on their individual effort. The student may become an analyst programmer in the future and may be involved in some large scale projects. Students often find it difficult to manage large software development jobs when they work in the industry. It is very important to provide students with experiences of software development in large scale applications while they are studying. However, there are practical difficulties to require students to develop a large application. First of all, students’ learning motivation drops very fast as the time they have to spend on study increases. If we require the student to code a large application, they are usually unable to see their results before the completion of the whole application. They will lose their interests in programming soon after they started. Lack of motivation is one of the major resistances to learning [1]. As we foresee the need, we consider large application development as an essential part of an advanced programming course. Some special arrangements have to be made to keep the students’ learning motivation. When we design a large application, we may divide the application into several modules. After the student has completed one module of the application, he/she can submit the modules to PASS. Some stubs or test drivers can be provided for testing their individual modules. It is important to reinforce the student’s success upon his/her completion of one module. This approach also increases the student’s confidence in learning. The intermediate results can keep students’ learning motivation constantly high. The students will develop the application in a progressive manner. After the students have completed the entire application, they can submit it to PASS, which will test all the modules together as a single integrated application. 18 M. Choy et al. 4 Conclusion This paper has described the design and functionalities of the latest version of PASS. PASS now allows the instructor to effectively locate the slow learners and identify the high achievers by the statistical data it collects while students are using it to test their own programs. We have also explained, with concrete illustrative examples, how classroom teaching and learning activities may be blended with practice exercises that are supported and monitored by PASS. By designing exercises at different levels of difficulty, PASS can provide stepwise learning experiences to students, such that they can solve problems pertaining to their corresponding ability levels. Instructors can also define problems in various ways in PASS so as to make students familiar with modules programming and be prepared for large projects. We have demonstrated just a few ideas on how PASS can be used to support blended learning. With the flexibility and extensibility of PASS, we anticipate that PASS will also facilitate other new and innovative strategies for blended learning. References 1. J.S. Atherton (1999), “Resistance to learning: A discussion based on participants in inservice professional training programmes”. Journal of Vocational Education and Training 51(1):77-90. 2. D.C. Berry (ed.) (1997). How Implicit is Implicit Learning. Oxford University Press. 3. B. du Boulay (1989), “Some difficulties of learning to program”. In E. Soloway and J.C. Spohrer (eds.), Studying the novice programmer. Hillsdale, N.J.: L. Erlbaum Associates. 4. B. Cheang, A. Kurnia, A. Lim, A., and W.-C. Oon (2003), “On automated grading of programming assignments in an academic institution”. Computers & Education 41(2):121131. 5. S.L. Chong and M. Choy (2004), “Towards a progressive learning environment for programming courses”. In R. Cheung, R. Lau and Q. Li (eds.), New Horizons in Webbased Learning: Proceedings of the 3rd International Conference on Web-based Learning (ICWL 2004), World Scientific Publishing Co. Ltd., pp. 200-205. 6. M. Choy, U. Nazir, C.K. Poon and Y.T. Yu (2005), “Experiences in using an automated system for improving students’ learning of computer programming”. In Proceedings of the 4th International Conference on Web-based Learning (ICWL 2005). Lecture Notes in Computer Science (LNCS no. 3583), Springer, pp. 267-272. 7. C.R. Graham (2006), “Blended learning systems: definition, current trends, and future directions”. In C.J. Bonk and C.R. Graham (eds.), Handbook of Blended Learning: Global Perspectives, Local Designs. San Franciso, CA: Pfeiffer Publishing. 8. D.A. Kolb (1984). Experiential Learning: Experience as the Source of Learning and Development. New Jersey: Prentice-Hall. 9. U. Nazir, C.K. Poon, Y.T. Yu and M. Choy (2005), “Automated assessment for improving the learning of computer programming: Potentials and challeges”. In Proceedings of the 9th Global Chinese Conference on Computers in Education (GCCCE 2005), pp. 634-639. 10. M. Schulman (2001), “Basic understandings for developing learning media for the classroom and beyond”. Learning Technology Newsletter 3(1). 11. Y.T. Yu, C.K. Poon and M. Choy (2006), “Experiences with PASS: Developing and using a programming assignment assessment system”. In Proceedings of the 6th International Conference on Quality Software (QSIC 2006), IEEE Computer Society Press, pp. 360-365. Web-based Logging of Classroom Teaching Activities for Blended Learning Joseph Fong Department of Computer Science, City University of Hong Kong, Hong Kong csjfong@cityu.edu.hk Abstract. Nowadays, eLearning has been adapted in all educational institutes, starting from kindergartens, primary schools, high schools, to the universities. Teachers are encouraged to use computer and Internet as a teaching medium in addition to the classroom teaching. The result is a blended learning which combines eLearning as a supplementary learning means to classroom learning. The issue becomes what is the best approach for blended learning for the effective and productive teaching methods. This paper suggests to record (log) all teaching activities into a web site for students to self study after classroom learning. The objective is to provide students more flexibility in learning with more facilities of eLearning, classroom learning and/or blended learning. A case study is illustrated in the paper for discussion. Keywords: blended learning, teaching activities, classroom learning, eLearning, Education 1 Introduction Education in general are undergoing rapid transition from the traditional learning and teaching to the more self-motivated mode constructivism through the application of IT. Many education institutes have made a tremendous step forward in the setting up of EDO (education office) in the promotion of eLearning. Efforts have been made on the implementation of a university wide educational Extra/Internet with the aim of facilitating a better learning and teaching environment as well as improving the overall student administration. Products such as WebCT [1] and Blackboard [2] have been in use for the past few years with a degree of success. As a rule, these products all suffer on two counts: 1. 2. They focus more on course management rather than learning process. Little consideration is given to areas such as workshops and projects. There is a complete absence of semi-automatic assessment facilities with performance tracking for facilitating an understanding of the strength and weakness of students. Basically, eLearning can be categorized into different areas as follows: Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 19-29, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 20 Joseph Fong eLearning for adaptive learning –In order to speed up the learning curve of learners, we aim to supplement classroom learning with eLearning. The students can do self learning from web-based learning exercises on the Internet, which will assess the students＇ academic level and provide them with suitable online exercises to work on. As a result, the students can learn everywhere, any time through Internet. eLearning for teacher helpers – In order to reduce teachers ＇ workloads in authoring exercises for their students, in getting student feedback and in communicating with students more effectively, many eLearning systems facilitate these functionalities for teachers as their helpers. For example, eLearning authoring tools can help teachers to prepare exercises, and BLOG journal files can help teachers get student feedback on a particular learning problem or subject. eLearning for distance learning –Besides traditional classroom learning, students can also access learning facilities remotely. For example, a virtual cyber laboratory can help students to perform their laboratory exercises at home through the Internet. An online tutorial session can help students access their tutors through their notebook computers. Consequently, students can learn in a very comfortable environment without incurring traveling time, cost and hassle. eLearning technology infrastructure – In order to make eLearning a success, technology infrastructure is a must. In fact, eLearning is more cost effective whenever it can be accessed by mass learners, because individual costs will be less with increased numbers of learners for eLearning while the eLearning development time and effort is fixed. In other words, once an eLearning package has been produced, it can be reused many times with minimal operating cost. However, such mass learner operations need the support of technology. Thus, research on eLearning technology infrastructure is another important area for researchers to explore. On the educational side, Blended Learning provides educators with the opportunity to showcase best practice by reporting the match between learning or outcome objectives and the development and implementation of this hybrid (blended) learning strategies and material to engage students in achieving those objectives. Ideally underpinned by learning theory, we should also attempt to demonstrate the value added to learning by using the blended approach. Both components of hybrid learning — the classroom and the eLearning contributions — should be presented to the students.. 2 Related work A definition of Blended Learning [3] is the combination of multiple approaches to learning, such as self-paced, collaborative or inquiry-based study. Blended learning can be accomplished through the use of 'blended' virtual and physical resources. Examples include combinations of technology-based materials, face-to-face sessions and print materials. Another definition of Blended learning [4] is a powerful training solution that combines elearning with a variety of other delivery methods for a superior learning experience. A third definition of Blended learning [5] combines face-to-face and computer mediated instruction. Web-based Logging of Classroom Teaching Activities for Blended Learning 21 Some educators [6] give their experience of implemented of Outcome-based learning as blended learning in University of Sydney. As standards for the use of Information and Communication Technology in teaching and learning are a relatively new aspect of the blended learning, their adoptions by the academics are still in its early stage. Other educators [7] valuated blended learning as a mixture of online and fact-toface learning using a variety of learning resources and communications options available to students and lecturers. In so doing a learning environment is created that is richer than either a traditional face-to-face environment or a fully online environment. ICWL2006 keynote [8] states that Learning theory, collaborative learning, and deep consideration of the technologies that may be leveraged should all be pulled together in defining techniques and guidelines for the creation of great Web-based contents and blended curricula. The objective of this paper is to implement the idea in the keynote. 3 Classroom teaching activities logging In general, classroom activities include lectures, tutorial session, laboratory, project, review, questions and answers, open forum and presentation etc. Each course focuses in certain aspect of activities due to its syllabus requirements. To the students, they are mainly interested in two issues: how much do they learn and what is their grade for the course. If the lecturer can log these activities into a web site for student’s reference, it will be easier for students to know the priority importance of each teaching activity as follows: z z z Lectures: - We can video record lectures for students to review them at home. However, this involves facilities management and operations. Many long distance education institutes provide such services for remote learning. Also, sometimes students complain about spending too much time copying the written notes from the blackboard. In this case, lecturer may consider storing the images of the written notes into the web site for students to download after the lecture. Furthermore, references can also be put into the web site for students to enhance their knowledge in the subject. Tutorial sessions – Students come to tutorial sessions for small group learning. They can ask more questions in the sessions for peer-to-peer learning, with more in-depth discussion of the subject. An effective approach is for teachers assess the students’ knowledge by giving tutorial exercises in the sessions. After marking the answers from students, the lecturers can discuss the answers with the students. Thus, the questions and answers are logged into the web site for students to review them. In this way, the students can learn even if the students are absent in the sessions. Laboratory exercises – Students use laboratory for their hands-on exercises. 22 Joseph Fong However, they may not have enough time to finish them in the laboratory. An alternative is to do the exercises on the Internet at home. Sometimes, for long distance education institutes, they allow students access laboratory facilities through remote logging. Both techniques can serve the students well. z Open forum – It is important for students discuss questions about the lecture among themselves. An open forum in the classroom and in a web site can be very helpful. A BLOG system is good to log these open forum conversation. Students can refer them even though they were not involved at the beginning of the discussion. This open forum is managed by the lecturer just in case incorrect message or information is passed among students. z Mind set diagram – A course in general covers many subjects. It is important for students to know the learning sequence of these subjects. A mind set diagram of the course can help students realize the position of each subject in the course. In other words, how the subjects are related to each other. Students should also be allowed to put their comments in the subjects which are the syllabus of the course. A picture is worth thousands words. The diagram is very helpful for the students’ understanding of the course. z Grading system – A computerized grading report system can assess students knowledge effectively and make them realize their understanding of the course materials immediately. The lecturers post the students’ grade of each assignment online along with the model answers so that the students can correct their mistakes at once. This will trigger students ask more questions in the class. z z Project management – Lecturers act as project supervisors to student’s projects. They define the project requirements and monitor student’s progress through face-to-face interview during the duration of the project. Furthermore, an eLearning system allows students to ftp their work to a departmental server on a regular basis. A test log of submitted work and simple version control tracks students’ performance. A particular server allows students to install software in pre-specified folders. This reduces the chance of hard coding, which is a bad practice normally committed by most students. Video conferencing facilities can be in place to allow formal communication between students and supervisor at pre-defined interval, such as once a week, and a log on students’ progress and supervisors can be maintained. . Assessment – The assessment system usually composes of two elements: quiz and test. Quiz provides immediate feedback to enhance the learning process whereas tests are for evaluation of students progress with the following functions: 9 Calculate marks automatically 9 Generate reports on the performance for quiz as well as test for the individual, the class and for different cohorts. 9 Students can learn at their own pace and are free to choose the time and level for their quiz with summary feedback on the performance indicating Web-based Logging of Classroom Teaching Activities for Blended Learning 9 z z 23 weakness in certain areas. Students’ learning pattern rules can be derived through statistical analysis of their learning results be the course web site using data mining approach. Course Work Management – Students can submit coursework with specific instructions and assigned readings. The collection of assignments is based on a predetermined schedule and at the pace of students’ progress. 9 Create lockers with unlimited number and level 9 Edit the lockers 9 Remove the lockers 9 Submit homework to the locker through the web browser 9 Acknowledgement on receipts to avoid disputes Scheduling – This acts as a communication channel between lectures and students. Course assignments, submission deadlines and requirements can be broadcasted to students by lecturers. The lecturers can also mark down project meetings and agreed work schedule with students. Shareware can be modified for this purpose. 4 A Case Study To illustrate the teaching techniques of blended learning, we choose teaching a course on “Data warehousing and Data mining” as a case study. The subject is rich in content, and each syllabus can be taught in depth for several lectures. The course aims to focus on the practical application of the data warehousing and data mining. The whole course web site hierarchy is designed as simple as possible. It is done according to Hick’s law [9]. It has predicting for a web site which consumes the user lesser time to browser if the hierarchy is small. In turns, the user will get the appropriate data easily with less short-term memory load. The home page covers the course teaching plan, scheduling, and outcome based teaching and learning objectives. The coursework consists of the project assignment, review and tutorial questions, hands-on and eLearning exercises, and references etc. The Grade web page is to list out students’ grades for each submitted assignment. The Open forum is for students communicating with each other and the lecturers for general discussion on the lecture. The Mind Mapping web page describe the learning sequence of syllabuses in the course. In the whole web sites, every web page contains a header and a navigation bar, the consistency layout let the user easily browse the web site without getting lost. Furthermore, there is no table in the layout. It is good for both search engine optimization as well as the accessibility of the users who use the screen readers as shown in Figure 1. 24 Joseph Fong Figure 1 System Architecture of a course web site In each lecture, lecture notes, written notes, references, review questions, tutorial questions, hands-on and eLearning exercises are all put into the course web for students to review and comprehend. For lecturers who will be the web masters, instead of uploading and downloading the files from the web site and local drive for each update, the system provides an easy way for the web masters to change the content of the website so as to make the website more user-friendly. For students who will be the end users, they can download lecture notes, written notes, references, review questions and tutorial questions from the course web site as shown in Figure 2 below: Web-based Logging of Classroom Teaching Activities for Blended Learning 25 Figure 2 Course material web pages for the case study In order to help students learn effectively, a mind set diagram is set up on the course web site for the students to know the learning sequence of the course. They are the syllabuses of data cleaning, data integration, star schema design, data cube loading, followed by different data mining techniques of Association Rules, Clustering, Decision Tree, Neural Network, Web Mining and Genetic Algorithm. XML technology is introduced in the course for report generation and transmission on the Internet. The arrows signs show the sequence of learning syllabus in the course. For students, there is a mind-map learning page provided for them as end users. They can give out their opinion about the structure of the course with fully understanding the basic structure of the web site as shown in Figure 3. 26 Joseph Fong Figure 3 Mind Set diagram for course on Data Warehousing and Data Mining Instead of uploading and downloading of excel file to update the grades of students, the system can provide dynamically function for the web master to record the grade with the address [10]. After marking the review and tutorial questions, the grades are put on the course web site. The score ranges from-0 (no score) to 1 (full score). The review questions are on theories and the tutorial questions are on the application of the theories taught in the class. Students are encouraged for peer-to-peer learning through discussion to each other before submitting their answers as shown in Figure 4. Web-based Logging of Classroom Teaching Activities for Blended Learning 27 CW = 25% |PT = 25% |Exam= 50% | Student ID Rev 1 Tut 1 Rev 2 Tut 2 Rev 3 Tut 3 Rev 4 Tut 4 Rev 5 Tut 5 25% CW 50184799 1 0.3 0.3 0.3 0.3 1 1 1 1 1 0.72 50453076 1 0.3 0.3 0.3 0.3 1 1 0.3 0.3 1 0.58 50478728 1 0.3 0.3 1 0.3 1 1 1 1 1 0.79 50797788 1 1 0.3 0.3 0.3 1 1 0.3 0.3 0.3 0.58 50832039 1 1 0.3 1 1 1 1 1 1 1 0.93 50854068 1 1 0.3 1 0.3 1 1 1 1 1 0.86 50860456 1 0.3 1 1 1 1 1 1 1 1 0.93 Average 1 0.6 0.4 0.7 0.5 1 1 0.8 0.8 0.9 F 4C a a y In order to encourage peer-to-peer learning, an open forum is put on the course web site for students discuss problems and solution among themselves. They can also ask lecturers on the lectures. The open forum applies blogging system such that students are free to append their comments to each particular subject that they are interested in. Security need to be implemented to block non-students intruders erase the blog messages as shown in Figure 5. 28 Joseph Fong Figure 5 Open forum for the course web site 5 Conclusion Ever since IT has been widely used in teaching, blended learning is the trend of teaching method for all levels of educational institutes. Combined with eLearning as supplementary learning avenue to class room learning, this paper introduces many ways of logging teaching activities into the web site which can help students learn more effectively inside and outside of the classroom. The result will be a very close partnership between teachers and students in teaching and learning. Web-based Logging of Classroom Teaching Activities for Blended Learning 29 Reference 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. URL http://www.WebCT.com URL http://www.Blackboard.com URL http://en.wikipedia.org/wiki/Blended_learning. URL: http://www.e-learningcentre.co.uk/eclipse/Resources/blended.htm Graham. Charles (2003), "Blended Learning Systems" in Bonk, C. J. & Graham, C. R. (Eds.). Handbook of blended learning: Global Perspectives, local designs. San Francisco, CA: Pfeiffer Publishing. (An Imprint of Wiley) Applebee, A., Elis, R., and Sheely, S.,(2004) "Developing a blended learning community at the U of Sydney", in R. Atkinson, C. McBeath, D. Joans-Dwyer and R. Philips (Eds), “Beyond the Comfort Zone” Proceedings of the 21st ASCILITE Conference, pp.58-66. Harding, a., Kaczynski, D., and Wood, L.,(2005) “Evaluation of blended learning analysis of qualitative data”, Proceedings of UniServe Science Blended Learning Symposium, pp.56-61. Won Kim (2006), “Directions for Web-Based Learning”, LNCS 4181, Proceedings of The 5th International Conference on Web-Based Learning, pp.1-7. URL http://www.hockscqc.com/articles/hickslaw.htm URL http://www.cs.cityu.edu.hk/~jfong/grade/CS5483 Blended Learning for Programming Courses: A Case Study of Outcome Based Teaching & Learning Fu Lee Wang, Joseph Fong, Marian Choy Department of Computer Science, City University of Hong Kong, Kowloon Tong, Hong Kong {flwang, csjfong, csmchoy}@cityu.edu.hk Abstract. Teaching of programming has created significant difficulties. Blended learning is a mixture of face-to-face instruction and computer-assisted instruction. This paper will share our experiences in City University of Hong Kong (CityU) as we teach programming courses by blended learning. Evaluation has showed that blended learning provide great flexibilities to both teaching and learning of computer programming. The students’ academic results have been greatly improved in programming course. Keywords: Blended Learning, Computer Programming, Outcome Based Teaching & Learning. 1 Introduction Computer programming is an essential fundamental skill required in many curriculums for higher education nowadays. It is commonly believed that the students would develop their general problem-solving skills through learning programming. However, teaching of computer programming has created significant difficulties for high-school and university students, and has failed to catalyze the development of higher order thinking skills [13]. A number of challenges have been identified for both teaching and learning programming [13]. Related research has showed that intelligent computer-assisted instruction (ICAI) technology can be a more effective way of teaching introductory programming courses - for certain populations [3]. Blended learning is to combine face-to-face instruction with computer-assisted instruction [7, 8]. Blended learning is the convergence of two representative learning environments. The traditional face-to-face learning environment has been used for centuries. On the other hand, the rapid development of technologies provides distributed learning environment as an alternative. In the past, these two environments are separated because they use different media/method combinations. Therefore, they are used to address the needs of different audiences [7]. Traditionally, distributed learning is used as an expansion to the face-to-face learning. Taking the university education as an example, the face-to-face learning environment is used in a teach-directed synchronous environment where the interpersonal interaction is a key component. On the other hand, the distributed Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 30-41, Pearson, 2007. Workshop on Blended Learing 2007, Edinburgh, United Kingdom. Blended Learning for Programming Courses 31 learning environment is usually used in distanced learning, which focus on asynchronous self-paced learning and learning-material is the key component [8]. The rapid development of technology has a significant impact on the learning environment. In fact, there is an increasing trend to integrate the two learning environments as a single system. Nowadays, more and more universities conduct the learning activities under both environments. As there is an increasing need for blended learning systems, efforts have been continuously devoted into the research of blended learning [7]. A number of blended learning platforms have been developed in real world. Blackboard Academic Suite is a well-developed learning platform [17] which has been widely used in educational institutes. WebCT is used to be a competitor of Blackboard [18]. At this moment, two leading companies co-operate to provide services to the education industry. The City University of Hong Kong (CityU) has put a lot of effort in development e-learning. The university has deployed the Blackboard Academic Suite (Bb) [17] as its unified campus-wide e-learning platform. Many e-learning activities are currently conducted in this platform. The Department of Computer Science is one of the departments who are devoted in e-learning development. Last year, the department underwent a major restructure in the course design in order to incorporate the implementation of Outcome Based Teaching & Learning (OBTL). We have successfully implemented blended learning to teach computer programming courses. This paper is going to share our experiences of blended learning. We have combined the advantages of both learning environments to deliver computer programming courses. The statistics has shown that students are greatly benefited with this mode of study. 2 Teaching & Learning Programming Course at CityU City University of Hong Kong (CityU) is one of the eight institutions of higher education which are financially supported by the government of Hong Kong. The University has achieved phenomenal growth since its establishment in 1984. The University ranks the top 154 world-wide [15]. The University is strong in technical science. It ranks the world’s top 51 in the field of engineering, technology and computer science [1]. We will share our experiences in teaching computer programming courses at CityU. The Department of Computer Science is one of the funding departments under the Faculty of Science and Engineering. In addition to the courses offered to the students who are majored in Computer Science, the department also offers a lot of service courses to the students from other departments. Computer programming is one of the common courses offered to students from various departments. We have implemented a blended mode of teaching and learning programming course at CityU. 32 Fu Lee Wang, Joseph Fong, Marian Choy 2.1 Outcome Based Teaching & Learning As a strategic plan, the City University of Hong Kong focuses on enhancement of teaching and learning qualities. The University has implemented Outcomes Based Teaching and Learning (OBTL), which is a student-centered approach for education. The curriculum topics in a program and the courses contained in it are expressed as the intended outcomes for the students to learn. Teaching is designed to directly encourage the students to learn those outcomes and assessments will then be done to confirm that. It is an approach in which the students themselves are actively engaged in their learning while the teachers are trying to facilitate them to do so. Derek Bok, President Emeritus and Research Professor at Harvard University, believes that we need a reformulation in the Undergraduate Education. He identified some core purposes of undergraduate education [5]. On the other hand, Harvard University has conducted a review of their entire undergraduate educational programs [10]. They are going to implement a new curriculum for undergraduates. The OBTL is developed on similar concept. The university identified some core outcomes (or objectives). Those outcomes state what the students are expected to be able to do at the end of a program. Based on the program outcomes, the teachers decide the course outcomes, which state what the students are expected to be able to do at the end of a course. The OBTL has been implemented in a number of institutes over many countries, for example, USA [16], United Kingdom [12], Australia [2], South Africa [11], etc. The OBTL is developed based on the concept of constructive alignment [4]. The key elements of a course, such as learning activities and assessment tasks, must be aligned to each other so that the intended outcomes may best be achieved. Teaching and learning activities are designed such that the students are required to enact the learning activities and therefore they will most likely to achieve the intended outcomes. The activities can be teacher, peer, or self-initiated. The students actively gain knowledge through engaging in appropriate learning activities. Traditional teaching starts from the perspective of the teachers. OBTL works from the perspective of the learners. Under a constructively aligned curriculum, students are actively engaged in their learning processes where the teaching are focused on what the students, rather than on what the teachers, are doing. The teacher needs to consider what outcomes have the students achieved, how to demonstrate those outcomes have indeed been achieved, and to what standards. It encourages all teachers to ask these questions by providing points for reflection on teaching. Unlike traditional teachings, the teachers have to pay a lot of efforts to design the course outcomes and related activities, instead of simply deciding on what topics to be covered. 2.2 OBTL with Blended Learning One of the major components in OBTL is the design of teaching and learning activities. These activities must be coherently aligned with the intended learning outcomes. However, the traditional classroom education does not meet the requirements of OBTL. Traditionally, the learning activities are limited inside the Blended Learning for Programming Courses 33 classrooms and assessment tasks are usually in the form of examinations. In order to enhance the qualities of teaching and learning, blended learning is introduced to the programming courses in CityU to implement OBTL. Blended learning combines classroom education with e-learning technology. It provides a large degree of flexibilities to the teachers for course design. With the support of technologies, the teachers at CityU are able to deliver the course materials in multiple channels. In the following sub-sections, we will explain more details how program courses are delivered in mixed channels. Activities in different formats can be provided to the students with time and geographical constraints. This approach not only promotes active learning, it also challenges the students to take control of their own learning. On the other hand, assessments play an important part in OBTL. Assessments are designed to align with the course outcomes to provide evidences on how well each student has achieved the outcomes. Moreover, assessments must be able to measure students in multiple dimensions. The students are evaluated by their performances in each outcome Such evidence could be provided by project work, case studies, assignments, examinations, laboratory work and reports, practicum, etc. 2.3 Computer Programming Course at CityU In the past, computer programming courses at CityU are taught in a traditional mode. The course was delivered in a mixture of large-sized face-to-face lectures and smallsized face-to-face tutorials (which might be in the form of laboratory sections). The students were evaluated by coursework and final examinations. The coursework was usually in the format of programming assignment or written quiz, and the final examination was in the format of paper-based written examinations. After implementation of Blended Learning, the courses are delivered in multiple channels: • The teachers present the primary course materials in the large-sized face-to-face lectures. • Small-sized face-to-face tutorials are conducted by the tutors to allow students to do some programming practices. • Supplementary course materials are delivered in Internet through the university elearning platform. For example, extensive examples are provided to help students to appreciate the good programming skills. • Computer programming clinic scheme (subsection 2.3) is setup to provide consultations to students. • Online intelligent computer-assisted instruction system (subsection 2.4) is developed to provide a programming practice platform to the students. There is one major difference between the OBTL and traditional teaching. The learning outcomes are clearly stated at the beginning of the courses. Each learning activity is aligned with the learning outcome. It provides a high level of transparency to the students. The students have a clear picture about the course structure. They know what learning activities they must enact in order to achieve the intended outcomes. On the other hand, the students are assessed by how well they have achieved the outcomes. The assessments are usually measured in multiple 34 Fu Lee Wang, Joseph Fong, Marian Choy dimensions. In CityU, the assessments of a programming course include the followings • Both online and offline short quizzes will be conducted to evaluate the student’s performances during the semester. • Students are required to do some programming assignments to demonstrate their programming capabilities. The e-learning platform allows a great flexibility in assignment design. For example, we provide some testing modules in the elearning platform. This allows the student to complete the assignments stepwisely. • The data collected in the intelligent computer-assisted instruction system may also be used to evaluate the students. • The e-learning environment in blended learning makes it feasible to conduct online programming quizzes. 2.4 Computer Programming Clinic Scheme at CityU For effective learning, it is important to provide the students with a good learning environment of computer programming. A pilot scheme of Computer Programming Clinic has been setup in CityU with the support of the Department of Computer Science. The clinic recruits students of senior years who are good at programming as programming consultants. The consultants will share their programming experiences with students of junior years. The senior students have similar backgrounds as the junior students. They understand clearly what problems the junior students are currently facing, and therefore be the most suitable persons to offer helps. This clinic is developed based on the idea of “help desk” system. We have setup a face-to-face clinic in the Computer Laboratory of Department of Computer Science. The junior students can visit the clinic for consultations during school hours. The consultants will perform the following tasks: • answer students’ questions related to general programming, • help students to identify the bugs in their program at high levels, • help students to formulate high-level pesudocode before programming, • suggest some appropriate readings for the students if necessary, • demonstrate a small segment of programming code to the students to help them to understand their program if necessary, • assign some simple tutorial problems to the students to help them to understand the programming concepts. Another obstacle for the students to learn programming is that they do not know where to seek help when they have encountered problems. Usually, the students encounter problems when they are doing some programming work after they returned home. Most students will put the problems aside and forget to solve their problems when they return to school. It greatly reduces their enthusiasm to study programming, if they lack of instant supports. In order to provide instant supports to the students beyond normal school hours, we have setup a virtual extension of the Computer Programming Clinic in Internet. CityU has deployed the Blackboard Academic Suite (Bb) as its unified e-learning platform [17]. To align with the e-learning strategic development of the University Blended Learning for Programming Courses 35 and to eliminate the development cost, we have implemented online clinic based on the Blackboard (Figure 1). During school hours and after school hours, consultants will be on duty to offer help to the junior students. We devote actively in monitoring the students’ learning. In addition to face-toface and on-line consultations, the consultants will analyze the coursework submitted in the programming courses and data in the electronic systems to identify the slow learners. Moreover, the instructors of the programming courses will also refer the students who have difficulties in programming to our clinic. Corresponding personalized learning program will be provided to the students. This project will greatly enhance the learning environment of computer programming. Fig. 1. Computer Programming Clinic – Online Clinic 2.5 Online Intelligent Computer-Assisted Instruction System for Programming Instant support to the student is a critical factor to the success of teaching and learning of programming. However, it introduces a huge pressure in the resources, and it may 36 Fu Lee Wang, Joseph Fong, Marian Choy not be affordable by some universities. It has been showed that intelligent computerassisted instruction technology can be a more effective way of teaching introductory programming courses [3]. We have implemented a computer-assisted instruction system to supplement our supports. Figure 2 shows the Programming Assignment aSsessment System (PASS). The PASS system is a web-based computer-assisted instruction system for computer programming developed at CityU [6]. The PASS system is a fully automated system to help students to study programming. Fig. 2. Programming Assignment aSsessment System (PASS) The PASS system allows the instructors to setup some tutorial problems. The instructors provide the input and the corresponding output to each test case. The students then submit their program for testing. The system automatically complies and executes the program submitted. By comparing the output generated by the students’ program and the expected output provided by the instructor, the system will then provide feedbacks to the students. For example, if the student gets wrong in certain type of inputs, the system will show the attached annotation provided by the instructor to give some hints of possible mistakes to the student. The instant feedback Blended Learning for Programming Courses 37 provided by the system provides concrete assistances to students to revise their programs, and debugging will become more interesting. On the other hand, some teaching strategies are incorporated with the intelligent computer-assisted instruction system: • In OBTL, a student need to demonstrate their skills in each intended learning outcome in order to pass the programming course. When the teachers design the programming activities in the PASS, they will associate each activity with specified outcome. This approach helps the teachers to review the effectiveness of their programming activates to each intended outcome. • The PASS system has a powerful statistical function of students’ performance in each programming problems. It can tell individual student’s performance in each outcome. Therefore, the system allows the teacher to measure students in multiple dimensions. Moreover, teachers have a close monitoring of the students’ learning progress. • The system allows the teachers to give test cases in different difficulty levels. For example, some normal cases will be tested as warm-up exercises to the students. Later on, some boundary cases will be used to test the robustness of the students’ programs. The students may even try some exceptional cases. This mechanism allows the students to control their paces of learning. On the other hand, we can cater the needs of students with various learning capabilities. • Traditionally, the students are required to complete a program before they can do some testing. Our system allows the teachers to provide the main body of the program, and students to submit their implementation of functions to the system, or vice versa. Then, the system will integrate the source codes together as a single program. This type of exercise is welcomed by the students, as they can speedily see the outcomes of their program without writing many lines of code. It significantly improves student’s incentive of learning. Moreover, it makes the students understand more about the concept of modular programming and also implementation hiding. • Finally, the system allows the students to submit components of a program as separate files. The components files may be developed by individual student. The system will integrate the source files together as a single project. This team-work style exercise makes the students understand the paradigm of software development. The PASS system allows a tailor-made learning pace and style for individual student. It has provided a quick and convenient channel for students to test their work without manual involvement. Instant feedback to students encourages them to enhance their programming skills. The introduction of PASS has made the learning of computer programming more rewarding than before. 3 Evaluation and Discussion A number of extensive evaluations have been conducted to measure the effectiveness of our teaching model. Evaluation results have shown that the blended learning is a promising approach in teaching and learning programming. 38 Fu Lee Wang, Joseph Fong, Marian Choy 3.1 Evaluation of Course Structure It is suggested that the blended learning can be measured by interviews and questionnaires [9]. We have conducted both evaluations to measure the effectiveness of the blended learning. A focus group session has been held with students who enrolled for any blended courses of computer programming in the Department. A set of interview questions are designed by professionals in education development. The students are interviewed by an independent interviewer and none of the course lecturers were presented. All the students in the focus group believe that the blended learning model can help them to learn the programming course more effectively. Most of the students appreciate the flexibilities provided by the blended learning. The students can selfcontrol their learning paces. The anytime/anywhere studying mode allows them to work at the time when they have the highest productivities. Moreover, the students become more independent and self-disciplined in their learning. Their time management skills are also enhanced. The preliminary results of interview suggest that blended learning is a good teaching and learning model. On the other hand, all the students appreciate interactivity of the online assessment system. However, some of the students are less satisfied. They hope that the online intelligent computer-assisted instruction system can provide more feedbacks to help them to debug their programs. This provides some directions for future enhancement of the system. Effectiveness of self-paced learning Effectiveness to identify student's weakness Effectiveness to learning programming Effectiveness to encourage continuous learning Effectiveness to achieve learning outcomes Effectiveness of tutorials Effectiveness of programming assignment Effectiveness of supplementary web 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.0 0 Fig. 3. Result of Questionnaires about the Course Structure In order to get a more quantitative measurement for the course structure of programming courses, we have conducted a survey by questionnaires. The questionnaires are designed by professionals in education development in the similar way as [9]. The students are asked to score each dimension of the course structure on Blended Learning for Programming Courses 39 the scale from 0 to 10, where a score of 10 represents the highest satisfaction, while 0 represents the least satisfaction. 250 students have participated in the survey. The results are summarized as Figure 4. In the Figure, we can clearly see that the students are highly satisfied with the course structure. The students are happy with the flexibilities provided by the blended learning. They most believe that the mixture of teaching channels, such as mixture of tutorial, assignment, supplementary web and online assessment system, can effectively teach them about computer programming. They help the students to identify their weakness and control their own learning paces. Therefore, the students can achieve the intended learning outcomes effectively (Figure 3). An ideal blended learning is a mixture of classroom learning and electronic learning. Self-paced learning is one of the major advantages of electronic learning [7, 8]. However, the students are generally less satisfied with the effectiveness of selfpaced learning (Figure 3). As a result, there is potential to further blend our courses. In the future, we will investigate on how to improve the self-pace learning in programming courses. 3.2 Evaluation of Students’ Performances In the past, the students taking programming courses in CityU are assessed by coursework and final examinations. The coursework was usually in the format of programming assignments, and the final examination was in the format of written examinations. After implementation of OBTL, the students are assessed in multiple dimensions. We have compared the results of the students before and after the implementation of OBTL. Table 1. Statistics of Course CS2362 Computer Programming for Engineers & Scientists Year 2004 Year 2005 Year 2006 Total no. of students 277 253 251 Grade Score Boundary % of Students % of Students % of Students A 69.5 7.94% 7.11% 26.00% B 54.5 16.25% 17.79% 22.40% C 39.5 35.38% 23.72% 23.60% D 34.5 9.75% 12.65% 4.80% F below 34.5 30.69% 38.74% 23.20% We have selected the course CS2362 Computer Programming for Engineers & Scientists as an example. This is a typical programming course at the introductory level. Because the class size of this course is very large, the statistical information of this course is worthy trusted. On the other hand, the materials of assessment are moderated by peer review to ensure the standard of assessment. No scaling of score has been conducted in this course. The score boundary for each grade has been fixed 40 Fu Lee Wang, Joseph Fong, Marian Choy by the department. As a result, this graded distribution of students is a very important indicator to show the performances of teaching and learning. In years 2004 and 2005, the programming courses were taught in traditional mode. Only a small percentage of students got grade “A”, while a large percentage of students failed the course in these two years (Table 1). These students who failed the course may retake the course in the next year. After the blended learning was implemented in year 2006, the percentage of grade “A” students increased dramatically from 7~8% to 26% (Table 1). At the same time, the percentage of failure decreased significantly. As shown in the table, the students’ performance in the programming course increase significantly after the implementation of the blended learning. This is a strong evidence to show the success of the blended learning. 4 Conclusion This paper has shared our experiences in implementing blended learning in teaching computer Programming in City University of Hong Kong. Outcome Based Teaching and Learning is a new pedagogy. However, traditional teaching model impose a lot of constraints in implementing OBTL. In contrary, blended learning provides great flexibilities to both the teachers and the students. At the same time, it can cater students with different learning paces. The interviews and questionnaires have shown that blended learning is very effective in teaching and learning computer programming. The students’ performances in the assessments have further confirmed our findings. We think that blended learning can be applied to other courses in the future as well. 5 Acknowledgement The PASS System is a System developed by C.K. Poon, Y.T. Yu, Marian Choy, Fu Lee Wang, Celine Chong, Usman Nazir, P. F. Tam, Isaac Yeung, Sam Lam and Leo Yuen at Department of Computer Science, City University of Hong Kong. The projected was substantially supported by a grant from City University of Hong Kong (Project No. 6000118). The Computer Programming Clinic is a project lead by Fu Lee Wang, Kenneth Lee and Jiying Wang at Department of Computer Science, City University of Hong Kong. We would like to thank individuals for their significant contributions to the paper. References 1. Academic Ranking of World Universities by Broad Subject Fields – 2007, Top 100 world universities in Engineering/Technology and Computer Sciences, available at http://ed.sjtu.edu.cn/ARWU-FIELD2007/ENG.htm Blended Learning for Programming Courses 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 41 Applebee, A.C., Ellis, R.A., & Sheely, S.D., (2004), Developing a blended learning community at the University of Sydney: Broadening the comfort zone, Proceedings of the 21st ASCILITE Conference, pp. 58-66, Perth, Western Australia, 5-8 December 2004. Anderson, J. R., & Skwarecki, E. (1986). The automated tutoring of introductory computer programming. Communications of the ACM, 29(9):842-849. Biggs, J. (2003) Teaching for Quality Learning at University, 2nd Edition, McGraw-Hill Education, Open University Press. Bok, D. (2005), Our Underachieving Colleges: A Candid Look at How Much Students Learn and Why They Should Be Learning More, Princeton University Press. M. Choy, U. Nazir, C.K. Poon and Y.T. Yu, Experiences in using an automated system for improving students learning of computer programming. In Proceedings of the 4th International Conference on Web-based Learning (ICWL 2005), Lecture Notes in Computer Science (LNCS), volume 3583, Springer, Hong Kong, 31 July 2005 - 3 August 2005, pages 267-272. Graham, C.R., (2005) Blended learning system: definition, current trends, and future directions. In Bonk, C. J. & Graham, C. R. (Eds.). Handbook of blended learning: Global Perspectives, local designs (pp 3-21), San Francisco, CA: Pfeiffer Publishing. Graham, C.R., & Allen, S. (2005). Blended learning: An emerging trend in education. In C. Howard, J.V. Boettecher, L. Justice & K.D. Schenk (Eds.), Encyclopedia of Distance Learning (99. 172-179). Hershey, PA: Idea Group Inc. Harding, A., Kaczynski, D. & Wood, L.N., (2005), Evaluation of blended learning: analysis of qualitative data, Proceedings of the Symposium of Blended Learning in Science Teaching & Learning, pp. 56-62, The University of Sydney, Australia, 28th-30th September 2005. Harvard University, Faculty of Arts and Sciences, (2004), A Report on the Harvard College Curricular Review, April 2004. Mason, M.B. Outcomes-based Education in South African Curricular Reform. Cambridge Journal of Education, Cambridge, UK, Carfax, 1999, 29(1): 137-143. Ross N & Davies D (1999). Outcome-based learning and the electronic curriculum at Birmingham Medical School, Medical Teacher 21(1): 26-31 Sleeman, D., (1986), The Challenges of teaching computer programming. Communication of the ACM, 29(9):840-841. Soloway, E., (1986), Learning to program = learning to construct mechanisms and explanations, Communications of the ACM, 29(9):850-858. Times Higher Education 2006 World University Rankings, available at: http://www.thes.co.uk/worldrankings Zahorchak, G.L. and Boyd, W.L., “The politics of outcome-based education in Pennsylvania”, Annual Conference of the University Council for Educational Administration, Philadelphia, PA, October 1994. Blackboard Academic Suite, http://www.blackboard.com/us/index.Bb WebCT, http://www.blackboard.com/webct Blended Learning with Webs, Wikis and Weblogs Philip Barker School of Computing, University of Teesside, Middlesbrough, TS1 3BA, UK. P.G.Barker@tees.ac.uk http://www.philip-barker.net Abstract. This paper discusses blended learning in terms of the optimal use of multiple learning channels and the educational messages that they can each embed. Blending algorithms are introduced as a mechanism for specifying the composition of blends and a learning spirals model is introduced as a vehicle to relate stimuli (embedded within educational messages) to the cognitive processes that these are intended to stimulate. Webs, wikis and weblogs are then discussed in terms of their importance as components within a blended elearning environment. Their significance both as a learning resource and as a knowledge management tool is discussed. Keywords: blended learning, blending algorithms, learning spirals, webs, wikis, weblogs 1. Introduction The fundamental axiom underlying this paper is that there is a need to pass skills and knowledge from one generation of people to another using whatever technologies that we have at our disposal. This could involve the transmission of material during a face-to-face encounter between a teacher and one or more students. It could involve the sending of an educational document from an author who is an expert in a particular area to someone else who is less well-versed in that subject. Such a transmission could involve a postal service or a computer network system. Of course, an educational transmission could also involve listening to a radio programme or a podcast or it might require watching a film or television programme. There are also many books, journals and other communication artefacts (both electronic and paper-based) that can be used to support the fundamental process of educational delivery. In this current age of ‘digital computing systems’ much learning now takes place through the medium of computers. These may be ‘stand-alone’ devices that exist within a learning organisation (such as a school or university) or in someone’s home or workplace. Increasingly, however, most computers are now networked together using either a private network system or a public facility such as the Internet. With Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 42-52, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. Blended Learning with Webs, Wikis and Weblogs 43 the advent of cellular wireless networks, we are moving towards an era in which we have ubiquitous electronic inter-connectivity. Bearing in mind what has been said above, it is important to realise that we now have a tremendous range of educational resources available. This raises the rather simple, but very important question of which ones should be used (in any given situation) in order to achieve a particular educational objective. Unfortunately, although the question is a simple one, finding an answer is much more complex. This complexity arises because of the many factors that need to be taken into account within any given teaching and learning situation. The primary concern of educators (that is, those involved in the educational delivery process) will be the effectiveness with which ‘educational messages’ can be created, delivered to and assimilated by a particular target audience. Of course, other stakeholders within an educational system (such as politicians, bureaucrats, lawyers and managers) will be more concerned with other issues such as the legality, validity, relevance and cost of the messages that are being broadcast to consumers. Being an academic, my primary concern is therefore with a consideration of how we generate educational messages, transmit them to students and then measure their effectiveness in terms of the behaviour changes that they induce. So, what is the best way of doing this? My honest answer would be: “I don’t really know”. I say this because the question must be made much more specific before an attempt can be made to answer it. Thus, what would work well in one situation might fail completely in another. In some situations, a lecture course could be used to encode relevant educational messages - and may work very well. In other situations, a text book may be a better alternative while in others, practical involvement may be a necessity for successful delivery. In my own discipline of computing, for example, it is inconceivable that I could teach computer programming without letting students actually become involved in programming activities. Effective teaching and learning in this particular context involves (minimally) two basic types of learning activity. First, attending lectures (or a suitable substitute) in order to learn about the theory and principles of computer programming and how to apply these using a particular programming language. Second, undertaking practical exercises that are designed to develop the skills needed to produce efficient and effective computer programs. Effective learning in this situation thus involves an appropriate ‘blend’ of two different types of learning activity. In this example, two ‘learning channels’ are being used in order to achieve the best possible learning experience for the people involved. More channels could be used (at extra cost) or either one could be used alone (but with reduced efficiency of learning). If we try to generalise what has been said above, we can define blended learning as an approach to pedagogy that mixes various types of educational resource - or learning channel - (that may originate from many different sources) in order to create an optimal learning experience for a consumer. The consumer could be an individual person or a group of people - such as a university class or an online community of learners. Of course, the larger the group of people involved, the more difficult it becomes to tune an educational message to the needs of particular individuals within that group. This is an age-old problem that has always plagued conventional face-to-face teaching (involving group sizes of more than one) but, for 44 Philip Barker which, solutions can be found within the domain of electronic teaching and learning. The way in which blended learning might address this issue is briefly discussed later. In this paper I would like to try to explain my model of blended learning and its relationship to the cognitive processes that this approach to learning is intended to stimulate and support. Naturally, an important aspect of learning is the acquisition of knowledge and skills. I am particularly interested in the ways in which people acquire knowledge, manage it and share it with others. The latter part of the paper will therefore deal with how electronic artefacts such as webs, wikis and weblogs can help to support blended learning and the acquisition, sharing and management of both private and public knowledge. 2. Underlying Theory According to Carman (2002) blended learning theory arises from the merging of three basic philosophies of knowledge and skill acquisition: cognitivism, constructivism and performance support. Each of these areas is concerned with a slightly different aspect of the ways in which individuals develop their knowledge and skills. Cognitivism is concerned with understanding the way in which the human mind works (cognitive behaviour) in relation to the various mental states in which it can exist. It can be used to explain thinking processes and creativity in terms of various ‘information processing’ models - see, for example, the ‘HIP’ model proposed by Card, Moran and Newell (1983). Constructivism deals with how individuals develop knowledge as a result of the various experiences to which they are exposed (Barker, 2004). The performance-support dimension of blended learning is more concerned with the development of skills (both physical and cognitive) and how different types of performance aid can be developed in order to overcome fundamental human limitations with respect to the different types of task that they undertake (Barker, 1995; Barker, van Schaik and Famakinwa, 2007). From a purely pragmatic perspective, the roots of blended learning lie within the multi-channel approach that was discussed in the previous section. This model of blended learning is depicted schematically in Figure 1. As can be seen from this diagram, each channel within a given learning environment can host a number of different ‘modes’ of educational communication. Each mode, in turn, will require an appropriate linguistic framework within which to encode the educational messages that were previously referred to in the Introduction to this paper. Of course, the messages that are embedded in a channel can also employ different styles or genres of communication. Naturally, it is hoped that the educational messages carried by the channels depicted in Figure 1 will embed appropriate stimuli that will activate and motivate a learner to assimilate the message content in an appropriate way, build new and/or augment existing cognitive structures and modify his/her behaviour accordingly. Naturally, an understanding of the principles and techniques of semiotics is an important asset in relation to designing educational messages and predicting their affect on human behaviour (Chandler, 2006; de Souza, 2005). The role of semiotics Blended Learning with Webs, Wikis and Weblogs 45 in relation to message transmission (in general) and educational messages (in particular) is discussed elsewhere (Barker, 2006a). Fig. 1. Blended Learning in terms of multiple learning channels. My own definition of blended learning and its realisation in terms of the activation of learning processes within the cognitive space of a learner is described in detail elsewhere (Barker, 2006a; 2006b). Within that work, an attempt was made to define blending algorithms in terms of the contributions made by the various learning channels that contribute to a particular learning scenario. It is my contention that the learning components of a given educational environment can be represented by the vertices of a polygon - the area contained within the polygon then defines the allowable blend space for the components concerned. Any given blend can then be represented by the geometric distances of the blend point from the vertices of the polygonal structure. These concepts are illustrated schematically in Figure 2 for a situation involving the use of three learning channels (components). Fig. 1. A blend space produced by three types of learning resource. By combining the geometrical distances depicted in diagrams like that shown in Figure 2 it is possible to derive blending algorithms similar to that shown below: Blend(X) = F({A}/a, {B}/b, {C}/c) 46 Philip Barker In the above equation [{A}, {B}, {C}] represents the detailed specification of the resource mix used to create blend X. Each item in this list (such as {A}) denotes the pragmatic resource set that makes up the conceptual learning mode denoted by A. Having identified a mechanism for specifying the composition of a blend, it remains now to explore how different points within the blend space influence the effectiveness of learning. It is therefore proposed that the blend (denoted by X in Figure 2) along with the cost of producing a course (having that blend composition) influences the course outcomes in the following way: Success(X, Attrition, Achievement) = F(Blend, Cost) An obvious extension of the model presented in Figure 2 would be to consider the implications of accommodating different learning styles and preferred mechanisms of learning (Honey and Mumford. 2000; Felder and Silverman, 1998). In this situation, the point X in Figure 2 would now need to represent a ‘generic blend’ for a particular course and a given cohort of students. This is referred to as X{generic} for the course/cohort combination. It assumes that exactly the same blend is used for every student in the cohort. An attractive advantage of using blended learning is its ability to use different combinations of resources for each individual student. A number of different variants of X{generic} therefore need to be made available in which the actual blend composition can be influenced by the learning styles and preferences of each individual student in a cohort. This is referred to as X{specific(i)}. This latter expression describes the specific composition of the learning blend administered to a particular learner (within a given cohort) that belongs to ‘student type i’. Using this approach it is now possible to consider the total set of blends for the whole cohort of students as forming an area (that defines the ‘mix variability’) within the underlying blending polygon. Naturally, there is likely to be some level of mis-match between any given student’s ‘ideal’ learning requirements and those suggested by the blending strategy for that student. It is therefore important to reflect the effect of this factor within the blending relationships that were presented earlier. This can be done by introducing an ‘index of fit’ (iFit) into the equations in order to represent how well a chosen X(specific) maps on to the ideal mix for a particular student. The new relationship between learning outcomes and blending variables therefore now becomes: Success(Xi, Attrition, Achievement) = F(iFit, Blend, Cost) It is important to emphasise that the detailed specification of the functionality referred to in the equations introduced above has yet to be identified through further research. We are in the process of exploring the implications of the above relationships within different types of learning environment. Initially, we are investigating the situations that prevail in small-scale studies that we are more easily able to monitor and control (Barker, 2006c; Webb et al, 2004). Of course, as has been suggested earlier in this paper, it is important to consider the effect that a blended learning mix has on the cognitive behaviour of those who are exposed to it. In our research we have been using the term learning spiral as a metaphor for a cognitive process related to a particular dimension of knowledge that an individual has. Our use of learning spirals is illustrated schematically in the model presented in Figure 3. This figure shows six cognitive processes (labelled P1 Blended Learning with Webs, Wikis and Weblogs 47 through P6) each one of which is represented by a corresponding learning spiral. Of course, the spirals shown in this diagram are not static (as they are depicted in the figure) - unless for some reason they have become dormant. Fig. 3. The learning spirals model for cognitive processes. Spirals are characterised by a number of important properties such as rotational speed, breadth of coverage, skewness, trajectory and depth of reach. Spirals can interact with each other and are strongly influenced by the underlying knowledge structures that reside in a person’s long-term memory. As was mentioned above, the state of a learning spiral can be perturbed by the stimuli embedded in the learning resources that a person uses. An individual’s preferred ways of learning - that is, his/her individual learning styles - can also have an effect on the properties of the learning spirals. When considering the design of blended learning resources, it is therefore imperative that we consider how the embedded stimuli (and their containing environments) can be targeted at specific learning spirals/processes. A simple Webbased model depicting our vision of the co-ordinated activity of learning spirals can be found at the URL specified in (Barker, 2006b). A detailed description and discussion of this model is presented elsewhere (Barker, 2006c). 3. Knowledge Sharing As was mentioned at the beginning of this paper, the sharing of knowledge and skills is a fundamental requirement of all teaching and learning systems. Naturally, within these systems, people (in general) and academics (in particular) have important roles to play in relation to being originators of educational messages. They must therefore be considered as an essential component of a blended learning environment. Depending upon the nature of the learning scenarios and the blending mix in which they are employed, people may play an active (primary or ‘front-line’) role or they may take on a more passive (secondary or indirectly supportive) function. For 48 Philip Barker example, a lecturer or tutor involved in delivering a face-to-face course plays an active role. However, the author of a course text book might only play a very passive role in the delivery of that course. It is important to remember that many academics spend a ‘life-time’ studying a particular discipline and, as a result of this they acquire vast amounts of knowledge, skills and experience which they make available to their colleagues, their students and the world at large. Bearing in mind the importance of this process of knowledge sharing, it is imperative to consider ways in which it might be optimised. The underlying concept of a shareable medium is central to a knowledge sharing processes. This is needed for two purposes: personal survival and community sharing. Because of the limitations of the human mind, we all need to use ‘external media’ in order both to remember things and to communicate our ideas (and knowledge) to others. This process is referred to as ‘exteriorisation’. Commonly used media include paper, audio tape and film (both celluloid and digital). Naturally, the ‘power’ and usefulness of a medium depends critically upon the range of material that it can ‘hold’ and the knowledge representation (and manipulation) facilities that it provides - these are denoted by the ‘linguistic frameworks’ that were referred to earlier in Figure 1. The problem with ‘real spoken words’ (as a shareable medium) is their transience or short livedness. Unless a listener can rapidly decode the educational message that is encoded in a spoken utterance, it becomes lost forever. Recorded words can be ‘rewound and replayed’ in order to get a second (or third or, even more) opportunity to capture the content of the utterance. Because of its more stable properties, paper has (for centuries) been the medium of choice for knowledge sharing. However, it lacks many of the attractive features that more modern electronic media possess such as ease of transmission from one location to another, inherent interactivity, ease of updating and the ability to easily represent change and movement (Barker, 2007a). Naturally, the transition from paper to electronic modes of writing will require the development of new types of writing skill (Barker, 2005a) Until the advent of computers and the ubiquitous availability of global communication networks, there has been no significantly dynamic, interactive and spontaneous way of sharing skills and knowledge - other than through face-to-face exchanges. Of course, nowadays this has all changed. The advent of online electronic artefacts such as those described in the following section of this paper, means that people can share knowledge and ideas literally ‘as they are being created’. Furthermore, through the ‘new’ electronic media, the participative (or communal) creation of shared knowledge has at last become a reality. However, having said this, it is very important that we consider the distinction between public knowledge (that we wish to share) and private knowledge (that we do not wish to expose to others). Naturally, each and every one of us has a ‘preferred set’ of knowledge-storage media and techniques that we use to support our everyday existence. For example, these may be conventional devices (such as notepads, the books we write, voice memo-pads, and so on) or they may be electronic tools such as a laptop computer, tablet device or a PDA. Unfortunately, however, when we want to share our material with others, we often come up against three basic types of problem - all of which are related to compatibility issues. First, one person’s media may not be Blended Learning with Webs, Wikis and Weblogs 49 compatible with those of another person. Second, even though a given medium may be commonly accepted and standardised across a given population of users, the ways in which each individual represents his/her knowledge (using that medium) may be different - see the range of knowledge representation frameworks depicted in Figure 1. Finally, even though the members of a sharing group may agree on a common knowledge representation framework, they may not necessarily agree upon how collections of knowledge items should be classified and aggregated into larger units. Again, like the ‘individualisation of learning’ problem that was described in the previous section, this is an age-old problem that has existed ever since people began to make knowledge collections public and shareable. However, the problem is magnified in globally available digital collections of public knowledge - primarily because of the tremendous number of people involved and the different views that they hold about how knowledge archives should be structured, classified and accessed. Of course, just as the problem is more difficult in the digital era, it is also easier to find workable solutions - such as the use of personal and shared ontologies. But these are not the only problems that we face with shared distributed knowledge systems. As I have discussed elsewhere (in the context of ‘distributed blogs’) a more difficult problem to solve is the emergence of the ‘knowledge holes’ that arise if knowledge is lost or retracted. Because of their importance within contemporary and future blended learning environments, it is important to consider how the new electronic artefacts (such as webs and wikis) can best be used to support learning processes. This issue is discussed in the following section of the paper. 4. Using Webs, Wikis and Weblogs As has been suggested earlier in this paper, when designing a course that depends upon the use of blended learning, it is important to consider the nature of the components that will be used, the educational functions that they will fulfil and the extent to which they will be deployed. A course designer therefore has to balance a complex inter-play between the different learning channels that are available for the implementation of a given learning scenario and the ways in which these are used to achieve given educational objectives. Undoubtedly, over the last decade or so, computer-based web structures have become an important mechanism for providing access to course resources that are available in electronic form. Because of their importance, the remainder of this section briefly outlines some of the major roles that webs, weblogs and wikis might play as resources within a blended environment. In general, there are two broad approaches that are commonly used for harnessing an electronic web facility; these are referred to as the disseminative and participative modes of use. Because of the advantages that are offered by the electronic mode of delivery, there is an increasing tendency to use web-based approaches for the dissemination of learning materials. This is an attractive approach since a major imperative in education has to be a reduction in the volume of paper that is currently 50 Philip Barker consumed. This can be easily achieved using web-based approaches - provided they are appropriately designed. As well as the disseminative role of webs, an equally important function that webbased systems can provide is mechanisms for the support of participation. This can take a variety of different forms depending upon the nature of the involvement that a course designer wants to develop for his/her course members. One of the simplest types of involvement involves participation in online testing and assessment procedures. Much more participative roles can be developed through the use of online conferencing using synchronous and/or asynchronous techniques. These approaches usually involve groups of students and staff forming an online community and using the conferencing system to communicate with each other. More advanced approaches to this type of participation can involve the use of powerful online computer-supported collaborative learning environments. These enable groups of people to work together in real-time in order to solve particular problems in a collaborative and participative way. Two other very important web-based tools that facilitate participation and collaboration are weblogs and wikis. Each of these provides a different type of participation. They can each be used in a variety of different ways to facilitate the collaborative development and assessment of electronic knowledge. The potential roles that wikis and weblogs can play within a blended learning environment have been discussed in detail elsewhere (Barker, 2005b; 2005c; 2007a; 2007b). This section therefore only summarises the essential features of each medium. A weblog (or ‘blog’ as it is often called) is a web-based structure that enables an individual (the blogger) to make postings to it. A posting is similar to an electronic mail message. These are then ‘visible’ to all those people who have read access to the blog. This visibility will depend upon whether the blog lies within the public domain (anyone can read it) or whether it resides within a closed user-group (where only members of the group can read it). Those people who can read the weblog will usually also be able to post comments relating to the particular entries that it contains - of course, these may be scrutinised and moderated before they are actually published. A typical example of such a blog (created using the Moveable Type system) can be found at the Web address given in (Barker, 2007c). The attractive feature of a weblog lies in the fact that its owner can get feedback from other people about the ideas and/or opinions that he/she posts to the blog. Weblogs therefore have obvious potential as an educational tool for providing feedback to learners - both tutor-based and peer-feedback. Within a given blog, the locus of control usually resides with the individual person who has created that artefact. In contrast, a wiki is a structure that enables a group of people to post material to a shared web-space. The locus of control now lies with a group of people rather than an individual. It is therefore a facility that supports truly collaborative knowledge development. As was mentioned earlier in this section, webs, wikis and weblogs are now starting to become a standard part of blended learning courses - see, for example, Gallop (2006) and Doolan (2006). It is therefore important to consider how these resources fit in alongside the other types of resource that are used for course development. This issue is discussed in more detail elsewhere (Barker, 2007b). Blended Learning with Webs, Wikis and Weblogs 51 5. Concluding Remarks There is a growing amount of empirical and pragmatic evidence to support the positive utility of the use of blended learning as an affective way of passing across knowledge and skills from one generation to another (Carman, 2002; Bielawski and Metcalf, 2005; Bonk and Graham, 2005). However, as important as these contributions are, there is no substitute for an adequate ‘theory of blended learning’ that is rooted firmly within its two primary parent domains: instructional design and cognitive science. The first of these should advise us about how we should prepare blended environments to meet particular types of educational need while the second should guide us with respect to being able to predict the effects that these environments will have on the states of mind of those who are exposed to them. References Barker, P.G., (ed) (1995). Electronic Performance Support Systems, Special Issue of Innovations in Education and Training International, 32(1), 1-73. Barker, P.G., (2004). Implementing Constructivism Using an e-Science Paradigm, 3803-3810 in EDMEDIA 2004: Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications, Lugano, Switzerland, Edited by L. Cantoni and C. McLoughlin, Association for the Advancement of Computing in Education, Norfolk, Virginia, USA. Barker,P.G., (2005a). Teaching and Assessing Electronic Writing Skills, 167-177 in Proceedings of the CBLIS 2005 International Conference on Computer-Based Learning in Science - Integrating New Technologies into Science, 2-6 July, University of Zilina, Slovakia, Edited by Z.C. Zacharia and C.P. Constantinou, University of Zilina Press, Slovakia. Barker, P.G., ( 2005b). A Role for Weblogs in Electronic Course Delivery; 677-682 in Proceedings of the EDMEDIA 2005 World Conference on Multimedia, Hypermedia and Telecommunications, Montreal, Canada, Edited by P. Kommers and G. Richards, AACE, Norfolk, Virginia, USA. Barker, P.G., ( 2005c). Potential Uses of Weblogs in Electronic Course Delivery, 515-522 in Proceedings of the E-LEARN 2005 World Conference on E-Learning in Corporate, Government Healthcare & Higher Education, Vancouver, Canada Edited by G. Richards, AACE, Norfolk, Virginia, USA. Barker, P.G., (2006a). Motivation, Learning Spirals and Blended Learning, 91-97 in Proceedings of the First Annual Blended Learning Conference: ‘Blended Learning Promoting Dialogue in Innovation and Practice’, 15th June 2006, University of Hertfordshire Press, Hatfield, Hertfordshire, UK. Barker, P.G., (2006b). Learning Spirals Model, Available online at: http://www.philipbarker.demon.co.uk/LearningSpirals/ Barker, P.G., (2006c). Motivating Students - Can Blended Learning Help? 1849-1854 in Proceedings of the E-Learn 2006 World Conference on E-Learning in Corporate, Government, Healthcare and Higher Education, Honolulu, Hawaii, 13-17 October, Edited by T. Reeves and S. Yamashita, AACE, Norfolk VA, USA. Barker, P.G., (2007a). Using Wikis as a Teaching and Learning Resource, Paper to appear in the Proceedings of the CBLIS 2007 International Conference on Computer-Based Learning in Science, Crete, Greece. 52 Philip Barker Barker, P.G., (2007b). Blended Learning the Wiki Way, Paper to appear in the Proceedings of the Second Annual Blended Learning Conference: ‘Supporting the Net Generation Learner’, University of Hertfordshire, Hatfield, Hertfordshire, UK. Barker, P.G., (2007c). The Stockton Rambling Club TS20 Weblog, available online at: http://ts20.gazettelive.co.uk/clubssocieties/stockton_rambling/ Barker, P.G., van Schaik, P. and Famakinwa, O., (2007). Building Electronic Performance Support Environments for First-Year University Students, Innovations in Teaching and International, 44(3), in press. Bielawski, L. and Metcalf, D., (2005). Blended e-Learning - Integrating Knowledge, Performance Support and Online Learning, HRD Press Inc, Amherst, MA, USA. Bonk, C.J. and Graham, C.R., (2005). The Handbook of Blended Learning: Global Perspectives, Local Designs, Pfeiffer, San Francisco, CA, USA. Card, S.K., Moran, T.P. and Newell, A.P., (1983). The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates, Hillsdale, NJ, USA. Carman, J.M., (2002). Blended Learning Design: Five Key Ingredients, KnowledgeNet Inc., this document is available online at: http://www.knowledgenet.com/pdf/Blended%20Learning%20Design_1028.PDF (accessed 22nd February, 2007). Chandler, D., (2006). Semiotics: The Basics, 2nd Edition, Routledge, London, UK. de Souza, C.S., (2005). The Semiotic Engineering of Human-Computer Interaction, The MIT Press, Cambridge, MA, USA. Doolan, M., (2006), Effective Strategies for Building a Learning Community Online Using a Wiki, 51-63 in Proceedings of the First Annual Blended Learning Conference: ‘Blended Learning - Promoting Dialogue in Innovation and Practice’, 15th June 2006, University of Hertfordshire Press, Hatfield, Hertfordshire, UK. Felder, R.M. and Silverman, L.K., (1998). Learning and Teaching Styles in Engineering Education, Engineering Education, 78(7), 674-681. Gallop, R., (2006). Blending by Blogging: how “Wikied” can it get?, 25-32 in Proceedings of the First Annual Blended Learning Conference: ‘Blended Learning - Promoting Dialogue in Innovation and Practice’, 15th June 2006, University of Hertfordshire Press, Hatfield, Hertfordshire, UK. Honey, P. and Mumford, A., (2000). The Learning Styles Questionnaire (80-item version), Peter Honey Publications, Maidenhead, UK. Webb, E.R., Jones, A.D., Barker, P.G. and van Schaik, P., (2004). Using E-Learning Dialogues Within a Blended Learning Environment, 2790-2796 in EDMEDIA 2004: Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications, Lugano, Switzerland, Edited by L. Cantoni and C. McLoughlin, Association for the Advancement of Computing in Education, Norfolk, Virginia, USA. Blended Learning: Beyond Web Page Design for the Delivery of Content Philip P Alberts1, Linda A Murray, Darren K Griffin & Julia E Stephenson 1: Learning & Teaching Development Unit, Brunel University, United Kingdom Phil.Alberts@brunel.ac.uk Abstract. The growing use of computers in all walks of life has inevitably raised the question of the efficiency of e-learning in higher education. Technology has entered university teaching in a variety of ways including content delivery, resources for additional reading, student interaction and learning assessment. It is evident that e-learning offers significantly increased flexibility to the student in terms of time, place, pace and process; however, the provision of too much flexibility may disadvantage the learning process and encourage inappropriate learning strategies. In this presentation, we consider that the uptake of e-learning is inevitable. Moreover, it can offer significant benefits to both lecturer and learner provided inappropriate practices are avoided. We propose that the design of the e-learning environment is allimportant, as is the imposition of structures and boundaries for the students. We conclude that e-learning strategies are likely to find widespread use if take-up offers significant advantages to lecturers as compared to their current practices. Introduction A lecture is the process by which the notes of the lecturer are transferred to the notes of the student without passing through the mind of either Attributed to R K Rathbun (Gilstrap and Martin, 1975) When the wit R K Rathbun first uttered these words, he could scarcely have imagined that this would become the mantra of many proponents of e-learning in higher education. The traditional lecture has been the mainstay of teaching in universities since their inception, but since at least 1931 (Holt, 1931) their efficacy has been questioned. Thus, before considering the benefits and prospects of e-learning as an alternative to lectures, it seems appropriate to consider why the traditional lecture has remained so popular and ask the question whether it is appropriate to consider replacing it, even partially. The main benefit of the traditional lecture is that it is efficient; one person can deliver a pre-determined body of knowledge to a large number of students and at the end of an hour each of them should at least have a cognitive reference structure for revision. Second, lectures are flexible in that they are only minimally constrained by time and location. Of course all the students have to be in one place at a given time, but this is only limited by timetabling and the availability of a large enough space for Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 53-65, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 54 Philip P. Alberts et al. all. Traditional lectures are by their very nature lecturer-centred; the notion of the lecturer as the “expert” delivering wisdom to the student body is central to this. Therefore such a system is attractive to the lecturer - he or she maintains significant control over the student learning experience and can implement learning strategies accordingly. In higher education, we therefore have a system for the delivery of a course that is efficient, flexible, tested and popular amongst lecturers. There should be no reason to consider a change in delivery, unless change is inevitable or desirable (to both lecturer and student). What follows is a view supported by literature that change is inevitable. We argue that there are three good reasons why this change will occur. We then go on to propose guidelines that lecturers should consider when adapting their preexisting course material. In particular, we will consider the increased use of information technology as a means of delivering lectures i.e. a workable alternative to traditional lectures. For the sake of clarity, the following definitions will be used: • Traditional lecturing: A lecturer makes a presentation to a group of students in a lecture room using visual aids such as the writing board and overhead projector, or even a digital visual aid such as PowerPoint projection. • E-learning: Information technology is used as the primary means of delivering courses and lecturer-student interaction takes place online. Distance learning courses are often delivered in this way. • Blended learning: Both traditional lecturing and information technology are used as the primary means of delivering courses. The lecturer decides on a ‘mix’ of components from each. Finally, we indicate what can be done to encourage lecturers to adopt e-learning. Three reasons why the adoption of e-learning is inevitable Reason 1: Use of computers and the Internet is widespread and growing in all walks of life The introduction of the Apple Macintosh computer in 1984 revolutionised the computing world. The integration of features such as the Graphical User Interface and a mouse for navigation to replace the cumbersome commands of MS-DOS was one of the first stepping stones to enable members of society to interact with computers in their own home and office environments. Its impact on education, however, was not immediate due to factors such as high costs (Alessi & Trollip, 2001). Over subsequent years, information technology diversified rapidly and the presence of computers in higher education institutions is now commonplace, as are personal computers in many societies. E-learning has experienced exponential growth in education over the last few years. The sophistication of software packages has increased and the Internet has expanded dramatically. All of these factors have provided and will continue to provide the opportunity for greater involvement of e-learning in higher education; they have already led to growth in online delivery of learning materials in many fields (Inglis et al., 2002). Blended Learning: Beyond Web Page Design for the Delivery of Content 55 Reason 2: There is an increased demand for higher education world-wide and increased pressure on lecturers to use technology to cope with it Many parts of the world are experiencing an increase in the demand for education and training; this is manifested in universities by larger intakes of students and a significantly increased proportion of members of the public benefiting from higher education (Dearing Report 1997; http://www.leeds.ac.uk/educol/ncihe/). In turn, this leads to increased demands on the lecturer, who on top on his or her dual teaching and research roles, may have been given insufficient extra time or resources to cope with the increase in demand. Daniel (1996) refers to this as an ‘educational crisis’ that can be resolved by the use of information technology. The use of information technology is already evident in Universities, as visual aids in lectures e.g. PowerPoint (Bartsch, 2003), video-streaming lectures (Shephard, 2003) computer-based learning programmes (Dewhurst et al., 2000), ‘data mining’ facilities, computer-based assessment (Thelwall, 2000), realistic simulation-based learning environments (Imber et al, 2003) and interactive presentation systems such as SMART boards (www.smarttech.com). Many lecturers are turning to the Internet as a means of aiding their teaching practices. The introduction of e-learning environments such as Blackboard FirstClass (www.firstclass.com), and Moodle (www.blackboard.com), (http://moodle.org/) has made technical issues involved in electronic delivery less onerous. There is also evidence that the tools provided by such systems are beneficial to learning. Romanov & Nevgi (2006) provide evidence that students using Blackboard (WebCT) with its accompanying tools such as ‘Discussion’ generated significantly higher assessment results compared to students using a conventional web site displaying the same course material but without the tools. A problem for university lecturers is that while many skills gained through experience of traditional lecturing can translate directly into effective skills for the design and delivery of online learning, others do not. Although many students are now “information technology natives” i.e. they have used computers from a very early age, most of their lecturers are “information technology immigrants” i.e. they have had to learn the use of computers in adulthood (Prensky, 2001). Thus lecturers may tend to use lecture notes, overhead transparencies and handouts as a starting point for the design of e-learning and often find it problematic to convert these into elearning resources from which students can learn effectively. Reason 3. Learning by means of information technology can now provide significant learning advantages for the student provided appropriate design is employed (four Ps) The proponents of e-learning as an alternative to the traditional lecture argue that such an approach provides significant advantages for student learning in terms of flexibility. For the purposes of this paper, we have dubbed these advantages the ‘four Ps’ as follows: • Place: Students have the flexibility to learn in a location of their own choosing (provided there is an available computer connected to the Internet). • Pace: Students can learn at the speed suited to them. • Peace: Students can determine the time in which they learn, choosing moments of peace and quiet most appropriate for learning. 56 Philip P. Alberts et al. • Process: Students can choose the means by which they learn, selecting the learning process most suitable for themselves. For instance, the non-sequential nature of most e-learning allows students to skip topics they are confident with and concentrate on unfamiliar topics. If we take an utopian ideal that “the widest possible choice is best”, it is easy to see how the above advantages might appear attractive to students and foster learner independence. Left to their own devices, however, and not constrained by the structure of a regular routine it may be argued that there is a risk for students that the advantages afforded by information technology may be misused: • Students may choose a place of learning that is not conducive to learning concentration. • In the absence of face-to-face encouragement and peer pressure and where time restrictions need to be self-imposed, the pace of learning may slow considerably. • The freedom to choose the time in which to learn may result in procrastination. • Given the freedom to choose their own learning process, students may choose one that is inappropriate for the learning task at hand. Information technology has now advanced to the point where the potential advantages of e-learning can be realised. In the following section we put forward recommendations for ‘good practice’ to facilitate the change from traditional lecturing to blended learning which maximises the benefits for both student and lecturer. It is our opinion that a reasonable first step would be the adaptation of existing course material to a useable electronic format. We therefore focus on realistic “good practice” that lecturers may follow in order to achieve meaningful blended learning. Ten good practices in the design of online course material The challenge for the designer of blended learning is to make the e-learning environment sufficiently engaging in order to promote the advantages of the ‘four Ps’, thereby promoting independent learning amongst students. Clearly, if students are learning via information technology rather than a traditional lecture, they are excluded from face-to-face human interaction including verbal and non-verbal communication. The design of e-learning should therefore seek to replace this by maximising the opportunities for interaction via information technology. Thus lecturer-centred traditional lectures will at least in part be replaced or supplemented by student-centred e-learning in a blended learning situation. In a previous study, Evans et al. (2004) examined the pitfalls of “simply putting notes on the web” as a means of delivering lecture material. (In fairness, such educational materials placed online are usually intended to be additional learning resources.) One possible solution is to use more sophisticated tools (Benyon et al., 1997). However this may not be practicable for financial reasons. The ten good practices provided below have been compiled through collating information from empirical studies. Some are based on Jakob Nielsen’s “Top Ten Mistakes in Web Design” (http://www.useit.com/alertbox/9605a.html). However, it is recognised here Blended Learning: Beyond Web Page Design for the Delivery of Content 57 that studying for a degree in higher education is different from gathering information by surfing the Internet, which was the focus of Nielsen’s work. Additional considerations for the use of the web technology in formal learning contexts are included. Good practice 1: Specify your learning outcomes and design your assessment strategies with them in mind Learning outcomes should be considered as a feature of any learning environment. However, we suggest that they are particularly relevant in the e-learning environment. Learning outcomes make electronic content more “three-dimensional” by allowing students to develop a “mind map” of the material and understand exactly what they should attain in their learning. Students subsequently do not have to guess the intentions of the materials and the depth to which they must be mastered. Race (1994) outlines reasons for specifying learning outcomes for open learners and we propose that all of these apply to e-learning. Learning outcomes are effective as they inter alia: • Build self-confidence in students • Alert the students to the standards that should be attained and the reasons why learning needs to occur in the chosen subject area • Alert the students to the challenges that are ahead of them. McMahon (2006) proposes seven maxims to promote deeper learning as opposed to surface learning, one of which is the incorporation of suitable learning outcomes. Empirical evidence suggests that learning outcomes need to be well-designed and that assessment strategies should be designed with them in mind. Good practice 2: Bear in mind that the lecturer is not physically available within the online environment to explain the environment to students; the environment itself should compensate for this. Text-based descriptions should be “stand-alone” and self-explanatory. Good practice here is to undertake ‘user-testing’ with selected students to determine understanding during the development of online material. According to Giardiello (2006) it is important to keep PowerPoint presentations simple and remember “less is more”; furthermore, when converting PowerPoint presentations to an online format, it is important to bear in mind that the bullet points were designed to be aides memoire to the verbal presentation. It is possible to add audio or video recordings / clips to a PowerPoint presentation. Unlike a traditional lecture, students can view the PowerPoint file as many times as they need to. Evidence has shown that providing students the opportunity to explore the material in their own time and pace reinforces the material and promotes active learning (Henly & Reid, 2001). Good practice 3: Make use of available multimedia facilities It is particularly important, especially in a highly visual subject, to exploit the potential of multimedia in the e-learning environment. The meaning and impact of the course content can be strengthened by the use of multimedia elements. Animations, graphics, digital photographs (Swartz, 2004), video clips and sound recordings can engage the learner and can often be utilised better in an electronic 58 Philip P. Alberts et al. rather than a paper-based format. Whitson et al., (2006) created online multimediabased lecture packets for surgical education using a combination of Macromedia Breeze (subsequently named Adobe Acrobat Connect Professional) and WebCT Vista. This study illustrates the powerful software programmes available which allow the production of high quality multimedia rich environments; PowerPoint can be synchronised with audio / video clips and also real-time video and audio conferences can be created. Presenting information in varied ways provides the learner with an enriched learning environment and an increased likelihood of maintaining interest. One such example is the “Intelligent Classroom” project (Winer & Cooperstock, 2002) that integrated various technologies such as audio, video, slides and digital notes on an electronic whiteboard / digital tablet during a live lecture which was recorded and made available for the students to access. The impact was measured qualitatively, and demonstrated overwhelming support amongst students for the use of this means of teaching. Another suggestion for using multimedia to maintain interest is reported in a study by Peachey, Jones and Jones (2006); results showed that the implementation of online quizzes and games applicable to the course material had the potential to increase student participation and also improve the learning process. Good practice 4: Structure material to more than one level of navigation and present information in “bite-size chunks”. Chapters in traditional textbooks were written with little structuring over and above the use of paragraphs. In recent years textbooks have become more visually interesting and have provided additional structure through the use of more subheadings, images, textboxes and so on. Similarly in providing content online, it is important that lecturers avoid lengthy scrolling web-pages in their e-learning environment. Rather, the learner’s visual memory should be engaged by introducing a hierarchy of topics and sub-topics. We suggest that material should have at least two levels of structuring in the hierarchy. This also assists learners to assess the relative importance of any given part of the material and enables the material to be placed in context (Evans & Edwards, 1999). Moreover, constructivist teaching theories suggest that students make mental connections and construct mind maps more readily when material is presented in this way (Novak, 1998). Without this, content is presented in a linear, one-dimensional fashion akin to a traditional textbook. Moreover, empirical cognitive psychology studies have demonstrated that learners benefit from material being sub-divided (a practice known as “chunking”: Lynch & Horton, 1999). The idea of chunking was first suggested by George Miller (1956) in a study on short-term human memory. Miller concluded that humans are capable of retaining between five and nine chunks of information. In incorporating the concept of chunking, scrolling should definitely be avoided (Nielsen, 1995 & 1998). Furthermore, a study by Lee (2005) showed that students least preferred the method of scrolling when compared with other computer text formats. Good practice 5: Provide sufficient navigational information to the student, particularly about where they are in the structure and from whence they came Blended Learning: Beyond Web Page Design for the Delivery of Content 59 Disorientation or being “lost in hyperspace” is a frequent problem associated with navigation in online teaching and learning materials (Conklin, 1987; Dix et al., 1998). Links leading to other pages may cause the user to get lost within an intricate structure and it may be impossible to retrace the learning path. In order to circumvent this, the e-learning environment should help the learner answer the questions “Where am I” and “From where did I come?” A frequently used practice on the Internet is to include “breadcrumbs”, which indicate the hierarchical steps that have been taken to reach the current page. This addresses the second question (“From where did I come?”) but not the first (“Where am I?”) because no indication is given as to where the user is in the hierarchy. Clear information about position within the context of the e-learning environment is all-important. A study by Gullikson et al., (1999) considered the effect of information architecture on an academic web site. The authors recommended including sufficient navigational aids at all times. Such navigational aids should be represented as a standard menu on each page and include the ability to: 1. get to the top-level menu from anywhere on the site (including sub-sites) 2. use any of the access tools from anywhere on the website 3. determine easily one’s location within the hierarchy. Ways to overcome problems of disorientation would be to incorporate navigation elements such as “next page”, “previous page” and links to the “main page” or “table of contents” as well as implementing a simple hierarchical structure with navigational tools such as maps, menus or toolbars. Fortunately, many e-learning environments provided by commercial vendors (or available as open source software) have these features built-in. This good practice is akin to the advice given to lecturers in relation to the traditional lecture. As well as providing an overview of the structure of the lecture at the outset, lecturers are encouraged to provide ongoing signposting throughout. When this is not done, the chance of a learner declaring “I am lost” arises – and in a traditional lecture students are rarely in a position to retrace their steps. Online, those features that make the learner safe in the knowledge of where they are and from whence they came, also provide confidence in addressing the question “Where can I go next?” Good practice 6: Provide an e-learning environment that strikes an appropriate balance between freedom of navigational choice and appropriate constraints An e-learning environment with too few navigational choices can be likened to the turning of pages in a book (“electronic page-turning” - Nielsen, 1998). This limits the choices open to the learner and is regarded as lecturer-centred. A student visiting online content for the first time may wish to read all of it; they may however be confident in their knowledge of certain topics and thereby decline the opportunity to visit the topics. A student revising content will most likely wish to visit only certain topics. In either event, the student should easily be able to go from topic to topic. In order to cater for individual learning requirements, it is appropriate to furnish the learner with choice in the selection of topics or sub-topics. This can inter alia be achieved by means of in-built menu options, easily recognisable buttons labelled with words or pictures, hot spots and hot text (Harding et al., 1995). 60 Philip P. Alberts et al. However, too much of a good thing brings its own disadvantage. One of the early celebrated advantages of hypermedia was that it provided complete control to the user (Nielsen, 1995; 1998). Imagine a scenario where the user is given a page with fifty links to other topics. It would take some time to determine which link to choose and reading the descriptions of all the links would be time-consuming. In the process of learning, this freedom can lead to the learner becoming overwhelmed or even straying from the e-learning environment. Benyon et al., (1997) describes this in terms of students becoming ‘bewildered’ and losing the coherence of the material. In subjects that involve a cumulative build-up of knowledge, it is advisable that a student does not encounter new material without first having mastered the prior material that is essential for its understanding. The assessment tools available in most e-learning environments provide the opportunity for learners to self-assess their progress and thus make sensible decisions in relation to the choices available to them. Use of these and the functions for conditional release of material allows the lecturer to ensure an appropriately structured learning experience. Good practice 7: Use hyperlinks only for additional information One of the disadvantages of hyperlinks is the possibility of disorientation (as mentioned above). Hyperlinks are unidirectional; this means that the e-learning environment can show the user the links that have the current node as their departure point, but not the ones that have it as their arrival point. In other words, the system will tell you where you can go next, but not in what alternative ways by which you might have arrived there. In this way, the structure is hidden and is in the mind of the designer. A partial solution is to open the page in a new window that can be closed easily, or consider the use of expanding hypertext (a newer hybrid hypertext format). Expanding hypertext is demonstrated in an empirical study by Lee (2005), placing the content of the hyperlink in the same page as the hyperlink, thereby decreasing disorientation as it retains a linear arrangement. The ability to add external hyperlinks with relative ease is considered to be a highly attractive feature, thereby taking the student to sites outside the e-learning environment. However, this can lead to problems because the quality of the external content may change and the links can become “dead” (Alessi & Trollip, 2001). A study by Cook & Dupras (2004) identified practical guidelines to develop successful web-based learning, one of which being “hyperlinks should be tested periodically”. Good Practice 8: Provide adequate feedback to the learner Feedback is generally defined as information about the quality of the student’s response. Interaction between the learner and the lecturer can be instant or delayed and feedback can be tailored to the individual learner. Feedback should be constructive and supportive and can be delivered through numerous media such as text, graphics, audio and video (Alessi & Trollip, 2001). Advice and guidelines on feedback have been suggested; for example a study on effective feedback by the UK Higher Education Academy proposes seven principles for good feedback (http://www.heacademy.ac.uk/806.htm). Feedback can bring a beneficial influence to the learning environment. For example, in a study by Thelwall (2000), a statistics test was delivered to students via a Blended Learning: Beyond Web Page Design for the Delivery of Content 61 computer-based assessment and it was concluded that the two factors that were thought to make the test a success was its ease of use and the fact that the 91% of the students found the feedback helpful. Feedback is a crucial aspect of interactivity in that it engages learners with the material. It should also be immediate in order to be effective. The more interactive a system, the more actively the students will learn (Schwier & Misanchuck, 1993). Different types of feedback should be implemented depending on the course content and the time available for the lecturer to formulate the feedback. Some methods are more time-consuming than others. Examples include personal feedback by the instructor, model-answers, peer-evaluation provided by the student(s) and automatic feedback provided by the computer system (Collis et al., 2001). Good practice 9: Provide some interactions in the e-learning environment and vary them E-learning environments provide courseware designers with a great deal of scope for interactivity. The level of interactivity is a factor that will determine to what degree the student engages with and remains interested in the environment. Interactivity can be very simple (Clarke, 2004). The lowest level may be considered as page turning or scrolling (Oliver, 1996). The highest level however may be considered as receiving input from the student who is active in the processing of information, for example interpreting graphs (Dewhurst et al., 2000). Research has revealed that interactivity can have advantageous effects for the learner causing their memory skills and understanding to increase to approximately 70% - in contrast to 20% when they only ‘see’, and 40% when they ‘see and hear’ (Lindstrom, 1994). Other levels of interaction have been identified. Moore (1989) identified three types of interactions within learning environments: a) interaction between learner and content, b) interaction between learner and other learners, and c) interaction between learner and teacher. The ultimate aim is to incorporate all three types of interaction in a blended learning approach. However, as a first step it is of primary importance to ensure interaction with the material by requiring some learner responses to questions and tasks. It is interesting to note that in the case of the traditional lecture, lecturers are increasingly challenged to incorporate these types of interaction. With larger class sizes and more heterogeneous groups, the management of lectures to use other than a ‘transmit-receive’ mode requires the development of a high level of skill. Good practice 10: Encourage students to put their learning into practice Interactive multimedia allows the learner to “try something out”, for i.e. by manipulating a simulation and allowing the possibility to “fail” in a safe environment. An example of this notion is simulating resuscitation training (Perkins, 2007) as the lives of patients are not put at risk. In addition, a randomised controlled study by Tanoue et al. (2005) demonstrated that the implementation of a simulation for the training of endoscopic surgery was considerably effective to improve skills and furthermore, such equipment will become increasingly essential because it seems the use of animals for surgical training will lessen worldwide in the future. A powerful learning situation is to provide the opportunity for the learner to interact extensively 62 Philip P. Alberts et al. with the content; a scenario where a response is invited from the user and feedback is provided. Paper-based teaching materials of course cannot provide such a degree of interaction. Perhaps the greatest need of students working independently, is knowledge of how they are progressing. In a traditional lecture it may be difficult for a student to know whether they have understood something because they cannot (or they feel they cannot) interrupt the lecturer. No such barriers exist within an e-learning environment. Interaction with the lecturer can be achieved using an online communication tool such as ‘Discussion’ and can also be done effectively by responding to self-assessment questions (SAQs) - such as multiple-choice, drag and drop, or text-entry (inviting a response from the learner and providing automated feedback). In this way common misapprehensions may be addressed. Race (1994) gives a total of eleven reasons for including SAQs and they are paraphrased below: They • give students the opportunity of experiential learning • confirm to students that which they have already grasped • help students to discover what they have yet to grasp • help “nip problems in the bud” • help students prioritise the importance of aspects of their course or module • help in the interpretation of the learning outcomes • provide a means of confidence-building • give students essential practice in responding to questions • help alleviate boredom • help in student orientation in terms of how much they have learned • help students regulate the pace of their learning Evans & Fan (2002) provide evidence that learners attach substantial value to the use of SAQs in multimedia courseware, suggesting that students should benefit from at least one SAQ for every sub-topic in the material. The “take-home message” therefore is that the e-learning environment need not be a “poor relation” of a traditional lecture; it could indeed be superior. What can be done to encourage lecturers to adopt blended learning? Although adoption of all or many of a the above ‘good practices’ can provide significant benefits for student learning, take-up by lecturers can be slow, unless a clear route to adoption can be found (Lieblein, 2000). We propose that the primary advantage to lecturers in adoption of blended learning is the reduction in time on their feet in the lecture venue. University class sizes are increasing, and universities themselves are expanding. As a result, lecturers can find themselves having to give their lectures to very large audiences with widely differing learning needs, or having to give the same lecture more than once because of timetabling, room size or other campus issues. We contend that adoption of blended learning should not reduce student-lecturer interaction; rather it should increase it - in spite of the lecturer and student spending Blended Learning: Beyond Web Page Design for the Delivery of Content 63 less time in the lecture venue. E-learning still requires the online presence of the lecturer. Free from the need to deliver every last part of content before the end of the lecture, the lecturer has the opportunity to talk with students (not at them) in the form of a seminar or tutorial – a process that can be far more rewarding. The switch from traditional lecture to blended strategies requires a significant time investment on the part of the lecturer, before he or she can experience the benefits. Lecturers need help in converting what they have already. We contend that existing teaching resources can be easily converted. Many lecturers already have PowerPoint presentations or have notes on the Internet. A second consideration is that there are increasing numbers of well-designed elearning resources available from the Internet. “Open source” repositories of learning objects are being developed very rapidly, but the take-up is slow. Further encouragement of their use from those charged with supporting e-learning seems to be needed. The following are practical suggestions for the lecturer in designing an e-learning environment: • Get some help; lobby your computer officer (if you have one) to provide e-learning support. Many institutions also have central support units devoted to the promotion of e-learning e.g. by providing infrastructure, mentoring development projects and providing introductions to colleagues who have successfully made the transition. • Take a training course in a web-authoring programme (e.g. Dreamweaver ). Such software is relatively simple to use. • Don’t re-invent the wheel! Make use of what’s out there already. Many e-learning resources are available on the Internet or through organisations such as the UK Higher Education Academy. • Adopt a procedure for the design of the e-learning environment. The minimum requirements can be regarded as providing learning outcomes, structure, chunking, navigational information and some SAQs. Be inventive if you feel like it, but establish the basics first. • Don’t abandon the lecture venue or your students - support your e-learning environment with seminars that might include more in-depth explanation and discussion of concepts. • Incorporate some online automated assessment; this can reduce your marking time. The pace of disciplinary research (measured through the increase in the number of journal articles and books) increases all the time. The present day lecturer needs to be a facilitator of learning, not an oracle of all wisdom in the discipline. Adoption of blended learning is by far the most promising means through which this can be achieved. The only remaining question is: despite the advantages it offers, how popular will blended learning strategies be given that so few of us appreciate change? References Alessi, S.M. & Trollip, S.R. (2001) Multimedia for learning: methods and development. Boston: Allyn and Bacon. 64 Philip P. Alberts et al. Bartsch, R.A. & Cobern, K.M. (2003) Effectiveness of PowerPoint in lectures. Computers & Education, 41(1), 77-86. Benyon, D., Stone, D., & Woodroffe, M. (1997) Experience with developing multimedia courseware for the World Wide Web: the need for better tools and clear pedagogy. International Journal of Human-Computer Studies, 47, pp.197-218. Clarke, A. (2004) e-Learning Skills. Hampshire: Palgrave Macmillian. Conklin, J. (1987) Hypertext: An introduction and survey. IEEE Computer, 20, 17–41. Collis, B., De Boer, W. & Slotman, K. (2001) Feedback for web-based assignments. Journal of Computer assisted Learning, 17(3), 306-313. Cook D.A. & Dupras D.M. (2004) A practical guide to developing effective web-based learning. Journal of General Internal Medicine, 19(6), 698-707. Daniel, J. (1996) Mega-Universities and Knowledge Media: Technology Strategies for Higher Education. London: Kogan Page. Dearing, R. (1997) The Dearing Report - The National Committee of Inquiry into Higher Education. London. Retrieved May 3, 2006, from the University of Leeds Web site: http://www.leeds.ac.uk/educol/ncihe/ Dewhurst, D.G., Macleod, H.A. & Norris, T.A.M. (2000) Independent student learning aided by computers: an alternative acceptable to lectures? Computers & Education, 35, 223-241. Dix, A., Finlay, J., Abowd, G. & Beale, R. (1998) Human-Computer Interaction (2nd edition) Prentice Hall Europe. Evans, C. & Edwards, M. (1999). Navigational Interface Design for Multimedia Courseware. Journal of Educational Multimedia and Hypermedia, 8 (2), 151-174. Evans, C. & Fan, J. (2002) Lifelong learning through the Virtual University. Journal of Campus Wide Information Systems, Multimedia Systems for Lifelong learning: Success and Challenges, 19(4), 127-134. Evans, C., Gibbons, N., Shah, K., Griffin, D.K. (2004) Virtual learning in the biological sciences – pitfalls of simply “putting notes on the web”. Computers & Education, 43(1-2), 49-61. Gilstrap, R. & Martin, W. (1975) Current Strategies for Teachers. Pacific Palisades, California, Goodyear Publishing Company. Giardiello, F.M. (2006) Powerful PowerPoint presentations. Gastrointestinal Endoscopy, 64(3) 393-394. Gullikson S., Blades R., Bragdon M., McKibbon S., Sparling M. & Toms E.G. (1999) The impact of information architecture on academic web site usability. The Electronic Library, 17(5), 293-304. Harding, R.D., Lay, S.W., Moule, H. & Quinney, D.A. (1995) Multimedia interactive mathematics courseware: The mathematics experience within the Renaissance Project. Computers & Education, 24(1), 1-23. Henly, D.C. & Reid, A.E. (2001) Use of the web to provide learning support for a large metabolism and nutrition class. Biochemistry and Molecular Biology Education, 29, 229233. Holt. H., 1931, cited in “The College Lecture, Long Derided, May Be Fading” by William Honan, New York Times, August 14, 2002. Imber, S., Shapira, G., Gordon, M., Judes, H. & Metzger, Z. (2003) A virtual reality dental simulator predicts performance in an operative dentistry manikin course. European Journal of Dental Education, 7(4), 160-163. Inglis, A., Ling, P., & Joosten, V. (2002) Delivering Digitally: Managing the Transition to the Knowledge Media (2nd edition) London: Kogan Page. Lee, M.J. (2005) Expanding hypertext: Does it address disorientation? Depends on individuals’ adventurousness. Journal of Computer-Mediated Communication, 10(3), article 6. Retrieved online from http://jcmc.indiana.edu/vol10/issue3/lee.html Blended Learning: Beyond Web Page Design for the Delivery of Content 65 Lindstrom, R. (1994) The Business Week Guide to Multimedia Presentations: Create Dynamic Presentations That Inspire, McGraw-Hill, New York, NY, Ch 2, pp. 33-46. Lieblein, E. (2000) Critical factors for successful delivery of online programs. The Internet and Higher Education, 3, 164-174. Lynch, P.J. & Horton, S. (1999) Web Style Guide: Basic Design Principles for Creating Web Sites. Yale University Press. Miller, G. A. (1956) The magical number seven plus or minus two: Some limitations on our capacity for processing information. Psychological Review, 63, 81-97. McMahon, T. (2006) Teaching for more effective learning: Seven maxims for practice. Radiography, 12(1), 34-44. Moore, M.G. (1989) Editorial: Three types of interaction. American Journal of Distance Education, 3(2), 1-16. Nielsen, J. (1995) Multimedia and Hypertext: the Internet and beyond. Boston, London: AP Professional. Nielsen, J. (1998) Electronic Books – A Bad Idea http://www.useit.com/alertbox/980726.html Novak, J. (1998) Learning, Creating, and Using Knowledge: Concept Maps as Facilitative Tools in Schools and Corporations. Lawrence Erlbaum Associates Publishers, Mahwah, NJ. Oliver, R. (1996) In C. McBeath and R. Atkinson (Eds), The Learning Superhighway: New world? New worries? Proceedings of the Third International Interactive Multimedia Symposium, 303-308. Perth, Western Australia, 21-25 January. Promaco Conventions. http://www.aset.org.au/confs/iims/1996/lp/oliver.html Peachey, P., Jones. P., & Jones, A. (2006) Encouraging student participation in an online course using “pull” initiatives. Electronic Journal of e-Learning, 4(1). Retrieved online from http://www.ejel.org/volume-4/v4-i1/v4-i1-art-9.htm Perkins, G.D. (2007) Simulation in resuscitation training. Resuscitation, 73, 202-211. Prensky, M. (2001) Digital Natives, Digital Immigrants http://www.marcprensky.com/writing/Prensky%20%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part1.pdf Race, P. (1994) The Open Learning Handbook. 2nd Ed., London: Kogan Page. Romanov, K. & Nevgi, A. (2006) Learning outcomes in medical informatics: Comparison of a WebCT course with ordinary web site material. International Journal of Medical Informatics, 75(2), 156-62. Swartz, M.L. (2004) Digital photographs and PowerPoint. American Journal of Orthodontics and Dentofacial Orthopedics,126(5) 639. Schwier, R.A. & Misanchuk, E.R. (1993) Interactive Multimedia Instruction. Englewood Cliffs, New Jersey, Educational Technology Publications. Shephard, K. (2003) Questioning, promoting and evaluating the use of streaming video to support student learning. British Journal of Educational Technology, 34(3), 295-308. Tanoue, K., Yasunaga, T., Konishi, K., Okazaki, K., Ieiri, S., Kawabe, Y., Matsumoto, K., Kakeji, Y., & Hashizume, M. (2005) Effectiveness of training for endoscopic surgery using a simulator with virtual reality: Randomised study. International Congress Series, 1281, 515-520. Thelwall, M. (2000) Computer-based assessment: a versatile educational tool. Computers & Education, 34(1), 37-49. Whitson, B.A., Hoang, C.D., Jie, T., & Maddaus, M.A. (2006) Association for Academic Surgery, 2006: Technology-Enhanced Interactive Surgical Education. Journal of Surgical Research, 136, 13-18. Winer, L.R. & Cooperstock, J. (2002) The “Intelligent Classroom”: Changing teaching and learning with an evolving technological environment. Computers & Education, 38, 253-266. Discovery of Educational Objective on e-Learning Resource: A Competency Approach Shi-Ming Huang, Hsiang-Yuan Hsueh, and Jing-Shiuan Hua National Chung Cheng University, ChiaYi, Taiwan {smhuang, hyhsueh, jshua}@mis.ccu.edu.tw Abstract. It is indeed important to implement e-Learning platforms for education and training purposes in a variety of domains. However, facing the enormous amount of learning resources, guidance on e-Learning platform for meeting the educational objectives is still a neglected issue. The learning experiences cannot be optimized and personalized. In this study, a semantic and systematic mechanism is proposed to discover the educational objectives of learning resources with competency information on e-Learning platform. To verify the feasibility of mechanism proposed in this study, the design and implementation of prototype system for the assessment of e-Learning effectiveness using the proposed mechanism are also discussed and demonstrated. By feasibility verification with system implementation for practical requirement, the effectiveness of the proposed mechanism can be basically certified. Keywords: e-Learning; Bloom taxonomy; Competency; O*Net competence database 1 Introduction It is indeed for importance of e-Learning implementations and platforms for education and training purposes in a variety of domains, including educational purposes in organizations or individuals and human resource development facilities in enterprises. It is indicated that the notion of e-Learning has widely been adopted as a proper solution for human resource development in enterprises [14]. The related investment of IT for e-Learning and web-based training programs is therefore expected to increase human resources and talents with better quality [6,8]. With the adoption of e-Learning platform, e-Learning mechanisms and strategies are expected to be dramatically switched from traditional advisor-lead instructions to learner-oriented paradigms [10]. It emphasizes the interaction among actors on eLearning platforms including learners, instructors, developers, and managers. Users' requirement and behavior played dominant roles during the development and refinement phases of e-Learning experiences [4,10,16]. However, facing enormous amount of learning resources on e-Learning platform, guidance on e-Learning platform for educational objectives is still a neglected issue. The learning experiences cannot be optimized and personalized, since learners cannot be guided to use learning Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 66-78, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. Discovery of Educational Objective on e-Learning Resource 67 resources to satisfy their personal learning objectives, particularly the overflowing learning resources on e-Learning platform. It is easy to retrieve any categories of learning contents from a computer-based e-Learning platform, but it is still necessary to look for additional solutions to determine the educational objectives for optimization. In such assumption, the importance rank of learning resource is needed to be established in a personalized manner in order to fit learners' requirement. In this study, the authors argued that additional information for educational objectives of learning resources should be systematically generated and appended on e-Learning platform. As for the standard of such additional information, one potential solution is the competency information specifications proposed as the O*Net competence database [13], which is the common standard for taxonomy, collection, description, evaluation, and publication of competency information with the form of dimensional factors. Several factors including a set of characteristics of employees, occupational opportunities, skills, and abilities are included in the standard. With an open descriptive meta-data and language, it is easy to express, identify, and clarify the competency of job searchers and competency requirements for one job opportunity in an enterprise with quantitative approaches. Information system solutions for competency management are also possible to implement with the aim of such underlying notion and open description language. In this article, a semantic and systematic mechanism is proposed to discover educational objectives, which is formulated by well-known Bloom taxonomy for educational objectives in this study, of learning resources on e-Learning platform. Strategically, the required educational objectives for competency information specified in O*Net metadata should first be determined. Educational objectives of learning resources in e-Learning platform, which reflect a set of competency information, can then be determined in a systematic and semantic manner. In order to verify the feasibility of mechanism proposed in this study, the design and implementation of a prototype system for the assessment of e-Learning effectiveness using the proposed mechanism are also discussed and demonstrated. By feasibility verification with system implementation for practical requirement, the effectiveness of the proposed mechanism can therefore be basically certified. 2 Theoretical Foundations 2.1 Computer-Based Learning Platforms Information Technology has played an important role in modern learning program, which is one of key enablers for individually tailored learning experience. As for the technical e-Learning platform, it is widely accepted that the advent of World Wide Web dramatically changed traditional computer-based technology. It provided a widely accessible communication topology based on open and common standards to access information and knowledge with unified manners [1]. The specification of Shareable Content Object Reference Model (SCORM) therefore provided a set of referential guideline to support the development of web-based e-Learning platform and collaboration of learning resources and contents. With the view of Learning Management 68 Shi-Ming Huang, Hsiang-Yuan Hsueh, and Jing-Shiuan Hua Systems (LMS), SCORM specification covers technical details in each layer of such information system. By definitions of technical foundations for web-based learning environment, it is possible to construct and deploy e-Learning platform that assumed the presence of strong, server-side, LMS-based learning content distribution [1]. 2.2 Competency Information for Characterizing Occupational Profiles The notion of competency management is important for human resource management and optimization in enterprises. The term competency can be defined as the measurable corresponding knowledge, skill, ability, and other behavioral characteristics (KSAOs), which always come out in reference to a given context, for the determination of efficiency and effectiveness of certain missions or tasks [7]. It is therefore applicable to determine distinguished talents in a systematical manner. Competency management therefore concerns the way in which competencies are properly organized and controlled [11]. It is responsible for the management of intangible assets of human resources, such as knowledge, know-how, and behavior, for competency either required by system or acquired by individuals. A variety of studies and standards are presented currently. One main category is the modeling and assessment of competency from the available competency information with the view of human resource management. For example, Linder investigated the status of competency assessment and human resource management performance as an empirical case study for the maturity measurement of competency information utilization [12]. An important standard of competency information is the O*Net, the Occupational Information Network, which is a comprehensive database for the occupational information of employee properties and job characteristics. It also provides a unified language for defining and describing the corresponding occupations. Its flexible design also captures various job requirements with the aim of information technologies [13]. In this article, the authors attempted to apply such complete competency information schema for expression and further analysis on competency management processes. 2.3 Bloom Taxonomy for Educational Objectives The term “educational objective” of learning resources or activities can be referred to as the educational goal which should be identified during the design phases of learning or tutoring activities. Educational objectives should also be reviewed after learning activities in order to determine the effectiveness of learning activities. The most well-known taxonomy of educational objectives is originated by Bloom [5]. In the taxonomy, educational objectives can be categorized in six levels including knowledge, comprehension, application, analysis, synthesis, and evaluation phases [5,15]. In this study, the authors recommended to formulate educational objectives by extended Bloom taxonomy of educational objectives, which is the refinement to the original taxonomy, proposed by Anderson [2]. Rather than one-dimension approach, the extended taxonomy involves two dimensions, including knowledge dimension and cognitive dimensions. The cognitive process dimensions can be categorized as: Discovery of Educational Objective on e-Learning Resource 69 „ Remember: Exhibit the memory of previous-learned materials by recognizing or recalling facts, terms, basic concepts and answers. „ Understand: Understanding of facts and ideas by interpreting, exemplifying, classifying, summarizing, inferring, comparing, and explaining main ideas. „ Apply: Using the available knowledge to execute and implement solutions in different ways. „ Analyze: Differentiating, organizing, and attributing knowledge by manipulating information using certain criteria. „ Evaluate: Checking and Judgments about information, validity of proposed ideas, or quality of work by certain criteria. „ Create: Generating, planning, and producing information or knowledge together and proposing new solutions. On the other hand, the knowledge dimension can be categorized as: „ Factual Knowledge: Knowledge about terminology and specification details. „ Conceptual Knowledge: Knowledge about generation, classification, and structural modeling of certain concept. „ Procedural Knowledge: Knowledge about workflows, algorithms, methods, procedures, and events. „ Meta-Cognitive Knowledge: Knowledge about strategies and decisional conditions. Strategically, the authors attempted to apply taxonomy as a tool to discover the characteristic of competency information and learning resources with the form of location in the taxonomy matrix. For example, the learning resources about “Normalization forms in relational data models” can be inferred as (Apply, Procedural). That is, its educational objective is to provide procedural knowledge to learners so that they can apply learned algorithms to improve the quality of database design. 3 Discovery of Educational Objectives using Bloom Taxonomy In this study, feature mapping involves the discovery of potential keyword from competency information specified in O*Net metadata and identification of educational objective for learning resources inferring from competency information. Figure 1 illustrates the mechanism. 3.1 Specification of Competency Information in Competency Reservoir In this study, the competency reservoir is required to store and retrieve competency information. According to competency definition from O*Net, training program or occupation requirement can be measured by 17 dimensions. In the simulation experience, with the view of learning facilities, 4 dimensions (tools, technologies, knowledge, and skills required by training programs or occupation requirement) are selected, as shown in Figure 2. In specifying O*Net metadata, the competency information can be categorized by occupational requirement. For example, the 70 Shi-Ming Huang, Hsiang-Yuan Hsueh, and Jing-Shiuan Hua competency information of Computer Programmer (No. 15-1021.00) defined in O*Net specification can be expressed by the following metadata. Table 1. Partial competency information for sample occupational requirement. Tools required Computer Servers, Desktop Computers, and Mainframe Computers, etc. Technologies required Analytical or Scientific Software, Application Server Software, and Compiler and de-compiler software, etc. Knowledge required Computers and Electronics, English Language, and Mathematics, etc. Skills required Programming, Critical Thinking, and Complex Problem Solving, etc. Fig. 1. Feature discovery and mapping on competency information. Fig. 2. Taxonomy of competency information. 3.2 Specification of Educational Objectives with Bloom Taxonomy On the other hand, the educational objectives of learning resources, which should be discovered by extended Bloom taxonomy, can be systematically identified by synonym thesaurus is provided in this study. Table 2 shows partial contents of synonym thesaurus for cognitive process dimensions [2]. Discovery of Educational Objective on e-Learning Resource 71 Table 2. Synonym thesaurus to discover educational objectives. Cognitive Process Dimension Remember Understand Apply Analyze Evaluate Create Partial Potential Synonyms (3 synonyms are selected) Recognizing, Recalling, Retrieving Interpreting, Clarifying, Representing Executing, Implementing, Carrying out Distinguishing, Discriminating, Organizing Critiquing, Testing, Monitoring Planning, Designing, Constructing On the other hand, synonym of knowledge dimensions should be manually defined, since knowledge is domain specific. Sample contents of synonym thesaurus used for Computer Programmers are shown in Table 3. Table 3. Synonym thesaurus to discover educational objectives. Knowledge Dimension Factual Conceptual Procedural Meta-Cognitive Partial Potential Synonyms (3 synonyms are selected) Computers, Servers, Mail Markup Languages, Data Models, Programming Languages Problem-Solving, Decision-Making, Troubleshooting Deductive Reasoning, Active Learning, Active Listening 3.3 Semantic Identification The first step of educational objectives discovery is the potential semantic identification from competence information, which is responsible to extract potential keywords which have more likelihood to reflect the educational objective defined in Bloom taxonomy. For example, considering the skill competency information defined in O*Net metadata for Computer Programmers: {Active Learning — Understanding the implications of new information for both current and future problem-solving and decision-making.}, potential keywords reflecting cognitive processes include Potential_ Cognitive_Keywords = {Learning, Understanding, Problem-Solving, Decision-Making}. On the other hand, in the sentence, a set of potential knowledge keyword Potential_Knowledge_Keyowrds = {Learning, Understanding, ProblemSolving, Decision-Making, Implementation, Information} can also be extracted. 3.4 Feature Discovery The second step of educational objectives discovery is the feature discovery by filtering the potential keyword lists using predefined synonym thesaurus. For example, the keyword lists generated in the last section can be pruned as Cognitive_Keywords = {Understanding} and Knowledge_Keyowrds = {Problem-Solving, DecisionMaking}. 72 Shi-Ming Huang, Hsiang-Yuan Hsueh, and Jing-Shiuan Hua 3.5 Semantic Mapping The last step of educational objectives discovery involves the mapping of competency information into educational objectives with Bloom taxonomy. In the example discussed in previous sections, it can be interpreted that the skill competency information “Active Learning” involves the “understanding” of “procedural knowledge” including “Problem-Solving and Decision-Making”. It should be noticed that for any competence information, it is possible to reflect a set of educational objectives. For example, the competency information “Active Learning” also involves the interpretation for “understanding” of “meta-cognitive knowledge” about “implication”. The following Figure 3 shows the corresponding matrix of the sample skill competency information “Active Learning” and “Complex Problem Solving”. Fig. 3. Interpretation of educational objective for competence information. 3.6 Discovery of Educational Objective on e-Learning Resource As the educational objectives of competency information are discovered, the educational objective of learning resources reflected a set of competencies can therefore be identified. For example, learning resource {R: Understanding Case Based Reasoning} reflects a set of competency information defined in O*Net metadata including {“Complex Problem Solving”, “Active Learning”}, the educational objectives can be inferred as shown in the following Figure 4. Fig. 4. Interpretation of educational objective for learning resource. 4 Application: Assessment of e-Learning Effectiveness In this section, the authors attempted to discuss the practical application applying the proposed mechanism to evaluate the effectiveness of e-Learning on users using Discovery of Educational Objective on e-Learning Resource 73 competency information and Bloom taxonomy in order to demonstrate the feasibility and practicality of proposed mechanism. The application of the proposed mechanism for the assessment of e-Learning effectiveness is based on the belief that: „ Effectiveness of users’ e-Learning experience can be measured by the status of educational objectives owned by learners. „ Users attempted to access learning resources for certain learning purposes which can be reflected by occupational requirements. „ Assessment of Learning experiences involves quizzing with questions reflected a set of competency information. „ Quiz, which reflects a set of competency requirements can also measure the status of educational objectives owned by learners. Figure 5 illustrates such belief. Fig. 5. Belief of proposed mechanism applying on assessment of e-Learning experience. 4.1 Assessment of e-Learning Effectiveness with Bloom Taxonomy In this study, the assessment of e-Learning effectiveness can be performed as in the following. A quiz is given with a set of questions randomly generated by Sequential Probability Ratio Test (SPRT) model [9]. Each question can be reflected by a set of competency information, and each competency can be reflected by a set of cognitive processes. That is, each question ( qi ) can be properly reflected by a set of cognitive processes as: q i = C C iT , ∀ i ∈ N (1) T , where C iT denotes the presence of cognitive process T on qi . For example, a question of a quiz may be reflected by a set of cognitive processes including “Apply, Analyze” of “Conceptual knowledge” for the competency “Complex Problem Solving”. 74 Shi-Ming Huang, Hsiang-Yuan Hsueh, and Jing-Shiuan Hua The mastery of a quiz reflected user-specified occupational requirement can be evaluated by the union of score of each cognitive process dimension that user obtained in the quiz: S T = C G ( qi ) × P ( qi ) (2) qi ∈T , where the function G ( qi ) means the score of question i gained by a testee and the function P ( qi ) denotes the importance of question i . The effectiveness of e-Learning can be measured by the mastery matrix with cognitive process dimensions, as shown in Table 4. Table 4. Measurement of e-Learning effectiveness by Bloom cognitive taxonomy. Remember Understand Apply Analyze Evaluate Create S t = ' REMEMBER ' S t = 'UNDERSTAND ' S t = ' APPLY ' S t = ' ANALYZE ' S t = ' EVALUATE ' S t = 'CREATE ' For instance, the mastery of a quiz reflected a testee’ occupational requirement, Computer Programmer, is measured. Given a quiz with three questions randomly generated by SPRT models for testing the mastery of the testee. The gained score, allotted score, importance, and reflected cognitive processes of each question are displayed in Table 5. The gained score which presented mastery for cognitive processes are shown in Table 6. It can be inferred from Table 6 that the effectiveness of learning on the testee is poor in the case illustration. Table 5. The related information of each question of the quiz in the case illustration. Question Gained Allotted Importance Reflected cognitive processes score number score “Apply, Analyze” of “Conceptual knowledge” for 1 7 10 5 the competency “Complex Problem Solving” “Apply” of “Factual knowledge” for the 2 0 15 10 competency “Analytic and scientific software” “Analyze, Create” of “Procedural knowledge” for 3 0 10 5 the competency “Programming” Table 6. Summarized the estimated mastery of cognitive processes in the case illustration. Cognitive processes dimension Gained score / Full marks Remember Understand Apply Analyze Evaluate Create Question 1 35/50 Question 2 0/150 Question 3 Total 35/50 0/50 N/A N/A 35/200 35/100 0/50 N/A 0/50 Discovery of Educational Objective on e-Learning Resource 75 4.2 Prototype System Architecture In this study, the authors have developed a web-based prototype system for assessment of e-Learning effectiveness using the proposed mechanism. Figure 6 shows the architecture. Fig. 6. Architecture of the prototype system. The prototype, which was developed for “Project of Personnel Training of Electronic Business” executed by the Ministry of Economic Affairs in Taiwan, has been deployed on Microsoft Windows server platform with Microsoft SQL Server and Internet Information Service (IIS). Some important features of the implementation include: „ Feature Mapping Module: This is the core module that implements the proposed mechanism. It is responsible for the systematic and semi-automatic discovery educational objectives of quizzes and competency information. „ User Interface: The interface is used for retrieving users’ learning requirements with the form of occupational requirements defined by O*Net metadata. „ Quizzing Module and Evaluation Module: These two modules are responsible for assessing the e-Learning effectiveness of users by applying SPRT based on user-specified occupational requirement. Figure 7 shows the screenshot of sample questions in the user interface generated by the quizzing module. Figure 8 shows the screenshot of mastery matrix with cognitive process dimensions after quizzing. Practically in the prototype system, the matrix can be visualized by the radar diagram for better interpretation by users. It should be noted that it is not necessary for all cognitive process dimensions to appear in the radar diagram. Dimensions involved are highly depended on the nature of questions in the quiz. 76 Shi-Ming Huang, Hsiang-Yuan Hsueh, and Jing-Shiuan Hua To test the preliminary feasibility of the proposed mechanism embedded in the prototype system, an internal test is performed by surveying twenty-two internal users and adopting T-test for statistical inference. The calculated t value is 8.57 > t 0.95 (21) = 1.721 under significance level α = 0.05, and therefore information satisfaction of users is significant. It means that the proposed mechanism in this paper is potential. Fig. 7. Screenshot of sample questions in user interface. Fig. 8. Visualized result of e-Learning effectiveness by Bloom cognitive taxonomy. 5 Conclusion Remarks In this article, a semantic and systematic mechanism is proposed to discover the educational objectives, which is formulated by well-known extended Bloom Discovery of Educational Objective on e-Learning Resource 77 taxonomy, of learning resources on e-Learning platform. In the first phase, the required educational objectives for competency information specified in O*Net metadata should be determined. The educational objectives of competency information can be visualized with a two-dimensional matrix. Educational objectives of learning resources in e-Learning platform, which reflect a set of competency information, can therefore be inferred in a systematic and semantic manner. In order to verify the feasibility of mechanism proposed in this study, the authors selected the assessment of e-Learning effectiveness as potential practical application using the proposed mechanism. A prototype system is designed and implemented for assessment of e-Learning effectiveness using the proposed mechanism in order to demonstrate the feasibility of the application using the proposed mechanism. By feasibility verification with system implementation for practical requirement, the effectiveness of the proposed mechanism can therefore be basically certified. As for future research with respect to the mechanisms of educational objective discovery and management, the authors recommended that the proposed mechanism could be widely adopted in individual or distributed learning facilities on e-Learning platform. With automatic discovery or retrieval of learning resources with userspecified educational objectives or competency requirements, it is possible for educational objectives as a behavioral guidance of learning resources to satisfy users learning requirement, since the characteristics and profiles of learning resource can be predicted by information provided by proposed mechanism. Acknowledgments. The work presented in this paper has been supported by National Science Council, Taiwan, R.O.C, under Grant No. 95-2524-S-194-004-. The authors deeply appreciate their financial support and encouragement. References 1. ADL: SCORM 2004 2nd Edition Overview [On-line]. Available: http://www.adlnet.org/ downloads/70.cfm (2004) 2. Anderson, W., Krathwohl, D.: A taxonomy for learning, teaching, and assessing: A revision of Bloom’s Educational Objectives. Longman, New York (2001) 3. Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley Press. (1999) 4. Baltes, C.: The e-Learning Balancing Act: Training and Education with Multimedia. IEEE Multimedia, Vol.8, Issue 4 (2001) 16-19 5. Bloom, B.: Taxonomy of Educational Objectives: The Classification of Educational Goals. Susan Fauer Company, Inc. (1956) 6. Brennan, M.: U.S. Corporate and Government e-Learning Forecast 2002-2007. IDC. (2003) 7. Catano, V.: Appendix 1: Competencies: A Review of the Literature and Bibliography [Online]. Available: http://www.cchra-ccarh.ca/en/phaseIreport/appendix.asp (1998) 8. Chang, K., Sung, Y., Lee, C.: Web-based collaborative inquiry learning. Journal of Computer Assisted Learning, Vol. 19, Issue 1 (2003) 56-69 9. Ferguson, R.: Computer-Assisted Criterion-Referenced Measurement (Report WP-41). Pittsburgh University, Learning Research and Development Center, Pittsburgh (1969) 78 Shi-Ming Huang, Hsiang-Yuan Hsueh, and Jing-Shiuan Hua 10. Friedman, R., Deek, F.: Innovation and Education in the Digital Age: Reconciling the Roles of Prdagogy, Technology, and the Business of Learning. IEEE Transactions on Engineering Management, Vol. 50, Issue 4 (2003) 403-412 11. Harzallah, M., Berio, G., Vernadat, F.: Analysis and Modeling of Individual Competencies: Toward Better Management of Human Resources. IEEE Transactions on Systems, Man, and Cybernetics, Part A, Vol. 6, Issue 1 (2006) 187-207 12. Linder, J.: Competency Assessment and Human Resource Management Performance of County Extension Chairs in Ohio. Journal of Agricultural Education, Vol.42, Issue 4 (2001) 21-31 Blended Learning: Experiences of Adult Learners in Hong Kong P. W. R. Lee and F. T. Chan HKU SPACE, The University of Hong Kong Pokfulam Road, Hong Kong, China {ruby.lee, ft.chan}@hkuspace.hku.hk Abstract. The development in information and communication technology brings along many impacts to education and training. Though delivery of courses solely based on e-learning on its own is not totally satisfactory to students and course providers, distance learning and face-to-face courses benefit from the use e-learning as a support in the teaching and learning process. Surveys of Hong Kong part-time students clearly indicated that they preferred a blended learning mode that retains some form of face-to-face teaching and utilizes e-learning at the same time. However, effective blended learning is not simply using the technology as an additional communication means or to organize extra learning activities. The pedagogy has to be adjusted to incorporate e-learning as part of the teaching and learning process. Key words: blended learning, adult learners, distance learning 1. Introduction The advancement in information and communication technology (ICT) brings a lot of hope to education institutes, especially those providing distance education. A wide range of e-learning tools, such as computer-mediated education software, online course materials, and online forum have been developed. Some institutes make use of the technologies to enhance their delivery of distance education courses. Some go for new ventures in offering courses solely using the electronic platform. However, these purely e-learning courses are largely not successful in delivering learning experience to the satisfaction of the students and in achieving cost effectiveness to the satisfaction of the institutes. The potential benefits of e-learning nevertheless flourish when used together with existing models of course delivery. This article reports the experiences and research findings of the authors in the last 12 years. A comprehensive discussion on the development of blended learning and utilizing e-learning to enhance teaching and learning effectiveness can be found in Macdonald (2006) and Naidu (2003). Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 79-87, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 80 P. W. R. Lee and F. T. Chan In the following sections, the development of e-learning activities introduced into distance education courses experienced by the authors is reviewed; the trend of students’ preferences on delivery modes is analyzed based on surveys conducted in various years; and a case study of comparing two blended learning approaches is discussed. Lastly, the Conclusion section summarizes the major findings about blended learning based on the surveys results and the case study. 2. Stages of Blended Learning Since 1990, the School of Professional and Continuing Education, The University of Hong Kong (HKU SPACE) collaborated with Charles Sturt University (CSU) in Australia their first joint course using a distance education delivery mode. Students were largely working adults and studied part-time. For each subject, the adult learners were given a set of distance learning package that consisted of a Subject Outline, a Study Guide, and a collection of required Readings. The Subject Outline stated the objectives and overview of the subject, information of the Subject Coordinator, the proposed self-learning schedule, the assessment items, the marking criteria, and the list of reference materials. About 8 sessions of face-to-face teaching/tutorial sessions of 2 to 3 hours each, depending on the subject, were organized by HKU SPACE using a local teacher to support the students. These sessions were scheduled after office hours or during the weekends to match the availability of the students. Students could communicate with CSU Subject Coordinator through email. At that time, email was far less popular than today. Most students have not used email before they joined the course. They needed to use a modem to dial in to a computer server operated by the University Computer Centre. Many did not have computers at home and few had modems. Before 2000, less than 50% of the households in Hong Kong had one or more PCs at home. Out of these households with PC, about 73% had access to the Internet (Census and Statistical Department website). The local Course Coordinator acted as the communication bridge between students in Hong Kong and the CSU Subject Coordinator in Australia. Since 1994, CSU started to supplement the printed learning package with a computer disk. Hyperlinks and computer animations of flowcharts and computer programs were added to increase the interactions between students and the learning materials. Students generally felt that printer materials had the advantage of physical portability. They could bring along the printed materials and read them during traveling or waiting at some places. Electronic materials simply mirroring the printed version was not welcomed. Additional features such as hyperlinks to relevant resources, computer animations, and multimedia presentations were needed to justify the development and production of electronic materials (Messing and Chan, 1999). Starting from 1999, a more comprehensive online support services were offered in the form of a Subject Web Page. In addition to CSU, Monash University (Monash) in Australia also jointly offered degree programs with HKU SPACE since 2002 using a Blended Learning: Experiences of Adult Learners in Hong Kong 81 web-enhanced distance education delivery mode, which supplemented by face-to-face teaching by local teachers in Hong Kong. For each subject, the set of distance learning materials was put online. In addition, a Forum was set up to enhance communication between the overseas University subject teacher and the students from different places. The Forum also facilitates discussions among the students themselves. Students from different places used the Forum as a platform to share their learning experiences. However, connection to the Internet in Hong Kong was largely through modems. Students needed to occupy a phone line at home and suffered from annoying unexpected disconnections. Recently more Hong Kong people had access to Internet at home. In 2003 the percentage of households that had PCs at home increased to 67.5% (more than 30% increase in three years) and out of these households 88.8% had access to the Internet. Effectively, the percentage of households that had access to Internet at home increased from 36.5% to almost 60%. An increase of 64% of households in Hong Kong had Internet access at home in the three years from 2000 to 2003. The latest survey in 2006 revealed that 71.7% of households had PCs at home and 93.6% of them had access to Internet. The percentage of households that had both PCs and Internet access rose to 67% (Census and Statistical Department website). The advancement in information and communication technologies supported the development of blended learning. Yet, to the students, there were learning elements of face-to-face sessions that could not be totally replaced by online learning. 3. Student Surveys on Preferred Learning Modes In order to understand learners’ preferences of various learning modes, surveys of students studied in the blended learning programs described above were conducted from 2000 to 2005. The first questionnaire survey was carried out in 2000 (Chan and Messing, Chan and Mills, 2000) with two groups of students. One group of 24 students studied a graduate diploma in library and the other group of 21 students studies a Master of Information Technology. Both courses utilized a blended learning mode that some subjects were taught face-to-face and some subjects were based on distance learning but with some supporting face-to-face tutorials. These students studied part-time while working full-time. They were asked to select the teaching mode they preferred most from the following options, assuming they were given a choice: - face-to-face teaching for all the subjects (FtF) - some face-to-face taught subjects and some distance learning subjects with supporting tutorials (BL1) - distance learning with face-to-face tutorials (BL2) - pure distance learning mode (DL) The survey findings revealed that almost all students rejected the pure distance learning mode. Yet, not too many of them preferred a purely face-to-face teaching mode, probably due to the fact that they were working full-time. The majority wanted 82 P. W. R. Lee and F. T. Chan to have the benefits of both world, having face-to-face sessions and distance learning at the same time. A summary of the findings is given in Table 1. IT students did not necessarily favored distance learning supported by technology when compared with the library students. Table 1. The Most Preferred Teaching Mode – Students’ Perspective in 2000 Learning Mode FtF BL1 BL2 DL IT Students n=21 9% 67% 24% 0% Library Students n=24 21% 50% 25% 4% Combined n=45 15.6% 57.8% 24.4% 2.2% Similar surveys on the learning experiences of part-time postgraduate students taking distance learning courses with blended learning were conducted in 2002 and 2003. In February 2002, a survey on the distance learning experience of the two Master’s degree programs jointly offered by HKU SPACE and Australian Universities was conducted. 58 successfully completed questionnaires were received for the February 2002 survey (Lee and Chan, 2002). During December 2002 and January 2003, students from a diploma course, an undergraduate program and five postgraduate programs that were jointly operated by HKU SPACE and universities from Australia and United Kingdom were invited to participate in the 2002-2003 survey. In the 2002-2003 survey, the full-time face-to-face study was refined to include both part-time and full-time study (Lee, Dooley and Chan, 2003). In early 2003, 130 returns were received. The results of all the surveys on thee most preferred learning mode are summarized in Table 2. Table 2. The Most Preferred Teaching Mode – Students’ Perspective in 2000 - 2003 Learning Mode FtF (part-time) FtF (full-time) BL1 BL2 DL 2002-03 n=130 15% 16% 52% 15% 2% Feb 2002 n=58 11% N/A 54% 35% 0% 2000 n=45 15.6% N/A 57.8% 24.4% 2.2% The latest survey was conducted in late 2005 with an extended scale (Lee, Dooley and Chan, 2006). More programs and recent graduates as well as active students from four undergraduate program and four postgraduate programs were included. 274 successfully completed questionnaires were received out of 1,357 distributions. The response rate was about 20%. As e-learning was much more developed and became so pervasive, the preferred teaching modes were redefined and the survey results on most preferred learning modes are shown in Table 3. Blended Learning: Experiences of Adult Learners in Hong Kong 83 Table 3. The Most Preferred Teaching Mode – Students’ Perspective in 2005 Learning Mode FtF supplemented with e-Learning BL1 BL2 Total online learning with some FtF support Total online learning with no FtF 2005 n=274 42% 22% 27% 6% 3% The findings of the 2005 survey about the preferred learning mode are similar to the previous survey results of 2000, 2002, and 2003. Blended learning modes are highly preferred by Hong Kong’s adult learners. The results of the 2000-2005 surveys indicated that blended learning modes with more face-to-face elements were more welcomed by the respondents. Web-enhanced distance learning with no face-to-face elements were not preferred by students respondents from 2000 to 2005 as only 2% to 3% of them preferred such learning mode. E-learning was simply viewed as a supplement in the blended mode. The results of these surveys confirm the predominant acceptance of the mixed delivery modes over pure distance learning and online learning. This further indicated that Hong Kong adult learners perceived faceto-face sessions as highly valuable. The findings of these surveys lead the authors to think what blended learning strategies can better integrate e-learning tools with face-to-face sessions to achieve higher teaching and learning effectiveness, especially for the part-time adult learners. The main features of the blended learning strategy that we have been adopting so far are as follows: - Distance learning package available online and in printed form - Face-to-face sessions at regular internals throughout the semester - Deployed asynchronous online environment (email, forum, subject webpage) for communication, discussion and access to resources The number of face-to-face sessions ranges from 6 to 10 for a semester-long subject. Each session lasts for 2 to 3 hours. The actual arrangement will depend on the nature of the subject and the level of the program. For example, more sessions will be organized for a bachelor degree program than a postgraduate program. Longer hours per session will be organized for a subject involving practical components. During the face-to-face sessions, the local teacher can choose to do one or more of the following activities: - teach the more challenging topics in the subject - discuss questions raised by students, - discuss the assessment items These face-to-face sessions also serve as checkpoints to keep students’ pace of progress and allow for peer-sharing and support. 84 P. W. R. Lee and F. T. Chan The key questions are how to integrate face-to-face sessions and e-learning to achieve the greatest synergy. For the good practice in blended learning, Macdonald (2006) stated that “if there is currently a recipe for a blended strategy, it is a broth of pedagogy, heavily peppered with pragmatism”. In the next section, we describe a case study, comparing two blended learning approaches in an education program. 4. Case Study of Two Blended Learning Approaches In 2005-2006, the delivery approaches of two subjects in an Education Diploma program were examined. The two subjects were taken by two groups of 20 students. Both subjects were supported by the same e-learning platform that supported the following functions online: ƒ Announcement – releasing of announcements related to the subject or program ƒ Resources – presentation slides, handouts, and other reference materials ƒ Forum – discussion forum for students to post views and questions as well as responses to other students’ submitted views and questions. For Subject A, students were encouraged to put up interesting teaching incidents on the Forum for sharing and discussions. Some in-class teaching activities had to put up to the Forum for the teacher and fellow students to read before the class. Fellow students could also post responses to the items posted. These postings, however, would not count as formal assessment items that contributed to scores affecting the final grades of the students. The posting was entirely voluntary. The number of postings of each lesson ranges from 6 to 37. The average number of postings per lesson was 14.7. The average number of postings per lesson per student was 0.74. For Subject B, students were divided into three groups. Students received a set of pre-class reading materials for the next weekly class meeting. Each group was assigned to work on an activity as stated in the materials. Each student was required to post his/her views or proposed solutions in the Forum. Students were encouraged to post responses or follow up questions to items posted by fellow classmates. Each activity provided a scenario for the students to analyze, they were then asked to prepare their responses to some questions. Typical questions, for example, read something like: What do you think about these views? Which one do you like more? How do these teachers see motivation differently? Or can you integrate these different approaches to motivate the class to learn? During the class meetings, members of each group would discuss their postings and any follow up postings. They formulated some concluding findings and presented these findings in the class. A summary report of the findings was then posted back to the Forum. As part of the assessment, the timely submission of the postings in the Forum and the presentations in class meetings contributed up to 20% of the final Blended Learning: Experiences of Adult Learners in Hong Kong 85 score. In addition, each student was required to compile the set of their own postings in the Forum together with their responses to other students’ postings as part of the final assignment for submission to the teacher at the end of the teaching term. The number of postings of each lesson for Subject B ranges from 30 to 50. The average number of postings per lesson was 36.8. The average number of postings per lesson per student was 1.84. This is 1.5 times more than Subject A. It is not only the quantity that implies Subject B has achieved a more effective use of the online Forum than Subject A. Students learnt more effectively in Subject B in several ways due to the design of the blended learning. In each class meeting, students were required to follow up on what they have prepared by reading the reading materials and posting on the Forum. Without such pre-class learning, they would not be able to work effectively the in-class activity. It is by design the integration of the face-to-face session and the use of the online Forum outside class that student learning was built upon. The grouping of students helps not only achieving collaborative learning but also exerting positive group pressure for each group member to submit his/her own posting on time. Otherwise, other members cannot read and prepare for the in-class discussion. It is also by design the weekly postings not only contribute to part of the final score but also form an essential part of the final assignment submission. The importance of using the online Forum is appropriately reflected in its weighting among assessment items. 5. Conclusion Learning at a distance from the campus was not a new thing. With the development of ICT, e-learning has firmly established its importance in education and training courses, no matter these courses are conducted in conventional face-to-face or through distance learning. In our studies, we find that throughout the last decade most students in Hong Kong studying part-time postgraduate and undergraduate programs indicated their preferences in retaining some form of face-to-face teaching while at the same time utilizing the advantages of e-learning. A pure form of delivery mode, whether it is face-to-face teaching or distance learning, is not appealing to the adult learners. Cheng (1998) commented that regarding students in Asian culture, it was uncertain whether they prefer to study at home and communicate electronically with their teachers. “Students in Asian culture are also not used to expressing themselves and exchanging views. They are more used to listening, keeping analyses in their minds, and express themselves only when it is very necessary. As such it remains to be seen whether the extension of the 86 P. W. R. Lee and F. T. Chan physical classroom to the cyberspace would further discourage or encourage interaction among students.” (Cheng 1998 p204) Through the 2000- 2005 surveys and the case study, it is concluded that Hong Kong students take a pragmatic approach towards e-learning. They can be active learners in the cyberspace if a proper pedagogy is adopted. To make blended learning more effective, it is more than introducing the technology component in the teaching and learning process. The right teaching approach and assessment strategies have to be employed. For example, it is evidenced that participation in online forum discussion becomes more active and fruitful when such activity is designed as an integral part of class teaching and contributing to the assessment. As mentioned by Macdonald (2006 p.54), different parts of a blended strategy were inter-related and there was much to learn about the ways to integrate e-learning with face-to-face support. With the high access rate to Internet at home nowadays, it is now more practical for Hong Kong teachers to explore how they can deploy online tools with their class teaching. References Census and Statistical Department, HKSAR Government website (March 2007): http://www.censtatd.gov.hk/hong_kong_statistics/statistics_by_subject/ Chan, F. T. and Messing, J. (2000) "A Joint Venture in Distance Education Program between Hong Kong and Australia", IVETA Conference 2000 Chan, F. T. and Mills, J. (2000) "Collaboration for Success in Open and Distance Education: A Case Study of Australia and Hong Kong", ‘Distance Education, an open question?’ Conference 2000 Cheng, Kai-Ming (1998) "Institutional collaboration in higher education: challenges of the information era", in Ronnie Carr (edn 1999), The Asian Distance Learner, p196-206, Open University of Hong Kong Press: Hong Kong, 1999. Lee, P. W. R. and Chan, F. T. (2002) "Mixed Mode of Delivery - An Effective Collaboration Model", ASAIHL 2002 Lifelong Learning Conference, Proceedings of ASAIHL 2002: Lifelong Learning, 17-19 June 2002, Nanyang Technological University, Singapore, 155163. Lee, P. W. R., Dooley, L. and Chan, F. T. (2003) "Enhancing Adult Learning via E-Learning: the Perspectives of Students and Teachers in Hong Kong", VIEWDET 2003 Vienna International Conference on eLearning, eMedicine, eSupport, 26 – 28 November, 2003, Vienna University of Technology, Austria. Lee P.W. R., Dooley L.S and Chan, F.T., (2006) "Effective E-Learning: Perspective of Adult Learners in Hong Kong” , HKITEC 2006: Hong Kong International IT in Education Conference – “Capacity Building for Learning through IT", Proceedings of HKITEC 2006, 6 – 8 February 2006, EMB, Hong Kong, pp. 66-75. Blended Learning: Experiences of Adult Learners in Hong Kong 87 Macdonald, J. (2006) Blended Learning and Online Tutoring: A Good Practice Guide, Gower, Hampshire Messing, J. and Chan, F.T. (1999) "Hands Across the Ocean: Using Information Technology in a Distance Education Course", Proceedings of the Fifth Hong Kong Web, F. Castro, R. Lai, and Sr. M. Wong (ed.), pp. 227-241. Naidu, S. (ed.) (2003) Learning and Teaching with Technology: Principles and Practices, Kogan Page, London. An Observational Study on Blended Learning for Japanese Language Studies in a Local University in Hong Kong Kenneth K.C. Lee1, Melody P.M. Chong 2 2 1 Department of Computer Science, City University of Hong Kong Department of Chinese, Translation and Linguistics, City University of Hong Kong {kenkclee, ctchong}@cityu.edu.hk Abstract. This paper discusses how educators and learners can be benefited from blended learning. The research employed a qualitative research method. An unstructured observational approach was used to assess how the current teaching activities can be improved efficiently and effectively with blended learning. Four dimensions of language learning in relation to vocabulary, listening, grammar and oral skills are addressed. The results reveal that vocabulary and listening skills of students can be improved by offering more input with web-based learning whereas grammar and oral skills are suggested to maintain more face-to-face interactions in the classroom between instructor and students. Implications for effective learning and recommendations for future research are addressed. Keywords: blended learning, language studies, Japanese language, e-learning 1 Introduction Thanks to the current advancement in the Internet technology, there have been revolutionary changes in our way of life. We can enjoy a bunch of advantages with reduced costs and improved services. The Internet shows its influences in the areas from business, mass media, entertainment and short messaging to e-learning. In Hong Kong like most developed countries, web-based learning or the so-called e-learning has been a success that the paradigm was widely adopted from university colleges to primary schools. Education is one of the major elements in governmental expenditure in Hong Kong. According to the governmental budget in 2006–07 [1], $56.5 billion was spent on education. For every university graduate who has passed through the education system, the government's total investment exceeds $1 million. A further $1.1 billion in total was injected into the language fund with a view to raising students' linguistic proficiency. Enhancing students’ learning performance with the etechnology is a vital mission that educational practitioners face today. Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 88-100, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. An Observational Study on Blended Learning for Japanese Language Studies 89 Yet a recent study conducted by Thomson and NETg [2] indicates that pure elearning approach does have its deficiencies. The study investigated the business skills of participants who received a sequence of e-training courses. The performance of participants was compared with another group who received “blended” training courses which consisted of a combination of online media and instructor-led supports. The study showed that the speed and accuracy performed by the “blended” group was considerably superior to that of the pure e-learning group by 30-40%. With the uprising urge of “enhanced productivity” and “cost effectiveness”, blended learning has become an ideal and only solution to educational practitioners and business training experts. What remains unsolved, however, is the question of “What is the correct blend?”. Similar to what the bartenders used to do, we are now doing a delicate blending with multiple media and thoughtless ah-Hoc blend giving us nothing but failure, as what Roger Schank commented [3]: “It is in vogue for a simple reason. No one wants to spend that much on elearning and people in general want to preserve what they have, so they have made up this nice name for not changing much and called in blended learning.” 1.1 Blended Learning in Language Studies Among all those academic and vocational subjects, language studies are probably the most challenging areas. The reason lies in the fact that language is not a single skill but a collection of literary and communicational skills which requires high degree of proficiency. It is a long and painstaking process that it takes years to train up the language ability of a student to a level which enables direct communication with native speakers. Language study is also distinctive in a way that the learning of high level concepts or syntaxes heavily depends on how well a student can grasp the fundamental ones. For example, to learn the passive voice in English, students have to master the past participles. This implies that foundation building is particularly important during the course of study. In this paper, we demonstrate that blended learning, which is characterized by a combination of free-form e-media and instructor lead teaching (ILT), is the natural and ideal solution to quality language learning. 1.2 Pros and Cons of Pure e-learning Approach In order to acquire “the best blend” for a course, one should begin with “knowing your subject, knowing your delivery media”. The e-learning paradigm provides us with a variety of instruction media and formats including websites, online discussion groups, knowledge base, video conferencing and mobile systems, to name a few. There are also well established Virtual Learning Environments (VLE) including the open-source ATutor [4] system and the proprietary WebCT/Blackboard [5] system which provide full solution to the learning and knowledge management processes. Considering the old days in which an instructor would like to release handouts to students, the only way to do so was to photocopy the handouts in mass volume and distribute it to students during scheduled tutorial sessions. Meanwhile, the use of hyperlinks and e-documents nowadays not only reduces the time and cost, but also 90 Kenneth K.C. Lee and Melody P.M. Chong allows rapid information dispersion at any time and anywhere. Apart from cost effectiveness and accessibility, e-learning can also solve two major problems in education in the long run in terms of “variance in progress” and “short memory duration”. As an educational practitioner, the greatest challenge comes from the control of class progress. Since students in class are not created equal, some students may need more time on certain topics. At times, the progress of class is "dictated by the slowest learners in the group" [6]. With e-learning, students can make use of non-contact hours to revise and refine their skills in particular areas they consider necessary, and hence progress among students could be synchronized. In addition, e-learning also excels in a way that it facilitates knowledge acquirement and makes students active learners. Traditional classrooms focus on the transfer of new knowledge from instructor to students without highlighting the revision process. (Revision is considered as something that students should handle themselves). According to a study on knowledge retention [7], the lack of revision and memory reinforcement may cause memory lost of 85% within a week. With online assessment and knowledge management systems, students can now take a more active role in the purchase of knowledge and self assessment. These can improve the quality of learning. However, e-learning is no silver bullet. Online resources and VLE do show their short sides from time to time. One major problem is that students are generally inexperienced and hence personal guidelines and study models are needed in order to keep them from wandering around the sea of online resources without target. Another issue is that e-solution generally lacks personal interaction which is important in disciplines such as second language acquisition [8]. As mentioned earlier, language involves a collection of skills. In this research, we attempted to investigate different learning dimensions to figure out a combination of pedagogic approach and teaching media for effective language learning. We started by knowing the subject. The next sections describe the methodology of the research and highlight the attributes of Japanese language. 1.3 Objectives of the Study This paper contributes by sharing our views on the application of blended learning in language studies. In particular, Japanese language study is discussed. The objectives of the study are to justify blended learning by analyzing the respective merits of traditional classroom instruction and e-learning. In addition, the paper discusses what it takes to help students master the Japanese language with reference to the existing online and e-resources adopted in a local university in Hong Kong. Finally, the paper also suggests possible development and directions for blended learning. It is hoped that our propositions of our recipe of blended learning could bring incites and insights to educational practitioners in similar academic disciplines. An Observational Study on Blended Learning for Japanese Language Studies 2 91 Methodology The present research employed a qualitative research approach known as observational study. Observational studies can be of either the nonparticipantobserver or the participant-observer type. Both of these can be either structured or unstructured. Observational study is that an observer predetermines a set of categories of activities or phenomena to be studied whereas an observer of unstructured observational study has no definite ideas of the particular aspects that need focus. The study employed the participant-observer type and unstructured study method. With this research method, the investigators entertain a set of tentative research objectives that serve as a guide as to who, when, where and how the individual will observe [9]. One of our investigators has joined a local university since 2000. The researcher is a member of a work team in the Japanese language section taking a role of coordinator and instructor, and observed the dynamics in classrooms for this study. This research method allows the researchers to study the central phenomenon in depth. The observation period was between 2004 and 2006. The courses observed included elementary to advanced Japanese language courses. All participants observed were university students across different disciplines and aged between 19 and 22. 3 An Observational Study for Japanese Language Study Contents Level Sections 1 Hours Criteria Points Writing-vocabulary 45min. Listening 45min. The examinee has mastered grammar to a 100points high level, knows around 2,000 kanji and 100points 10,000 words, and has an integrated command Reading-grammar Total 90min. 180min. 200points of the language sufficient for life in Japanese 400points society. This level is normally reached after studying Japanese for around 900 hours. Fig. 1. Japanese Language Proficiency Test (Level 1) Source: Society of Japanese Language Education, Hong Kong In this study, we use the Japanese Language Proficiency Test (JLPT) [10] as a benchmark to generate the ideas and examine how the current teaching activities can be benefited from blended learning. The JLPT is a well-known international language test consisting of four levels. Each test is made up of three sections including writingvocabulary, listening and reading-grammar. The contents and criteria of the test are presented in Fig. 1. To highlight the importance of blended learning, we should first understand the nature of Japanese language. Like other languages, the study of Japanese requires intensive training in different aspects such as grammar and oral skills. However, the picture is further complicated by distinctive linguistic features which exist in the Japanese language. For example, Japanese words can be written as kanji (Chinese character) or kana (Japanese character) and contain a huge amount of 92 Kenneth K.C. Lee and Melody P.M. Chong homophones. This section describes the problems faced in Japanese language study with respect to vocabulary, listening, grammar and oral skills. 3.1 Study of Japanese Vocabulary In most instructor-led classes, instructors teach students about 180 new vocabularies in one semester. Whereas those advanced learners have to remember over 6,000 to 10,000 words when they intend to attend the level 1 or 2 of the JLPT. For decades, compilation of personal wordlist is the only means for students to master a huge amount of vocabularies. But yet, compilation of word list is not a no-brainer and considerable efforts are required. As mentioned in the “Body & Brain” magazine [11], it would be beneficial for recollection if information is “organized into a smaller set of subcategories”. This implies that a practical wordlist must be well categorized. For instance, it is more efficient to study the words with similar (synonym) or opposite (antonym) meanings. As the mental database of vocabularies grows, memory tends to “interfere with each other”. Our observations reveal that many Japanese learners find the vocabularies difficult to memorize because some Japanese words may contain two or more different pronunciations. For example, the word ‘human-being or person’ has at least three different pronunciations. It includes hito, jin and nin. The situation is complicated due to the existence of homophones. Some examples are given in Fig. 2. Pronounciation koto kōtō kōdo kōdō Possible meanings ‘capital’, ‘affair’ or ‘piano’ ‘oral’, ‘nice pitching (baseball)’, or ‘high level’ ‘height’, ‘brightness’ or ‘hardness’ ‘action’, ‘highway’, or lecture hall’ Fig. 2. Homophones in Japanese 3.2 Study of Japanese Listening Skill In traditional in-class training, most instructors find difficulties in improving the listening skills of their students. Given that the contact-hours are limited, most of the times have been spent on vocabulary and grammatical explanations. Nowadays, standard teaching aids used are those companion tapes or CDs included in textbook. Each chapter accounts for about 5-7 minutes of recording, including the recitation of main text/article, sound-only roleplay and listening questions. Considering that time needed for students to write down the answers and for instructors to reveal the correct answer together with vocabulary and grammatical explanations, one could easily draw a conclusion that instructors are simply not able to afford playing the tape many times within the standard 50-minute listening lab session in the classroom. Needless to say, playing real-life recordings such as news broadcast or radio program in class is simply out of the question. The outcome of all those is that the listening abililty among An Observational Study on Blended Learning for Japanese Language Studies 93 students in class could be highly asynchronus, which in turn, affects the progress of class. It is especially problematic that a considerable number of Japanese courses are taught in Japanese itself by native teachers. 3.3 Study of Japanese Grammar Most Japanese words are derived from Chinese. The situation is similar to English in which some sixty percent of English words are derived from Latin languages. However, Japanese grammar did not significantly change [12]. When teaching a foreign language like Japanese, we found that most of our instructors spend most of the times to teach the grammatical concepts to students. Some scholars [13] also urge that, in particular, the Japanese particle system is a major hurdle for second-language learners in college-level Japanese courses, due to its complexity and its absence from the learners’ first language. In Japanese, each noun phrase is followed by a particle that indicates its grammatical and semantic role. It is essential for most students to understand the rule of the particles especially for those who intend to attend the JLPT. Examinees are always asked to fill in the blanks by selecting a correct particle from a list of multiple choice questions. Most elementary students find the rule of particles difficult to understand. 3.4 Study of Japanese Oral Skill Like most languages, the accent and intonation of spoken Japanese may be the most difficult parts to learn. In Japanese language, there are many words with same pronunciations. However, a change in the accent would change the meaning of the ____ __ word. For instance, ame can be ‘candy’ (ame) or ‘rain’ (ame). Unlike English which has stress accent, Japanese has pitch accent which means that after an accented syllable, the pitch falls. Another distinct feature of Japanese language is the honorific expressions. Japan is well-known as a hierarchical society. Japanese has an extensive system to express politeness and formality. The position of a person is determined by a variety of factors including job, age, experience, or even psychological state [14]. Japanese polite expressions are divided into two types that are honorific words and modest words. The former is used to express a speaker's respect while the latter is used to express a humble attitude of a speaker. This makes Japanese expressions difficult for the overseas learners [15]. 4 Discussion So far, we have discussed the four dimensions of Japanese language learning. In this section, we go further to discuss how educators and learners can be benefited from blended learning with proposed activities in relation to vocabulary, listening, grammar and oral skills. Suggested activities are highlighted in Fig. 3 whereas each dimension is discussed sufficiently in the followings. 94 Kenneth K.C. Lee and Melody P.M. Chong Ability Distinctive attributes z Numerous! Blending of media z Is best memorized with synonym / antonym z Requires dictation Vocabulary online resour ces hardcopy wordlist/ dictionary z Closely related to oral skill z Upgrade teaching aids from cassette tape to digital media with visual aid and/or speed control online radio/AV programs tapes and CDs z Idea can be presented in a thousand and one ways with different tones and styles z Distribution of distinctive articles, templates & teaching aids via internet / VLE z Learning via reading model articles with good rhetoric z Replace part of assessment with online quiz (especially MC questions for lower grade) z Practice makes perfect Writing and Grammar Oral Internet as a platform for discussion and assessment (MC) z Discussion or Q&A Manual using online forum guided reading by e-mentors or e-tutors (as and self-learning / knowledge marking anagement) z Marking of students’ work by experienced instructor, yet hardcopy submissions are replaced by electronic ones submitted through VLE z Stress on interaction z Face-to-face classroom gatherings with role-play practices z Complex intonation rules z Existence of homophone z Current Artificial Intelligence and speech recognition technologies could not catch up with the complexity of natural language z Remember vocabularies anytime anywhere with mobile technology z Utilize online radio / AV broadcast via steaming technology through internet z Ability improves if constantly “immerse” in that language. z Best training aids possibly come from reallife dialogues & announcements from native speakers z Use of online dictionaries which overwhelm the traditional ones in terms of word count and related resources z Online quiz (dictation) with word database z Repeated listening is needed Listening Suggested activities z Share and exchange compiled wordlist via internet and VLE digital audio/ speech (supporti ve) face-to-face guidance z Recording of students’ speech into digital form (e.g. mp3) and submit to instructor via VLE. Fig. 3. Suggested blending model for Japanese study An Observational Study on Blended Learning for Japanese Language Studies 4.1 95 Vocabulary With respect to the vocabulary dimension, it is expected that the e-technology will create synergy by speeding up the wordlist compilation and classification process for students’ self-learning. The “huge information volume” and “high searching speed” properties of e-resources match perfectly to the demanding wordlist compilation task. Students can store the words in an electronic form instead of hand-written text on small paper cards. It allows students update the personalized wordlists and review them at anytime and anywhere. Students could also share or exchange wordlists in VLE platforms, discussion forums, blogs or even personal wiki solutions [16] with their fellows. Existing web-based systems such as Yahoo! Dictionary Service [17] (Fig. 4) provide related synonym and antonym in terms of a large word counts and related resources. Fig. 4. Yahoo! Japan – Dictionary Service Fig. 5. ii-Kanji: a mobile system for vocabulary learning Fig. 6. Kanji-Step: an online quiz system with detail statistics 96 Kenneth K.C. Lee and Melody P.M. Chong Mobile system such as ii-Kanji [18] (Fig. 5) facilitates kanji (Chinese character) learning by enabling memorization and revision at anytime and anywhere. eTechnology also enables students to have self evaluation through the online dictation systems. Systems such as Kanji-Step [19] provide students with vocabulary quizzes with words classified into categories and levels. As shown in Figure 6, statistics would be generated after quizzes and it could act as clues for further improvement. In the JLPT paper, candidates are always being asked to choose the right pronunciation of a kanji. Students have to select the correct one from a set of similar pronunciations (Fig. 7). The computerized grouping system suggested above will certainly help students to solve the learning problems. To further facilitate the learning process, we suggest incorporation of query system for similarly-pronounced words into e-learning environments. Although such system is yet to be seen in public, there should be no technical difficulty with free Japanese resources [20] and a server side programming language such as PHP, ASP .NET or JSP. Fig. 7. JPLT Paper 1: Writing-Vocabulary Source: The 1997 Jap. Language Test Lvl 1, published by Society of Jap. Language Education, HK 4.2 Listening As mentioned earlier, the difficulties of listening skill training lie in “limited contacthour” and “asynchronous progress in class”. The use of e-resource could certainly alleviate the current situation. First of all, the media itself is inherently superior. Digital audio in wav, mp3 or wma formats allow random access, as opposite to the sequential access method in traditional audio tape. Student could easily set bookmark to appropriate phrase or word in which they consider more attention is needed. Another advantage over traditional media is that students could adjust the playback speed according to one’s personal listening ability. There are existing pedagogic tools for audio indexing together with speed control for language learning [21]. With classical signal processing techniques such as resampling and pitch-shift, students could enjoy clear and slowdowned replay of listening material without acoustic distortion. Popular media players such as PowerDVD [22] and GomPlayer [23] An Observational Study on Blended Learning for Japanese Language Studies 97 already have the variable-play-speed features incorporated and are applicable to most Audio/Visual (A/V) media. Students can also enjoy listening of real-life programmes via streaming technology on the Internet. Japanese mass media such as NHK [24] broadcasts free radio programmes on the Internet. Students can refine their listening skill by immersing themselves in real-life language and native Japanese dialogues. Students can also learn jargons in specialized channels such as Daiwa Securities [25] and Nikkei Business [26]. In addition, there are many Japanese learning programs packaged with visual content and description such as the “ShinNihongo de Kurasou” programme (新 日本語で暮らそう) provided by NHK. With the advanced Internet technology, today students can take more active role in self-learning at their own pace. 4.3 Oral It is believed that the best way to enhance oral ability may be to maintain face-to-face interactions between tutors and learners. As Yamazaki [27] suggests, despite the great expectations of society for such an ideal speech translating system, there remains a great number of difficult problems related to acoustic and linguistic phenomena that need to be overcome. This is because spoken language is characterized by the use of colloquial idioms, occasional omission of words and inversion of word order. It seems that face-to-face interactions in the classrooms with role-play practices remain to be the most effective way to polish the oral skill. Nevertheless, in reality, it is infeasible to have intensive training between teachers and students because this involves a huge investment in terms of budget and human resources. Based on our observations and experiences, it is suggested that students may record their speech into a digital form (e.g. mp3) and submit it to their instructor via email or VLE for feedbacks. This is especially effective for those advanced level classes that the instructor may not have enough time in the classroom to train up their students’ speech and presentation skills. 4.4 Grammar In order to improve students’ writing and grammar skills, a combination of a blend of e-technology and traditional tutoring is suggested. Nagata’s [13] experimental study suggests that the computer’s metalinguistic feedback program can lead learners to develop general grammatical competence in the use of particles. The program can guide adult learners with explicit grammatical instruction and detailed, metalinguistic feedback about errors. Instructors can also post some good articles on VLE environment (Fig. 8) [5] for students’ self-learning. With on-line discussion forum, instructors can also encourage their students to write Japanese diary and share their writings with their fellow students. Online forum supporting alias and anonymous identity may stimulate students’ motivation. It is expected that students would be more willing to post their works and ask for comments. In addition, forum or chat room can be set up allowing students to discuss the issues in relation to grammar, culture or experience sharing. Instructors can also assign senior students to act as e- 98 Kenneth K.C. Lee and Melody P.M. Chong helpers or e-mentors to help the junior students on their particular learning problems. It can provide practical chances for those senior students to learn and also lead to better utilization of resources. With the above suggestions, instructors can spend more time on creating teaching materials and help those students with learning difficulties. Fig.8. Japanese discussion forum on Blackboard (VLE system) 5 Conclusion The effectiveness of blended learning depends on whether a teacher can match the appropriate delivery media to existing teaching activities and to catch up with the elearning training. In this paper students are suggested using online dictionaries and quizzes to assist vocabulary learning. The informative online resources not only save time for knowledge management and retrieval but also free students from scheduled and geographical restrictions. E-resources also benefit listening training by providing acoustic features which are absent from traditional media. The use of media streaming through the Internet allows students to access real life programmes and broadcasts from Japan, which are perceived as ideal training materials for listening. For grammatical training, we suggest the use of VLE and online forums with assistances from e-helpers or e-mentors. Instructors may also employ the computer assisted software with intelligent feedback systems which may help students solve the sophisticated grammatical problems. Finally, with respect to oral skill training, the preservation of face-to-face instruction with activities such as real life role-play shall be maintained. As suggested by Chaudron [8], the interaction in the classroom between peers is one of the most significant factors for successful learning of the second language acquisition (SLA). However, recording of learning progress in a digital form is also suggested to supplement with the classroom interactions because it allows valuable clues of room for improvement. To conclude, the use of e-technology and blended learning brings about effective learning and cost effectiveness. However, like other revolutionary institutional policies, ‘human’ is the ultimate factor which governs make or break. Despite the availability of robust hardware (technology), we also need appropriate software (instructors) to make things happen. Instructors must be well trained, both technically An Observational Study on Blended Learning for Japanese Language Studies 99 and mentally, and accommodate themselves to the new-generation instructional media. In addition, facilitators shall also adjust their teaching mode rested on the motivation and feedbacks of their students, and be able to encourage and maintain active and well-mannered learning atmospheres on the web-based learning. 6 Limitations and Recommendations for Future Research This study has some weaknesses that future researchers might consider. The study relied on information observed from a single organization. More extensive analysis with larger sample size would serve to strengthen understandings and conclusions on blended learning for language studies. Future research shall also examine the effects of blended learning on students’ performance based on the empirical data and analysis. More research into this area will contribute to formulate a better teaching mode for educational practitioners. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] The 2006-07 Expenditure, The Government of Hong Kong SAR http://www.budget.gov.hk/2006/eng/budget17.htm Thomson Job Impact Study - The Next Generation of Corporate Learning http://www.netg.com/NewsAndEvents/PressReleases/top Blended Learning - Ask the Arthur – CIO http://www2.cio.com/ask%5Cauthor/2001/questions/question232.html ATutor Learning Content Management System http://www.atutor.ca/ Blackboard – Educate. Innovate. Anywhere http://www.blackboard.com/us/index.Bb Masterworks International - Distance Learning http://www.masterworksinternational.com/faq.asp Jeff Snipes, “Blended Learning: Reinforcing Results”, Sept 2005 http://www.clomedia.com/content/templates/clo_article.asp?articleid=1070&zoneid=25 Chaudron, C. “Second Language Classrooms: Research on Teaching and Learning.”, Cambridge: Cambridge University Press, 2004. Cavana, R.Y., Delahaye, B.L. and Sekran, U., “Applied Business Research: Qualitative and Quantitative Methods.”, Australia: John Wiley & Sons, 2001. The Japanese Language Proficiency Test (JLPT), Japan Educational Exchanges and Services http://www.jees.or.jp/jlpt/en/index.htm Body & Brain Magazine http://www.bodynbrain.com/01_magazine/news_view.asp?SeqNO=36 The Japanese Language, http://www.wsu.edu/~dee/ANCJAPAN/LANGUAGE.HTM N. Nagata, “The Effectiveness of Computer-Assisted Metalinguistic Instruction: A Case Study in Japanese”, Foreign Language Annals, 30(2), pp. 187-200, 1997. Wikipedia – Japanese Language – Politeness http://en.wikipedia.org/wiki/Japanese_language#Politeness H. Ogata, and Y. Yano, "How ubiquitous computing can support language learning", In Proceedings of KEST 2003, Honjo, Akita, Japan, pp. 1-6, 2003. List of wiki software, http://en.wikipedia.org/wiki/List_of_wiki_software 100 [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] Kenneth K.C. Lee and Melody P.M. Chong Yahoo! Japan – Dictionary, http://dic.yahoo.co.jp/ Today’s ii-Kanji, http://www.ii-kanji.com/ Kanji-Step – Japanese Language Resource Center, http://www.kanjistep.com/ EDICT - Japanese/English dictionary project (Monash University), http://www.csse.monash.edu.au/~jwb/edict_doc.html D. Rossiter, G. Lam, B. Mak, “Automatic Audio Indexing and Audio Playback Speed Control as Tools for Language Learning”, ICWL 2006, pp. 289-297. CyberLink – PowerDVD, http://www.cyberlink.com/ GOM Player, http://www.gomplayer.com/ NHK News, http://www.nhk.or.jp/news/ Daiwa Internet TV, http://www.daiwatv.jp/ NBOnline (Nikkei Business Online), http://business.nikkeibp.co.jp Yamazaki, Y, “Research activities on spontaneous speech translation”, Denshi Tokyo, 33, pp. 109-114, 1995. Structured Blended Learning Implementation for an Open Learning Environment Jason K.Y.Chan, Ken C.K.Law Department of Computer Science, City University of Hong Kong, Tat chee Ave., Kowloon, Hong Kong, jasonchan@cm13.mit.edu & cskckl@cityu.edu.hk Abstract. This paper proposes a structured blended learning for providing elearning strategies adopted by the Open University of Hong Kong (OUHK). The paper identified the factors that are likely to impact on the use of learning technologies in future. By introducing the structured blended learning with the concept of learning cycle, the problem of being lost in hypermedia environment can then be solved. Knowledge can be delivered more effectively and efficiently to meet student learning needs. A scenario example on the use of elearning technologies and user interface are illustrated. The implications of the learning strategy adopted at OUHK will also be discussed. Keywords: blended learning, learning preference, Open Learning Environment (OLE), proactive planning, asynchronous and synchronous online learning. 1 Introduction This paper proposes a structured blended learning mode for providing distance education by the Open University of Hong Kong (OUHK) in the few years. According to Iverson (1993), traditional methods of curriculum development usually require four-to-seven year cycle in the process from draft originals to final revised curriculum implementation. In the world of fast changing information technology and ever changing applications, this development cycle will encompass an enormous amount of changes. Such a period would also involve the assessment of current strengths and weaknesses of the learning environment and the identification of new educational trends. In Hong Kong, most higher education institution's strategic plan adopted a 5-year development cycle. (OUHK, 2002; HKU, 2002; CityU, 2002) In this paper, the factors that are likely to impact on the potential use of learning technologies in the OUHK over the next few years will be described. In the first section, we will provide a brief teaching and learning environment at the OUHK, reviewing the factors that are likely to impact on the potential use of learning technologies in the OUHK over the next few years. In the second section, we will introduce the proactive plan about what learning technologies will be likely to be employed at that time. In the third section, we will discuss the implications of this change, followed by a brief conclusion in the fourth section. Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 101-113, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 102 Jason K.Y.Chan, Ken C.K.Law 1.1 Teaching and Learning The OUHK was the first university, as a self-financing and non-profit making institution, to offer open and distance education in Hong Kong. Since its inception in 1989, the University has striven continuously to achieve its mission of providing flexible further education opportunities for adults aged over 17. The University has offered more than 130 postgraduate, degree, associate degree and sub-degree programs. (OUHK, 2005) 1.2 Blended Learning The OUHK offers studies with flexible form of learning environment, for students to choose, where and when, and also provides them with carefully structured materials to guide them through the courses. The learning style offered by OUHK is a blended learning mode via printed materials, CD-Rom, non-classroom face-to-face tutorial sessions held in the evening and weekend and also online learning. Students can decide on their study path and pace, making use of a variety of print and interactive multimedia materials, tutorial support and also access to electronic library resources. The OUHK provides classroom sessions around Hong Kong for tutorial lessons through rental means. There are also trained tutors to conduct tutorial sessions, tutorial discussions and course work assessments, and also responded to students' queries by phone and e-mail. The adoption of Online Learning Environment (OLE), students can access a comprehensive range of learning resources and personalized services to help them study more effectively without the constraints of time and place. (OUHK, 2005) 1.3 Readiness of Student with Information Technology To use electronic means of communication, OUHK need to know if the students are familiar and be able to access the necessary facilities and equipment. The OUHK carried out surveys on students regarding their information technology readiness on a regular basis. One of the surveys revealed that an increasing trend of PCs ownership by students, access to the Internet, and willingness to use email for communication. (OUHK, 2005) Structured Blended Learning Implementation for an Open Learning Environment 103 Source: Chung, 2003 Another recent survey in July 2003 (Chung, 2003) indicates that: 99.0% of students have PCs; 97.5% of students have access to the Internet; 78.5% use broadband, and 63.5% use email more than once a day; 78.0% of students prefer to use the Internet for communication with the university, tutors and other students (8.0% do not prefer to do so; the rest are neutral); and 84.0% of students prefer to receive information or materials from the University’s Registry via email (8.8% do not prefer to do so; the rest are neutral). The results reveal that the majority of students are technologically prepared and have access to information technology equipment. Fewer than 10% of our students do not want to use electronic means as the primary form of communication with OUHK. (Yuen, 2004) 1.4 Student Learning Preferences When the students have access to the related facilities, another thing we need to know is their willingness or preference to use learning technologies. Another OUHK survey (Vermeer and Murphy, 2004) revealed that there is a strong student preference for using multiple forms of media in learning. Student Preferences for Using Different Forms of Media in Learning Source: Vermeer and Murphy, 2004 104 Jason K.Y.Chan, Ken C.K.Law OUHK students’ first preference is clearly face-to-face contact. Yet two forms of online learning support -- informational course websites, and online discussion boards -- still ranked very high, with over 50% of respondents considering them either ‘effective’ or ‘highly effective’. Since other data, both research-based and anecdotal, show a highly variable commitment to the use of online learning tools in OUHK courses, these positive results are significant. Equally significant is students’ disappointingly low opinion of course CD-ROMs: only 36% of respondents found them effective or highly effective, and 26% considered them ineffective or highly ineffective. (Vermeer and Murphy, 2004) OUHK course studies still utilize two traditional forms of media, i.e. video and audio tapes, as many students suggested the OUHK to provide online, streamed versions of as Audio/Video content. Numerous respondents in fact suggested that the University provide ‘video lectures’, ‘tutorial notes in PowerPoint’ and ‘highly interactive Flash notes’. Another significant theme in respondents’ comments was repeated requests for multiple forms of access to course content. The OUHK students want a varied media mix, including overlapping provision of content using different media, and they want it delivered using the latest technologies. However, the University need to consider the key factors for the change involved in time, effectiveness and also financially viable. (Vermeer and Murphy, 2004) 1.5 Insufficient Interaction and Guidance in Existing OLE Successful e-learning environment involve interactivity between teachers and students, between students and the learning environment, and among students themselves, as well as active learning in the classroom (Sherry 1996). Multimedia equipment such as graphics, video, animation, and sound, can add richness in contents to materials, but they do not add interactions. An animation sequence or colorful graphical scene attracts the learners’ attention, but does not engage them in making decisions or immerse them in the program. Interactions make learning active rather than passive, and they provide learners and the instructor with feedback. Interaction is a cyclic process in which two actors alternatively listen, think and speak (Crawford, 2000). It is important to develop teaching materials with a variety of interactive techniques. Interactivity takes many forms; it is not just limited to audio and video, nor solely to teacher-student interactions. It also represents the connectivity the students feel with aides, facilitators and their peers. Garrison (1990) argued that the quality and integrity of the educational process depends upon sustained two-way communication. Without connectivity, e-learning degenerates into the old correspondence course model of independent study. The student becomes autonomous and isolated, procrastinates, and eventually drops out. Effective distance education should approach Keegan's ideal of an authentic learning experience. However, the existing web-based learning does not provide sufficient interaction means. Structured Blended Learning Implementation for an Open Learning Environment 105 Moreover, the existing OLE connects hypermedia such as text, graphics, animation, audio and video in a nonlinear manner. The major problem with hypermedia leads to feel lost in learning hyperspace. Learners may get lost in complex hypermedia connected webs. It is difficult for learners to find a way to organize the learning material as a guided learning in an orderly manner. Besides, there is the unpredictable nature of how readers link to hypermedia elements. If essential information is located in a hypermedia link that is not structured, the learner will not master the objectives in that lesson. Finally, a learning gap will exist between learners and teachers, teachers and knowledge delivery, knowledge delivery and learners. The interactions and communications between teachers and learners are not sufficient and efficient. (Chan, 2003) 2. Proactive Planning University policy makers, nowadays are required to have proactive thinking of how the university will be lead, managed and structured the learning environment. They also need to face many important issues and decisions concerning the impacts of information technology on their institutions such as organization, governance, management, and its relationships to students, faculties, and staffs. All of them will require careful reevaluation, forecast and almost certain changes. (Duderstadt, Atkins and Houweling, 2002) 2.1 Scenarios of Information Technologies in OLE The OUHK imitated an e-learning project in January, 2004, which aimed to develop a WEB-based Interactive Tutoring System (Webits) that enabled teachers and students to interact effectively online, supported educational multimedia data transmission in real time such as video, audio, text, and enhanced white-board data, presentation of educational materials. The project was developed based on the following scenario imagination and assumptions. Assume that technology developments, that most people will have access to a wireless network with bandwidth of at least one gigabyte per second within a few years. Most computers will be equipped with webcam with face tracking technology as the basic standard device. The developments in mobile computing technologies, the university server will remind the students’ course schedule via SMS or MMS messages. Some course introductions might be delivered in video and audio via cell phones. The new communication means will blur the distinction between traditional and distance education. The technologies have potential uses in both situations with little noticeable differences. The main pedagogical issue is to understand where the new technology will have real impact on learning effectiveness. Some of the technologies will ease the constraints of time and distance by bringing the possibilities of face-toface tutoring to the students, in the home, on the street or anywhere, as long as the students have access to a Internet connected device. 106 Jason K.Y.Chan, Ken C.K.Law 2.2 Combination of Asynchronous and Synchronous Online Learning The advantage of asynchronous OLE is flexible learning especially for global distance education, as it does not require the learners or instructors to be online at the same time. However, learners may get lost in complex hypermedia information webs. Learners may find it difficult to orient to find their way to the learning topics. (Chan and Law, 2004) It is concluded, apart from adapting the asynchronous OLE, interactive tutoring systems will be introduced to provide synchronous interaction for more efficient moderating of learners’ discussion and immediate feedback from instructors. The asynchronous and synchronous online learning platforms do not exclude each other but provide complimentary ways of communications and interactions for different teaching and learning purposes. The range of tools available in an interactive tutoring system makes complex topics manageable. Complex topics can be explained directly by using tools such as whiteboards, application sharing, textchat, real-time audio, and video-conferencing. These synchronous tools will be combined with asynchronous tools such as video clips, text, images and animation. Instructors and learners will be able to illustrate their ideas in both directions interactively. 2.3 Learning Cycle in the OLE The major challenge for distance education is to address the issues of dialogue across the response and psychological distance between teachers and students. (Uys, 1999) Chizmar and Walbert (1999) argue that pedagogy must drive the choice of instructional technology. To provide successful teaching, teachers must be considerate and design a series of steps that make connections with the students’ prior understanding, actively process new information or practice skills, and set the stage for demonstration of the learning outcomes. Teachers can adjust these steps in the light of the ‘classroom’ experience as it unfolds. (Chan, Yim and Chen, 2005) And hence, the blended learning model can be more structured by having a four-stage Learning Cycle in the OLE platform as illustrated in Figure 1. The four stages are: Pre-learning; Lecturing; Tutoring; and Assessment. We can see the flow of user interfaces and how they are interrelated with the four learning stages to accomplish the learning cycle. Structured Blended Learning Implementation for an Open Learning Environment 107 Figure 1 Web-based Learning and Interactive Tutoring System with Learning Cycle. (Chan, 2003) 2.3.1 Pre-learning The activities in Pre-learning stage will capture the student's attention, stimulate their thinking and help them access prior knowledge. The teaching materials at prelearning stage can be asynchronous. Figure 2 illustrated Pre-lecturing Stage with a simulated screen shot on a topic. 108 Jason K.Y.Chan, Ken C.K.Law Figure 2 Example of Pre-learning Stage on a topic. 2.3.2 Lecturing Lecturing will be taken as a one-to-many stage. Students will be introduced to concepts and skills/abilities using familiar material. This is to make it easier for students to concentrate on the concept development. Students can input their queries and comments during the virtual lecture. Figure 3 illustrated Lecturing Stage with a simulated screen shot on a topic. Figure 3 Example of Lecturing Stage on a topic. Structured Blended Learning Implementation for an Open Learning Environment 109 2.3.3 Tutoring Tutoring stage will give students the opportunity to expand and solidify their understanding of the concept. Students will be involved in an analysis of their exploration. Their understanding will be clarified and modified because of reflection via synchronous interactive activities. In the tutoring stage, all participants connected by web-cams to their computers will show their video images to the system and all participants. Figure 4 illustrated Tutoring Stage with a simulated screen shot on a topic, noticed the video-conference feature (with web-cam devices) to enable discussions between tutor and students. Figure 4 Example of Tutoring Stage on a topic. 2.3.4 Assessment Finally, the learning cycle will close with an assessment stage that answers the questions: “Was the instruction effective?”, and “What is the next step for the learner?”. In this stage, the instructor will deliver a prepared assignment in multiplechoice mode to the students to complete in a given period of time. The results and statistics of the assignment or quiz will be sent to the participants immediately after the given period. Figure 5 illustrated Assessment Stage with a simulated screen shot on a topic. 110 Jason K.Y.Chan, Ken C.K.Law Figure 5 Example of Assessment Stage on a topic. 3. Meeting the Learning Needs It is important to point out that it is not necessary to apply the four-stage learning cycle in all the courses. (Chan, Yim and Chen, 2005) The above stimulated synchronous interactive online learning system should allow the instructor to use any stage and in different order in real practice. By introducing the concept of a learning cycle, the blended learning model will become more structured. The problem of becoming lost in hypermedia information web can then be solved. Knowledge can be delivered more effectively and efficiently. The system design may be more creative by providing the following features and arranging them in different learning stages: z allow teachers to schedule their tutorials in a given period of time; z provide video indexing function for teachers to correspond the lecture notes’ content with video playback; z allow teachers and students to interact in a virtual face-to-face environment; z allow teachers to display and make remarks on the files they uploaded; and z allow teachers to show special items such as 3-D objects through the connection to a visualizer. On the other hand, as asynchronous e-learning tools, such as WebCT and Blackboard were being used for several years, users may become familiar and comfortable with the existing course delivery mode and would resist the introduction of synchronous online learning platform. In certain cases, users may believe it will conflict with the value of flexible learning. However, as mentioned, the e-learning approach by the OUHK is a blended learning mode via printed materials, CD-Roms, lecture-less face-to-face tutorial sessions and OLE. In practice, instructors often give lectures in a one-to-many mode. Structured Blended Learning Implementation for an Open Learning Environment 111 However, in tutorial sessions, it is often that students need guidance from instructors in order to understand the teaching materials (in the printed text, CD-Roms or websites), one-to-one interactive mode would be more desirable. OUHK adopted a synchronous communication medium for the student to raise questions and obtain immediate feedback and interaction with tutor. As suggested by McLuhan (1989) in the work, quoted, “the medium is the message”. Moreover, there is a need for a more flexible and practical schedule of tutorial sessions since most of the OUHK students have full-time work during the day. It may be difficult to attend the course in the evening, as many of them regularly need to work overtime. With the synchronous online learning platform, the tutorial sessions can be arranged with higher priority, such as starting at late evening to enable students to cope with their overtime demand from employers. This is clearly more flexible for OUHK students and their work life style. 4. Conclusion By using a more structured blended learning platform with learning cycle, students can gain the more guidance in learning and the University can provide more flexible e-learning environment. However, “e-learning is not a cheap alternative to face-toface teaching” (Bates, 2002). But, for the OUHK, the combination of asynchronous and synchronous online learning platform will reduce the need to have classrooms around the city, (OUHK, 2005) with some cost saving in classroom rentals. This has direct impact on the tuition fee reduction, to attract more students and helps to increase enrolments. Although the “contact-hour course” is still labor-intensive (Graves, Henshaw, Oberlin and Parker, 1997), it is still acceptable to the OUHK which needs to provide mandatory face-to-face tutorial sessions. But the challenge is, as Bates describes, “to develop a system that encourages teaching units to be innovative and able to respond quickly to changes in subject matter, student needs, and technology. At the same time, redundancy and conflicting standards and policies across the institution must be avoided” (Bates, 2000). The University will need to provide additional resources for tutor training and briefing sessions to students for adoption of the new OLE, if the University is determined to have the full-scale phase in of using information technology in the learning environment. References Anderson, T. (2003) ‘Getting the Mix Right Again: An Updated and Theoretical Rationale for Interaction’, International Review of Research in Open and Distance Learning, 4(2). Retrieved February 10, 2005, from http://www.irrodl.org/content/v4.2/anderson.html Bates, A.W. (1993). Theory and practice in the use of technology in distance education, In Keegan, D. (Ed.). Theoretical principles of distance education. London: Routledge. Bates, A.W. (2000). Managing Technological Change: Strategies for College and University Leaders. San Francisco: Jossey Bass. Bates, A.W. (2002) National Strategies for e-learning in Post-Secondary Education and Training. Paris: UNESCO/International Institute for Educational Planning. 112 Jason K.Y.Chan, Ken C.K.Law Chan, J. K. Y. (2003). Web-based Learning and Interactive Tutoring System with Learning Cycle, Thesis of Master of Science in Computing, City University of Hong Kong. Chan, J. K. Y. (2005) Experience in developing interactive conferencing tool (voice chat), Communications Technology Forum, Island Learning Centre, OUHK, January 15, 2005. Chan, J. K. Y. and Law, K. (2004). A need for a fine tuned interactive online learning system with learning cycle, presented in the conference of the 21st ICDE World Conference on Open Learning and Distance Education, Hong Kong, February 18-22, 2004. Chan, J. K. Y., Yim, P. Y., and Chen, L. (2005). Web-based interactive learning system and learning cycle. China Educational Technology. 2005 (3). Chizmar, J. F., and Walbert, M. S. (1999). Web-Based Learning Environments Guided by Principles of Good Teaching Practice. The Journal of Economic Education, 30, 248-259. Chung, L (2003). Survey on OUHK students’ use of IT -- 2003, unpublished report, Hong Kong: The Open University of Hong Kong. City University of Hong Kong. (2002). Strategic Plan 2003 to 2008: Meeting the Challenge of Change. Retrieved March 20, 2007, from http://www.cityu.edu.hk/op/plan/plan_20032008_01.htm Crawford, C. (2000). Understanding Interactivity. Self published, 2000. Republished 2002, by No Starch Press as The Art of Interactive Design. Duderstadt, J. J., Atkins, D. E. and Houweling D. V. (2002). ‘Higher education faces a brave new world’. Higher Education in the Digital Age, pp.3-22. Graves, W., Henshaw, R., Oberlin, J. and Parker, A. (1997). ‘Infusing information technology into the academic process’. In Peterson, M., Dill, D., Mets, L.A. and Associates (eds.) Planning and Management for a Changing Environment. San Francisco: Jossey Bass, pp. 432-452. Gunn, C. (1998). Virtual technologies in higher education: vision or reality? In Peters, M. & Roberts, P. (Eds.). Virtual technologies and tertiary education. London: Routledge. pp. 134145. HKU. (2002). The University of Hong Kong Strategic Development 2003-2008, Retrieved March 20, 2007, from http://www.hku.hk/strategic-booklet/text/eng-html.htm McLuhan, M. and Fiore, Q. (1989). The Medium is the Message. New York : Touchstone. OUHK. (2002). Business Plan 2003–2006, Retrieved November 30, 2005, from https://staff.intranet.ouhk.edu.hk/items/po/bp/index.htm OUHK. (2005). The Open University of Hong Kong Staff Intranet. Retrieved February 13, 2005, from http://www.ouhk.edu.hk Raskin, J. (2000). The Humane Interface. New Directions for Designing Interactive Systems. Addison-Wesley. Vermeer, R. and Murphy, D. (2004) ‘What students want – students get?’. Paper presented at the 18th Annual Conference of AAOU, Shanghai, November 28-30, 2004. Uys, P. M. (1999). Towards the Virtual Class: Technology Issues from a Fractal Management Perspective. Proceedings of the ED-MEDIA 99-World Conference on Educational Multimedia, Hypermedia & Telecommunications. Seattle: AACE. Retrieved February 10, 2005, from http://www.globe-online.com/philip.uys/www.globeonline.com,philip.uys,edmedia1999.htm Uys, P. M. & Siverts, S. A. (2001). Managing Technological Transformation in Higher Education: A Southern African Perspective. Proceedings of the 22nd World International Council for Distance Education Conference. Dusseldorf, Germany. Vermeer, R. and Murphy, D. (2004). ‘What students want - students get?’. Paper presented at the 18th Annual Conference of AAOU, Shanghai, November 28-30, 2004. Wong, Wai-man. (1999) ‘Electronic Library Services for Distance Learners - Its Developments in the Open University of Hong Kong.’ In [Proceedings of the] ICDE Librarians' Roundtable, 11-12 October, 1999, The Open University of Hong Kong. Hong Kong: Open University of Hong Kong, 1999, 82-85. ERIC ED 438 832. Structured Blended Learning Implementation for an Open Learning Environment 113 Yuen, K. S. (2004). ‘Replacing postal mail by electronic mail for communication in distance education: how is this achieved at the OUHK?’, Paper presented at the 18th Annual Conference of AAOU, Shanghai, November 28-30, 2004. The Disruptive Effect of Technology: a University Case Study Norah Jones Centre for Excellence in Learning and Teaching (CELT), University of Glamorgan, United Kingdom, CF37 1DL. njones2@glam.ac.uk Abstract. For many, e-learning and blended learning are seen as a technical solution to improve teaching. It is commonly viewed as neutral - just another tool in the lecturer's kit bag. This we believe is a naïve view and hides the extent and complexity of change required at universities [8, 9]. Technology is not just another way of delivering course content. Blended learning is challenging our education practices and underlying epistemologies and theories. The design of blended learning needs to be grounded in sound education theory. We need to ensure that we blend technological and pedagogical advancements. If we design programmes on-line and ignore education theory then we are in danger of leaving learning to chance. Keywords: blended learning, higher education 1 Aim The aim of this paper is to explore the impact of embedding blended learning in a case study university in the Higher Education sector in the UK. The paper provides an organisational context, explores the definitional complexities associated with blended learning; and the way in which blended learning challenges the hierarchies of universities. 2 Introduction E-College Wales (ECW) was established in 2001 at the University of Glamorgan as a vehicle for the delivery of distance learning courses, from the University of Glamorgan Business School and six further education colleges. The courses that were developed for ECW were technologically enhanced versions of programmes that were available by traditional means on-campus and new courses developed only for online delivery. After three years of operation, ECW had recruited over 1,000 students onto programmes, a considerable success in terms of e-learning programmes. The project however disrupted many aspects of the university’s cultures. Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 114-122, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. The Disruptive Effect of Technology 115 When I agreed to manage this project I had assumed I would build on my experiences. I quickly learned that this was not as simple as it had seemed; every aspect of University life needed to be reconsidered and I found I was challenging existing hierarchies and traditions[9, 10]. It became clear at the start of the project that a range of staff, academic, technical, administrative and staff with new composite skills from different departments across the University and across the Welsh further education network needed to work together. Existing administrative structures were problematic and hindered interdisciplinary arrangements. There were very few examples of staff from so many different areas of the University working together in one group; the boundaries between academic departments and support departments were well protected by tradition and culture needed to change 3 The Blurring of Boundaries The development of an e-learning environment led to the creation of multidisciplinary teams, including staff from Academic Registry, Learning Resources Centre, Human Resources, Marketing, Student Information Systems, Information Systems, Student Services, the partner colleges and the academic schools. It is clear that the success of the project rested upon an integrated team involving all the University's support departments working alongside the Business School from the beginning. This is because in an e-learning environment the support is required at the start and is immediately transparent to the e-learner when they log on-line [18]. Nunan et al. [16] highlight the importance of integration, "Information technologies are bringing structural change to service areas, causing a convergence of roles and functions between registry, library, corporate services, production and teaching support and student services" (p.72). The blurring of traditional departmental boundaries has been particularly evident in the development of modules on-line and represents an important change within universities. This however is posing challenges, especially to heads of department who may see this blurring as a threat to their power. Sloman [19, p.14] emphasises, "Connectivity is especially powerful. Boundaries separating different organisations and activities have become increasingly irrelevant (or blurred)". The new paradigm has also created new relationships between further education partners and between further education and higher education. Further education staff, and colleges, are assuming new roles which will necessitate new agreements, funding models and approaches to quality assurance. E-learning requires much higher levels of quality assurance including issues of copyright; intellectual property rights need to be sorted out at the beginning of the process in the development of on-line courses [12]. These changes will also threaten the prevailing hierarchies and pre-conceptions about the status of further education and the collaborative models were re-examined [5]. 116 Norah Jones It is clear that the impact of e-learning, and in the future mobile learning, will require universities to re-think fundamentally their strategies, in a range of areas including human resources, estates, pedagogy, quality assurance, funding, management and commercial and educational partnerships. Inglis et al. [7, p.189] confirm our findings: “For most organizations, the transition to electronic delivery will represent a significant shift. It will involve major changes to the organization: changes in staffing, procedures, infrastructure, and most of all to the culture of the organization”. Information and communications technologies (ICT) are forcing major changes in the location, development, methods, delivery, support, evaluation and timing of education delivery. Although it provides real opportunities it also poses threats that need to be addressed. Grasping the full potential of ICT will require a substantial shift in human resources policies including recruitment, contracts, training and development and innovative payment systems. As far back as 1967, McLuhan [15] visioned 'the global village' and Hanna [6, p. 8] prophesises that "McLuhan's concept of the global village is about to come to life for every person on the planet". E-learning and the Internet are seen as the ideal medium to create a global village [7, 13, Evans and Nation 2000) and universities are expected to be at the centre of the move towards the 'global village'. [20, p.39] highlights this as follows: " ..some of the biggest changes for universities will stem from further advances in I.T… a capacity for interactive networking which will connect any university to a global audience". Communication technologies that are free from time or place constraints provide new challenges to universities on how they should be organised. It is clear that universities need to change to accommodate the impact of technology on learning. Very early in the E-College project it became apparent that e-delivery offered exciting opportunities for delivering to the ever more diverse backgrounds of students which the University recruits in response to the Government’s aim of extending participation to 50%. The challenge for the University would therefore be to mainstream the managed learning environment with all of its attendant questions of funding streams and HR issues. Unless we adopt more flexible forms of delivery we will lose students to more responsive universities including private universities and virtual universities. The economic argument, although compelling at this time, is not sufficient to incorporate ICT into universities. Lea and Nicoll [14, p.6] highlight "ICT is commonly promoted through governments and the media as mere 'technical improvements to the learning systems'…. However this view masks the extent and complexity of the changes taking place, of the requirements for substantial institutional change and of reconfigured practices and understandings of pedagogy". The value of e-learning needs to be shown on pedagogical grounds. Laurillard [13, p.241] argues that the delivery infrastructure should never be in the foreground; rather it should be supporting the dialogue on learning. At the end of the four year project, the University needed to make decisions about the future of e-learning. The next section of the paper describes the ways in which the University embedded technology into learning and teaching. The Disruptive Effect of Technology 4 117 Moving from e-learning to blended learning At the time when the ECW project was first initiated in 2001, elearning was at its height. Although the idea of blended learning was starting to emerge, Universities across the globe were focussing on offering courses completely online. The development of large scales elearning projects such as the UKeU, dominated the sector. Many large scale elearning projects failed, and one of the criticisms was that these projects pursued a narrow concept of e-learning and failed to realise that students required human contacts and support in their learning [11]. Thus the outcomes of the ECW Project led directly to the university establishing blended learning as a strategic goal and creating a blended learning unit to support the academic delivery of courses through Blended Learning. The Vice-Chancellor’s vision for enhancement of learning and teaching placed e-learning amongst the highest gaols of his strategy fro the University. Glamorgan is committed to the delivery of a first class learning environment incorporating the highest standard of elearning, tutor facilitation and use of cutting edge learning facilities… The focus of this activity was to move from distance approaches to online learning to the development of on-campus blended learning delivery. The University recognises the transformative effects of e-learning and is proceeding from a position of strength which has led to a major commitment to blended learning across all faculties. Although the work in E College Wales (ECW) has been at the forefront of e-learning developments, colleagues across all academic schools are now involved in blended learning innovations. 4.1 Definitional complexity There are definitional complexities and ambiguities surrounding such terms as elearning and blended learning. Blended learning as a term has gained considerable currency in both business training and educational contexts (Oliver and Trigwell, 2005). Despite its widespread use the way in which it is interpreted varies and requires further exploration and analysis [1]. From the outset, it is important to understand that the terms ‘e-learning’ and blended learning are used in many different and frequently confusing ways; definitions cover any activity from the simple use of email and PowerPoint presentations delivered on campus through to sophisticated multimedia simulations for use in stand alone study at any location in the world. It should be noted that blending different approaches to learning is not a new idea, in business schools traditionally a variety of pedagogic approaches have been used, for example lectures, seminars, tutorials, case studies, role play, residential week end course, adventure training and action learning groups. The difference here is that Information Technology and the development of Virtual Learning Environments (VLEs) are used to support the learning process. 118 Norah Jones 4.2 Continuum of Blended Learning At the University of Glamorgan we have adopted a continuum of blended learning in which the ‘blend’ is a spectrum that moves from basic ICT usage to intensive ICT usage: The University has opted for a blended learning approach, whereby each Faculty is expected to embed technology into the learning experience within their own frames of reference/context. This decision reflects the impact of external drivers such as the growing familiarity with technology by many of our potential clients, Government and employers’ initiatives, competition from other universities and HEIs as well as the University’s desire to enhance the learning experience of its students and providing accessible higher education. 5 Overcoming Barriers The ECW project identified a number of barriers which impacted on the development of e-learning in the case study university. The next section provides the main issues arising form the project; this includes technical, cultural and learning and teaching issues. These issues will be analysed next; this forms the main agenda for change. 5.1 Technical Issues As a result of ECW the Vice Chancellor of the University invested in a central esupport unit based in its central IT department and a blended learning team of educational experts were appointed to join the Learning and Teaching Central team (CELT). The e-support team (eST) comprises staff who have a breadth of professional and operational experience in e-learning and the use of information and communication technology (ICT). Based in the central information systems department and the learning resources centre, the team is proficient in developing and supporting the deployment of technology to enhance learning across a variety of areas. The eST is also responsible for the delivery of staff development activities and The Disruptive Effect of Technology 119 support, such as the Blackboard training sessions and a series of ‘How to…’ guidance documents. The eST team offers: Customer Support Services One-stop-shop for all ICT and e-learning support — customer focused with access to online solutions and other support teams. Experience includes customer service centre set up, proactive student support and provision of support and advice 24/7. Facilitation & Publishing Practical advice for utilising technology to enhance learning within pedagogically proven frameworks. Includes developing online content from both an instructional and publishing view point. Multimedia Development Experienced in providing a range of graphical, audio and animated e-learning solutions, technical knowledge of software tools, development capability for games, quizzes, interactive simulations and case-studies. Able to advise in the following areas: accessibility, technical issues, creating e-learning and multimedia, good practice guidance, layout and design. Systems Development and Support Develop maintain and support the Blackboard and bespoke virtual learning environment (VLE) systems. Provide advice on system functionality and development. Assess software capabilities and provide in-house solutions to development issues. The LRC eST offers: eResources Management The team’s main aim is to help staff integrate into their teaching — in the classroom or online — the most appropriate existing learning resources from the Learning Resource Centre’s collections and beyond, to create a resource-rich and easy to use learning environment for students. Advice is provided on the availability of learning resources in different formats, and on the options for linking to external resources from Blackboard. Guidance is offered on the copyright implications of using content and permissions can be obtained on your behalf if required. Media Production This team manages and supports the LRC’s media production facilities and has extensive experience of creating video, audio and photographic material. Production can be undertaken for staff, in the studio or on location. If staff prefer to develop their own material the team then provides equipment and advice on how to get the best results. 120 Norah Jones 5.2 Learning and teaching issues The Centre for Excellence in Learning and Teaching (CELT) comprises of blended learning pedagogical advisors, research and evaluation staff and staff involved in providing advice on teaching and learning. CELT is committed to ensuring that blended learning will be not be driven by technology but by the needs of the University, its staff and students. It is important to note that although face to face teaching will be the norm for most students it is anticipated that there will be a continued growth and development in the use of blended learning. During 2006/07 we were engaged in an HEA project to benchmark our blended learning activities. Part of this benchmarking involved a survey capturing the experiences of teaching staff in blended learning. The most common pedagogical use of technology by lecturers was to post lecture notes, this accounted for 79% of respondents. The next most common usage (67%) was to use the VLE (BlackBoard) to make announcements to students. Over half the sample (56%) used the VLE for testing and assessment. There was also a growing percentage of staff (45%) who used the discussion group features. One could arrive at a negative conclusion with the highest usage on information storage but this was well received by students, they wanted their lecture notes online and accessible outside of the lecture times. In addition they valued the opportunity for discussions to be held in face to face meetings. This however may reflect current students’ demands but not be typical for future students. What will students expect in the future? There are great changes being made in both primary and secondary schools with regard to Information Technology (IT) and in addition more and more young people are used to resource based information gathering. We need to be aware of changes in the pre-university sectors, professional bodies and in the work place with regard to IT. As blended learning becomes core rather than peripheral and optional, it is important that all staff are engaged in reflection on their curriculum development and the learning and teaching methods best used to deliver the curriculum. 5.3 Cultural Issues The blurring of traditional departmental boundaries continue to pose many challenges, especially to heads of department some of whom see this blurring as a threat to their power. Collaborating with staff who are outside the security of shared disciplinary assumptions expose staff to new and different ideas and approaches. The overall conclusion from Christensen’s analysis [2] is that it is extremely difficult for established organisations to adopt and embrace disruptive technologies. A small number of organisations manage it, but a much larger number fail in the attempt. “If e-learning is a disruptive technology, then the next two decades will require a dramatic restructuring of higher education. We would prefer to see a planned transition in which universities planned to learn how to implement e-learning, than to wait for universities to be put out of business by new organisations that have been quicker to understand what e-learning can be used for” [14]. There is no doubt that changes in the methods of teaching through the use of technology require a changed management structure to support the change. As Elton The Disruptive Effect of Technology 121 [3, p.219] stresses ‘new ways of learning require new forms of institutional management’. It is clear that the impact of e-learning, and in the future mobile learning, will require universities to re-think fundamentally their strategies, in a range of areas including human resources, estates, pedagogy, quality assurance, funding, management and commercial and educational partnerships. 6 Conclusion It is very important that the technology is not incorporated into programmes uncritically. Many students especially those aged between 18-25 will probably not want entirely e-learning courses or to study at a virtual university. The key features of university life for many young people are the social and recreational activities. In addition for any age group sustaining motivation in a virtual environment is problematic. As I have indicated earlier there are a variety of e-learning models and including in this is blended learning which offers one solution so that on-line learning enhances the best of face to face provision. Nonetheless the advantages of blended learning are clear; we now have immediate access to information sources from around the world, we can more easily collaborate with others globally and we can keep in touch with our friends and family. ICT has changed so many aspects of our lives and as the networks continue to grow so will our dependence on them grow. Already we have seen how ICT allows us to create simulations and allows for powerful visualisation and software offers new forms of interactivity through computer mediated conference facilities. Rogers [17, p.358] emphasises, "When each student has his or her own portal to the world's knowledge, the role of the teacher as giver of information will forever change. It is not a matter of if but when this transformation will occur". Like Rogers I believe this transformation is imminent and lecturers need to be prepared and developed to accommodate changes to their roles. The challenge for the university is to use blended learning and in particular our experiences from ECW as an opportunity to re-examine pedagogies and focus on the needs of the learners. The needs of learners have changed due to financial pressures an increasing number of students are now working and studying full time; students are pressing for more flexible delivery to accommodate these demands. In addition the Government’s agendas on life-long learning, widening access and increasing participation rates to 50% has increased the need for universities to respond more flexibly to the requirements of a more diverse group. Blended learning allows students to use interactive materials flexibly at any time. Advances in technology are converging with the requirement for universities to be more flexible. As with all IT driven change, e learning has developed more slowly and less smoothly than its advocates predicted, but with far greater consequences, ultimately, than most 122 Norah Jones assume. E-learning is challenging our education practices. The design of blended learning needs to be grounded in sound education theory. We need to ensure that we blend technological and pedagogical advancements. If we design programmes on-line and ignore education theory then we are in danger of leaving learning to chance. Lewin is purported to have said that nothing was as practical than a good theory, I would add that there is nothing worse than a theory misapplied! We need to get the model right, become more adventurous in terms of timing, structure, assessment etc. There is no doubt that unless we adopt more flexible forms of delivery we will lose students to more responsive universities including private universities and virtual universities. It is clear that technology disrupts but we must learn from each other the important lessons. We cannot build knowledge without sharing information and good practices. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. Bonk, C. and Graham, C. (eds.) The Handbook of Blended Learning, Pfeiffer San Francisco (2006) Christensen, C.M. The Innovator’s Dilemma, Harper Collins New York (2000) Elton, L. New ways of learning in higher education: managing the change, Tertiary Education and Management, Vol. 5 (1999) 207-225 Connolly, M., Jones, N. and Turner, D. E-learning: a fresh look, Journal of Higher Education Management and Policy, Vol. 18 (3) (2006) 135 -147 Connolly, M., Jones, C. and Jones, N. Managing Collaboration Across Further and Higher Education: a case in practice, Journal of Further and Higher Education, forthcoming 2007 Hanna, D. and Associates, Higher Education in an Era of Digital Competition, Atwood Publishing, Madison (2000) Inglis, A., Ling, P. and Joosten, V. Delivering Digitally, Kogan Page London (2002) Jones, N. From Here to E-ternity. Professorial Inaugural Lecture, University of Glamorgan (2004) Jones, N. and O’Shea, J. Challenging Hierarchies”, Higher Education Vol. 48 (3) 379395 (2004) Jones, N. and O'Shea, J. A Case Study in Managing Change: introducing e-learning into a university, Management, Vol. 29 (8) 29-35 (2003) Jones, N. and Peachey, P. The Development of Socialization in an On-line Learning Environment, Journal of Interactive Online Learning, Vol. 3 Winter (2005) Jones, N., Morgan, A. & Turner, D. The E-College and Quality Assurance: the irresistible meets the immovable, Quality Assurance in Education", Vol. 10 (4) (2002) 229-237 Laurillard, D. Rethinking University Teaching, Routledge London (2002) Lea, M. & Nicoll, K. (eds) Distributed Learning: Social Cultural Approaches to Practice, Open University / Routledge Falmer London (2002) McLuhan, M. The Medium is the Message, Penguin Harmondsworth (1967) Nunan, T. Ricmor, G. and McCausland, H. 'E-learning Futures' in Rich, T. (2001) Perspectives, Policy and Practice in Higher Education, Vol. 5 (3) (2000) 68-77 Rogers, C. Freedom to Learn, Prentice Hall New Jersey ( 1994) Sloman, M. The E-Learning Revolution, CIPD London (2001) Wilson, A. Strategy for Management Development in Scott, P. (eds,) Higher Education Reformed, Falmer London (2000) 29-44 The Marriage of Rousseau and Blended Learning: An Investigation of 3 Higher Educational Institutions’ Praxis Esyin Chew1, Norah Jones1, David Turner2 2 1 Centre for Excellence in Learning and Teaching (CELT), Education Department at Faculty of Humanities and Social Science, University of Glamorgan, United Kingdom, CF37 1DL. {echew, njones2, dturner}@glam.ac.uk Abstract. This paper sets out the central problem of current blended learning research that it does not have an appropriate focus on educational theory. The blended learning praxis in higher education can be understood in terms of Rousseau’s educational theory is explained. The research methods for collecting qualitative data from 28 academics in 3 universities are explained concisely. The analysis and discussion of institutional practices are used to identify best practice for blended learning. The role of the educator and student on holistic learning are examined. Overall, the author urges that the focal point of blended learning research should not merely on the innovative technology and instructional design issues; educational theories remain the fundamental foundation for any educationalist as well as for any educational technologist. Keywords: Blended learning, educational theory, educational technology, higher education. 1 Introduction “…blended leaning could become one of the most significant developments of the 21st century.” [27, p.26] Blended Learning is a phrase which is increasingly being used in higher education. However, Whitelock [29] claims that blended learning has not gained ground with theorists but is embraced by practitioners. Practitioners of technology and education show more interest in blended learning than educational theorists do. Boyle [6] further explains that the design and development of blended learning solutions should be pedagogically driven. Blended learning represents a real opportunity to create learning experiences that can compensate for shortcomings in face-to-face delivery of programmes or in technology-mediated learning. There has been an increasing number studies on blended learning, which have examined such aspects as its effectiveness and its impact learning in modern higher educational [22, 16, 21, 24] and how it promotes the innovative dialogues and practices in the learning environment [26, 5]. The increasing interest shown by researchers in these topics has gone hand in hand with the Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 123-135, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 124 Esyin Chew, Norah Jones and David Turner development of technology, especially the emergence of the internet, rich media objects [26], mobile technology [28] and artificial intelligence [30, 7]. Loveless [18] asserts that technological development is a cultural artifact in the experience of students and educators. Technology is a tool of cultural artifact to educator or student who applies it. It is varying from one individual to another and one discipline to another. The usage and effectiveness of the technology is merely cultural variety. Dewey [11] argues that if the learner is not trained in the right use of the tools, there is grave danger that he may deprave himself and injure others. Croft [9] explains Dewey’s idea in the following statement: “…individuals inappropriately acquainted with the technologies in contemporary use risk deterioration of self and damage society itself…” (p. 302) Otte [21] further emphasises that the tool, technology, should be thought of merely as a means, and that pedagogical ends should be paramount. Matthews [19] suggests an interesting metaphor, namely that an educator without the pedagogy end is like a sailor without a rudder - blown around by whatever fashions and technology which dominate the current educational direction. In the blended learning context, the learning process may be insignificant if the technology adopted does not perform consistently or if it is not coherent with the educational considerations. The main focus of work on blended learning is the convenience of educators and students, together with certain considerations related to pedagogical issues. However, this results in two critical questions: Are such pedagogical considerations underpinned by adequate educational theories or are they merely common sense but labeled as “learning theories”? Does such theory speak to the needs of educators or students from different disciplines? Hence, there is a need to explore educational theory and its relationship with blended learning. The educational philosopher, Jean-Jacques Rousseau was chosen in this research because he is the earliest and influential educationalist who scattered the traditional educational ideas as to the replacement of tabula rasa or banking concept by investigation and reason [10]. Blended learning practice in three higher educational institutions is studied in this research. The paper explains the research methods used to obtain the qualitative data. The principles of blended learning form the perspectives of both practitioners and theorists are analysed and discussed. 2 The Marriage of Rousseau and Blended Learning The systematic integration of technology into education may only take place when the technologist is able to understand educational theories and embed them to meet the needs of educators and students [4]. Technologies and effective content development methods are the major concern in most of the blended learning practices, at the expense of the educational theories underneath. How blended learning practice is driven by educational theories is rarely considered. In general, current blended learning practices in higher education underline Dewey’s [12, 25] and Vygotsky’s [8, 14] conceptions of social interaction, social The Marriage of Rousseau and Blended Learning 125 constructivism and a guided learning environment. However, this paper uses the theoretical framework taken from Rousseau, who claimed that the goal of education should be to cultivate humanity’s natural tendencies [23, 13]. This overarching philosophy can be linked to the development of holistic blended learning practice. Rousseau's central idea is to reject the authority of the teacher and emphasis on individual reasoning and reflection. The student must be developed in the consciousness of complete freedom. The student must committed to the reasoning willingly. The learning process is not instilling by an authoritarian teacher but developing according to the dictates of human nature, not to the authoritative instructions from educator. Rousseau also contended that the three great teachers of man were nature, man, and experience, and that the second and third tended to destroy the value of the first [10]. The challenge for the educator is to create such a free and natural learning environment, but with certain constraints, without learner’s recognising the constraint. The constraint is meant to protect the learner from disruption from “man and experience”. This is the challenge for the educators. This perception should be inevitably a major consideration in blended learning design and development. Student-led instruction using educational technologies can be used to create an open and free discussion space within a context. The collaborative tools such as online discussion boards and wikis allow participants to commit to open reasoning, open investigation and reflection on knowledge. The educator or the editorial board plays the role of facilitator to create the constraint, for instance to delete disruptive information or to suggest guidelines to the learner in such an e-environment in a non-authority manner. The learner is the decision maker and he or she can choose which information to agree with or to take issue with. Rousseau's educational idea is to educate people to bring out their natural goodness, self-esteem, independence, compassion and equality [15]. This is difficult aim and cannot be realised in the face-to-face classroom. The process of education is thoroughly developmental and it shall follow the growth of human ‘nature’. Blogs and Wikis allow the learner to practise social interactions as well as to express their candid views and reflections. Through the reasoning and investigation process, learning is cultivated in a natural way. The learner will thus be fully developed not only in relation to their knowledge but also in relation to their ethical development because the education process is by internal nature and not by external forces. Whatever is posted on the web would always be there and open for review and criticism. The learner will be conscious of her/himself as an independent individual, responsible for her/his own thoughts and actions, completely independent from others, yet able to interact with others and direct her/his life by reason. Before the time of Rousseau educationalists had opined that the education is a kind of top (educator) - down (learner) instilling concept. Rousseau enlightens the later educationists to “autonomy as an educational environment”. Hence, the aim of this paper is to study Rousseau’s pedagogical practice, and to consider how it might be developed in the context of blended learning. Based on this view of pedagogical practice, it will go on to investigate three current higher education institutions which are embedding blended learning into their teaching, both formally and informally. 126 Esyin Chew, Norah Jones and David Turner 3 Research Method Anderson [2] defines education as a process and therefore a research method which is flexible and process-oriented is needed. According to Adelman [1], case study methodology can be used to identify and evaluate the flexibility of reality especially in the variety and complexity of educational purposes and environments. With these boundaries, case study method is an appropriate choice for the study of blended learning research. The present study incorporated case studies with qualitativequantitative interactive continuum methodology [20] because such a method integrates the strengths of both qualitative and quantitative strategies. First, the arguments of Rousseau are studied. In order to study the practice in higher education institutions, three universities were visited and observed. 28 academic staffs and 6 students from varying disciplines were observed and interviewed. Interviews lasted between 40 minutes to 2.5 hours. Qualitative as well as quantitative data has been collected from their teaching and learning experiences. The names of the interviewees as well as the institutions are anonymous due to considerations of confidentiality and ethics. The principal criterion in the selection of exemplary higher educational institutions was not “which HEI represent the totality?”, but rather, “which group of HEIs can provide a better understanding of the research questions?” and “which group of HEIs reflect strong, positive and constructive examples of the research interest?” Given these criterions, a diverse group of HEIs and faculties were needed. For instance the traditional old universities and the new universities upgraded from polytechnics, and the contrasting nature of disciplines related to technology such as a Faculty of Computer Science and Faculty of Education; or the Faculty of Information and Communication Technology and the Faculty of Humanities and Social Sciences are proposed to meet the criteria stated above. To maximize the findings in a case study, a range of formal and informal data collection instruments were used. These included recorded face-to-face interviews and site visits to the case study institutions, in addition to direct observation during the site visits. The responses have been analysed and discussed in the following section. 4 Discussion and Analysis 28 academic staffs were formally interviewed and their disciplines are summarized in the table 1. The major technologies and e-learning system used in these three universities were investigated and observed. Overall, the effort on e-learning development is mainly initiated at institutional level. However, awareness o, and commitment to, blended learning varies from one institution to another. The discussion on this section focuses on cross-institution comparison based on Rousseau's educational ideas. The data were used to identify and analyse institutional policies and pedagogical issues in three main areas: (1) Blended learning awareness and best practices which exemplify their institutional policies, together with actual implementation; (2) The role of lecturer after embedding blended learning (3) The value of blended learning in promoting The Marriage of Rousseau and Blended Learning 127 holistic learning. Table 2 illustrates the overview and the preliminary observations from the four months of data collection. Table 1. The Disciplines of the Interviewees University Category University A / Disciplines (Old university with 102 years of history) Science, Engineering, 6 Computing and IT Art, Language, 4 Education, Business, Humanities and Social Sciences Total 10 University B (New university with 6 years of history) 8 University C (New university with 14 years of history) 3 17 3 4 11 11 7 28 Table 2. Overview of the Preliminary Data Collection University A University B Less awareness on Awareness on eBlended blended learning as learning space than Learning well as e-learning. blended learning. Awareness Blended Learning Practices Not an institutionalwide commitment. The blended learning practices are up to the faculty and individual academic interest. VLE Implemented Across Institution Elearning, an inhouse built web-based learning management system but not well publicized to the academics and students. Many lecturers and students are not aware of the existence of such system. Technology or Pedagogy? Emphasis on the traditional instructional methods. Made a commitment to web-based mediated learning. Aimed to promote the use of ICT and ITintensive learning through innovative courses and learning support systems. Support the web-based leaning space intensively with workshop and training. WEBLE, an in-house built virtual learning space and courseware for the academics and students. Known by most of the lecturers and students. Emphasis on technological concerns. the Total University C Awareness of blended learning across the university. Made a commitment to the adoption of Blended Learning across the institution. A three-year project to embed Blended Learning across the University’s provision with high publicity. A clear model, named continuum of elearning is used. Blackboard, Questionmark Perception, in-house built rich media or web-based applications based on bidding process initiated by the academics. Known by most of the lecturers and students. Emphasis on both technological and pedagogical concerns. 128 Esyin Chew, Norah Jones and David Turner 4.1 Blended Learning Awareness and Best Practices Figures 1 and 2 depict the awareness of blended learning in these three universities. From the interview, 63% of the interviewees were aware of the concept and definition of blended learning, although one interviewee out of the 63% has a misconception about the definition of blended learning: Interviewee B3: In my opinion, blended learning is same as hybrid learning, I think they are similar. N o t Aw a re o f B le n d e d L e a rn in g , 1 3 , 37% Aw a re o f B le n d e d L e a rn in g , 2 2 , 63% Fig. 1. The Awareness of Blended Learning 15 16 13 14 12 9 10 No Of Interviewee 8 6 4 2 7 6 Yes No 4 2 0 0 Univers ity Univers ity Univers ity A B C Total Fig. 2. The Awareness of Blended Learning by Institution A large numbers of the academics in University B have never heard of the term “Blended Learning” as their institutional policy aimed to promote the use of ICT and IT-intensive learning. Less concern was focused on the pedagogical issues, as the institutional ethos was shaped by the fact that the institution has a background as a vocational college, and it only became a university few years ago. Most of the academics who understand and agree on the benefits of teaching mediated by technologies were from University A and C. University C has obviously made a commitment to the adoption of blended learning across the institution’s provision, driven by a three-year project with high publicity such as a well-design blended learning website with exemplars of case studies, monthly seminars and a road show. It has a clear institutional policy to embed blended learning across the campus compared with the other two universities. Such practice is inspired by a continuum of The Marriage of Rousseau and Blended Learning 129 e-learning model shown in the figure 3. This blended learning model shows the concise blend of an individual as well as institutional growing in blended learning. The self evaluation state and the direction ahead are well-defined for anyone who adopts this model. Fig. 3. Continuum of E-Learning [17] The clear model embedding across the University C has raised the awareness of the academics as well as students on blended learning. Such awareness is the initial and substantive move for the changes. Few of the positive responses from the academics after using technology to complement the face-to-face classroom are clearly presented in the following: Interviewee C3: I have used Blackboard as support material and engaging dialogue with students. I found that very helpful. Interviewee C2: I am quite excited about the prospect to be able to use Questionmark Perception…I would like to be able to use something like that to give students formative feedback and summative feedback as well…I am quite interested in the technology that can be interactive. Interviewee C6: Yes, I started to use Blackboard more this year compared with last year...at the moment it is easier for me to go through the lecture, it's all there...If there is something that I forgot, they are going to be there, I know all the information is there. However, sufficient resources such as technical and IT staff as well as educationalist, financial support from the management are the key factors for embedding blended learning across the institution after awareness rising. These are the major concerns raised from the academics: Interviewee C3: I think you got to sort out this resource…there must be some sense of resources available; the university is either bite the bullets and pay for it, or forget all about this. 130 Esyin Chew, Norah Jones and David Turner Interview B8: …depending on the implementation, depending on the management wanted the lecturer to be as a designer or executor. It would be a good thing if he is a designer and back-up by a group of technical team…For example the lecturer is the director of a movie and back up by a team of people, this is excellent. If he is the director and the cameraman, and also in charged of the lighting yet worrying about the sound effect, he will kill himself at the end. The idea of “director and support team” has precisely explained the successful mechanism in blended learning. Interestingly University A has no clear institutionalwide policy to embed blended learning however their awareness came from individual research interest, the culture and facilities at the faculty level, and above all, from the individual passion for enhancing the learning and teaching quality: Interviewee A2: …when you get your hands on the digital one such as computer, I think you can't go back anymore. I mean you just have to use it…it is a so effective! Most of the interviewees agree with the teaching and learning will be enhanced when they are mediated by technology. However it emphatically plays as a supplementary tool to compliment face-to-face rather than replacing it, technology can never replace the face-to-face instruction. This point is precisely stated by the following interviewees: Interviewee B2: To me, there is always the main stream and supplementary in education. And the main stream will never change, from the very beginning till the end, the things which are changed are the supplementary and tools. Regardless the technologies, or chalks or paper, the main stream would never change. On the other hand, these two elements can be also seen as a symbiosis or conflating for better teaching and learning experience: Interviewee A3: Blended Learning make use technology and also the humanity values, face-to-face… the technology alone is not enough, with the human alone is still also have certain constraint, as we are now in a technological world. So we need to combine both. University B aimed to offer the learning experiences that instill in students a consciousness of their role in a rapidly changing and technological oriented world with a strong sense of professional and social responsibility. This aligned with Rousseau’s view on education is how to prepare a better individuals to construct a better society, not how to teach or learn effectively. However, such ideal is too theoretical and yet to be imparted to the academics and students practices. The current implementation of blended learning across the university is still in the “unawareness” and technological-focus stage. The Marriage of Rousseau and Blended Learning 131 4.2 The Role of the Lecturer after Embedding Blended Learning One of the research questions is to obtain the interviewees’ opinion on whether the role of lecturer will be or will not be changed by embedding educational technologies into the teaching and learning practices. The below table describes the quantitative findings: D epending , 2, 5% N ot C hanged, 13, 33% C hanged , 25, 62% Fig. 4. Will the Role of the Lecturer be Changed after Embedding Blended Learning? 25 academics opine that their role would be changed after embedding blended learning, whereas 13 academics assert an opposite view. This is a debating issue from the pedagogical perspective. There is an essential view offered from academic discussed next: Interviewee A5: With the advancement of the Internet, the information rich society, nobody claim that we know everything. We have students who are hardworking, they know more than us. The lecturers are no longer dictate, we merely facilitate. The role of the lecturer will be changed from merely knowledge deliverer to knowledge facilitator. In facts, Rousseau against this kind of “top-down” knowledge instilling process in education centuries ago and it should not have happened today. The academics who argue that the role will not be changed fall into the group of senior and experienced educator. They view technology-driven teaching as the different form of instruction method without changing the main role as an educator: Interviewee B2: ... Before the time of white board and marker, people were using black board and chalk...You will use Power Point after such technology had been invented. In the future, you may use other new things when they are invented. However, these technologies are helping us to conduct our classes in a better way.... who is the one who organise and conduct the class? The lecturer is.. The argument here is: what are the fundamental role(s) of a lecturer? According to Rousseau, the role of the educator is to create an autonomy and nature learning environment, not by any techniques to instill the knowledge to learner. The learner learns from self reflection and reasoning, not to the authoritative instructions or “knowledge transfer” from educator. Hence, 132 Esyin Chew, Norah Jones and David Turner “It's depending the role in the past. If your role is the "baby-sitter" to the students then you would change. But if your role isn't spoon feeding at the beginning then it would not be changed. To me, my role never change in the past ten plus years. I don't care there is Internet or no”. ~ Interviewee A8. Relatively an experienced e-moderator concludes this with an insight view: Interviewee C7: When we first started delivering e-moderating course, we have a very firmly view, that if you are a good tutor offline and you would be a good tutor online…you are a good tutor means you are a good tutor whatever the medium is. This view shows that good teaching is driven by pedagogy and it is not platform dependant. The technology as a means, and that pedagogical ends should be paramount. 4.3 The values of Blended Learning in Promoting Holistic Learning N ot S ure , 4, 14% N o, 9, 32% Y es , 15, 54% Fig. 5. Blended Learning help in Holistic Learning? Blended learning does help in evaluating soft skills and promoting holistic learning in the sense of not only assess then task accomplishment but the hidden and high order thinking and communication skill. The following academics’ explicate such perspective: Interviewee C7: if I monitoring their progress using some kind of online discussion forum or blog for their experiences, I can see who is participating and who is giving leadership and in which the leadership is being challenged and where the conflict is…so I think they are very powerful tools…not about the task but about the individual and powerful learning tools. This experience actually aligned with Rousseau’s cultivation of human kind natural tendencies. It can be realised and enriched educational technologies such as blog and wiki in order to stimulate the natural tendencies such as social interaction, knowledge The Marriage of Rousseau and Blended Learning 133 sharing, free and open discussion and etc. It is merely difficult to be realised in the face-to-face classroom with space and time constraint: Interviewee A7: If the number of students is not very big. If you have a very big class you will not have the time to read through whatever the students' post. Rousseau’s philosophy of education, therefore, is not related simply at particular techniques that best ensure that the students in a big class will absorb information and concepts. His goal was to produce an education that maximized individual human potential rather than restricted it. From such basis, the educator shall make sure of the current technologies such as blog and wiki to create an open and independent learning environment to reflect and to develop the human potential such as critical thinking and to throw out their ideas. Through the dialogues and ideas exchange, the relationship between educator and learner are brought closer: Interviewee A3: To if you have the wiki or blog, they become more interactive in the sense, more creative and encourage students for throwing out their ideas… Interviewee B8: I join some of the students’ blog from Journalism study. …Sometimes I do give some comments and I feel my relationship with them is much closer. I think this is helpful as I know their life and what happening around them, as well as their views at political issues…This blog is unique as they really stimulate the critical thinking in life. Time constraint indeed is above all the critical considerations for the lecturer: Interviewee B5: I am not sure how far such technologies can help but one thing I can sure is with such things the students would ‘troubling’ me more than the past. Without the understanding of the educational theory and individual passion in education, such obstacles would lead to frustration and the most, drop out from blended learning. 5 Conclusion The new technologies have the potential to be both helpful and harmful, as do the educational systems. Educators and educational technologist cannot merely geared by all sorts of e-learning technologies but left behind the educational theory. Instead, quick responses and reasonably understanding of educational theories are required [3]. The heart of the blended learning praxis lays the need for a deep understanding of the trends and educational theories, only through that understanding can emerge initial interest and passion. Rousseau’s educational theory was chosen for this research. His central idea is to reject the authority of the teacher and emphasis on individual reasoning and reflection. 134 Esyin Chew, Norah Jones and David Turner The student must committed to the reasoning willingly and in an autonomous environment. The learning process is not instilling by an authoritarian teacher but developing according to the dictates of human nature, not to the authoritative instructions from educator. Thus, this research investigated the current praxis of blended learning in higher education based on Rousseau’s idea. The qualitative data is anlaysed and compared. The result reflected a clear institutional policy will promote the awareness across the university, follow up individual interest and passion on teaching and learning. Resources and support team in place is the critical successful factor. The role of the lecturer will be enhanced if the educator does not realise the values of “creating an autonomy and nature learning environment, not by any techniques to instill the knowledge to learner” initially. In addition to this, blended learning does help in holistic learning with bringing closer the relationship between educator and learner with autonomous communication, and to stimulate the natural tendencies without space and fixed-time constraint. In contrary, language barriers and the abuse usage may cause frustration and above all, time constraint is the major challenges for implementing blended learning. As a conclusion, it is asserted that the educational theories remain the fundamental foundation for any educationalist as well as educational technologist. With such understanding and passion would triumph over the challenges mentioned above. Blended learning researcher, therefore, is urged to meekly shift the focal point from innovative technology and instructional design issues, to educational foundation, as it is. References 1. Adelman, C., Kemimis, S., Jenkins, D. Rethinking Case Study: Notes from the Second Cambridge Conference. IN H., SIMONS (Ed) Towards a Science of The Singular. Centre for Applied Research in Education, University of East Anglia, (1980) 45-61 2. Anderson, G. Fundamentals of educational Research (2nd Edition). RoutlegeFalmer London (2004) 3. Aspy, D., N., Aspy, C., B. Toward Effective Advocacy for Humanistic Values, Vol. 37(2), Journal of Humanistic Education and Development (1998) 85-95 4. Bailey, G., Pownell, D. Tecnology Staff-Development and Support Programs: Applying Abraham Maslow’s Hierarchy of Needs, Learning and Leading with Technology, Vol. 26(3) (2006) 48-64 5. Barker, T. Computer Adaptive Assessment and Its Use in the Development of a Student Model for Blended Learning, Blended Learning – Promoting Dialogue in Innovation and Practice, University of Hertfordshire, First Annual Blended Learning Conference. Learning in Higher Education and Workplace Learning Settings’ Hatfield (2006) 6. Boyle, T. A Dynamic, Systematic Method for Developing Blended Learning, Education,Communication and Information, Vol. 5(3) (2005) 221-232 7. Chew, E., Jones, N, Generic Model of Computation for Intelligent Computer-aided Assessment Progress (iCAP), University of Loughborough: Tenth International Computer Assisted Assessment (CAA) Conference, 4-5th July, Loughborough (2006) 107-124 8. Cortazzi, M., Hall, B. Vygotsky and learning. Education Libraries Journal, Vol. 42 (1999) 17-21 9. Croft, R. S. What Is a Computer in the Classroom? A Deweyan Philosophy, Journal of Educational Technology Systems, Vol. 22 (1994) 301 10. Cubberley, E. P. The History Of Education, Blackmask Online (2003) The Marriage of Rousseau and Blended Learning 135 11. Dewey, J. Democracy and Education: An Introduction to the Philosophy of Education, The Macmillan Company New York (1922) 12. Ehrlich, T. Reinventing John Dewey's "Pedagogy as a University Discipline". The Elementary School Journal, ProQuest Psychology Journals, Vol. 98 (1998) 489 13. James, J. D. Jean-Jacques Rousseau In: James, F. & Bradley, D. (ed.), The Internet Encyclopaedia of Philosophy (2006) URL: http://www.iep.utm.edu/ [Accessed: 3.3.2007] 14. Jennifer, A., J. S., Monfries, M. Motivation to Learn in University Students: Links with Vygotsky's Assisted Discovery. European Conference on Educational Research. Bath (1995) 15. Johnston, I. Introduction to the Eighteenth Century and Rousseau's Emile. Malaspina University-College, Media Relations & Publications Department. (1996) 16. Johnson, J., Tang, M. Integration of e-Management, e-Development and e-Learning Technologies for Blended Course delivery, AACE Journal, Vol. 13(2) (2005) 185-199 17. Jones, N. Chapter 13:E-College Wales, A Case Study of Blended Learning. In: C. J. Bonk & C. R. Graham (eds.). Handbook of blended learning: Global Perspectives, local designs. Pfeiffer Publishing CA (2006) 18. Loveless, A., M. Where do You Stand to Get a Good View of Pedagogy?, Journal of Technology and Teacher Education, Vol. 8(4) (2006) 337-349 19. Matthews, M. Knowledge, Action and Power. IN MACKIE, R. (Ed.) Literacy and Revolution: the Pedagogy of Paulo Freire. Pluto Press London (1980) 20. Newman, I., Benz, C., R. Qualitative-Quantitative Research Research Methodology: Exploring the Interactive Continuum, Southern Illinois University Press IL (1998) 21. Otte, G. Using Blended Learning to Drive Faculty Development (And Vice Versa), SloanC Series, Vol. 6 (2005) 71-84 22. Ross, B., Gage, K. Global Perspective on Blended Learning: Insight from WebCT and Our Customers in higher Education. In: C. J. Bonk & C. R. Graham (eds.). Handbook of blended learning: Global Perspectives, local designs. Pfeiffer Publishing CA (2006) 23. Rousseau, J.-J. Emile, The Project Gutenberg EBook (2004) 24. Salmon, G., Lawless. Chapter 28: Management Education for the Twenty-First Century, In: C. J. Bonk & C. R. Graham (eds.). Handbook of blended learning: Global Perspectives, local designs. Pfeiffer Publishing CA (2006) 25. Simpson, D. J. John Dewey's Concept of the Student. Canadian Journal of Education, Vol. 26 (2001) 26. Smith, C. A Rich Media Rapid Prototype Approach to Developing Student Online Resources, Blended Learning – Promoting Dialogue in Innovation and Practice, University of Hertfordshire, First Annual Blended Learning Conference. Hatfield (2006) 27. Thorne, K. Blended Learning: How to Integrate Online & Traditional Learning, Kogan Page London (2003) 18 28. Wagner, e., d. Chapter 4: On Designing Interaction Experiences for The Next Generation of Blended Learning. In: C. J. Bonk & C. R. Graham (eds.). Handbook of blended learning: Global Perspectives, local designs. Pfeiffer Publishing CA (2006) 29. Whitelock, D. Blended Learning: Forget the Name But What About The Claims? In: Whitelock, D., & Mason, R. (eds), Bledned Learning. Special Issue of Education, Communication and Information (2004) 30. Zhang, T., Z., Fu, Y., G., Shen, R., M. Improve Question & Answer System By Applying Genetic Algorithm, Proceedings of the 3rd International IEEE Conference on Madune Learning and Cybernetics, Shanghai (2004) 2317-2321 Developing an On-line Medical Curriculum Management Platform Jenny Fang, Francis Wong, Raymond Chu Medical Information Technology, Faculty of Medicine, The Chinese University of Hong Kong, HKSAR {jennyfang, franciswong, raymondchu}@mit.cuhk.edu.hk Abstract. In the context of medical education, the current manual system of managing and updating teaching activities is not only time-consuming but also inefficient and inconvenient. In this paper, we present an account of the development of a web-based Medical Curriculum Management Platform (MCMP) that provides a flexible and comprehensive e-management environment designed to assist in the management and facilitation of the learning process from an administrative perspective. The system solves most, if not all, problems related to the administration and integration of a web-based multi-dimensional curriculum map. It also provides functions that cannot be achieved manually, such as a bird’s eye view of the entire Faculty curriculum map and students’ group/rotation roster. This overview can be accessed using a simple tap, update and distribute approach, whereby various kinds of information can be processed and posted with a minimum of effort. Keywords: medical education, administration, e-management, curriculum map. 1 Introduction This paper reports on the development of a web-based medical education management environment that embraces, from an administrative perspective, an innovative approach to the creation and management of complex organizational data and the retrieval and use of this data in a simple way. Medical education provides a particularly unique learning environment because of its multi-faceted nature. In comparison with other disciplines, a medical curriculum is extremely complex, with teaching and training activities scheduled in multiple locations arranged across the various academic departments, hospitals and wards. For selected groups of students, various teaching activities need to be arranged on the same day but in different locations and time-slots. Since many members of the medical teaching staff are also practicing doctors in the teaching hospitals, it is quite common that changes to course schedules have to take place whenever urgent medical cases arise. This necessitates regular updating of teaching schedules. Similarly, student group arrangements need to be updated or changed according to the different modules (or time periods) and teaching programs the students are undertaking. Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 136-149, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. Developing an On-line Medical Curriculum Management Platform 137 The profiles of those involved in a medical education programme are many and varied, and include not only the students and lecturers (who can be practicing doctors, specialists, nurses and so on), but also university administrators and hospital staff. The complexity of a curriculum programme involving such a diverse range of stakeholders and constantly varying factors necessitates an extremely efficient administration system, in which the role of the administrators is particularly crucial to the successful management and implementation of the educational programme. In the past, the arrangement and management of all teaching activities relied on a somewhat outmoded manual system of updates, such as memos on notice boards, mass emails or phone calls. Essentially this meant that all staff and students had to regularly view all notices and announcements, in case there was something relevant to their own programme of activities. Due to the complexities of time and location, traditional updating methods also made it difficult to establish clear and effective communication among departments. If course coordinators, course designers and senior managers of faculty wanted to get an overview of the entire curriculum arrangement, they had to obtain and consider multiple documents from different departments. The difficulty of simply gathering the relevant information meant that course content overlap often occurred despite the best efforts of the responsible management. Under the traditional curriculum management organization, lecturers basically taught their subject areas, and had little chance of getting to know or understand what was being taught in other departments or in different student years. In order to try and tackle such problems as course content overlap and the complexity of interdepartmental communication, the Faculty of Medicine decided to implement a new curriculum commencing in 2001/02. The revised curriculum is organized and delivered as one that is faculty-owned (not departmentally developed), integrated (within and among years for basic and clinical sciences) and system-organ based (not discipline-based). However, the success of the new curriculum approach was significantly dependent on the contributions of the administrative staff in facilitating the organizational arrangements necessary to ensure that staff and students were not only in the right place at the right time, but also had access to essential learning assets. The role of administration was extended to assume organizational responsibility for the entire curriculum, and it was recognized that comprehensive management support was essential. It was therefore decided that the development of a web-based management system would be the most efficient way to proceed. The challenges faced in the development of such a system were how to use technology to: • develop a robust curriculum management structure to provide the necessary data/information via a central platform; • provide easy accessibility to the data/information; • make the organizational structure transparent to all users; • improve the management of administration and learning assets (repositories of learning objects/outcomes, teaching materials, timetables, students’ work and assessment, and so on). Although established environments such as WebCT and Blackboard have been invaluable in enhancing the presentation modes that facilitate learning, and have 138 Jenny Fang, Francis Wong, Raymond Chu consequently broadened our concepts of the possibilities of web-based learning environments, their major focus is the provision of an e-learning environment and the delivery of course materials at the individual course level. The new medical curriculum was prefaced on the management of the curriculum across all year levels and across multiple credentials. Current educational-support technology fails to adequately cope with the challenges of maintaining, in an effortless way, the necessary interactions and information essential to facilitating the creation and management of a medical educational learning environment. [1] Medical education embraces a wider learning community than that normally envisaged in the development of online learning platforms. It is particularly unique in that the notion of ‘classroom’ is extraordinarily dynamic. Medical students learn within an environment that is constantly varying. Learning takes place in and around lecture theatres, laboratories, tutorial rooms and hospital wards, as well as via on-site experiences and a multitude of other exposures that all contribute to the understanding of the discipline, and of the profession, of medicine. The Medical Curriculum Management Platform (MCMP) was therefore developed to cater for these diverse and demanding requirements. It was designed not only to provide easy access to data for various users, but also to make the entire medical curriculum transparent for the whole learning community. This system reflects a unique approach an administrative point of view to the management of a medical education curriculum. 2. Overview of the MCMP Model The Medical Curriculum Management Platform was designed for the Faculty to facilitate management of the new curriculum activities, as well as to embrace information technology to reduce the administrative workload. The MCMP was developed as a platform for students, lecturers and administrators to share teaching and administrative information. The primary aim was to facilitate the management of the personnel and resources involved in medical education environment rather than to focus on the pedagogical programme of the degree. It is not an e-learning system, and does not provide any features to create learning planning, learning activities design, or learning flow control. However, it does provide a link function to other e-learning systems provided by the university; as well as a platform for student collaboration, and for teachers to distribute learning materials and view students’ work. The system serves five different types of user, namely, medical students; professors, lecturers and tutors; adjunct tutors; administrators; and site administrators (Fig. 1). Developing an On-line Medical Curriculum Management Platform 139 Fig. 1. System overview The general functionalities of the system are extensive; some examples include: • providing a web-based GUI for ease of interaction with the database; • Providing users with an overview of all courses within the curriculum; • providing a function for the administrator to create and post announcements for specific student years or lecturers; • providing a function to allow the administrators and lecturers to upload or distribute teaching materials to specific students; • allowing users to use the Form to Mail function to create a HTML form that sends email to specific student groups, lecturers or administrators; • providing a Teaching Event calendar and table to enable users to easily check their teaching/learning schedules; While the importance of the software analysis and design process is indisputable with respect to ensuring that a constructed computer system realizes all its functionalities, the design of the user interfaces is equally important to the success of a web application as it largely determines the extent of user acceptance. As a consequence, a great deal of time was spent in designing the user interface, with special regard to the factors of usability, visualization, functionality and accessibility. [2] The layout of each user interface was predefined in the design documents, which were developed to ensure that all user interfaces were intuitive and easy to use with help documentation. Jenny Fang, Francis Wong, Raymond Chu Fig. 2. A administrator homepage 140 Developing an On-line Medical Curriculum Management Platform 141 The MCMP interface is primarily divided into three modes: ¾ Administrator mode: The administrator mode displays the GUI for the department administrators and site administrators to manage the website and its contents. The content of each user interface in this mode is dependent on the type, and the granted access control, of the administrator. Each administrator can only access those modules that they are allowed to use. A sample administrator page is shown in Fig. 2. ¾ Lecturer mode: The lecturer mode displays the GUI for lecturers to view their own schedules and the teaching schedules of other years or panels. The interfaces are designed to support lecturers in the management of their teaching activities and their communication with students, administrators and other members of staff. ¾ Student mode: The student mode displays the GUI for students to allow them to access their own schedules, logbooks, exam results, and other learning-related activities (Fig. 3). The interface content for each student is dependent on their study year and their learning group. The general functionalities of the system are extensive; some examples include: • providing a web-based GUI for ease of interaction with the database; • providing users with an overview of all courses within the curriculum; • providing a function for the administrator to create and post announcements for specific student years or lecturers; • providing a function to allow the administrators and lecturers to upload or distribute teaching materials to specific students; • allowing users to use the Form to Mail function to create a HTML form that sends email to specific student groups, lecturers or administrators; • providing a Teaching Event calendar and table to enable users to easily check their teaching/learning schedules; The flowcharts for lecturer and administrator access interfaces are decidedly similar, but these users have more comprehensive access rights. Two typical functions included in the administrative mode are the Teaching Event management and Module/Panel Mark management features. The Teaching Event management feature is designed to: • show updated teaching events and provide an overview of the schedules of each teaching group; • provide an interface where administrators can easily create teaching events for specific student groups in a particular frequency period (e.g., once only, once per week, every weekend, once per module, etc.); • allow administrators to modify event content and update variables (e.g., location, time, date and tutors) when schedules need to be refined. The scope of this feature is captured in Fig. 4. Fig. 3. System flowchart of student homepage 142 Jenny Fang, Francis Wong, Raymond Chu 143 Fig. 4. Add Event(s) page for administrators Developing an On-line Medical Curriculum Management Platform Fig. 5. Student exam report for administrators 144 Jenny Fang, Francis Wong, Raymond Chu Developing an On-line Medical Curriculum Management Platform Fig. 6. Layout of printed student report 145 146 Jenny Fang, Francis Wong, Raymond Chu The Module / Panel Mark management feature is designed to: • assist administrators in the management of student results (the system allows on-line input of results, upload of results using an upload function in Excel format, as well as distribution of results to Faculty and students); • provide on-line report viewing and report printing function (administrators can print out the student examination report for faculty use). The scope of this feature is captured in Fig. 5 and Fig. 6. The Teaching Event management and Module/Panel Mark management features are invaluable in reducing the workload of administrators and providing a standardized framework for the centralization and presentation of data. 2.1 Development tools The MCMP is a dynamic web application, and as such necessitates the implementation of several server side programs in order for the system to retrieve data from the central database server. We are using an MS SQL Database Server as the central database server. The whole system was mainly developed using Adobe Coldfusion Markup Language (CFML), combined with javascript, HTML and CSS. CFML is a development tool for creating web-based applications and provides a set of functions to assist the web developer to develop complex interactive web applications. Common features provided within CFML include: • dynamic, data-driven web pages; • database connectivity; • sophisticated data collection and processing; • powerful report generation; • email function; • rich and engaging user interfaces; • access to any back-end system. Additional tools used by the development team included Macromedia Dreamweaver, Flash and Firework. 2.2 Limitations of the system Although extensive time was spent on the design and development of the system, including the design of a comprehensive test plan and consequent exhaustive testing, there are a number of factors that can affect the performance of the MCMP and which need to be constantly monitored by the development and maintenance team. For example: • the accessibility of the GUI may vary depending on the client OS and browser (beyond our control); • the performance of the system can be affected by factors such as network speed, configuration of the client machine, types of application and database server, and the server configuration and tuning; Developing an On-line Medical Curriculum Management Platform 147 • the size of materials files to be uploaded and downloaded vary depending on the limitations of network speed, server timeout setting and the uploading component used. 3. Implementation 3.1 Student perspective A medical curriculum is translated into a teaching programme by means of system panel and skill modules, medical clerkship, surgical clerkship and rotation among clinical departments. Students are divided into several core groups according to clinical rotation, and then sub-divided into sub-groups under each rotation. Each student therefore usually has more than five identities and needs to deal with group members from different panels. Before the launch of the MCMP, curriculum and course administration materials were primarily circulated as paper handouts. In order to have an overview of their daily schedules, students had to check handbooks from different panels and then prepare their own timetables. As even those classes in the same panel or clerkship are independently categorized into different sub-topics and listed in separate tables, students experienced great difficulty in preparing their timetables at the beginning of each school year. The MCMP relieves these difficulties by preparing a timetable for each student dynamically according to their rotation period. Students can check schedules and contact group members online via forums and group email. Student logbooks can also be updated online. The online logbook was especially designed to collect and store data relating to the procedures and examinations that students have taken part in during their medical and surgical clerkship. This data can be a useful reference for teachers and administrators when monitoring student progress and suggesting further activities relevant for each student. 3.2 Lecturer perspective Lecturers in the Medical School are not only teachers but also doctors in the hospital. Their daily schedule is tight and the time that they have available for students is limited. It is hard for lecturers to contact students after class as students disperse to other classes or other panels. It is also difficult for lecturers to remain up-to-date with the academic progress of their students. The MCMP relieves these problems by providing an extensive web-based collection of student curriculum information, such as student photographs, grouping structures, contact information, logbooks, assignments and event schedules. Lecturers can easily retrieve information about the students in their classes directly from the web. The MCMP’s online forum also provides a platform for lecturers and students to communicate and keep in touch. This is important as both lecturers and students have 148 Jenny Fang, Francis Wong, Raymond Chu extensive teaching/learning and hospital responsibilities which make it hard to schedule face-to-face appointments. 3.3 Administrator perspective The MCMP recognizes that administrative staff play an important role in the functioning of the whole curriculum by performing clerical tasks and data entry, serving as a bridge between students and lecturers, and generally making sure that work and communication occur efficiently and speedily. In the past, student information was not disclosed inter-departmentally. This meant that it was hard for administrative staff to access the information they needed when arranging student events involving other departments. With MCMP, all student information related to the curriculum is collected together in one platform, thus enabling administrative staff to easily retrieve the information they need. Instead of circulating memos, administrative staff can now simply post announcements to the web using MCMP announcement tools. Announcements can be posted to a specific audience (e.g. Year 2 and Year 3 students and teaching staff). If amendments are needed, administrative staff can edit MCMP announcements online after they have been posted. Through MCMP, administrative staff can also update schedules and distribute exam results to a target audience in real-time. These tools help administrators deal efficiently with their day-to-day work responsibilities and simplify communication with lecturers and students distributed over a large number of locations. Feedback from administrative staff indicates that the MCMP assignment submission function has also proved helpful in simplifying their work and saving time. Administrators no longer need to email students individually or collect assignments through a collection box, but can simply check online to ascertain whether students have handed in assignments or not. If students need to be reminded of assignment due dates, it is no longer necessary for administrative staff to telephone them using possibly outdated contact lists, as announcements can now be posted or sent as emails to selected individuals or groups of students. 3.4 Challenges Although the MCMP can schedule all teaching events before the commencement of the school term, re-scheduling due to class cancellation throughout the school year is unavoidable due to the fact that lecturers are also doctors responsible for handling emergency cases in the teaching hospital. Although use of the MCMP can ameliorate this situation, sometimes hospital cases are so urgent that even if administrative staff contact students individually through the system, class cancellation occurs too late to properly inform students of the rescheduled class. A tool such as the MCMP has many useful applications. However, the major factor determining its acceptance is the user-friendliness of its interface design. Experience has shown that end-users are reluctant to use a new system if the operation is complicated or requires a lot of steps. [3] If the user is uncertain about what to do or Developing an On-line Medical Curriculum Management Platform 149 what the outcome will be, they will refuse to use the new tool and will very quickly revert to a manual mode of operation. In order to prevent this early rejection, we must clearly recognize the needs of the user and design the interface from a non-technical user’s point of view. Not only do we need to provide a useful and efficient tool, we also need to help educate the user in the use of that tool. Providing support workshops to end-users allows us to obtain feedback and make system amendments where necessary. A system can only be successful if it is accepted by its users. 4. Conclusion The MCMP is an unique and innovative administrative platform that has increased the efficiency of the medical department and decreased the workload and work hours of both staff and students. The MCMP helps lecturers to communicate more easily with students, administrative staff to work more efficiently. It also enables students to organize and schedule their daily routines with less time and effort. The result is that throughout the whole department less time and resources are wasted and more time can be spent on teaching and studying. Although the MCMP was not developed to provide new methods for e-teaching and e-learning directly – this requires initiatives from the teaching staff themselves – however, it does provide support for e-learning in as much as it is able to access any back-end system. What the MCMP has done is to centralize curriculum information from different departments so that all users – students, lecturers and administrative staff – can easily retrieve the information they require online, rather than by making phone calls or sending memos to different departments. The information collected in the MCMP is extensive and complex, it positively enhances the learning environment and facilitates communication within the Faculty. The MCMP provides an essential platform to underpin any educational pedagogy. 5. References 1. 2. 3. Alexander, D., (2002). An accessibility audit of WebCT. Downloaded from http://ausweb.scu.edu.au/aw02/papers/refereed/alexander/paper.html March 21, 2007. The World Wide Web Consortium (W3C) - Introduction to Web Accessibility http://www.w3.org/WAI/gettingstarted/Overview.html Krug, Steve, (2006). Don’t Make Me Think: A Common sense Approach to Web Usability, New Riders Concepts of Blended Learning for Different Content Types Andreas Henrich and Stefanie Sieber University of Bamberg, Faculty of Information Systems and Applied Informatics, Chair of Media Informatics, D-96045 Bamberg, Germany {andreas.henrich, stefanie.sieber}@wiai.uni-bamberg.de Abstract. The immense effort for creating e-Learning content is one of the limiting factors in the proliferation of e-Learning technologies. In the present paper we therefore describe two concepts for Blended Learning that try to achieve a good balance of creation as well as maintenance effort and the benefit for the students. These two concepts are applicable for different classes of courses. A text-based concept is described for courses dealing with fundamental and mature contents and a recording-based concept for more advanced topics with a high alteration rate. The concepts as well as their technical implementation and evaluation results are presented. Keywords: course types, LaTeX, recordings, moderate creation and maintenance effort 1 Introduction It has become a common practise to support learning with different kinds of electronic systems, no matter if it comes to face-to-face study paths with full face-to-face lectures or virtual study paths without any face-to-face phases. The spread of electronic support reaches from just preparing digital learning material to providing complex learning management systems (LMS) implementing elaborated didactic concepts. There are many studies showing the advances of e-Learning as well as evaluating e-Learning challenges and how to exploit its full potential, e.g. [6]. Of course a lot of research has been done in this area so far. Existing and implemented concepts and studies cover all aspects of e-Learning. There are among others theoretical concepts dealing with special issues, research reports on strategies and concepts improving Blended Learning like [2], or methodologies for designing higher-education by implementing complex systems, e.g. [1]. However a lot of educational institutions suffer from a lack of resources, concerning financial and personnel support, and lots of known concepts and systems cannot be set up or used due to these restrictions. In spite of that it should be possible for smaller institutions to set up a quality system supporting teaching and learning in a satisfying way. Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 150-161, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. Concepts of Blended Learning for Different Content Types 151 The overall aim of this paper is therefore to present an approach that is wellelaborated, combining traditional face-to-face teaching and evolving technical possibilities, covering important factors of teaching and learning and most importantly not having an immense need for additional resources. The technical basis we used to implement our concept is an Open Source LMS, in our case Moodle1, which is used university-wide2 and administrated by one person employed part-time. The presented concepts should therefore be easily adaptable and practical for a lot of institutions without special claims on resources. The nature of Blended Learning always depends on various factors like scope of faceto-face and virtual phases as well as obtainable working time. Since having a unique concept for every single lecture cannot be efficient we were developing a concept to transform plenty of practises, conditions and requirements to a few well-elaborated but homogeneous forms. Needless to say there are some general conditions Blended Learning should meet and that therefore should be covered by every instance of the desired concept. Put another way, we can define those conditions as general goals to be accomplished with our concept in order to make it usefully applicable. Those goals have to be determined at least from two sides, the lecturers' and the students' side. Concerning the lecturers' side it is indispensable that lecture notes, respectively provided learning material, can be created with little additional effort and maintained with even less effort. In addition there should be a productive environment for lecturers to absolutely eliminate dispensable time invested in the building process but extend the time that is available for qualified supervision of students. Taking a look at the students' side, an important demand is the request for printable material. Despite the various advantages of online content, students have a general need for printable material. Naturally students in Blended Learning courses require familiar advantages of e-Learning as well. Another goal we added that cannot be accredited one of the sides is the possibility to integrate the result into our university-wide LMS. The concept we were developing should cover a wide, though specific span of Blended Learning forms. To avoid restrictions due to special characteristics of study paths and courses and to allow a detailed concept fitting the needs of all participants, we decided to split our courses into two different types by the means of different content types that are presented in the courses. It is a practical experience that on the one hand there are courses having almost the same content over the years without being obsolete, and on the other hand there are courses that can hardly come through one semester without the need to be changed. Depending on the subject, there are usually courses or course fragments containing very fundamental and mature issues. The content of those courses is therefore almost 1 2 Official Moodle-Website http://www.moodle.org Virtual Campus of the University of Bamberg is available at http://vc.uni-bamberg.de/moodle. 152 Andreas Henrich and Stefanie Sieber static with a quite low alteration rate. Of course most subjects also have courses covering more advanced and therefore often newer issues, e.g. actual research topics and insights. Permanent monitoring of treated topics and subsequently a high alteration rate are essential to uphold the quality of those courses. Since it has been shown that delivering well-elaborated learning material itself is not sufficient to provide high-quality teaching and that there are other factors like monitoring the students' success and understanding that need to be concerned, we are adding auxiliary components to our courses. There are various analyses on how to ensure profound high-quality teaching, e.g. [4]. As mentioned before there are different factors like time and resource restrictions for a lot of smaller educational institutions that will not allow a widespread approach to cover all factors desirable. Therefore we are trying to consider the most important factors like ensuring profound understanding and supporting the students' ability to critical self monitoring and review as far as possible by adding further components —such as self-test— to our concept for both types of courses. In the following two sections the differing concepts for the two types of courses as characterised above are explained and reasoned. As finally shown in section 4 the concepts turned out to be successful and can easily be adapted. 2 Text-Based Courses Like outlined above, one type of courses are courses containing fundamental issues setting up the base for following advanced courses. It is obvious that basics of a specific well delved subject do not change very often or very fast so that the presumption of almost static courses is realistic. The idea for this kind of courses is to structure the subject matter like a book, to take the specific characteristics mentioned into account. Accordingly it is feasible to provide the subject matter in different digital forms. To realise this concept we needed a building tool that is at best commonly known, easy to use and of course providing the desired outputs. Conceivable outputs include especially PDF- and HTML-documents. For this reason we evaluated a “single source, multiple publishing” approach that builds different kinds of output from one XML source [3]. An approach like this seemed to bring the brightest prospects. Unfortunately creating and updating the text-sources directly in XML is pretty inconvenient. The effort needed to maintain text itself is additionally amplified, e.g. due to lots of formulae and graphics to be included. At the same time we also verified the idea to provide the learning material in different digital forms by questionnaires handed out to our students. The result strengthened the assumption that our web server statistics allowed. Students not only definitely preferred printable content but also saw no need for additionally offering the curriculum as interactive online content. Concepts of Blended Learning for Different Content Types 153 All these factors and experiences lead us to discarding the “single source, multiple publishing” approach in the end and brought about the decision to offer PDFdocuments as learning material concerning the curriculum. The decision for PDFdocuments is motivated by the fact that the PDF-format is an established platform independent format that has good prospects. PDF already offers mature possibilities to provide comfortably structured and interactive documents. Besides, further development is already taking place and in particular the interactivity options are expected to be improved and increased in future versions. The following conclusion to use LaTeX as building tool was just a logical step. LaTeX is a well known and widespread format especially for academic writing and it also provides very good integration of formulae and extensive possibilities to include graphics. We employ a specific format that provides additional benefits, e.g. marginalia, which are complemented to improve the structure and clearness of the text. The whole material is also completed by a wide index and an extensive bibliography where literature is directly linked if possible. Besides, we focus on adding lots of additional links to benefit interactivity; there are anchors referring to further parts of the learning material, concrete links to related parts of our LMS and a large number of links to external websites. We are convinced that generating PDF-documents from LaTeXsources is a good choice because there is the possibility to have a really comfortable interactive version of text. PDF-documents therefore can be used as online and print version all at once, providing the benefits of both versions. Generally the PDFdocuments are uploaded to our LMS and provided to the students this way. The concept to use the material of those courses embraces two phases. First, students are working through the provided learning material on their own. This approach replaces the normal face-to-face lectures and therefore, as assistance, a schedule showing the optimal allocation of learning is handed out to the students. Second, to supplement the self-organised learning there are weekly face-to-face meetings with the lecturer. Mainly the material scheduled for the last week is discussed in these meetings, but the discussion is also open for special issues or questions emerged. To provide additional interactive parts to our text-based courses and encourage the students' learning possibilities there are additional components supporting a broader view on the content of the curriculum and expanding alternative ways for students to learn and understand the essence of the subject. We are adding two interactive components: Java Web Start applications and self-test tasks. Java Web Start applications are an important part of our courses – especially where a profound understanding of sophisticated concepts is desirable. We figured out that there are several concepts in each course where such an application can provide useful visualisation and additional understanding. Applications are uploaded to our LMS and provided within the system. Due to the standalone nature applications are independent from any system and can be used not only on our current LMS but on probably different future versions too. 154 Andreas Henrich and Stefanie Sieber Fig. 1. Progression of a clustering applet visualising a clustering process Depending on the content of the courses there is a variety of applications provided. To name but a few, one course we are offering is Information Retrieval [5] and we are e.g. providing applications on Recall and Precision graphs as well as applications to explain the process of stemming methods or to combine understanding and implementation of algorithms for pattern matching. As an example you can see the time progress of an application visualising the clustering process of a specified clustering algorithm in figure 1. Self-test tasks represent the second component we chose to provide additional options and interactivity for students. Self-test tasks constitute an instrument for students to get an estimation of their current knowledge and understanding of important concepts as well as a possibility to improve their current state of knowledge by accomplishing the provided tasks. Self-test tasks are normally realised as a sequence of multiple choice questions. The type of questions ranges from questions on theoretical aspects to sophisticated computations. Concepts of Blended Learning for Different Content Types 155 Fig. 2. Integrated self-test task on signature trees Other than Java Web Start applications self-test tasks are generated and provided directly in the LMS. This approach brings a potential risk concerning possible chances in the use of the LMS, but because our actual system offers a backup functionality that exports selected data to XML we are willing to take the risk of doing some migration work in case of a change of the system. Besides, the LMS offers wide evaluation possibilities of test results, a factor that also confirms our considerations. Abiding with the Information Retrieval course there are, among others, self-tests on principle Information Retrieval questions and models of Information Retrieval like the vector space model or the binary independence retrieval model. In figure 2 you can see a screen shot of a self-test task on signature trees. The specific problem is searching for a precise signature in a given signature tree; there are 3 pre-formulated answers to choose from. The main goal of integrating and providing PDF lecture notes, Java Web Start applications as well as self-test tasks as parts of one course in our overall LMS is therefore to meet the requirements of students and lecturers all at once: providing well elaborated and printable lecture notes that are easy to create and modify as well as a balanced degree of interactivity to benefit students' success. To round the concept for text-based courses we are providing weekly exercise lessons. The exercises mainly consist of implementing the concepts presented and learnt during the lecture-like parts of the course. We consider this as an important part of 156 Andreas Henrich and Stefanie Sieber our concept, respectively courses, because students learn to study and apply the theoretical concepts practically. As a further possibility students get the chance to collect bonus-points for the exam concluding the course due to biweekly tests that are offered. In total there are 5 voluntary tests with tasks about different concepts that are part of the course and 4 points can be reached each time. The best 3 test results are added to the exam result if the exam is initially passed. Since the best score in the exam is 90 points no matter if bonus-points are considered or not students can markedly improve their exam results by collecting bonus-points. Our LMS offers a good support for those tests. The detailed test is provided on our LMS and students are first able to download the test and build their solution wherever they want to. Second, they are able to deliver the results directly in the system by an upload mechanism. Lecturers are then able to view, judge and score the students' solutions also directly in the system, while students can check their test results and score as well as a usually provided suggested solution after the lecturers' scoring is completed. The thoughts behind those tests are that students are really getting into the subject not just shortly before the exam but during the whole semester because they need to understand the content of the course to process the tests. Of course this might not apply to all students because it is a voluntary offer but normally students are motivated to do the tests in order to improve their exam results this way. Looking at the whole concept at a glance we are trying to regroup the face-to-face phases. That implies reducing face-to-face phases where they are not absolutely necessary, e.g. to work through provided learning material, but extending face-to-face phases where further supervision and contact can be useful, e.g. by additional exercises.. 3 Recording-Based Courses In contrast to courses consisting primarily of fundamental and therefore static content, there are courses containing current standards and systems as well as actual research topics and results. Since we attempt to keep our courses up-to-date, these are obviously courses that need to be changed more often. For this reason an appropriate building and update strategy is needed to keep the effort within a manageable limit. As a consequence we were looking for an approach that allows generating and providing courses very easily with barely additional tasks apart from delivering the face-to-face lecture itself. Obviously this brought us to Rapid e-Learning and we started pursuing an appropriate Rapid e-Learning approach. In contrast to text-based courses we were keeping the traditional way of face-to-face lectures attended by students for recording-based courses. We chose to record the lectures [7,8] and provide the recordings afterwards because the lecture has to be delivered anyway and if it can be recorded in the same time there is apparently almost no need for additional time and effort. The thoughts behind this concept are to provide Concepts of Blended Learning for Different Content Types 157 the lectures for students even after the face-to-face lectures were actually delivered. This way, recordings can be used for post processing or exam preparations. Students can run the recordings to repeat the whole curriculum or just parts they probably missed or still have questions on. Of course no one is directed to use the recordings at all, it is a voluntary offer. We had thoughts about a smaller number of students attending the face-to-face lectures at the beginning because students could use the recordings instead of taking part in the lectures but our thoughts were proven wrong. Students are still attending the lectures and there is no noticeable smaller number except for those missing due to inevitable conflicting dates with other lectures. So far we see this as an advantage of our recordings since they are attenuating these problems. The requirements and preparations needed to accomplish this approach are neither demanding nor expensive. First of all lecture notes are created as slides. There is no need for any additional formatting or settings, and since slides are used as base to deliver the lectures anyway the slides can be created as usual. Furthermore the plain slides can be stored as PDF-documents so that they can be provided as additional material for students to meet the requirement of printable material. The actual recording process takes place when the face-to-face lecture is delivered. The lecture is thereby delivered as usual but the lecturer records his lesson. The technical equipping needed to run this system is imaginably low: A tablet PC connected to a projector is used to display the slides and to provide additional information and assistance on the slides as well as extra slides created off the cuff if desired, we are using the Rapid e-Learning tool Lecturnity 3 . The only additional requirement is an ordinary web camera. The camera records a video of the lecturer while delivering the lesson, including the audio stream. Of course there are restrictions concerning the quality of the recordings if recording is done with a simple web camera, but since the recording result is integrated into and provided on our LMS the restrictions correspond with the restrictions due to bandwidth and storage (in our setting a recording of a 90-minutes lecture requires about 120MB). Lecturnity is able to combine the recordings of the web camera and the recordings of the slides. Furthermore thumbnails of the slides are displayed as table of content and can be used to navigate through the video. The standard output format is a proprietary format, but there are lots of standard outputs possible. It is e.g. possible to produce Flash- or SCORM-files. 3 Official Lecturnity-Website http://www.lecturnity.de 158 Andreas Henrich and Stefanie Sieber Fig. 3. Screen shot of a recording available for students To view the standard output format a free player is available. To make it even simpler, we are providing the player on our LMS, therefore students only need to download and install the player. The technical requirements to install the player are very low, so that except for very few cases no additional costs occur for the students. There are some students having problems due to video configuration, but the rate is below 1 percent. In total the lecturer only needs about 5 to 10 minutes of extra time for each lecture to import the slides into Lecturnity, to export the final recording after the lecture and to publish the recording in the LMS. In figure 3 you can see a screen shot showing the result of the recording that is available for students after the face-to-face lecture, viewed with the corresponding player. As you can see the lecturer, the slides and the table of contents for navigation are visible all at once. The concept of recording-based courses is also supplemented with exercises and tests to collect bonus-points for the exam. The realisation of these two features is basically like in text-based courses. Concepts of Blended Learning for Different Content Types 159 3 Evaluation After extending the scope of our Blended Learning concept, we were reorganising all our provided courses and mapping each to the types described above. The resulting distribution is quite balanced – there are 7 courses that we regularly provide, 2 of them are basic courses and therefore provided as text-based courses, 4 of the courses lie in the advanced field and are realised as recording-based courses, and 1 of the courses has basic and advanced parts and is therefore realised as a hybrid form. Evaluation concerning the lecturer-side is restricted to a self-critical review at the end of each semester, showing that the deliberations concerning needed effort and available time were generally correct. Since nobody but us is using the concept by now there are no further evaluation results available. Concerning the view of students, the two introduced course types are evaluated against different criteria at the end of each semester. We are presenting exemplary evaluation results of one text-based and one recording-based course that took place in the last semester – the results are based on 25 evaluated questionnaires. There are some general questions that are similar for text-based and recording-based courses. Students were asked general ratings for the lecture, respectively the lecturelike part, and the exercises; the scale for the general questions reached from 1 (very good) to 6 (very bad). The grading for the lecture concept as a whole and the associated exercises ranked from 2.1 to 2.6 for text-based courses and from 1.6 to 2.0 for recording-based courses. The special focus for text-based courses was on questions about organisation and structure of the learning material and the weekly face-to-face meetings. The scale available reached from 1 (disagreement) to 4 (agreement). The learning material was rated in different criteria, e.g. structure and understandability as well as personal estimation on how good the learning material covered the underlying area, and how helpful the learning material was for understanding concepts of that area; the average values of ratings in this area reached from 3.1 to 3.8. Concerning the weekly face-toface meeting students basically had to rank the degree of help that was offered by these meetings; the average value was 3.1 in this area. The focus for recording-based courses was of course on rating the use of recordings. The plain result was that all participants were using the recordings, some for equalising missed lectures, some for post processing difficult parts and some for exam preparation. Drawing an overall resume all values are positively above average and students seem to be comfortable with the concept of text-based as well as recording-based courses. After mapping our Blended Learning courses successfully to the evolved concept, we were thinking about further efforts and developments. Since we are participating in a 160 Andreas Henrich and Stefanie Sieber joint offer of a virtual study path4 it was obvious to transfer our existing concept to the courses offered in this environment. We realised that the concept could easily be adapted to virtual courses and there is actually almost no alteration needed. Concerning the text-based courses all parts of the courses could be used just the way they were, because the virtual courses are provided via our LMS too. The PDF lecture notes can be provided as usual and the same applies to Java Web Start applications as well as self-test tasks. Solely the weekly meetings to discuss concerns or questions emerged for students while working through the learning material are dropped. Instead of arranging face-to-face meetings the discussions are entirely moved to the forums included in our LMS. Transforming the recording-based courses slight changes or possibly additional time is needed. In principle the recordings can be provided as usual and the same applies to the additional lecture notes provided to students. The decisive advantage is that the recordings make students of virtual courses feel like they are actually attending faceto-face lectures. The problem is admittedly that the recordings have to be created although there is no face-to-face class. In our case this problem does not occur because the recording-based courses of the face-to-face study path are identical to the recording-based courses of the virtual study path so that we can reuse the recordings arising out of the face-to-face lectures. More concrete, the face-to-face course is given in the winter semester and the corresponding recordings are used for the virtual study path in the following summer semester. We are aware that this a special case and that the additional time needed to create the slides and recordings without a corresponding face-to-face lecture can turn out to be a problem due to time restrictions. Questions and discussions normally covered during the face-to-face lecture are moved to forums for recording-based virtual courses as well. Both types of courses therefore proceed on the assumption that the lecturer spends time on supervising the forums and answering questions arising in an appropriate time. Obviously the face-to-face exercises have to be dropped for both kinds of courses because there is no possibility for students to attend such classes. Nevertheless it is possible to offer the tests to collect bonus-points for the exams as usual so that students get for once an idea of practical issues related to the theoretical concepts learnt before. Detailed evaluations have shown that students are pretty satisfied with this approach. Evaluations are taking place at the end of each semester and each proceeded course is evaluated properly. The evaluation results presented below refer to two text-based and one recording-based course of the last two semesters, that is about 40 fully completed and evaluated questionnaires. We restricted the results to the last two semesters to present the results as actual as possible. Evaluation criteria were on the one hand criteria concerning the quality of supervision, e.g. supervision of content-related or technical problems as well as organisational support. On the other hand criteria 4 The virtual study path is a joint offer of the University of Duisburg-Essen and the University of Bamberg, further information is available at http://www.vawi.de. Concepts of Blended Learning for Different Content Types 161 concerning the content of the courses, e.g. the intermediation of knowledge, comprehensibility or structure of the presented content were evaluated. The evaluation scale allowed grading every criteria on a scale from 1 (low) to 7 (high), additional space for comments on especially positive or negative features as well as further comments was available. Although the students admitted that the effort needed to pass the courses was above average by tendency, we got very positive responses on average. All criteria concerning the quality of supervision were ranked very high, at least 6.0 or above, above all courses. There is no recognisable difference between recording-based and text-based courses. The criteria concerning the content of the courses are rated little worse but still very high relating to the evaluation scale. Concrete values vary from 5.5 to 6.7 for text-based courses and lie closely around 6.0 for recording-based courses. Analysing the spaces for free comments it became obvious that especially the provided recordings were appreciated when looking at the recording-based course. There were a few students that had start-up difficulties because the learning material for recording-based courses is organised as a structured collection of bullet items and not continuous text, but the recordings and possibilities available by forums smoothed these problems. Concerning the text-based courses lots of students commended the systematic book-like structure of the learning material. Some students always go to the bother of suggesting concrete improvements and as far as it is possible we are trying to consider the suggestions for the next semester. The most salient overall result was that a few simple but effective facts were mentioned very positive across all courses. Extremely positive responses gained the fact that the lecturer itself was actively attending the forum discussions and answering questions of every kind. The short response time provided by lecturer and tutor is also a circumstance that was appreciated a lot. References 1. Andy Hon Wai Chun: The agile teaching/learning methodology and its e-learning platform. In ICWL, pages 11–18, 2004. 2. Michael Derntl and Renate Motschnig-Pitrik: Patterns for blended, person-centered learning: strategy, concepts, experiences, and evaluation. In SAC, pages 916–923, 2004 3. Max Dunn: Single-source publishing with xml. IT Professional, 5, 2003. 4. Joseph Fong, Irene S. Y. Kwan, Margaret Ng, Ivan Li, and S. K. Chan: An applicationoriented e-learning system with self-monitoring and adaptive exercises. In ICWL, pages 325–332, 2004. 5. Andreas Henrich and Karlheinz Morgenroth: Information retrieval as elearning course in german - lessons learned after 5 years of experience. In TLIR, 2007.7. 6. Won Kim: Directions for web-based learning. In ICWL, pages 1–9, 2006. 7. Tobias Lauer, Rainer Müller and Stephan Trahasch: Learning with lecture recordings: Key issues for end-users. In ICALT, 2004. 8. Thomas Ottmann, Stephan Trahasch, and Tobias Lauer: Systems support for virtualizing traditional courses in science and engineering. In Quality Education @ a Distance, pages 73–82, 2003. Blended Learning: Towards a Mix for SMEs Stakeholders and their Priorities Sabine Moebs and Stephan Weibelzahl National College of Ireland, School of Informatics, Mayor Street, Dublin 1, Ireland {smoebs, sweibelzahl}@ncirl.ie Abstract. While blended learning seems to be quite suitable for small and medium sized enterprises (SMEs), current uptake of this learning method is low. In this paper we propose a research design to examine the requirements for blended learning in SMEs. It is based on a three-round ranking-type Delphi study. Participants for the panels were carefully selected. Our method takes into account that the area and the term of blended learning are discussed in very different, partially contradicting connotations. For this purpose, we first provide the background of the initial research question and describe our research design. Next, we present preliminary results of the Delphi study and the steps in preparation of round 2. Participants were selected for the online-Delphi and grouped into panels of SME learners, trainers and providers of e-learning as well as learners from large companies as a control group for the SME learners. Keywords: Blended Learning, SME, Delphi study, Requirements 1 Introduction Originating in the corporate training sector the term blended learning refers to the provision or use of resources which combine e-Learning with more traditional educational resources [1]. Our research combines an exploration of blended learning and methods of learning operation in small and medium-sized enterprises (SMEs). Blended Learning is often used as a buzz word with vague and varying meanings. It has often been described as a way to get the best out of the two worlds of technology enhanced learning and traditional classroom-based learning, benefiting from the advantages of technology enhanced learning while compensating for its disadvantages through additional in-class sessions [2, 3] and vice versa.. Blended learning is a combination of any form of technology enhanced learning with face-to-face instructor-led learning [3]. Blended learning has been proposed as a solution for training needs in SMEs [4]. SMEs are often innovative, but under high economic pressure. This economic pressure is a threat to ongoing learning activities although continuous training and learning is necessary to stay competitive. Learning in the form of e-Learning is not in high demand with SMEs although one could expect that it is highly suitable to the learning demand at short notice [5] which is typical for SME learning. Research suggests that blended learning can significantly improve learning satisfaction, Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 162-173, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. Blended Learning: Towards a Mix for SMEs 163 improve accessibility and increase participation [6]. It therefore has the potential to alleviate a number of issues that arise in SMEs and act as constraints to the use of blended learning, namely lack of social interaction during learning programs and exchange of ideas. Customized in-house solutions used by multinational companies (e.g., corporate LCMS or learning portal, content tailored to specific needs of company) are usually not feasible for smaller organizations. Therefore SMEs rely on what providers offer, i.e., instructor led training or off-the-shelf solutions. This is one of the reasons why learners in SMEs are reluctant to use blended learning for their learning and training needs [4]. 2 Aspects of Blended Learning Blended learning describes a learning environment that either combines teaching methods, delivery methods, media formats or a mixture of all these. In the literature the term is used to describe the integrated combination of traditional off-line methods of learning with intranet, extranet web-based or internetbased online approaches [7]. To accentuate the fact that the concept is learner centered, blended learning can be described as a mix of delivery methods that have been selected and fashioned to accommodate the various learning needs of a diverse audience in a variety of subjects [8]. Blended learning aspects are covered in a variety of research papers as well as in very practical instructions for blended learning. The most prominent are briefly outlined here. 2.1 Dimensions of Blended Learning To describe the variety of interaction Graham [9] introduced the so-called four dimensions of interaction in face-to-face and distributed learning environments. The four dimensions are space, time, fidelity and humanness. Space can range from live or physical and face-to-face over mixed reality to virtual reality. The time dimension develops from live synchronous with a very short lag time to asynchronous, which has a long lag time. Fidelity reaches from a high level that is rich in senses, which means it can incorporate sound, pictures, text and even fragrances, and the other end of the dimension is using only one of the senses, e.g. text only. The humanness dimension addresses the ratio of human interaction and machine interaction. 2.2 Frameworks in blended learning Poor design of blended learning material can lead to much poorer learning results in a blended environment compared to a single method delivery. Several authors developed frameworks to react to this challenge. Wenger and Ferguson [10] describe a framework to guide the design and deployment of company trainings and courses. The framework reflects the idea that most learning environments are blended already, considering that even a classroom- 164 Sabine Moebs and Stephan Weibelzahl only course incorporates a variety of different learning modalities. Their approach consists of three steps: In a first step the learning ecology matrix is developed. The x-axis illustrates the focus on the delivery of instruction that varies from “content delivery focus” to “experience and practice focus” and the y-axis illustrates who controls the navigation of the learning process varying from “guided navigation” to “learner self-navigation”. In a second step four general learning modalities are included: studying, practicing, teaching and coaching. These modalities do not refer to either classroom or e-learning, but are rather applicable to both. In a last step the matrix is completed with distinct instructional, learning and knowledge elements. The learning ecology matrix aims at delivering a high quality learning experience and at providing control over the learning experience for both, the learner and the instructor. It strives at combining formal and informal learning rather than positioning them as opponents. The social nature of learning has to be considered in all learning elements. The aspect of cost-effectiveness is recognized, but merely in the sense that any project aims at a combination of learning outcomes at a total minimum cost. 2.3 Success factors for blended learning A variety of teaching methods, as well as a variety of different learners with different preferences and needs determine success of blended learning. Several success factors have been identified in the literature: Design of the blend: A well designed blend of teaching methods can provide an appropriate learning experience for most learners. The characteristics of the audience have to be considered. This includes recognition of the amount of time they will have to access the content, which includes connectivity issues [8, 11, 12]. Time flexibility: The flexibility in scheduling and format is critical to success. Availability of the system enables them to study when they are ready to do so. Mix of media and learning styles: The flexibility in media formats provides optimum learning experiences based on personal preference. To select the right methods and formats the learning styles and the education level of employees has to be considered as well as the motivation of the learners [11, 13]. Student support: Response from tutors, subject matter experts as well as technical or logistical support staff needs to be posted within 24 hours, which corresponds to a rule of thumb for effective e-communication in general. The positive effect of a timely response can be intensified by additional phone calls and face-to-face conversations and will provide a sense that there are real people behind the online environment [13]. Executive support: Blended learning needs executive support for the introduction just as any other major change in a business environment. The decision to change to a blended solution from the system that was in use before cannot be left to individuals who are not in charge. Content: The kind and quality of learning content is critical for success. Apart from choosing the appropriate kind of content and the decision whether learning activities are intended to inform people, develop skills, or build competencies, the consideration of the time before information is out-of-date is of high importance [11]. Blended Learning: Towards a Mix for SMEs 165 2.4 Blended Learning in SMEs The mix of ICT based training in combination with face-to-face interaction has been identified as a success factor for blended learning in SMEs. A second success factor is trust in the safety of the training environment, online as well as face-to-face. SME learners prefer an informal and ad-hoc approach to learning which suits their busy schedule. Lack of confidence in ICT on the other hand is an obstacle for taking on blended learning by learners in SMEs as well as the lack of an immediate payback of the training. Activity-based learning is clearly preferred over knowledge-based learning. The size of the company has a strong impact on any kind of learning and training activities which leads to small firms often lacking a lifelong learning culture [14, 15]. A study of e-learning in continuing vocational training with emphasis on SMEs came to the conclusion that there is not much information available on elearning in SMEs [16]. 3 Statement of Purpose of Research Blended Learning can combine the positive aspects of the two learning environments, classroom based learning and e-Learning [2]. While blended learning seems to be quite suitable for small and medium sized enterprises (SMEs), current uptake of this learning method is low [4]. An on-line Delphi study and several in-depth interviews are used for data collection. The study involves main stakeholders in blended learning in SMEs: trainers, providers and researchers from the area of e-learning, blended learning and lifelong learning, learners in SMEs will be compared with a control group of learners from large companies. This study will explore what is a good mix in blended learning - a mix of online and face-to-face teaching - for learners in small and medium-sized enterprises (SMEs). 4 Delphi Study There are different types of Delphi studies out of which a ranking-type Delphi is most suitable for the research question. A pre-Delphi study provides a method to determine the required type of study and other characteristics of the study. The quality of the study is highly dependent on the selection of the participants and the analysis of the data from the first round. 4.1 Pre-Delphi-Study To determine whether a Delphi study would be a suitable approach and which type would be most appropriate, we conducted a pre-Delphi [17] study. Initially we extracted a set of questions from a literature review of the two areas blended learning 166 Sabine Moebs and Stephan Weibelzahl and learning in small and medium-sized enterprises as well as recommendations from the Delphi technique [18]. In the pre-study we included a small group of potential study participants: a representative from an SME, a provider, a trainer and a researcher. At first we considered asking for feedback eventually leading to consensus on different topic areas such as learning styles, different dimensions of interaction, delivery modes and learning modalities. The pre-test showed that the research as planned would not provide input from panelists about their priorities, but rather about agreement or disagreement with the researchers’ perspective or previous research. It became obvious that the study has to be open to the stakeholders’ selection of important topics; otherwise we might run into the problem of simply confirming our own ideas rather than eliciting the opinion of the panelists [18]. Moreover, we excluded the idea of using a mind map to collect initial input in the first round, because it seemed likely that a number of participants might not be familiar with the technology and therefore be distracted or even turned off from the study topic itself. Instead we decided to develop an online form with a clear navigation, a good usability that takes as much work from the participants as possible and which is at best self-explaining how to use. The pre-Delphi study provided us with a decision to conduct a ranking type Delphi study and set the limit for the amount of issues to be asked from the panelists as an input as well as the number of rounds that seem to be feasible without strongly increasing drop-out rates from the participants. 4.2 Participants of the Delphi Study The Delphi study involves a total of 50 participants with the sub-panels small and medium sized enterprises (SMEs) from the IT industry, and from the tourism industry, large companies, as well as trainers, providers and researchers from the areas blended learning, e-learning and lifelong learning. SMEs were selected following the current SME definition of the EU [19]. SME panels: Selection for the SME panels followed a general pattern for all SME participants and an additional industry specific pattern. All SME participants were selected following the pattern being employee of an SME, an interest in or experience with blended learning or e-learning, internet access, management responsibility (team leader, project manager, department head) and the number of years of experience in the industry. The participants from the IT SME panel have an average of 7.6 years experience in the industry. They are managing directors, owners, manager, network technical staff and software developers. The tourism SME panel includes the following areas: hotels, B&B, tourism information, travel agent and tourist guide or instructor (ski, snowboard, sailing, etc.). The participants from the tourism SME panel have an average of 18.8 years experience in the industry. They are owners, managing directors and project managers and work for B&Bs, hotels, an outdoor events provider, travel agencies and tourism information. Blended Learning: Towards a Mix for SMEs 167 Large company panel: The large company panel participants were selected following the pattern being an employee of a large company with more than 249 employees, experienced with blended learning or e-learning and having internet access. Again we looked for team leaders, project managers and department heads for this panel. Trainer panel: The participants for the trainer panel were selected according to a pattern [20] that required several years’ experience in blended learning, experience with learners in SMEs and unrestricted internet access. The trainer panelists have an average experience of 11.1 years in blended learning; the median is 8 years and years of experience ranges from 5 to 22 years. The panelists are mainly trainers in the IT sector, some in general education. Provider panel: The participants for the provider panel were selected for their experience in blended learning and the positions in e-learning companies. The provider panelists have an average of 9.9 yrs experience in blended learning; the median is 4 years and years of experience ranges from 4 to 16 years. The panelists’ positions are head of product development or education management, learning design manager, technical staff, project manager and managing director. Research panel: The participants in the research panel were selected for their research area, work and publications in the areas of e-learning, blended learning and lifelong learning. The research panelists have an average of 11.6 yrs experience; the median is 10 years and years of experience ranges from 9 to 16 years. The research topics are for instance evaluation of TEL, access to learning, digital learning styles, social media, open educational research, business models in e-learning and e-learning standards. 4.3 Ranking-Type-Delphi Study To select a suitable Delphi application we turned to a taxonomy proposed by Day and Bobeva [21]. Our Delphi study design can be described using their taxonomy. There will be three rounds, one for discovery of issues and the two following rounds to determine the most important issues and to rank them. The participants will be heterogeneous since there will be five sub panels of participants with different expertise, researchers and providers of e-learning, online and face-to-face trainers, learners in SMEs and learners in big companies. Within the sub panels we aim at a best possible homogeneity. The study will be conducted as an on-line survey and all communication will be conducted electronically using e-mail, website and VoIP. We aim at single-blind anonymity of the panelists while conducting the study. In addition to these criteria we decided on a ranking type survey. The ranking-type Delphi aims at finding an agreement between groups through a ranking of self-selected issues. The ranking type Delphi study requires that the researcher focuses on three initial decisions: 1. when to stop polling 2. how many issues to carry over to the next round 3. use of statistical techniques to support their conclusions. 168 Sabine Moebs and Stephan Weibelzahl Literature suggests that these answers have to be decided individually, depending on the study design, number of participants, area of interest, etc. We follow Schmidt’s [22] and Couger’s [23] examples and considered the results of the pre-Delphi. The polling will stop after round 3. Initially the panelists are asked to list the 5-10 important aspects of blended learning. Participants have to add a description and a rationale for putting the item on the list. The total input from round 1 is consolidated into a list size short enough to be accepted by the participants in the next round. The full list of all aspects, including duplicated and synonyms can easily overstrain the participants and might result in high drop-out rates. A too short list on the other hand can result in loss of information. Where panelists use different terms for the same issue the researchers have to provide a summary matching the different terms and one common description of the issue. In the study an initial list of 225 items from round 1 was condensed to a list of 59 items as input for round 2. In the second round the panelists rank their “Top 20” issues, ties not allowed, out of the consolidated list of round 1. The second round is aggregated into a list of “Top 20” items for each sub panel. This 20-item panel specific list is presented in the third round as a list of the “Top 10” with ranks from ten to one. All other items on the list (11-20) are equally ranked “0”. The panelists now rank their “Top 10” issues from the 20-item list. The rank is calculated by combining the percentage of selection and relative rankings by the individual participants. An approximation of the mean ranks has been produced by multiplying each percentage of mention by its first-round rank. A combined measure of the ranks in the second and third round provide a value for each item and provide the final evaluation of each item on the 20 item list. 4.4 Results of Round 1 of the Delphi-Study Round 1 collects a minimum of five and a maximum of ten aspects (items) of blended learning considered most important for the topic from each participant. A consolidated list with all the items from all participants will be given to the participants in round 2. In round 1 a list of approximately 200 unsorted items were collected. The following list highlights the aspects that were mentioned by five participants or more. The items are briefly described, reflecting the variety of meanings. Accessibility: 24-hour accessibility of online parts, the option to work from home or while traveling were mentioned was mentioned. Materials, tutor, IT and classrooms have to be convenient as possible for the learner to encourage maximum participation. The learner has to be able to decide, where and when to learn. Time Flexibility: Trainers and participants should have the time to get to know the system and the combination of online and face-to-face teaching and learning. Online learning should enable to learn when it suits the learner and moreover enable completely independent learning. It should enable participants to decide on suitable learning times suitable to other activities and to use times in between normal work. Online learning should support the learner to make best use of their own time. Blended Learning: Towards a Mix for SMEs 169 Cost Efficiency: Blended learning as an option to keep the price of the training solution as low as possible was mentioned. Participants also see a potential to find a good mix by emphasizing the lower-cost elements of the blend (e.g. off-the-shelf elearning entities, on-line books, etc.). Some participants believe that blended learning often means lower costs. The option to use course materials on a regional level is expected to reduce travel costs. In general participants expressed the opinion that online learning is more cost-efficient immediately as well as in the long-term. Student Interaction: The ability to interact at different levels and through different media should allow a more adaptive approach to learning. Participatory opportunities for students to have a voice e.g. using VoIP was mentioned several times, but also taking the student through a number of learning routes rather than a given sequencing of learning materials. Support Mechanisms: Personal support for every participant by mail, phone or chat or mentoring is considered important. Collaboration tools are seen as possibilities to greatly improve the team work that can be allocated and performed. The ability to work in teams or virtual teams is perceived as a supporting function. To provide ample opportunities for students to obtain help with specific problems was mentioned as well as the need to explore topics that might be a little off the curriculum. The online assessment is considered helpful for motivating the learner, because it gives immediate feedback, but the social interaction in the classroom is also required. Mix of methods and media: The use of different media, different learning media, face-to-face and online are the characteristics mentioned most often. A selection of media for specific parts of a course, e.g. test or interactive content that is read to the learner or includes images and text as well as the classical reading of books, is mentioned several times. The user should have printed/offline media supplementing the online learning content. Learning with the suitable media and to make the right choice seems crucial for this area. Online modules allow flexibility, but can be very generic. Face-to-face phases in contrast can be very intensive and don't allow for distraction. They are considered more restrictive. Mix of learning styles: A mix of learning styles is expected to support learning success. A basic mix of online and offline activities are expected as well as a mix of different ways to present and teach. This mix is also expected to increase motivation. Workplace-related learning: Learning must be relevant and useful to the learner, otherwise it is just an exercise soon to be forgotten. The course needs to be relevant to the skills / information gap that the organization has. Learning content has to be upto-date and important to the user. Individuality: Instruction should be designed to adapt to the individual learner and it should provide different kinds of learning experiences. The e-learning enables learners to set an individual focus. Trainers or facilitators need to be able to deal with different personalities and heterogeneous groups. Knowledge Base Internet: The internet provides the most recent knowledge to everybody in forums and there is no way to beat the internet as an easy to use dictionary for any topic. Accessibility and easy search functions make the internet a vast source of teaching materials and enables trainers to provide access to lots of related material to build student enthusiasm. It provides a number of elements that we can add into a blend, like blogs, wikis, mobile, podcasts etc. 170 Sabine Moebs and Stephan Weibelzahl Recognition of traditional learning: Formal class room training which follows a specific training guideline tends to cover the topic in question in greater detail. From experience this is suited to the more committed. Formal classroom/college is ideal also for employee networking. Self-Paced Learning: One needs to learn self-paced learning. Nowadays everything is presented to students rather than letting them figure it out themselves. Provide learning and practice experiences that are available over a continuum of time, versus all within a short timeframe. Spaced learning and practice helps cement new knowledge into long term memory, and provides additional cues for retrieving the knowledge and skills under different circumstances. Course enables participants to select order of topics and modules. The pacing of the learning process is placed to a certain extent with the student and can suit their time needs and commitment. Selfpaced learning is suitable for shift workers and those on time constraints who may not be able to attend a conventional timetable class or course. Blended Learning's main benefit for students and employers is the flexibility to do the course at your own time and pace. Technology: Keep technology simple - If and when technology driven solutions are part of blended programs, keep it as simple as possible. The switching between mask/pages should be kept at a minimum. The learning environment has to be kept simple. Many LMS have lots of different functions and features that an average learner in an SME does not need. It is better to reduce the number of functions so that learning of content plays the main role, not learning to use the system. Offer tools of Social Software for more experienced learners. Make it easy to access and operate any technology components of the blended solution (web page, on-demand course, pod-cast, virtual classroom, etc.). 4.5 Getting from Round 1 to Round 2 of the Delphi Study We used techniques of the coding phase in grounded theory [24] to analyze the results of round one of the Delphi study. Constant comparison of the aspects collected eventually revealed common properties, categories and eventually identified core categories. We continually checked whether new categories or concise concepts emerged. Table 1. Consolidation of List Items from Round 1 to Round 2 R1 Activity No. Items 225 Æ Merge doubles Æ Merge synonyms & word radicals 176 145 Æ Join similar context Round 2 59 The results were analyzed in three steps. First an alphabetical list of all items revealed doubles. If the descriptions of the items actually described the same aspect they were merged. In the next step the reduced list was checked for synonyms and Blended Learning: Towards a Mix for SMEs 171 word radicals. In case they revealed more doubles in the descriptions, one of the aspects was kept on the list. In the third step similar contexts or differently named aspects with an identical or similar description were summarized and designated umbrella terms. 4.6 Next Steps A ranked, consolidated list is prepared from the results of the first round. In the second round the panelists rank their “Top 20” issues, ties not allowed, out of the consolidated list of round 1. The second round is aggregated into a list of 20 items for each sub panel for the final round. This 20-item list is presented in the third round as a list with the “Top 10” with ranks from 10 to 1. All other items on the list (11-20) are equally ranked “0”. The panelists now rank their “top 10 issues” from the 20-item list. The rank is calculated by combining the percentage of selection and relative rankings by the individual participants. An approximation of the mean ranks has been produced by multiplying each percentage of mention by its first-round rank. A combined measure of the ranks in the second and third round provide a value for each item and provide the final evaluation of each item on the 20 item list. To visualize the results of round two and three, concept maps [25] are provided to enable the participants to get a quick overview of the results, although we are not using the complete process of concept mapping. The statement maps, concept maps which locate the statements in a coordinate plane of highest rank and percentage of agreement, will be prepared for each of the sub panels. 5 Expected findings This paper highlights the results from the first round of the ranking-type Delphi study. The final results are expected to show which of the characteristics from the total list of characteristics of blended learning from the first round are the most important for the Delphi panel in total. It will also provide results for the individual panels, how the selection process within the panel evolved and how the panel results differ. The final results of the study will give an indication which topics to explore towards a mix in blended learning for SMEs. Acknowledgement The work described in this paper is part of the Up2UML project [Project No. DE05-B-F-PP-146369] and is partially funded by the Leonardo da Vinci Agency / European Commission. 172 Sabine Moebs and Stephan Weibelzahl References 1. Driscoll, M.: Blended Learning: Let's get beyond the hype (2002) Retrieved April 10 2006 from http://www-8.ibm.com 2. Bonk, C.J.; Graham, C. R.: Handbook of Blended Learning. Pfeiffer, San Francisco (2006) 3. Rosenberg, M.J.: E-Learning Strategies for delivering knowledge in the digital age. McGraw-Hill (2001) 4. Hamburg, I.; Lindecke, C.; Terstriep, J.: eLearning - A chance for small and medium sized enterprises (2005) Retrieved Apr 24, 2006 from http://eleed.campussource.de/archive/2/236/ 5. Wood, D.; Watson, J.: Factors Limiting the Proliferation of E-learning within Small to Medium Sized Enterprises. In: G. Richards (Ed.): Proceedings of World Conference on ELearning in Corporate Government, Healthcare, and Higher Education 2002. AACE, Chesapeake (2002) 1049-1055 6. Daniel, B. K., Matheos, K.; McCalla G.: Blended Learning Approach for Technology Enhanced Learning Environments. (2004) Retrieved March 23 2006 from http://oltfile.qut.edu.au/ 7. Garavan, T.N. & O’Donnell, D.: eLearning in Irish Organizations? Retrieved April 14, 2006 from www.cipd.ie (2003) 8. McSporran, M. & King, C.: Blended Is Better: Choosing Educational Delivery Methods (2005) Retrieved March 20, 2006 from hyperdisc.unitec.ac.nz/ 9. Graham, C.R.: Blended learning systems: definition, current trends and future directions. In: Bonk, C.J. & Graham, C. R.: The Handbook of Blended Learning. San Francisco Pfeiffer (2006) 10. Wenger, M.S. & Ferguson, C.: A learning ecology model for blended learning from Sun Microsystems. In: Bonk, C.J. & Graham, C. R.: The Handbook of Blended Learning. Pfeiffer, San Francisco (2006) 11. Bersin, J.: What works in Blended Learning. (2003) Retrieved April 12, 2006 from http://www.learningcircuits.org/2003/jul2003/bersin.htm 12. Saunders, P. & Werner, K.: Finding the right blend for effective learning. (2004) Retrieved March 15 2006 from http://www.wmich.edu/teachlearn/new/blended.htm#move 13. Serveau, K.: Examining an Online Corporate Study Group. In: G. Richards (Ed.): Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education.. AACE Chesapeake (2004) 940-945 14. Stokes, A.: Using telemonitoring to deliver training to SMEs: a pilot study. In: Education + Training, Vol. 43, No. 6. (2001) 317-324 15. Lawless, N., Allan, J., O’Dwyer, M.: Face-to-face or distance training: two different approaches to motivate SMEs to learn. In: Education + Training, Vol. 42, No. 4/5. (2000) 16. Page, K.: A preliminary study on the current state of e-learning in lifelong learning. Luxembourg Office for Official Publications of the European Communities (2006) 17. Haeder, M.: Die Expertenauswahl bei Delphi-Befragungen. In: ZUMA How-To Reihe, Nr. 5. (2000) Retrieved October 15 2006 from www.gesis.org 18. Linstone, H.A.; Turoff, M.: The Delphi Method – Techniques and Applications. (2002) Retrieved March 10, 2006 from http://www.is.njit.edu/pubs/delphibook/ 19. European Commission: The new SME definition: User guide and model declaration. (2006) Retrieved April 24, 2006 from http://europa.eu.int/ 20. Alexander, C.: A Pattern Language. Oxford University Press, New York (1977) 21. Day J.; Bobeva M.: A Generic Toolkit for the Successful Management of Delphi Studies. In: Electronic Journal of Business Research Methods, Vol. 3, Issue 2 (2005) 103-116 22. Schmidt, R.C.: Managing Delphi Surveys using nonparametric statistical techniques. In: Decision Sciences, Vol. 28, No. 3 (1997) 763-774 Blended Learning: Towards a Mix for SMEs 173 23. Couger, J.D.: Key Human Resource Issues in IS in the 1990s: Views of IS Executives versus Human Resource Executives. In: Information & Management, Vol. 14 No.4 (1988) 161-74 24. Strauss, A., Corbin, J.: Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Sage Publications, Newbury Park London New Delhi (1990) Supporting Self-Regulated Learning in a Blended Course Giuliana Dettori, Donatella Persico Istituto per le Tecnologie Didattiche del CNR, Genova – Italy dettori@itd.cnr.it persico@itd.cnr.it Abstract. This study investigates the practice and development of selfregulated learning abilities in a blended course for trainee teachers, where traditional, face-to-face sessions alternate with online collaborative modules. Based on data emerged from an end-of-course questionnaire and on the analysis of the online interactions, the study suggests that the potential of the online CSCL component of the course was especially valuable as concerns the social aspects of SRL. The face-to-face sessions, instead, seem to contribute in particular to the development of the cognitive sphere. This study suggests that a balanced blend of presence and online activities may result synergetic from the point of view of SRL. Keywords: Self-Regulated Learning, Computer Supported Collaborative Learning (CSCL), Interaction Analysis, Blended Learning 1 Introduction Self-Regulated Learning (SRL) is a learner-directed process which transforms mental abilities into operative capabilities, in relation to a task at hand [22, 23]. It allows people to autonomously handle their own learning process and to face the needs to learn and to get updated that arise, with increasing frequency, in professional carriers and in everyday life [19]. Hence, self-regulation skills support lifelong learning by making people active learners and by favouring the transfer of knowledge and problem solving methods among different learning situations. For this reason, developing students’ self-regulatory skills is currently considered a major goal in education, and it is deemed as important as acquiring content knowledge. It is widely accepted that self-regulation entails an active and conscious control of one’s own activity from the point of view of meta-cognition, motivation and behaviour, both individual and social [22, 23]. The same author also points out that SRL entails the cyclical repetition of three phases, that is, planning or forethought, performance (which includes execution monitoring), as well as evaluation of the work done and achievements reached. Controlling cognition, motivation and behaviour throughout these three phases allows the students to take advantage of their knowledge and skills in the environment where learning takes place, progressing towards the achievement of their personal learning objectives. Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 174-185, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. Supporting Self-Regulated Learning in a Blended Course 175 Concerning the development of self-regulation, the literature indicates that some aspects, such as meta-cognitive knowledge and skills, generally improve as students get older. It is also acknowledged, however, that the acquisition of SRL competence is not automatic nor does it take place in a short time [2], but should be supported by suitable teaching and practice. Developing SRL abilities is a complex task and depends on many factors. The literature underlines the importance of creating and structuring learning environments that allow and facilitate control of the essential dimensions of learning [16]. The importance of receiving formative feedback [13] and being encouraged to reflect and revise one’s work [20] is also stressed. The literature suggests that SRL abilities need to be adapted when moving to different learning contexts [6], as learners do not seem to self-regulate in the same way on all tasks. SRL in Computer Supported Collaborative Learning (CSCL) has been investigated by several authors [1, 7, 8, 14, 19]. From these studies, it appears that CSCL environments are suited to practice SRL or even foster its development, both in online and in blended courses, because of the great emphasis given to collaborative activities and to the fact that in such courses meta-cognitive skills are often among the explicit or implicit objectives of the learning process. Moreover, CSCL is essentially based on textual interactions, and this seems to lead to a deeper reflection and involvement, since the messages exchanged are recorded in the environment and learners can access them at any time [12]. Finally, in CSCL learners are usually free to choose when and where to work on a course. This gives them a perception of freedom of choice, which is reported to encourage self-regulation [3]. The relationship between CSCL and SRL, however, has a twofold nature. If SRL can be seen as a useful side-effect brought about by CSCL, it also turns out to be one of its requirements. Effectively participating in CSCL, as a matter of fact, entails possessing a good amount of self-regulation, not only for time management and work organization but also for handling the transition from the classroom model of instruction to that of distance, collaborative learning [1]. Moreover, a number of meta-cognitive skills are necessary to take full advantage of this learning method. The relationship between SRL and CSCL is therefore quite complex and worth further investigation. The study of SRL in blended learning, is even more necessary, especially since there is not a single way to mix face-to-face and online activities [20]. Since in this case learning always takes place in two different environments (one online and one face-to face), SRL is possibly clearly put into play in more than one way; it is therefore worth investigating if, and under what conditions, the support to SRL provided by the online and face-to-face modules may result synergetic, or independent or even in conflict with each other. In order to help understand this relationship, we analysed the potential support to SRL of a blended course for teacher training. We analysed the online component of the course, by applying two complementary methods. The first consisted in asking course participants to fill in an end-of-course questionnaire (the TELESTUDENTS questionnaire developed by the TELEPEERS 1 European project). The aim of this 1 “Self-regulated Learning in Technology Enhanced Learning Environments at University Level: a Peer Review”, Grant agreement 2003-4710-/001-001 EDU-ELEARN, http://www.lmi.ub.es/telepeers/ 176 Giuliana Dettori, Donatella Persico survey was to investigate if the SRL-supporting features of the online course component had been perceived by the participants as effective and useful in regulating their own learning. The second was based on data obtained by carrying out a quantitative analysis of the messages exchanged by a group of students in two activities of the course. This small scale, more in-depth study provided us with complementary information based on the observation of learning dynamics. Finally, we considered if and to what extent the face-to-face component resulted synergetic with the online one in determining the SRL potential of the whole course. 2 The course analysed We analysed a course in Educational Technology, called TD-SSIS, run by the Institute for Educational Technology (part of the Italian National Research Council) within the Teacher Training School of the University of Genoa. The course objectives were to promote the development of instructional design competence, with special focus on the evaluation and selection of learning strategies, techniques and tools and on the implementation of educational technology in the school context. This course has been shaped differently over the past years, experiencing different blending designs. In this paper, we focus on the course run in 2005, that addressed 94 prospective teachers of all disciplines. Most students (94,4% of the respondents) did not have previous experience with online courses. The course consisted of five blended modules lasting a total of twelve weeks, with a face-to-face meeting at the beginning of each module to start up its activity. In addition, the course featured two transversal activities, respectively devoted to socialization and to meta-reflection on the course and its method, which were run completely online, in parallel to the other modules. The five blended modules consisted in: 1. 2. 3. 4. 5. a familiarisation game aimed at getting acquainted with each other and with the software environment; an activity devoted to online educational resources, including a phase of individual work and one of peer review; a role play where students were requested to spot strengths and weaknesses of a WebQuest; a case study focusing on a best practice project where several classes and schools collaborated to develop a common artefact; a conclusive activity promoting reflection on the course itself and on the learning achieved. The course adopted a social-constructivist learning model and employed Centrinity FirstClass™ as Computer Mediated Communication (CMC) platform to support collaborative activities. The main interaction modality was written and asynchronous, but in some cases chats were also used. Guidance and facilitation was provided by a team of 7 tutors plus the course coordinator. Online activities were carried out in groups of about 10 students and took place within discussion areas set up on purpose. Supporting Self-Regulated Learning in a Blended Course 177 3 Indicators of self-regulation from the questionnaire The questionnaire used to collect data on trainees’ appreciation of the SRL features was an adaptation of the TELESTUDENTS-SRL 2 . This questionnaire, which was developed based on the SRL-characterization arising from the literature, especially from the work of 22, 23], considers students’ perception and appreciation of several aspects of a learning environment: ƒ the support provided to the users in finding their way in the environment (facilities of the interface and possibilities of personalization), ƒ the support to planning the learning process (availability of planning tools like calendars and progress reports, history, indication of prerequisite, etc.), ƒ the support to execution (availability of materials in different formats, of tools for collaboration and communication, of different learning paths and levels of difficulties, of formative feedback), ƒ the support to self-evaluation (evaluation tools, possibility to compare one’s work with that of peers and with given models). For each of these areas, the possibility to control cognitive, motivational, emotional and social aspects was considered. All questions were to be graded on a 0 to 4 scale. In the analysis of students answers, we took as positive the grades above 2, which express an appreciation more than average. The data gathered with this questionnaire come from 72 respondents (out of 94 students, amounting to 76,6% of the course participants). Its main outcomes are reported and discussed below. 3.1 Cognitive and meta-cognitive aspects The students expressed their appreciation in relation to the cognitive aspects as concerns the possibility to: ƒ establish personalised learning goals: mean 2,53, sd 0,96; ƒ plan their own learning 2,16, sd 0,84; ƒ switch to a new learning strategy if needed: mean 2,60, sd 0,84; ƒ pace one’s learning activities: mean 3,44, sd 0,77; ƒ learn from home 3,63, sd 0,59; ƒ personalise the interface: mean 2,51, sd 1,19. From these data, students appear to feel fairly in control of the cognitive aspects related to planning, and in particular to take advantage of the freedom offered by the environment in terms of time and work organization. The lowest average score was given to the possibility to pace one’s learning activities. This relatively low rating was probably due to the fact that most of the planning decisions had to be negotiated with both the course designers and peers, and the learning pace was somewhat imposed by the need to collaborate. 2 http://www.lmi.ub.es/taconet/tools.php 178 Giuliana Dettori, Donatella Persico The study also investigated meta-cognitive aspects related to performance and evaluation; the students expressed as follows their appreciation for the possibilities offered: ƒ encouragement to actively participate in learning: mean 2,79, sd 0,88 ƒ help to reflect on problem solving: mean 2,80 sd 0,98; ƒ help to reflect on learning progress: mean 2,72, sd 0,95; ƒ help to recognize the achievement of learning goals: mean 2,49, sd 0,92; ƒ availability of appropriate feedback on work done: mean 2,37, sd 1. In this case as well, the data are rather positive, especially as concerns the support provided to meta-reflection. These data support the claim of several authors [10, 15] that asynchronous written communication has considerable potential for reflection since written messages are permanent, provide multiple viewpoints from group members and force people to strive for clarity. 3.2 Emotional and motivational aspects Concerning emotional and motivational aspects, the help provided by the environment was deemed valuable as to: ƒ re-establish a positive working attitude after difficulties: mean 2.67, sd 0,96; ƒ keep up a positive working attitude: mean 2,40, sd 0,97; ƒ work out strategies to keep up motivation: mean 3,43, sd 0,96. ƒ have more confidence in their own abilities: mean 2,21, sd 1,06. These data highlight a weakness of the online environment: the online help facilities provided by the CMC platform used are rather poor and in some cases the novelty of the approach generated anxiety and a sense of inadequacy. Both problems were also reported by the tutors. The first was effectively counter-balanced by a special conference devoted to technical help, where one expert was available to provide support in a very short time. The second is typical of online activities and hence unavoidable, but it can be alleviated by creating a comfortable social climate. This was done by encouraging a friendly atmosphere in the online discussion areas, that was further reinforced during the face-to-face meetings. 3.3 Social aspects The social aspects concern the ability of course participants to interact with the other members of the community in an efficient and effective way, and take advantage of such interactions to improve their learning process. These aspects include help-seeking, communication ability, effective collaboration, comparison with given models or with peers’ performance and achievements, etc. The questionnaire reported the following appreciation for the various social aspects of the online activities: ƒ working with peers: mean 3,04, sd 0,92; ƒ communicating with peers: mean 3,00, sd 0,94; ƒ comparing results with peers: mean 3,21, sd 0,84; ƒ contacting and receiving help from tutors: mean 2,68, sd 1,01; Supporting Self-Regulated Learning in a Blended Course 179 ƒ discussing work with tutors: mean 2,41, sd 1,03. According to the above data, the students’ judgement about the potential of the course to develop social SRL aspects appears particularly positive, especially concerning contact with peers. Appreciation of this aspect also emerged from several participants’ messages, where students expressed the opinion that the virtual environment had much helped to create a sense of belonging to the course community. They also pointed out that the use of the two delivering modalities had created an opportunity to interact with more course mates, in comparison with other purely faceto-face courses, in that the online collaborative activities had led them to interact with peers they did not know before and would have not thought to approach in normal conditions, and subsequently felt motivated to look for them during the face-to-face classes. 3.4 Summary of questionnaire results The data presented distinguish the three main components of SRL: the cognitive/meta-cognitive one, the emotional and motivational, and the social behaviour. This distinction derives from the literature on SRL, but it is clear from the above discussion that the borders between them are quite blurred and that the various aspects investigated within each of them are strictly intertwined. Nevertheless, it may be useful to try a rough comparison of the support perceived by the students in relation to the three components. If we compare the mean value of the above data we obtain the histogram shown in Fig.1, which suggests that the social component of SRL was perceived to be supported more than the cognitive/meta-cognitive component, which in turn was deemed to be supported more than the emotional/motivational one. This general evaluation is confirmed by the data obtained by applying content analysis to part of the messages exchanged in the course, that are presented in the following section. 4 Indicators of self-regulation from interaction analysis We resorted to interaction analysis in order to see if a different way to analyse the course would confirm or not the questionnaire data. Content analysis of interactions appears a suitable tool to this end, because it combines qualitative and quantitative analysis of messages. This technique has been increasingly used by researchers to gain insight about CSCL dynamics, taking advantage of the permanent nature of computer conference transcripts [17]. 180 Giuliana Dettori, Donatella Persico 4 3,5 3 2,5 2 1,5 1 0,5 0 cognitive and metacognitive aspects emotional and motivational aspects social aspects Figure 1. Summary of the data from the end-of-course questionnaire We analysed the messages exchanged by a group of eight students and one tutor in the work areas of Modules 3 and 4, looking for expressions that could indicate the use of SRL in some activities. Each activity lasted three weeks, for a total of six weeks and 249 messages exchanged. The sample chosen had more or less the same features of the whole cohort of students, as concerns ratio between males and females, mixture of backgrounds, average grade earned in the final assessment. Both activities were collaborative. Two coders examined separately all students’ messages of the sample. One coder was one of the course designers, while the other was an external rater. The chosen unit of analysis was the message. After coding, the inter-rater reliability was calculated. The percent agreement resulted to be 88% for the fist activity and 80% for the second one. The indicators of SRL we used are derived from the literature, checked against our experience. They can be classified along three dimensions. The first dimension has to do with the phases of the self-regulated learning process, that is, planning, monitoring and evaluating. The second has to do with the distinction between self regulation in individual work vs self-regulation in team work. The third distinguishes between cognitive or meta-cognitive aspects and emotional or motivational ones. More details about this distinction and the related indicators are reported by Dettori and Persico [9]. If a message contains reference to self-regulated actions, then we can deduce that the author of the message, having taken those actions, has practised self-regulation to some extent. For example, if one of the students proposes to his/her peers a given workplan and asks for feedback, we can interpret the message as a planning action (in that the author of the message must have done some planning to write the proposal) and also as an attempt to socialise his/her own efforts, in that he/she is asking for feedback and is not trying to impose his/her own ideas to the group. The opposite, however, can not be claimed, because if self-regulation does not emerge from students messages, this doesn’t mean that it did not occur, but simply that it was not openly expressed. For example, a student might have done some planning in individual work, but he/she might have decided that it was not relevant to tell his/her peers and tutors about those plans. In conclusion, the interpretation of the results of the study must be carried out bearing in mind that what has emerged from them may just be the “tip of the iceberg” of a wider phenomenon. Supporting Self-Regulated Learning in a Blended Course 181 The main results of the content analysis are reported in Fig. 2 to 4. The two activities had the same duration, but both the total number of messages exchanged and the percentage of messages that were found relevant to study SRL were higher in Module 4 (a case study) than in Module 3 (a role play). This can partly be explained by the nature of the two activities and by the increased familiarity among the group members, but it can also suggest that self-regulation had improved over the course. Most likely, all hypothesis contributed to determine these data. Module 3, being a role play, proposed an inherent plan (who does what) that partially relieved students from taking individual decisions on how to participate (this is confirmed by the data in Fig. 2, where indicators of planning events in Module 3 are significantly less than those of Module 4). However, Module 4 shows a generally higher concentration of SRLrelated events, therefore supporting the idea that students self-regulated their learning better in this phase, as concerns not only planning, but also monitoring and evaluation tasks. Message categorisation according to "process" model Message categorisation "individual" vs "social" 70 30 60 50 25 20 Module 3 15 Module 4 10 5 40 Module 3 30 Module 4 20 10 0 0 Planning Monitoring Evaluation Figure 2. Coding results along the categories of the process model Individual Social Figure 3. Coding results individual vs social categories along the Message categorisation: cognitive and metacognitive vs emotional and motivational 40 30 Module 3 20 Module 4 10 0 cognitive/metacognitive Emotional/motivational Figure 4. Coding results along the categories cognitive and meta-cognitive vs. emotional and motivational 182 Giuliana Dettori, Donatella Persico 5 From online to blended learning The above data mostly focus on the potential support to SRL of the online component of our blended course. What about the support provided by the whole course? Does the inclusion of some face-to-face activities increase, or decrease, or leave unchanged, students’ possibilities to practice and improve SRL? If any change is induced, what aspects are affected? At first sight we might think that the presence of a face-to-face component would decrease the global support to SRL, since students’ freedom to organize their learning activities was obviously limited by the fact that the face-to-face activities had a fixed schedule. SRL, however, is not much about being free to organise one’s learning activity, but rather to be able to make good use of the amount of freedom allowed by the learning situation. In this respect, some authors claim that total freedom in taking decisions concerning one’s own learning is not really essential for self-regulation, indeed learners need to have some minimal knowledge related to the learning activity at hand in order to be able to make informed choices [18]. Therefore, some guidance is necessary to allow learners to exercise control [11]. In the case of our blended course, the face-to-face classes at the beginning of the 5 modules had the function to start up the modules’ activity by providing a general, content-related framework for the work to be carried out online. Giving introductory classes in presence turned out to be particularly efficient, since the presentation of the relevant content knowledge and the clarification of collective doubts on the topic could be made more quickly than it would be possible online. Hence, the face-to-face component of the course contributed to the SRL support on the cognitive/metacognitive level, by providing the guidance necessary to help the students move at ease through the online activity. Moreover, the face-to-face classes also served the purpose to encourage and motivate the students who were less familiar with technology and collaboration tools, therefore determining a positive perception of this aspect in relation to the whole course. In the authors opinion, on the other hand, the online component supported the social level more than the face-to-face one. A course with about 100 students, and with a dense schedule of activities, can in fact hardly provide frequent student-centred and highly interactive activities. Since the lack of a suitable social experience appears to be a primary source of self-regulatory dysfunctions [4], the online component of the course served a purpose of great relevance to the overall quality of the learning experience. In order to better understand the relation between online and face-to-face in this course, we will consider again some of the data coming from the end-of-course questionnaire, and in particular the answers to questions aiming to distinguish between the contribution given to the course by the different learning modes. These data (Fig. 5) show that the overall course quality was positively judged by the students, and that the online component of the course significantly contributed to such quality, more than the face-to-face one. Consistently, the students found (Fig.6) that learning collaboratively in the CMC environment was more important than individual study of learning resources, which in turn was deemed more important than the face-to-face sessions. Supporting Self-Regulated Learning in a Blended Course 183 A further remark can be made about the students opinions on how much they felt responsibility for their own learning. Again, the end-of-course questionnaire provides a mean rating of 2,72 in a 0 to 4 scale (sd 1,06), which is quite satisfactory. Fig.7 also gives an insight about students perceptions of motivation improvement along the course. A significant increase in students motivation was also perceived by the tutors. In conclusion, both components appear to have played different but important roles in determining the SRL potential of the whole course. Each of them contributed to overcome what could be seen as a weakness of the other component. Moreover, it is important to recall what emerged from the students’ messages, mentioned in Section 3.3. The work online with previously unknown peers, followed by face-toface sessions where they had an opportunity to meet, had led to the creation of a warmer social atmosphere, which supported the emotional aspects of SRL more than it would have been possible in a purely online or purely face-to-face course. Overall, the balance of the online and face-to-face components was appreciated by the students (mean 2,9; sd 0,94). 4,00 3,50 3,00 2,50 2,00 1,50 1,00 0,50 0,00 4,00 3,07 2,92 3,21 2,82 3,50 2,56 3,00 2,85 3,00 2,30 2,50 2,00 1,50 1,00 0,50 the course, globally the course contents the blended metthod the face-toface component the online component Figure 5. Question “express your opinion about the quality of the course and its components (0=very low/4=very high)” 0,00 working in the CMC environment face-to-face meetings individual study of learning material Figure 6. Question “Rate the importance of the following modes of study in your learning process (0=very low/4=very high)” 4,00 3,50 3,00 2,94 3,12 2,48 2,50 2,00 1,50 1,00 0,50 0,00 in the initial phase in the intermediate phase in the final phase Figure 7. Question “Rate your motivation during the course (0=very low/4=very high)” 184 Giuliana Dettori, Donatella Persico 6 Concluding remarks The case study analysed suggests that blended courses can offer good opportunities to foster students SRL skills. We must be cautious, however, to generalise these positive data. The same outcomes can obviously not be expected from any blended course, because the variety of possible ways to blend presence and online activities affects the synergy between the two component. When these cover different parts of the course program, for instance, or the face-to-face meetings have just an organizational aim, the online activity can not take advantage of the cognitive guidance provided by a face-to-face introduction. On the other hand, having only rare face-to-face meetings or concentrating them at the beginning of the course, may fail to create the positive social atmosphere induced by the alternation between online collaboration and face-to-face interaction with peers. Further research is therefore needed to study different course structures, analysing the impact of components with different aims, organizations and lengths, as well as of different combinations of them, in order to shed further light on the relationship between SRL and blended learning. References 1. Arbaugh, J.B. (2004) Learning to learn online: A study of perceptual changes between multiple online course experiences. Internet and Higher Education 7, pp. 169-182. 2. Boekaerts, M. (1997) Self-regulated learning: a new concept embraced by researchers, policy makers, educators, teachers and students. Learning and Instruction 7 n.2 , pp. 161186. 3. Boekaerts, M. (1999) Self-regulated learning: where we are today. International Journal of Educational Research, 31, pp. 445-457. 4. Boekaerts, M., Pintrich, P. R., Zeidner, M., (2000). Handbook of self-regulation. Academic Press., San Diego. 5. Bolhuis, S. (2003) Towards process-oriented teaching for self-directed lifelong learning: a multidimensional perspective. Learning and Instruction 13, pp. 327-347. 6. Corno L. (2001). Volitional aspects of self-regulated learning. In Zimmerman B.J. , Schunk D.A. (eds.), Self-regulated learning and academic achievement: theoretical perspectives (pp. 191-225). Lawrence Erlbaum Associates, Mahwah, N.J. 7. Dettori G.; Giannetti T. & Persico D. (2005). "CMC environments supporting self-regulated learning". Proc. Internat. Conference m-ICTE 2005, Caceres (E), 7-10 June 2005, 379-383. 8. Dettori, G., Giannetti, T. & Persico, D. (2006). SRL in online cooperative learning: implications for pre-service teacher training. European Journal of Education, vol.41 (3/4), 397-414. 9. Dettori G. & Persico, D. (2007). Using Interaction Analysis to reveal Self-Regulated Learning in Virtual Communities. Proceedings of the International Educational Technology Conference 2007, Crete, Sept. 2007, LNCS, Springer, Berlin, (in print). 10.Dillenbourg P. (1999), Collaborative-learning: Cognitive and Computational Approaches. Oxford. Elsevier. 11.Jones, A. & Issroff, K. (2005) Learning technologies: Affective and social issues in computer-supported collaborative learning. Computers & Education 44, pp. 395–408. 12.Kanselaar, G., Erkens, G., Jaspers, J. & Tabachneck-Schijf, H. (2001) Essay review: Computer supported collaborative learning, Teaching and Teacher Education 17, pp. 123129. Supporting Self-Regulated Learning in a Blended Course 185 13.Kramarsky B., Zeichner O. (2001). Using technology to enhance mathematical reasoning: effects of feedback and self regulation learning. Educational Media International, 38 (2-3), pp.77-82. 14.Lynch, R. & Dembo, M. (2004) The relationship between self-regulation and on-line learning in a blended learning context. International review of Research in Open and distance learning, 5, n.2,. Retrieved from http://www.irrodl.org/content/v5.2/lynchdembo.html in Jan. 2006. 15.Palloff, R. & Pratt, K. (1999) Building learning communities in cyberspace: Effective strategies for the online classroom. San Francisco: Jossey-Bass Publishers. 16.Rosario P., Nuñez Perez J.C., González-Pienda J.A. (2004). Stories that show how to study and how to learn: an experience in the Portuguese school system. Electronic Journal of Research in Educational Psychology, vol.2, n.1, pp. 131-144, http://www.investigacionpsicopedagogica.org/revista/index.php3. 17.Rourke L., Anderson T., Garrison D.R. & Archer W. (2001). Methodological Issues in the Content Analysis of Computer Conference Transcripts, International Journal of Artificial Intelligence in Education, vol. 12, 8-22. 18.Schraw, G. (1998) Promoting general metacognitive awareness, Instructional science 26, pp. 113-125 19.Van Den Boom, G., Paas F., Van Merrienboer, J. J.G. & Van Gog T. (2004) Reflection prompts and tutor feedback in a web-based learning environments: effects on students’ selfregulated learning competence, Computer in Human Behaviour 20, pp. 551-567. 20.Vye N. J., Schwartz D. L., Brasford J. D., Barron B. J., Zech L (1998). SMART Environments that support monitoring reflection and revision. In D. J. Hacker, J. Dunlosky, A. C. Graesser (Eds.), Metacognition in educational Theory and Practice. Laurence Erlbaum Associates, Mahwah, NJ. 21.Yoon, S.-W. & Lim D. H. (2007). Strategic blending: strategic blending to improve learning and Performance. International Journal. on E-Learning, 6 (3), 475-489. 22.Zimmerman B. J. (1998). Developing Self-fulfilling cycles of academic regulation: an analysis of exemplary instructional models. In Shunk and Zimmermann (eds) Self-regulated learning. From teaching to Self-reflective practice, The Guildford Press, New York, 1-19. 23.Zimmerman B. J. (2001). Theories of self-regulated learning and academic achievement: an overview and analysis. In B.J. Zimmerman & D.A. Schunk eds., Self-regulated learning and academic achievement: theoretical perspectives. Lawrence Erlbaum Associates, Mahwah, NJ., pp. 1-37. A Study on the Impact of the Use of an Automatic and Adaptive Free-text Assessment System during a University Course Diana Perez-Marin, Ismael Pascual-Nieto, Enrique Alfonseca, Eloy Anguiano, and Pilar Rodriguez Computer Science Department, Universidad Autonoma de Madrid, C/ Francisco Tomas y Valiente, 11, Cantoblanco, 28049, Madrid, Spain {Diana.Perez, Ismael.Pascual, Enrique.Alfonseca, Eloy.Anguiano,Pilar.Rodriguez}@uam.es Abstract. Students should be given the opportunity of being assessed with computers not only with objective testing such as multiple-choice questions or fill-in-the-blanks items, but also with open-ended questions. Moreover, given the new possibilities that adaptation brings to e-learning, the benefits of incorporating adaptation into the assessment should also be tested. Recently, the automatic and adaptive free-text scorer called Willow has been developed. In this paper, we present the first study done on the impact of the use of Willow during a course by student volunteers of our home university. The type of students who use this system, how they use it, the impact of its use on their final scores and the students and teacher degree of satisfaction have been analyzed. It is concluded that students like and find useful Willow because it helps them to review concepts and, the teacher likes the idea of having a system that provides more exercises and feedback to the students. Keywords: free-text scoring, e-assessment, e-learning, Natural Language Processing 1 Introduction Assessment is essential to learn. However, in the traditional educational environment, teachers do not usually have enough time to accomplish a formative assessment or assessment for learning (to give feedback about the results of the evaluation) and summative assessment or assessment of learning (to give the numerical score as result of the evaluation) [1]. Thus, the formative aspect is sometimes neglected. As a possible solution to this problem, automatic computer based tests are being used since many decades ago, starting a new area known as Computer Assisted Assessment (CAA) with objective testing such as fill-in-the-blank exercises and Multiple Choice Questions (MCQ). However, some pedagogues consider that objective testing questions do not always measure the higher cognitive skills of the students [2]. In answer to this problem, in the mid-sixties, CAA was extended with the possibility of automatically assessing free-text answers [3]. Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 186-195, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. A study on the impact of the use of an automatic and adaptive free-text assessment 187 The advances in Natural Language Processing (NLP) have made possible a favorable progress of this field. In fact, statistical and NLP techniques, Information Extraction (IE), clustering, and integrated-approaches are being used to tackle this problem with increasingly better results [4]. However, the automatic assessment of students’ answers has not been fully solved yet, due to the complex nature of the task proposed. Many of the current systems are limited to restricted domains, need to be trained before they can be used, and do not take into account information of the students to select the questions and to modify the evaluation [5]. On the other hand, it is more and more common to find applications that are adapted to the users’ models. Moreover, the popularization of e-learning has brought a new perspective to education, in which it is not the teacher the one that controls the tutoring process, but the students who learn at their own pace, and according to their knowledge background and personal features. An Adaptive Hypermedia (AH) course presents the contents and modifies the possibilities for navigating the lessons, in a way that is adapted to the students’ models. Recently, the Willow system [6] has been developed. It is the first free-text Adaptive CAA system (ACAA) able to process students’ answers written in Spanish or in English in an automatic and adaptive way. It uses techniques from the NLP and AH fields. Besides, it is also able to generate from the students’ answers, their conceptual model (i.e. the structure of concepts and relationships among them). In this paper, we present a study on the impact of use of Willow when it has been used by a group of students in our home university. In particular, the type of students who use this system, how they use it, the impact of its use on their final scores and the students’ and teacher degree of satisfaction will be analyzed. It is concluded that students like and find useful this kind of systems because it helps them to review concepts (particularly in the days before the exam) and, teachers like the idea of having a system that provides them more exercises and feedback to the students. The article is organized as follows: Section 2 describes the Willow system; Section 3 details the experiment carried out, focusing on the most relevant items under study (type of students, how they use the system, scores achieved and degree of satisfaction); and, finally Section 4 ends with the main conclusions and lines of future research. 2 Willow Willow is an automatic and adaptive free-text CAA (ACAA) system1. It gets as input a short answer (one or two paragraphs) written by a student in Spanish or in English in free text (without templates or any kind of restriction) and a set of correct answers written by the teachers in free text or references (again without templates or any kind of restriction) and produces as output: a numerical score, the processed answer and the correct answers of the teachers. Figure 1 shows an example of its interface when asking a question to the student. 1 Available on-line at http://orestes.ii.uam.es:8080/ateneaAdaptativa/jsp/loginAtenea.jsp. 188 Diana Perez-Marin et al. Fig. 1. A question delivered by the system according to the settings provided. The main aim of Willow is not to substitute the teacher but to allow the students to have more training available before exams. Its core idea is to compare the student’s answer to a set of correct answers called the references. The more similar a student’s answer is to the references, the higher the score the student achieves. The references can be written by several teachers (each teacher writes a reference per question, so that finally, there are at least three different references per question to cover as much as possible paraphrasing in the students answers) or they can be the best students answers of the last year training. The system is underpinned by a combination of Natural Language Processing (NLP) and Adaptive Hypermedia (AH) techniques. The NLP techniques are included in the wraetlic NLP tools 2 , including stemming (to transform each word to each canonical form, e.g. cars to car), closed-class words removal (to ignore meaningless words such as prepositions or determiners), term identification (to group multiwords and get a better knowledge of the students’ concepts, e.g. to consider OperatingSystem instead of Operating and System as two independent words), Word Sense Disambiguation (WSD) (to identify the sense in which a polysemous word is being used) and Latent Semantic Analysis (LSA) (to capture the semantic similarity between the words) [7]. The comparison between the student’s answers and the references is performed by the statistical module called ERB. This module uses a modified version of the Bleu algorithm [8]. This algorithm does not require any training or special processing to the texts. It accepts as input the student’s answer and the references in plain text and produces as output a score of the similarity between them. However, the texts can be processed before entering the ERB module with the techniques selected in the 2 Available at www.ii.uam.es/˜ealfon/eng/download.html A study on the impact of the use of an automatic and adaptive free-text assessment 189 configuration step in order to make them more comparable. Besides, the score given by ERB can be combined with the similarity score given by the LSA module [7]. The optimum combination found is stemming, removal of closed-class words, LSA and statistical techniques achieving up to 56% average Pearson correlation between the scores given by the teachers and Willow’s scores to the same set of questions. The AH techniques comprise the management of static students’ models including a set of features such as language or age. Teachers are asked to choose which features should be considered in each new collection of questions and, to write several versions of the statement and references for each question according to the features chosen. That way, Willow is able to present the right statement of the question and assesses it using the right versions of the references for each student’s model. A dynamic procedure that selects the next question to ask depending on how the student has answered the previous questions and the difficulty level of the questions have also been incorporated in Willow. Each question has been associated a difficulty level from 0 (easy) to 2 (difficult) by the teacher. Next, each student is assigned a zero level of difficulty in all the topics of the course. During the assessment session as the students answer questions of different topics chosen according to their difficulty levels, the values are modified to adjust the level of difficulty of the questions to the level of knowledge that each student has in each topic addressed in the collection. In particular, students are promoted to a higher level of difficulty in a topic when they pass a certain percentage (whose value is configurable) of the questions in the level in which they are. Besides, when they fail a certain percentage (also configurable) of the questions, they will be dropped to the lower level (if they have not passed all the questions of that level and they are not in the lowest level). Besides, to foster reflective thinking students are not permitted just to give an answer in blank. Moreover, whenever a student fails a question, it is not directly provided the teachers’ references. Instead, a set of clarification questions is presented: − The first clarification question (CQ) just asks for more information about the question (to distinguish if the student knew the answer to the question but did not bother to answer it or, indeed, if the student did not know the answer). − The second CQ asks about a certain concept that is related to the question. − The third CQ is a yes/no question in which the system presents a statement extracted from the references and the students have to say whether it is true or false. There is an automatic negation module that is in charge, of randomly changing the verb to a negative form or using the antonyms of certain adjectives, so that students do not learn that they just have to always say that the statement is true as it is copied from the references. A sample set of clarification questions would be as follows: Willow’s question What is an operating system (OS)? Student’s answer It is a process with threads. Please, explain your previous answer more. Willow’s 1st CQ Student’s answer It is the first application executed in the computer. Please, explain more about application. Willow’s 2nd CQ Student’s answer The OS is an application that serves as interface between the hardware and the user. Is it true that Unix and Windows are examples of OSs? Willow’s 3rd CQ Student’s answer Yes. 190 Diana Perez-Marin et al. To sum up, once the answer has been introduced in Willow, it is applied a set of NLP techniques so that it can be compared against the references that have also been processed by the same set of references. Next, if the question has been passed, a new question is presented of the same level of difficulty if the student has not passed the percentage of questions necessary to go up to a higher level of difficulty. On the other hand, if the question is failed, the set of clarification questions starts and the student is asked until s/he passes the question or, the question is marked as failed to be asked in the future. 3 Experiment During the course 2006-2007, an experiment was carried out with a group of students of the Operating Systems subject of the Telecommunications Engineering career. The goal of the experiment was to study how students and, which type of students use Willow during the course when they are given free access to the system, accessible from any computer connected to Internet at any time. The spirit of the experiment was of blended learning. That is, to complement the traditional teaching with the new technologies (in this case, the use of Willow after class). Besides, to avoid that students think that it was extra homework, the use of Willow was voluntary and individual. In fact, the motivation given by the teacher to the students was that it will count positively in the final score of the students who were between failing and passing the subject. Additionally, we also explained the students that the questions introduced in the system were gathered from previous year exams and, that they contain the references (correct answers) written by the teachers. Willow was set up to use stemming, removal of closed class words and ERB. LSA was not used because although it provides a higher correlation, it also needs a big corpus to train the system and, we did not have any available at the time of experiment. Nevertheless, the improvement in the correlation when using LSA is quite low (54% average Pearson correlation without LSA and 56% average Pearson correlation with LSA). The Operating System course in Willow consists of 5 topics, with 4 questions per topic. The teacher only chose for this course, the language as feature. Thus, each student had to register in the system to give the information about his or her personal data and language. Next, we asked the students to try to complete 4 questions per week (i.e. a topic per week) and, finally to fill in a satisfaction questionnaire that was also on-line. 3.1 Type of students From the 59 students enrolled in the course, 24 logged into the system (i.e. 40.7%) summing a total of 172 answers provided to the system. From these 24 students, it can be considered that 9 (37.5%) completed the use of Willow as they finished at least half of the total number of answers in the system. We expected this value (less than 50% of the students completing 10 questions or more) as in a preliminary A study on the impact of the use of an automatic and adaptive free-text assessment 191 questionnaire in which we asked students how many hours they study Operating Systems per week, all of them answered less than 5 hours as they had too much work with other subjects. Therefore, we were unsure whether they would do any activity at all, that was not compulsory. In fact, as one of the goals of the study was to analyze which type of students are willingly to use automatic and adaptive free-text scorers, at the beginning of the experiment, we fixed two features to classify the students of the class: skill and interest. Both of them were taken in a range from 0 (complete lack of this feature) up to 1 (complete existence of the feature). Skill was measured according to the numerical scores given by the teacher in the exam of the course (i.e. a student who achieves the maximum score in the exam will have 1 in skill). Interest was measured according to the number of questions answered in Willow (i.e. a student who has answered the 20 questions will have 1 in interest). Fig. 2.A plot of the skill and interest of the students who did not complete the use of Willow. Fig. 3.A plot of the skill and interest of the students who completed the use of Willow. 192 Diana Perez-Marin et al. Once the experiment finished, we implemented an automatic classifier of types of students that scales the number of questions answered in Willow and the score achieved in the exam to the values of skill and interest. These values are plotted for the students who did not complete the use of Willow in Figure 2 and for the students who completed it in Figure 3. It can be seen that students who used Willow had a higher level of skill than students who did not complete the use of Willow. However, it should also be noted the cases in which students with low interest has been able to achieve a high score and, the case of students with high interest but unable to achieve a high score despite their effort. Regarding students who failed the last year the subject and had to course it this year again, we were unsure about whether they would use the system more than other students (as they wanted to make sure that this year they will pass the subject) or, if they would not use the system (as they are students with low level of interest). In total, 9 students from the 59 were in this situation (retakers). From them, only 2 used Willow. That is, they did not show a higher trend of using this kind of support systems than non-retakers students. 3.2 How the students use the system Willow keeps a detail log of all the actions that the students perform with the system. Table 1 shows a summary of the logs gathered. A session is each time that a student has logged in the system. It can be seen how only six students have used an end-ofsession condition different of just closing the application and, in average, they have answered eleven questions per session. It is also interesting to observe how these students have exploited the Willow’s personalization features as nearly all of them have changed the text area size and the font size. Table 1. Summary of the logs gathered in the experiment of the use of Willow. A study on the impact of the use of an automatic and adaptive free-text assessment 193 Fig. 4. Number of questions answered week by week from October 2006 to January 2007 by the students who used Willow. Another interesting fact is how the number of sessions before important exams dramatically increases as can be seen in Figure 4. Week 1 corresponds to the first week of the experiment (October 16th to 22nd). It is considered that the number of students who entered the system the first weeks is just because they were curious about the system they have been told in class. Next, the second peak in the graph is around the weeks 5-7 given that the first exam was the November 28th in the seventh week of course. After that, despite students have been advised to complete at least four questions per week to have a complete training before the final exam, there is less use of the system until the end of December - beginning of January when students are again reviewing, in this case, for the final exam at the end of January. Thus, it can be seen how the system is used by the students to get more training before their exams. 3.3 Impact of Willow’s use First of all, it is important to keep in mind that Willow is not a system to teach new concepts but just to review concepts (formative assessment). In order to validate this supporting function, we have examined the results achieved by the students (both using and not using Willow) in the final exam of the course. From the 59 students enrolled in the course, 51 (86.44%) took the final exam. If we just focus on the students who completed the use of Willow, all of them took the final exam. Furthermore, the two highest scores in the class are among the students who completed the use of Willow and, the retakers who used Willow, passed the subject while most of the retakers who did not use Willow failed again the subject. 194 Diana Perez-Marin et al. In fact, the mean score (in a scale from 0 to 10) of the 42 students who took the exam without having used Willow is 4.6 (1.6 deviation), whereas the mean score in the same scale of the 9 students who took the exam having used Willow is 5.2 (1.5 deviation). That is, in average, students who reviewed with Willow got more than half point in the final score than students who did not review with Willow. 3.4 Degree of satisfaction During all the experiment, students have stated how much they like the idea of using an automatic system to reinforce concepts even when they did not have much time to use it. In the middle of the course, we asked the whole class to fill in a voluntary questionnaire in which two items about the use of Willow were included (I have used Willow and I think that... and I have not used Willow because...). From the sixteen students who answered the questionnaire, six students gave an answer to the first item. In general, all of them agreed that Willow is a very interesting system that gave them the possibility of getting more training before their exams. Furthermore, students did not only state that they think that it is useful for them but, also, for other subjects and, that they would recommend it to their colleagues. Ten students answered that they have not used Willow yet and, the reason was unanimous: lack of time as the amount of compulsory work from the practical part of the subject is too high. However, they were all also aware that they need to study more, if they want to pass the theoretical part of the subject. At the end of the course, we passed another voluntary questionnaire to gather the final conclusions of the students about the use of Willow and, how they have enjoyed using it. From the five students who answered the questionnaire, it can be stated that: − All the students have Internet at home and they prefer to use Willow from their home. − They regard the dynamic adaptation in the order of the questions as very useful and, like that the difficulty of the questions starts in a low level. − All the students find very easy to use Willow and useful to review concepts. − 80% of the students say that the system lives up their expectative, feel satisfied with its use and would recommend its use (even in other subjects). − The other 20% claimed that they cannot say that, because they do not like to be presented again and again questions that cannot pass. This feeling of repetition could be explained because there were only 4 questions per topic. − All the students highlight the possibility of having immediate feedback as very helpful. − The teacher has also supported the use of Willow during all the course. In fact, he considers very interesting the opportunity of giving extra training to their students with a system that also gives them immediate feedback. A study on the impact of the use of an automatic and adaptive free-text assessment 195 4 Conclusions and future work It can be concluded that students and teachers enjoy and find helpful the use of an automatic and adaptive free-text scoring system. Furthermore, that all students who used Willow to review (answering more than 12 questions) took the final exam and passed it with higher scores than students who did not use it. However, it has also been observed how students tend to dedicate less than 5 hours per week to study Operating Systems at home. Therefore, despite they consider that they have to review more, and we recommended them to answer 4 questions per week, most of them only used massively the system in the days before exams. In the future, as it has been concluded that using Willow helps the students, we intend to make compulsory the use of Willow in one subject (not related to computers so that we also studied the impact of Willow on non-technical students) and to force them to organize their study so that they do not leave all the review in the previous days of the exam. Finally, it is also planned to improve the classification of students to analyze if the use of Willow can motivate and keep engaged a student with low interest (albeit at the beginning s/he did not want to use the system) up to a medium or high level interest. Acknowledgments. This work has been sponsored by Spanish Ministry of Science and Technology, project no. TIN2004-03140. References 1. Gardner, J., ed.: Assessment and Learning, SAGE Publications (2006) 2. Birenbaum, M., Tatsuoka, K., Gutvirtz, Y.: Effects of response format on diagnostic assessment of scholastic achievement. Applied psychological measurement 16 (1992) 3. Page, E.: The imminence of grading essays by computer. Phi Delta Kappan 47 (1966) 4. Valenti, S., Neri, F., Cucchiarelli, A.: An overview of current research on automated essay grading. Journal of Information Technology Education 2 (2003) 319–330 5. Perez, D.; Alfonseca, E. & Rodriguez, P. Can computers assess open-ended questions?, Revista Novática 183 (2006) 6. Perez-Marin, D., Alfonseca, E., Rodriguez, P.: On the dynamic adaptation of computer assisted assessment of free-text answers. In proceedings of the Adaptive Hypermedia Conference, LNCS 4018, Springer-Verlag (2006) 7. Perez, D., Gliozzo, A., Strapparava, C., Alfonseca, E., Rodriguez, P., Magnini, B.: Automatic assessment of students’ free-text answers underpinned by the combination of a bleu-inspired algorithm and latent semantic analysis. In: Florida Artificial Intelligence Research Society conference, FLAIRS-2005, American Association for Artificial Intelligence (AAAI) Press (2005) 8. Papineni, K., Roukos, S., Ward, T., Zhu, W.: Bleu: a method for automatic evaluation of machine translation. Research report, IBM (2001) A Wiki-Based Collaborative Learning Design and Its Effect in Secondary Math Studies Liming Zhang1, Chan Lam Wong2 1 Faculty of Education, University of Macau, Macao SAR, PR China 2 PuiChing Middle School, Macao SAR, PR China 1 lmzhang@umac.mo, 2pcmsricky@yahoo.com Abstract. This paper promotes use of wiki in order to facilitate the collaborative learning in secondary math studies. An experiment was presented in this paper to analyze the correlation between the students’ posts in the wiki environment and their math achievement. A questionnaire was designed after the experiment to collect the opinions from the student view. The experiment results are positive and promising. It is concluded that after school collaborative practice towards the homework could be one of the solutions to improve the students’ math learning. Keywords: Collaborative Learning, Wiki Technology, Math Learning 1 Introduction The concept of collaborative learning has been widely studied and advocated throughout the professional literature [5]. Collaborative learning is a method of teaching and learning in which students make a team and work together to explore the answer for a significant question or create a meaningful project [11]. The students are responsible for one another's learning as well as their own. Thus, the success of one student helps other students to be successful. Math is one of the most difficulty subjects in the secondary studies. Many researchers have been seeking for new pedagogies to improve the student performance in math [10]. Collaborative learning is one of the methods emphasized in the studies. Benefits from a collaborative environment include [3]: z Interpersonal Development. Students learn to relate to their peers and other learners as they work together in groups. This can be especially helpful to students who have difficulties with communication skills. They can benefit from structured interactions with others. z Actively Involving Students in Learning. Each member has opportunities to contribute in small groups. Students are apt to take more ownership of their material and to think critically about related issues when they work as a team. z More Opportunities for Personal Feedback. Because there are more exchanges among students in small groups, students receive more personal feedback about their ideas and responses. Such feedback is often not possible in large-group Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 196-205, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. A Wiki-Based Collaborative Learning Design and Its Effect in Secondary Math Studies 197 instruction, in which one or two students exchange ideas and the rest of the class listens. Proponents of collaborative learning also claim that the active exchange of ideas within small groups not only increases interest among the participants, but also promotes critical thinking [5]. Some researchers concluded that there is persuasive evidence that collaborative teams achieve at higher levels of thought and retain information longer than students who work quietly as individuals [6]. The shared learning gives students opportunities to engage in discussion, taking responsibility for their own learning, and thus become critical thinkers [14]. However, some mathematicians doubt whether collaborative learning could be used in math classrooms without considering the major purpose of math instruction, which is to help students learn to think mathematically [4]. They think that under the collaborative learning class, teachers cannot share their insights with students or warn them against pitfalls, and at the same time students cannot learn enthusiastically from their teacher [16]. How to take the advantage of the collaborative learning and meanwhile to release the mathematicians’ mind of doubt becomes one of our research topics. Carroll [2], in his classic paper, hypothesized that learning should be viewed as a ratio of time spent in the learning activity to the actual time needed to learn. Many empirical investigations [10] indicated that higher rates of learning are related to the amount of time students spend actively engaged in their academic tasks. Coursework may represent a way to increase academic learning time. Researchers have also investigated the effects of math coursework on math achievement, examining the impact of the numbers (amount) and the types (content) of math courses [8]. Sebring [13] noted that the quantity of math coursework had an effect on math achievement. Gamoran [18] reported that students performed better on standardized math achievement tests if they took more math courses; and math coursework particularly differentiates math achievement among students. It comes to us that a collaborative learning based coursework might be an effective way to improve the student math performance. A wiki-based collaborative learning is studied in this paper. In the learning environment, a teacher guided coursework was designed which is close to the homework in the style. The coursework could be done after school in a collaborative way in accompanying with the traditional teaching in the classroom. The homework still should be done by the students individually. The advantage of the collaborative learning is emphasized in this design, and at the same time the students’ active learning time is effectively extended. The experiment showed positive result as expected. A wiki is a serve-based collaborative tool that allows any authorized user to share information by writing new content, adding to existing content, and editing or commenting on content. There are some applications of wiki technology reported in the areas, including student writing [9], knowledge management systems [12], and graduate course work design [1]. Few applications could be found in the secondary math studies. This paper aims to investigate an effective and efficient learning strategy as a supplement in the traditional secondary math studies. 198 Liming Zhang, Chan Lam Wong 2 Overview of Wiki Technology 'Wiki' is the Hawaiian word for 'quick'. Leuf and Cunningham [7] define a wiki as "a freely expandable collection of interlinked Web pages, a hypertext system for storing and modifying information-a database where each page is easily editable by any user with a forms-capable Web browser client". A wiki enables documents to be written very collaboratively, in a simple markup language using a web browser. A single page in a wiki is referred to as a "wiki page", while the entire body of pages, which are usually highly interconnected via hyperlinks, is "the wiki". A wiki is actually a very simple, easy-to-use user-maintained database for creating, browsing and searching information [15]. The technical attributes of wiki technology includes [12] : z Wikis run over the World Wide Web and can be supported by any browser. z The technology is governed by an underlying HTTP protocol that determines client and server communication. z Wikis are able to respond to both requests for data (GET) and data submission (POST), in a given Web front, based on the HTTP protocol. From a functional dimension, three major attributes of wiki technology were offered in the same paper: z Any member of a wiki community can edit any pages in that community's wiki Web site. The editing of wiki pages does not require any additional functions in the Web browser. z Members of a wiki community can build and develop "meaningful topic associations" by creating numerous links between wiki pages. The linking of wiki pages is simple to do. z Originally the technology was not meant to engage casual visitors; rather, the technology was designed to enable users to regularly update the wiki pages in a collaborative fashion, thereby continuously changing the nature of the wiki Web site. Ward Cunningham [19] who developed the first wiki web site shares his design principles at his web site, including the following: • Open - Should a page be found to be incomplete or poorly organized, any reader can edit it as they see fit. • Incremental - Pages can cite other pages, including pages that have not been written yet. • Organic - The structure and text content of the site are open to editing and evolution. • Mundane - A small number of (irregular) text conventions will provide access to the most useful page markup. • Universal - The mechanisms of editing and organizing are the same as those of writing so that any writer is automatically an editor and organizer. • Observable - Activity within the site can be watched and reviewed by any other visitor to the site. A Wiki-Based Collaborative Learning Design and Its Effect in Secondary Math Studies 199 Wiki has the following features and functionalities [17]. • Ability to edit a page • View recently changed pages • ‘History’ feature to view or roll back to previous versions of a page Most wikis also provide the user with a set of navigation or utility tools such as [17]: • ‘Discuss’ offline changes or proposed changes to a page • A ‘backlinks’ function (view all the pages that link to the page currently displayed) • Search function McKiernan [17] lists eleven wiki advantages including the capability of incorporating the knowledge of experts, peers and other professionals, the means to annotate evolving issues, and a way of increasing communication efficiency and productivity. He also lists seven wiki disadvantages including installation of wiki engine software, Issues of legal liability, privacy, reputation, and security, and Content accuracy, balance, comprehensiveness, and consistency, and reliability. 3 Research Method An experiment was presented in this paper to analyze the correlation between the students’ posts in the wiki environment and their math achievement. The math topic factorization was selected as the research topic partly because it is easy to be described using the wiki tool. One class in Form 8 was selected as the experiment group. There were 54 students in the class. The wiki environment was designed and developed by their math teacher himself. A questionnaire was designed and distributed to the students after the experiment. There were 14 questions in total, including the frequency and the purpose of the students to visit the wiki environment. All the students completed the questionnaire. 3.1 Wiki Environment Design and Development There were two stages in this study. In the first stage, a wiki environment was designed and developed. MediaWiki is selected as a development tool in this research. MediaWiki is a free server-based software, that is licensed under the GNU General Public License (GPL). It is an extremely powerful, scaleable software and a featurerich wiki implementation, that uses PHP to process and display data stored in its MySQL database [15]. Pages use MediaWiki's wikitext format, so that users without knowledge of XHTML or CSS can edit them easily. When a user submits an edit to a page, MediaWiki writes it to the database, but without deleting the previous versions of the page, thus allowing easy reverts in case of vandalism or spamming. MediaWiki can manage image and multimedia files, too, which are stored in the filesystem. A typical wiki allows any visitors to add, remove, edit and change content without the need for registration. In order to manage and evaluate the contents posted by the students in the study, part resource program codes were amended. Only the students 200 Liming Zhang, Chan Lam Wong who registered in the course were allowed to login to the proposed wiki environment. Other visitors can browse the wiki pages, however, editing and posting are not permitted. Figure 1 illustrates the first wiki page in the proposed wiki environment. Fig. 1. The first wiki page in the wiki environment The contents of the wiki pages were designed based on some important and difficult problems discussed in the class. In most cases, the teacher provided certain exercise topics in the wiki pages, which were closely related to their homework. The students could either try to answer the questions if they though they were able to, or post further questions to discuss with other students. If a solution was not correct, or not perfect, it could be further amended by other students or by the teacher, if necessary. The students worked on the wiki pages individually at home through Internet. The discussions and the whole solutions posted in the wiki pages could help them to finish their homework by themselves. Figure 2 shows the solutions of some math exercises posted by the students. The exercise topics posted by the teacher in the wiki pages were not the homework itself; however, they were in the similar styles. Though neither the students were requested to provide the solutions to the posted exercises, nor the answers were recorded as part of the students’ final scores, there were still quite a lot students getting involved in this learning activities. The main reasons claimed by the students later included that the discussed topics were closely guided by the teacher and were helpful for them to finish their homework. A Wiki-Based Collaborative Learning Design and Its Effect in Secondary Math Studies 3.2 201 Experiment Design The second stage of the study was to investigate the correlation between the students’ participating in the wiki based collaborative learning and their math achievement. Pearson correlation was used to test whether there was significant correlation between the above two parties. Fig. 2. An example of the wiki pages posted by the students The students’ math achievement in the semester priori to the experiment was used as pretest result. The experiment lasted two and a half months. A posttest was conducted after the experiment. A questionnaire was also provided to the students to collect their opinions towards the collaborative learning experience. 4 Data Analysis and Discussion The students’ pretest and posttest data were analyzed in two aspects. i) The correlation between the students’ pretest achievement and their posting amount in the wiki environment. Table1 illustrates the statistics results of the students’ posting amount based on their pretest achievement. Table 2 shows the correlation results conducted by SPSS. 202 Liming Zhang, Chan Lam Wong Table 1. Statistics results of the students’ posting amount and their pretest achievement Pretest Scores Range (0~100) 30-40 40-50 50-60 60-70 70-80 80-90 90-100 Number of the Students 2 2 4 17 13 7 9 Posts in the wiki environment Frequency 34 27 38 540 229 397 681 Average Frequency Standard Deviation 17.00 13.50 9.50 31.76 17.62 56.71 75.67 22.63 3.54 6.66 40.17 31.23 54.68 83.59 Table 2. Pearson Correlation between the students’ posting amount and their pretest achievement Posting Amount Pearson .369(**) Correlation Sig. (2-tailed) .006 N 54 ** Correlation is significant at the 0.01 level (2-tailed). Pretest Achievement The result from Table 2 shows that there is a significant correlation between the students’ pretest achievement and their posting amount in the wiki environment. It indicates that the better the students’ math performance before, the more their interesting in posting to the wiki environment. ii). The correlation between the students’ posting amount in the wiki environment and the students’ posttest achievement. Table 3. Statistics results of the students’ posting amount and their posttest achievement Posttest Scores Range (0~100) 30-40 40-50 50-60 60-70 70-80 80-90 90-100 Number of the Students 0 2 1 12 10 12 17 Posts in the wiki environment Frequency 0 15 16 240 247 462 966 Average Frequency Standard Deviation 0 7.5 16 20 24.7 38.5 56.8 0 2.5 0 32.7 38.2 43.2 66.3 Table 4. Pearson Correlation between the students’ posting amount and their posttest achievement Posting Amount Posttest Achievement Pearson Correlation .355(**) A Wiki-Based Collaborative Learning Design and Its Effect in Secondary Math Studies Sig. (2-tailed) N 203 .008 54 ** Correlation is significant at the 0.01 level (2-tailed). Table 3 illustrates the statistics results of the students’ posting amount based on their posttest achievement. The result from Table 3 shows the students’ achievement in the posttest was improved. Table 4 shows the correlation results conducted by SPSS. It shows that there is a significant correlation between the students’ posting amount in the wiki environment and the students’ posttest achievement. It indicates that the more the students’ posting amount in the wiki environment, the better their posttest achievement. In the study, the students participating to the experiment were classified into three groups based on their posting amount to the wiki environment. They were named as frequently posting group, question-asking group, and silent group. z In the frequently posting group, the students actively got involved into the learning. Most posts submitted by them were solutions to the questions, though sometimes the answers were not fully correct. Most students in this group were top students in the class. z In the question-asking group, the students usually posted some questions requiring additional explanation to the solutions already posted by other students. It showed that those students could not fully understand the solutions posted by others, but they would like to make an effort towards understand them. The math level of the students in this group was mostly in the middle of the class. z There were a few students who were relatively quiet in the wiki environment during the whole experiment period. They just posted very few questions. Most of the students’ math level in this silent group was low in the class. In the questionnaire, we investigated the effects of the proposed wiki based collaborative learning from the students’ view. z The students from the frequently posting group claimed that they were very excited when using the wiki because there was a kind of competition between them. They would like to be the one who posted the correct answer first. But they also claimed that they didn’t think that the wiki had much effect on their math achievement. z The students from the question-asking group claimed that although they didn’t post as many as the ones from the frequently posting group, they got much benefit from the question answers posted by others. It helped them to finish their homework more smoothly. They claimed that the wiki was of great positive effect on their posttest math achievement. z The students from the silent group claimed that although they didn’t post many in the wiki, they followed others’ posts in the whole experiment period. It helped them to understand the math contents better. They also admitted that their posttest performance was better than before and they were more confident in math after the experiment. 204 5 Liming Zhang, Chan Lam Wong Conclusion The promising experimental result through wiki environment shows that the collaborative learning seems to be one of the effective and efficient ways to improve the students’ math achievement in secondary school. The web-based after school collaborative environment could become a bridge connecting the knowledge students learned in class with their own practice. The design of the after school collaborative learning in accompanying with the traditional teaching in the classroom could not only release the mathematicians’ doubt, but also emphasize the advantage of the collaborative learning itself. This is in consistency with the learning theory constructivism. The experiment was designed and conducted in high school in Macao with the specific topic factorization. Nevertheless the method and results may be extended to topics and students of wide range and without geographical restriction. References 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Bold, M., “Use of Wiki in Graduate Course Work”, Journal of Interactive Learning Research, 17, 1, (2006) 5-14. Carroll, J.B. ,A model of School Learning, Teachers College Record, 64, (1963) 723-733. Educational Broadcasting Corporation, Concept to Classroom – Workshop cooperative and collaborative learning, Available at: http://www.thirteen.org/edonline/concept2class/coopcollab/index.html. (20/03/2007). Ding, M., Li, X., Piccolo, D. and Kulm, G. “Teacher Interventions in Cooperative Learning Math Classes”, The Journal of Educational Research, 100, 3, (2007) 162-175. Gokhale, A.A. , “Collaborative Learning Enhances Critical Thinking”, Journal of Technology Education. 7, 1, (1995). Johnson, R. T., and Johnson, D. W., “Action Research: Cooperative Learning in the Science Classroom”, Science and Children, 24, (1986) 31-32. Leuf, B. and Cunningham, W., “The wiki way. Quick collaboration of the web”, Addison-Wesley, (2001). Ma, X. and McIntyre, L.J., “Exploring the Differential Effects of Math Courses on Math Achievement”, Canadian Journal of Education, 28,4, (2005) 827-852. McPherson, K., Wikis and Student writing, Teacher Librarian, 34, 2, (2006) 70-72. Melia, M. C. and Rosenberg, M.S., “Effects of Cooperative Homework Teams on the Acquisition of Math Skills by Secondary Students with Mild Disabilities”, Exceptional Children, 60,6, (1994) 538-548. Panitz, T., “A Definition of Collaborative vs Cooperative Learning”, Deliberations on Learning and Teaching in Higher Education, Available at: http://www.city.londonmet.ac.uk/deliberations/home.html (19/03/2007). Raman, M., Ryan, T., and Olfman, L., “Designing Knowledge Management Systems for Teaching and Learning with Wiki Technology”, Journal of Information Systems Education, 16, 3, (2005) 311-320. Sebring, P.A., “Consequences of Differential Amounts of High School Coursework: Will the New Graduation Requirements Help? ”, Educational Evaluation and Policy Analysis, 9, (1987) 258-273. Totten, S., Sills, T., Digby, A., and Russ, P., “Cooperative learning: A guide to research”, New York: Garland, (1991). A Wiki-Based Collaborative Learning Design and Its Effect in Secondary Math Studies 21 22 23 24 25 205 Wikipedia--the Free Encyclopedia, Available at: http://en.wikipedia.org/wiki/Main_Page. (20/03/2007). Wu, H., “The Math Education Reform: Why You Should Be Concerned and What You Can Do”, The American Mathematical Monthly, 107, (1997) 946-954. McKiernan, G., “Wikis: Disruptive Technology for Dynamic Possibilities”, Available at: http://nl.wikipedia.org/wiki/Hoofdpagina (20/03/2007). Gamoran, A., “The Stratification of High School Learning Opportunities”, Sociology of Education, 60, 3, (1987) 135-155. Cunningham, W., “Wiki Design Principles”, Available at http://c2.com/cgi/wiki?WikiDesignPrinciples (20/03/2007). Implementation of an E-Learning System – Learning Design and Learning Objects Management and Evaluation through Standardization of Resources Hugo Rego, Tiago Moreira, Francisco José Garcia University Of Salamanca, Plaza de la Merced s/n 37008 Salamanca, Spain hugo_rego05@yahoo.com,thm@mail.pt,fgarcia@usal.es Abstract. AHKME e-learning system main aim is to provide a modular and extensible system with adaptive and knowledge management abilities for students and teachers. This system is based on the IMS specifications representing information through metadata. Metadata is used to satisfy requirements like reusability, interoperability and multipurpose. The system provides authoring tools to define learning methods with adaptive characteristics, and tools to create courses allowing users with different roles, promoting several types of collaborative and group learning. It is also endowed with tools to retrieve, import and evaluate learning objects based on metadata, where students can use quality educational contents fitting their characteristics, and teachers can of use quality educational contents to structure their courses. The metadata management and evaluation play an important role in order to get the best results in the teaching/learning process. 1 Introduction In the era we live in, information and its transformation into knowledge became crucial that’s why standardization became important because it provides a semantic representation of knowledge through ontologies in which concepts are clearly and unambiguously identified, also providing a set of semantic relation types which allow representing meaning by linking concepts together [15][5]. In order to develop our system, Adaptive Hypermedia Knowledge Management Elearning system (AHKME), we had to choose from among several existing technological standards and specifications the ones that best fit our needs in order to reach our objectives of multipurpose, independence of the learning domain, reusability and interoperability of resources and courses, since several standards and specifications have been developed to structure pedagogical contents and to allow the characterization of a wide variety of learning environments [20]. AHKME uses both knowledge representation and management based on metadata described by specifications where teachers can create, evaluate, import and retrieve quality educational resources, and students aquire knowledge through quality learning objects (LO), as well as through the most appropriate learning technique based on their Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 206-216, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. Implementation of an E-Learning System 207 characteristics and learning style, available learning activities, instructional design and LO characteristics. The goals of AHKME and main contributions are: the LO management and quality evaluation; Use of the IMS specifications to standardize all the resources of the platform; the interaction of subsystems through the feedback between them allowing the platform to adapt to the students and teachers characteristics and to new contexts. In this paper we will start to present an analysis of e-learning current approaches and a standards and specifications comparative analysis to find the best to develop our system. Then we describe the system and focus on applications that provide the LO management and evaluation. Finally we present the conclusions and future work. 2 Current Approaches In order to start developing AHKME we have analysed several e-learning systems, though this analysis has to be an evaluative methodical process of several aspects, where we have to consider the system’s main target, if it is an enterprise or educational environment and a training or educational implementation, the students’ pre-knowledge and IT skills, the platform’s support in terms of resources and infrastructures, and the criteria basis should be quality. An e-learning platforms/systems analysis can be done empirically, which is a more technical analysis considering the platforms’ features, tools and potentialities, in terms of availability and quality (9-available, x–not available). So, we have analysed Blackboard, WebCT, IntraLearn, Angel, Atutor, Moodle, Sakai and DotLRN like shown on Table 1, in order to identify strong points and weaknesses, so we could try to use them in the development of our platform [4][7][10]. From Table 1 we found that almost all the platforms have good administrative and communication tools, compliance with standards, high implementation level and good documentation, though they have some problems regarding LO management, sharing, reusability and quality evaluation, resources adaptation to the students’ characteristics, among others. From the comparison of commercial and freeware/open-source platforms we found that the commercial ones have more difficulty integrating with other systems and supporting different kinds of pedagogies and of course the costs. These weaknesses are mainly traduced in problems regarding interoperability, reusability and quality of resources, learning domain independence, extensibility of the platforms, meeting some of our goals already presented. In order to solve these problems we have decided to develop an open source platform focused on these issues. 3 Standards and Specifications Comparative Analysis One of the biggest difficulties of e-learning systems/platforms is in structuring content and information using nowadays pedagogical models so they can reach a wider range of educational systems and obtain a greater teaching quality, that is why standards were developed for. As we know the use of standards have become very useful because automatically makes everything cross systems providing this way common 208 Hugo Rego, Tiago Moreira, Francisco José Garcia knowledge. The use of a standard helps to achieve more stable systems, reduces the development and maintenance time, allows backward compatibility and validation, increases search engine success, among many other know advantages [18]. Table 1. Analysis of e-learning systems Platforms Technical Aspects Interoperability/integration Standards and specs. Compliance .LRN Sakai ATutor Open Source Angel IntraLearn WebCT BB Comercial Moodle Tools/Features 9 9 9 9 9 9 9 9 (1) (2) (3) (6) (1) (1) (2) (3) (4) (5) (1) (6) (1) (2) (1) (6) (6) 9 9 9 9 Extensibility x x x x Adaptation and Personalization 9 9 9 9 9 9 9 Interface custom. and personalization x 9 9 9 9 9 9 9 Choose interface language x Students previous knowledge x x x x x x x x Courses and resources adaptability x x x x x x x x Administrative 9 9 9 9 9 9 9 9 Student manage. / monitor. tools 9 9 9 9 9 9 Database access mechanisms x x 9 9 9 9 9 9 9 9 Admin. workflows quality & functio. 9 9 9 9 9 9 Tracking users x x Resources Management 9 9 9 9 9 9 9 9 Content authoring and editing 9 LOs and other types of content mng. x x x x x x x 9 9 9 9 9 9 9 Templates to aid on content creation x 9 LO search and indexation x x x x x x x Evaluation of quality of resources x x x x x x x x 9 Learning objects sharing/reuse x x x x x x x Communication 9 9 9 9 9 9 9 9 Forum 9 9 9 9 9 9 9 Chat x 9 9 9 9 Whiteboard x x x x 9 9 9 9 9 9 9 9 Email 9 Audio and video streaming x x x x x x x Evaluation 9 9 9 9 9 9 9 9 Self assessments 9 9 9 9 9 9 9 9 Tests 9 9 9 9 Inquiries x x x x Costs H H H H N N N N 9 9 9 9 9 9 9 9 Documentation SCORM-(1);IMS-(2);AICC-(3);LRN-(4);Section 508-(5);Some IMS Specifications-(6);High–H;None–N Having detected the main problems of current e-learning approaches, we’ve started to analyse several aspects of several standards and specifications to choose the one(s) that would best fit our needs, like presented in Table 2. Implementation of an E-Learning System 209 We have analyzed the IMS Specifications [12], AICC, SCORM [1] and Dublin Core [8], and from this analysis we’ve chosen the IMS specifications, since they allow most of the issues we’ve analyzed and that we consider important for our goals. Table 2. Standards and specifications comparative analysis IMS 9 9 9 9 9 9 Features Metadata Learner Profile Content Packaging Q&T Interoperability DR Interoperability Content structure Content Communication Learning Design Simple Sequencing Accessibility XML Bindings RDF Implementation handbooks Learner registration 9 9 9 9 9 9 9 AICC SCORM 9 9 9 9 9 9 9 Dublin Core 9 9 9 9 9 9 9 9 4 AHKME Description AHKME, presented on Figure 1, is an e-learning system that is divided in four different subsystems: Learning Object Manager and Learning Design subsystem, Knowledge Management subsystem, Adaptive subsystem and Visualization and Presentation subsystem. Fig. 1. AHKME’s structure 210 Hugo Rego, Tiago Moreira, Francisco José Garcia These subsystems were structured taking into account the following: First we have the LO creation and management process, followed by the course creation process through learning design (LD). In parallel the Knowledge Management subsystem evaluates LOs’ and courses’ quality that then pass through an adaptive process based on the students’ characteristics to be presented to them. These subsystems are web applications that were developed using Asynchronous JavaScript And XML (AJAX) to create interactive web applications [3], HTML and CSS for the Web pages’ design, PHP (PHP: Hypertext Preprocessor) to run on server side to make the manipulation of XML files, Javascript to implement mechanisms in Web forms and .NET and C to implement several software agents. All of these subsystems use XML as standard for file storage and knowledge representation which allows content interchange between different applications and platforms, facilitating content publishing [6]. All LO management and LD subsystem tools include a mechanism that packages the generated information, at the level of LOs, courses as well as at the level of the adapted courses. We will now focus on the parts of this system that provide the LO management and evaluation through metadata. 4.1 LOM and Learning Design Subsystem The Learning Object Management and Learning Design subsystem is mostly used by teachers where they can develop, search, retrieve, import and analyze resources and also create courses. We will now describe the tools and features of this subsystem and how they are related with the IMS specifications. 4.1.1 LO Manager The Learning Object Manager tool, presented on Figure 2, allows teachers to define/create metadata to describe LOs through IMS Learning Resource Metadata specification which is based on the IEEE Learning Object Metadata (IEEE LOM) [11] standard that allows the knowledge management and representation through LOs. Fig. 2. LO Manager Architecture Implementation of an E-Learning System 211 This tool structures all information in a XML manifest, that gathers all XML files with LO’s metadata and resources facilitating the learning contents’ management. It allows the creation of general metadata that can be associated to any LO and the creation of packages with their manifests and LOs and their storage in a MySQL database, enabling their management. All these files and packages pass through a validation process to check their conformance with the specifications, and the communication between tools and databases is based on the XML Document Object Model. The LOs are not static in the repositories, but they are in constant evaluation by the KM subsystem. After the evaluation it may be needed to change the LOs’ cataloguing or the way they are related with others to get better associations letting these changes to be reflected until the packages’ creation, taking into account the user’s wishes, granting a higher level of flexibility. The main advantage of using the IMS specification for LOs is that through the association of descriptive tags, we can better index, find, use and reuse them. 4.1.2 LD Editor The subsystem’s part referring to the Learning Design (Figure 3) provides a tool where teachers can create and structure courses using level A of the IMS LD specifications defining courses’ activities, sequence, users’ roles, student or staff, and metadata. It generates a XML manifest gathering all the courses’ XML files, LOs, metadata and resource files. With the usage of XML files information can be reused in the construction of other courses facilitating the learning information portability [13]. Fig. 3. Learning Design Tool architecture This tool also provides the package creation with courses integrating them in a data repository, to reach a more efficient management and, also, communicates with the 212 Hugo Rego, Tiago Moreira, Francisco José Garcia KM subsystem to evaluate the courses created. After the evaluation this tool allows the courses’ restructuring allowing the user to interact in the learning design process. 4.2 Knowledge Management Subsystem We’ve decide to create a subsystem which main objective is to assure quality to the information in the platform through the LOs’ and courses’ evaluation, to get the best courses and the best resources to reach to the best learning/teaching process. We will now describe how the learning object evaluation is processed. 4.2.1 LO Evaluation The quality of learning resources is becoming an aspect with great importance on elearning environments, since when e-learning systems emerged there was a massive production of resources without taking into account their quality. Vargo, et. al states that a systematic evaluation of learning objects must become a valued practice if the promise of ubiquitous, high quality Web-based education is to become a reality [19]. To archive an optimal evaluation of LOs, it’s necessary to consider quality criteria from different kind of categories, for this reason the criteria with the respective weight presented on Table 3 were proposed [16]. Table 3. Evaluation criteria categories and matching with the IMSLRM educational category Eval. criteria categories Weight Psychopedagogical 30% Didactic-curricular 30% Technical-aesthetic 20% Functional 20% IMSLRM Ed. elements intended end user role; typical age range; difficulty learning-resource type; context; typical learning time; description semantic density; language interactivity type; interactivity level Description Criteria that can evaluate, for example, if the LO has the capacity to motivate the student for learning; Criteria to evaluate if the LO helps to archive the unit of learning objectives, etc; Criteria to evaluate the legibility of the LO, the colors used, etc; Criteria to evaluate LOs accessibility among other aspects to guarantee that it doesn’t obstruct the learning process; The final evaluation is the sum of all category classifications multiplied by their weight and has following rating scale: 0=not present; 1=Very low; 2=Low; 3=Medium; 4=High; 5=Very High. With these criteria, we’re developing two different tools to evaluate LOs’ quality. One tool allows teachers and experts to analyze, change and evaluate LOs through a Web application and after the individual evaluation, all the persons involved gather in an on-line forum to reach to the LO final evaluation [16].The other tool is an intelligent agent that automatically evaluates LOs which architecture is shown in Figure 4. Implementation of an E-Learning System 213 Fig. 4. Architecture of the agent The agent starts to import the LO to evaluate and others already evaluated, then applies data mining techniques (decision trees) to the LO educational characteristics defined in the IMSLRM specification to calculus its final evaluation and agent stores it on a database and on the annotation element described in the specification. For this evaluation we matched the IMSLRM educational category elements and the evaluation criteria categories like presented on Table 3. We have just considered the educational category because it has almost all LOs’ technical and educational aspects we found important. With these two tool LOs are constantly being availed of their quality, playing an important role in the reusability of the LOs for different contexts. 5 Integration with Other Systems The objective of integration with other systems is to give an opportunity for a LMS to benefit from this Learning Design Back-Office system, as well as to give a FrontEnd to AHKME as presented on Figure 5. Fig. 5. Presentation Architecture 214 Hugo Rego, Tiago Moreira, Francisco José Garcia Our objective is to benefit from all the LMSs’ strong points already mentioned on our analysis adding the tools we have developed by merging/integrating systems being possible depending on the LMSs’ integration tools. For example if you have an open source system it can be directly integrated or if you have Blackboard it can be done through building blocks. 6 AHKME LOM Vs Similar Tools We have also done an analysis of key features of some metadata tools along with AHKME’s LO Manager (LOM Editor [14], ADL SCORM [2], Reggie [17] and EUN [9]). To make this analysis we have defined a set of tasks like the ones described on Table 4 and tested if the different tools supported them. The analysed tools provide functionalities for meeting specific requirements like XML validation and metadata files creation, lacking important issues like: educational orientation, by not providing a list of available educational metadata; require that the person who edits metadata must know XML; functionalities regarding the user’s needs to characterize several learning environments; resources’ management. 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 EUN AHKME LOM Creation of new metadata files Modification of data in metadata files Support any educational metadata standard, specification Modification of structure of metadata files Validation in terms of data values Validation of structure of metadata Support of the XML Packaging of LOs metadata Evaluation of LOs metadata LO Search and Indexation Allow metadata document management LOM Editor ADL SCORM Task Reggie Table 4. Comparative analysis between AHKME LOM tool and similar tools 9 9 So, AHKME LOM distinguishes itself from the others by introducing an abstraction level to the user from the technical aspects in terms of the XML language and is more focused on the user needs, by facilitating the metadata annotation of the LO through a metadata automation process and the search and retrieval of the LO, for the user to reuse the LO in another scenarios. Because of AHKME’s LO quality evaluation, the user may choose the best LOs that best fit his educational scenario. Implementation of an E-Learning System 215 7 Conclusions In this article we have presented how AHKME uses metadata for learning resource management and evaluation through the IMS specifications, which use combine potentialities of metadata and XML, providing LOs’ description through metadata allowing them to be catalogued, localized, indexed, reused and interoperable. The main AHKME contributions are: the LO management and quality evaluation; the usage of the IMS specifications to standardize all the platforms’ resources in order to reach learning components interoperability and compatibility; All subsystems interaction through feedback allowing the platform to adapt to the students and teachers characteristics and to new contexts; Being a multipurpose system it can be applied to several kinds of matters, students and learning strategies in both training and educational environments; As modular and open source system allows developers to add new modules and extend the system or integrate it with already develop elearning systems and tools; through knowledge management the content has continuous evaluation, granting quality to all resources in the platform for teachers and students to use. In terms of future work, we will include in the learning design tool, the level B of the IMS LD specification that allows the inclusion of properties and general conditions. In the adaptive subsystem we will add some functionality according to the IMS Question and Test Interoperability and Enterprise specification. In the knowledge management subsystem we will add the feature of course quality analysis, through the development of a standardization knowledge model and evaluation tools. Acknowledgments. This work has been partly financed by Ministry of Education and Science as well as FEDER KEOPS project (TSI2005-00960). References 1. ADL: Sharable Content Object Reference Model (SCORM)® 2004 3rd Edition - Overview Version 1.0. Advanced Distributed Learning (2006) 2. ADL SCORM Metadata Generator (2005): http://www.adlnet.org 3. AJAX: Asynchronous JavaScript And XML (2007). http://www.w3schools.com/ajax/default.asp, 4. ANGEL (2005): http://www.angellearning.com/ 5. Berners-Lee, T., Hendler, J., Lassila O.: The Semantic Web. Scientific American, 284(5), pp.34-43 (2001) 6. Bray, T., Paoli, J., and Sperberg-MacQueen, C.M.: Extensible Markup Languaje (XML) 1.0, 3rd Edition, W3C Recommendation (2004) 7. Colace, F., De Santo, M, Vento, M.: Evaluating On-line Learning Platforms: a Case Study, HICSS’03 – Hawaii International Conference on System Science (2002) 8. Dublin Core Metadata Initiative (2005): http://dublincore.org 9. EUN (2005): http://www.en.eun.org/menu/resources/set-metaedit.html 10. Graf, S., List, B.: An Evaluation of Open Source E-Learning Platforms Stressing Adaptation Issues. ICALT 2005 - The 5th IEEE International Conference on Advanced Learning Technologies (2005) 216 Hugo Rego, Tiago Moreira, Francisco José Garcia 11. IEEE LTSC Working Group 12: Draft Standard for Learning Object Metadata. Institute of Electrical and Electronics Engineers, Inc. (2002) 12. IMS Specifications, IMS Global Learning Consortium, Inc. (2004) http://www.imsglobal.org/specifications.cfm 13. Koper, R., Olivier, B., Anderson, T.: IMS Learning Design Information Model - Version 1.0 Final Specification. IMS Global Learning Consortium, Inc. (2003) 14. LOM Editor (2005): http://www.kom.e-technik.tu-darmstadt.de/~abed/lomeditor 15. Mendes, M.E.S., Sacks, L.: Dynamic Knowledge Representation for e-Learning Applications. In Proceedings of the 2001 BISC International Workshop on Fuzzy Logic and the Internet, FLINT'2001, Memorandum No. UCB/ERL M01/28, USA, Aug., 176-181 (2001) 16. Morales, E., García, F. J., Moreira, T., Rego, H., Berlanga, A.: Units of Learning Quality Evaluation. In SPDECE 2004 Design,Evaluation and Description of Reusable Learning Contents. Proceedings (Guadalajara (Spain), October 20-22, 2004). CEUR Workshop Proceedings Vol. 117. http://ceur-ws.org/Vol-117. ISSN 1613-0073 (2004) 17. Reggie Metadata Editor (2005): http://metadata.net/dstc 18. Totkov, G., Krusteva, C., Baltadzhiev, N.: About the Standardization and the Interoperability of E-Learning Resources. CompSysTech’2004 - International Conference on Computer Systems and Technologies (2004) 19. Vargo J., Nesbit J.C., Belfer K., Archambault A., Learning object evaluation: computermediated collaboration and inter-rater reliability. International Journal of Computers and Applications, 25 (3) (2003) 20. Wiley, D.: Connecting learning objects to instructional design theory. In D. Wiley (Ed.): The Instructional Use of Learning Objects,10 May 2003 (2003) Visualisation of Learners’ Contributions in Chat Conversations Stefan Trausan-Matu1,2, Traian Rebedea1, Alexandru Dragan1, and Catalin Alexandru1 1 “Politehnica” University of Bucharest, Department of Computer Science and Engineering, Splaiul Independetei nr. 313, Bucharest, Romania 2 Research Institute for Artificial Intelligence of the Romanian Academy Calea 13 Septembrie nr.13, Bucharest, Romania trausan@cs.pub.ro, traian@createit.ro, alexd18@yahoo.com, cata@ew.ro Abstract. In this paper is presented a novel dialogistic, socio-cultural perspective and an associated software tool, which provide structured visualisation and analysis means of Computer Supported Collaborative Learning chat conversations. The implemented tools use knowledge-based techniques and are based on Bakhtin’s dialogistic paradigm. They visualize the threading of topics and utterances in the conversation and the contributions of the participants in collaborative learning during instant messenger chats. Natural language processing based on the WordNet lexical ontology and semantic distances are used for detecting topics in the chat and their threading. The experiments with the developed application were performed with students at a course on Human-Computer Interaction in Bucharest Politehnica University. Keywords: Computer Supported Collaborative Learning, Dialogism, Chat Conversations, Ontologies, Natural Language Processing 1 Introduction In recent years, in conjunction with the omnipresence of the Internet and to the increasing number of collaborative tools like discussion forums and instance messaging (chat conversations), Computer Supported Collaborative Learning (CSCL) became an extending and promising way of learning on the Internet, which could supplement traditional classroom learning. However, CSCL needs particular supporting tools, for example for tackling and reviewing complex dialog threads in collaborative learning in small groups using chat conversations. The paper describes such a tool, that offers visualisation means to analyse the threading of dialog topics and the contribution of each learner in a chat conversation. A theoretical background for developing tools for supporting CSCL using chat conversations is the socio-cultural paradigm, stating that knowledge is socially built in communities [7] (including the case of small groups of students that learn together). Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 217-226, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 218 Stefan Trausan-Matu et al. This new paradigm is imposing itself not only due to technology advances but also because the individual cognition perspective of classical artificial intelligence (stating that knowledge should be considered as being in the mind of individuals) did not fulfil all its expectations [4,14]. However, knowledge-based technology, combined with natural language processing, has some important applications (e.g. in text mining) and we should not throw away its potential facilities. Therefore, the approach presented in this paper is integrating both the knowledge-based (ontology-based), cognitive paradigm and the socio-cultural one. Learning paradigms have also changed in a similar way, from Computer-Assisted Instruction and Intelligent Tutoring Systems to Computer-Supported Collaborative Learning (CSCL) [4,7]. As a consequence, learning is now conceived as discourse building, as Sfard remarked: “rather than speaking about ‘acquisition of knowledge,’ many people prefer to view learning as becoming a participant in a certain discourse” [6]. The way learning is considered has implications on the nature of the computer tools designed to support it. For example, the tools described in this paper, which offer the possibility of visualising the discourse in chat conversations, are based on Bakhtin’s dialogistic theory [1,2], which may be seen as extending Vygotsky’s sociocultural ideas [12]. Knowledge-based processing techniques and the lexical ontology WordNet (http://wordnet.princeton.edu) are used for the identification, delimitation and visualisation of the inter-animation of the voices of the learners. In addition, an assessment of the competence of each learner is provided. There are chat environments for CSCL containing facilities like whiteboards and explicit referencing. Such an environment is ConcertChat [3], used in this paper. There are also applications that use natural language processing for abstracting (e.g. speech acts identification [9] and summarization [10]) or knowledge extraction from chats and forums. However, these facilities are limited, and one assumption of the research whose results are presented here is that the limitations are due to the neglecting of the socio-cultural paradigm. The experiments for validating the developed application were performed with students from the final year at the Computer Science Department of the Bucharest Politehnica University, at a course on Human-Computer Interaction. For performing the chat conversation, the ConcerChat was used. The paper continues with a section introducing the socio-cultural and Bakhtin’s dialogism paradigms. The third section discusses the knowledge-based ideas used in the present approach. The next section contains the description of the visualisation tools. The paper ends with conclusions and references. 2 A Dialogical, Socio-Cultural Paradigm of Learning The socio-cultural paradigm is based on the work of the Russian psychologist Lev Vygotsky, who emphasized the role of socially established artefacts in communication and learning [12]. Mikhail Mikhailovici Bakhtin brought a lot of details to the ideas of Vygotsky, analysing the role of language and discourse, and especially of speech and dialog. Bakhtin focuses on the idea of dialogism, making it a fundamental philosophical category, named dialogistic: “… Any true understanding is Visualisation of Learners’ Contributions in Chat Conversations 219 dialogic in nature.” [11]. Moreover, Lotman considers text as a „thinking device” [13], determining that: “The semantic structure of an internally persuasive discourse is not finite, it is open; in each of the new contexts that dialogize it, this discourse is able to reveal ever new ways to mean” [2]. In forums and chat conversations, group knowledge arises in discourse and is preserved in linguistic artifacts, whose meaning is co-constructed within group processes [5], and has a dual nature. Communities of voices, in parallel to the trend towards unity, have an additional differential, unmerged, character: “The intersection, consonance, or interference of speeches in the overt dialog with the speeches in the heroes’ interior dialogs are everywhere present. The specific totality of ideas, thoughts and words is everywhere passed through several unmerged voices, taking on a different sound in each” [1]. This dual nature of community and individuality of voices is expressed by Bakhtin also by the concept of polyphony, that he considers the invention and one of the main merits of Dostoevsky novels [1]. The relation of discourse and communities to music was remarked also by Tannen: “Dialogue combine with repetition to create rhythm. Dialogue is liminal between repetitions and images: like repetition is strongly sonorous” [8]. In chat conversations, different voices are obvious recognized. However, starting from Bakhtin’s ideas, in our approach the concept of voices is not only limited to the number of participants in the chat. A voice is, from this perspective, something said by a participant in a given moment and that it may be reflected in many subsequent utterances. Also, each utterance may contain an unlimited number of voices. 3 Knowledge-Based Text Processing Ontologies like WordNet or FrameNet (http://framenet.icsi.berkeley.edu) are very successful inheritors of knowledge representation research in artificial intelligence. They are semantic networks or frame structures built starting from human experience and, in fact, they are ways of sharing experience. Any collaboration using natural language, any discourse needs to start from a common vocabulary, a shared ontology. The word “ontology” is used in philosophy to denote the theory about what is considered to exist. Any system in philosophy starts from an ontology, that means from the identification of the concepts and relations considered as fundamental. Ontologies capture fundamental categories, concepts, their properties and relations. One very important relation among concepts is the taxonomic one, from a more general to a more specific concept. This relation may be used as a way of “inheriting” properties from the more general concepts (“hypernyms”). Other important relations are “part-whole” (“meronym”), “synonym”, “antonym”. Viewing knowledge bases as ontologies determines important advantages for developers of knowledge-based systems. First of all, an ontology is developed as a coherent framework for the reality and therefore it facilitates knowledge acquisition and machine learning. A new concept is easy to add in such a framework by finding one or some more general concepts and defining some differences between the new concept and the more general ones. 220 Stefan Trausan-Matu et al. Ontologies are very important in text mining. For these kind of applications they offer the substrate for semantic analysis and, very important, the possibility of defining a measure of semantic closeness, based on the graph with concepts from ontologies as nodes and their relations as arcs. This semantic closeness is very important in text analysis for example in the retrieval of texts that do not contain a given word, but they contain a synonym or a semantically related word. 4 Visualization of Users’ Competences The approach presented here integrates Bakhtin’s socio-cultural ideas with knowledge-based natural language processing for the visualisation of the contributions of each learner. The procedure consists in the identification of the topics discussed in the chat, the separation of the contributions of each participant to a topic (the voices) and, eventually, the measurement and visualisation of these contributions. 4.1 Identification of Chat Topics The chat topics are identified in several ways in the present approach. A first method id to detect the list of concepts (words) that appeared most frequently in the conversation, by using statistical natural language processing methods. Accordingly, the importance of a subject is considered related to its frequency in the chat. The first step in finding the chat subjects is to strip the text of irrelevant words (stop-words), text emoticons (e.g. “:)”, “:D”, and “:P”), special abbreviations used while chatting (e.g. “brb”, “np”, and “thx”) and other words considered of no use at this stage. Fig. 1. A fragment of a chat for a Human-Computer Interaction course, using the ConcerChat facilities of referencing The resulted chat text is then tokenised and each different word is considered as a candidate concept in the analysis. For each of these candidates, WordNet is used for finding synonyms. If a concept is not found on WordNet, mistypes are searched. If successful, the synonyms of the suggested word will be retrieved. If no suggestions are found, the word is considered as being specific to the analyzed chat and the user is Visualisation of Learners’ Contributions in Chat Conversations 221 asked for details. The last stage for identifying the chat subjects consists of unifying the candidate concepts discovered in the chat. In addition to the above method for determining the chat topics, a surface analysis technique is used. Observing that new topics are generally introduced into a conversation using some standard expressions such as “let’s talk about email” or “what about wikis”, a simple and efficient method is used for deducing the topics in a conversation by searching for the moment when they are first mentioned. The process of identifying a pattern in an utterance is done using the synset for each word that has already been extracted from WordNet. This technique will be improved in a future version of the application by using machine-learning methods for detecting the patterns specific to the introduction of new topics. Another option is to consider the extension of the simple patterns described above to more complicated parsing rules. 4.2 The Graphical Representation of the Conversation The graphical representation of the chat was designed to permit the best visualization of the conversation, to facilitate an analysis based on the polyphony theory of Bakhtin, and to maximize the straightforwardness of following the chat elements. For each participant in the chat, there is a separate horizontal line in the representation and each utterance is placed in the line corresponding to the issuer of that utterance, taking into account its positioning in the original chat file – using the timeline as an horizontal axis. Each utterance is represented as a rectangle aligned according to the issuer on the vertical axis and having a horizontal axis length that is proportional with the dimension of the utterance. The distance between two different utterances is proportional with the time passed between the utterances. Of course, there is a minimum and a maximum dimension for each measure in order to restrict anomalies that could appear in the graphical representation due to extreme cases or chat logging errors. The relationships between utterances are represented using coloured lines that connect these utterances. The explicit references that are known due to the use of the ConcertChat software are depicted using blue connecting lines, while the implicit references that are deduced using the method described in this paper are represented using red lines. The utterances that introduce a new topic in the conversation are represented with a red margin. The graphical representation of the chat has a scaling factor that permits an attentive observation of the details in a conversation, as well as an overview of the chat. The different visual elements determined by our application – such as utterances in the same topic, topic introducing utterances and relationships between topics – can be turned on and off in the graphical representation by use of checkboxes. At the bottom of the graphical representation of the conversation, after the line corresponding to the last participant in the chat, there is a special area that represents the importance of each utterance, considered as a chat voice, in the conversation (see figure 2). How this importance is determined is presented in a further section. 222 Stefan Trausan-Matu et al. Fig. 2. The threads of references in the chat 4.3 Discovering the Implicit Voices Considering each chat utterance as being a voice that has a certain importance in the conversation, it is obvious that each utterance generally contains more than a single voice, as it includes the current voice and probably at least one referring voice. As we are working with ConcertChat transcript files, we acknowledge the voices that are explicitly pointed out by the chat participants during the conversation, using the software’s referencing tool. Nevertheless, because users are seldom in a hurry or simply not attentive enough, part of the utterances do not have any explicit references. Thus, it is necessary to find a method for discovering the implicit references in an utterance; in this way, we shall identify more relationships between the utterances in the chat. The method proposed here is similar to the one presented above for determining the introduction of new chat topics. We are using another list of patterns that consists from a set of words (expressions) and a local subject called the referred word. If we identify that an utterance matches one of the patterns, we firstly determine what word in the utterance is the referred word (e.g. “I don’t agree with your assessment”). Then, we search for this word in the predetermined number of the most recent previous utterances. If we can find this word in one of these utterances, then we have discovered an implicit relationship between the two lines, the current utterance referring to the identified utterance. We have also implemented two empirical methods, which provide very good results when utilizing any chat software. One of these empirical methods is based on the following fact: if between three utterances there are two explicit relationships from the first to the second and from the second to the third and the second utterance is a short agreement or disagreement, then between the first and the third utterance there exists an implicit relationship. For example, consider the following example, Visualisation of Learners’ Contributions in Chat Conversations 223 where there are explicit references between A and B, respectively B and C, it is clearly we have an implicit relationship between A and C. In the last utterance, we have influences from both A and B: A – I think wikis are the best (…) B – I disagree (…) C – Maybe we should talk about them anyway REF A REF B 4.4 Determining the Strength Value of an Utterance Starting from existing references within the analysed conversations, both those explicit, offered by the used chat environment, as well as those implicit determined by the program using the previously presented methodology, one could assemble a conversation graph. This graph may be used both for determining the strength value of each utterance in the chat considered as a separate voice, as well as for emphasizing certain subjects (threads) of the conversation. The importance of an utterance in a conversation can be calculated through its length and by the number of key (important) words. Another approach was also investigated: an utterance is important if it influences the subsequent evolution of the conversation. Using this definition as a starting point, we may infer that an important utterance will be that utterance which is a reference for as many possible subsequent utterances. Even if this approach could be extended to include the types of subsequent references (implicit or explicit, agreements or disagreements), in the present case we have preferred a more simplistic approach, without making allowances for the types of references to the utterance. Consequently, the importance of an utterance can be considered as a strength value of an utterance, where an utterance is strong if it influences the future of the conversation (such as breaking news in the field of news). When determining the strength of an utterance, the strength of the utterances which refer to it is used. Thus, if an utterance is referenced by other utterances which are considered important, obviously that utterance also becomes important. As a result, for the calculation of the importance of every utterance, the graph is ran through in the opposite direction of the edges, as a matter of fact in the reverse order of the moment the utterance was typed. Utterances which do not have references to themselves (the last utterance of the chat will certainly be one of them) receive a default importance – taken as the unit. Then, running through the graph in the reverse order of references, each utterance receives an importance equal to that of the default plus a quota (subunit) from the sum of the importance of the utterances referring to the current utterance. Another modality to calculate could be 1 plus the number of utterances that refer to the present utterance, but this choice seemed less suitable. By using this method of calculating the importance of an utterance, the utterances which have started an important conversation within the chat, as well as those 224 Stefan Trausan-Matu et al. utterances which begin new topics or mark the passage between topics, are more easily emphasized. If the explicit relationships were always used and the implicit ones could be correctly determined in as high a number as possible, then this method of calculating the importance of a voice would be successful. 4.5 Assessing the Competencies of the Learners in the Conversation In order to determine the competences of the chat users, we first searched the most important topics in the analyzed chat conversation. The generated graphics evaluate the competences of each user, starting from the list of subjects determined as explained above and using other criteria such as questions, agreement, disagreement or explicit and implicit referencing. The graphics are generated using a series of parameters like: implicit and explicit reference factors, bonuses for agreement, penalties for disagreement, minimum value for a chat utterance, penalty factors for utterances that agree or disagree with other utterances as these utterances have less originality than the first ones. Fig. 3. The evolution of the competence degree During the first step of the graphics generation, the value of each utterance is computed by reporting it to an abstract utterance that is built from the most important concepts in the conversation determined as described above. When constructing this utterance, we take into account only the concepts whose frequency of appearance is above a given threshold. Then, all the utterances in the chat are scaled in the interval 0 – 100, by comparing each utterance with the abstract utterance. The comparison is done using the synsets of each word contained in the utterance. Thus, this process uses only the horizontal relations from WordNet. An utterance with a score of 0 contains no words from the concepts in the abstract utterance and an utterance with a score of 100 contains all the concepts from the abstract utterance. On the Ox axis the graphics hold all the utterances in the chat and on the Oy axis the value attributed to each participant in the conversation, representing each user’s Visualisation of Learners’ Contributions in Chat Conversations 225 competence (see figure 3). Accordingly, for each utterance, at least the value of a user competence is modified – the value for the user that issued that utterance. For each utterance in the chat, the values of the users’ competences are modified using the following rules: 1) the user that issued the current utterance receives the score of the utterance, eventually downgraded if that utterance is an agreement or disagreement in relation to a previous utterance (in order to encourage originality); 2) all the users that are literally present in the current utterance are rewarded with a percentage of the utterance value, considering that they have some merit in the value of this utterance, as being mentioned in the text of the utterance encourages us to think so; 3) the issuer of the utterance explicitly referred to by the current utterance is rewarded if this utterance is an agreement and is penalized if the utterance is a disagreement; 4) the issuer of the utterance explicitly referred to by the current utterance that is not an agreement or a disagreement, will be rewarded with a fraction of the value of this utterance; and 5) if the current utterance has a score of 0, the issuer will receive a minimum score in order to differentiate between the users that actually participate in the chat and those who do not. All the percentages and all the other factors used for computing the competence of each user are used as parameters of the process and can be easily modified in the application interface. The process described above builds competence function graphics for each participant in the chat. At the start of the process, each user has a null competence. It should be mentioned that the competence of a user is not a strictly increasing function, as users are penalized for utterances that are in disagreement with the other users’ opinions. 5 Conclusions The paper presents an application that visualizes the voices (following Bakhtin’s ideas) of the participants on forums or chat conversations, similarly to music scores. In addition, some other diagrammatic representations are used for viewing the influence of a given speaker’s voice. The application may be used for inspecting what is going on and in what degree learners are implied in a forum discussion or a chat conversation. Moreover, the competence of each participant may be measured, that means that learners may be assessed in collaborative learning on the web. The application uses the WordNet ontology. Knowledge acquisition for concepts that are not present in this ontology is provided through dialogs with the user of the analysis system and by caching the results. Natural language technology is used for the identification of discussion topics, for segmentation and for identifying implicit references. Further work will consider more complex semantic distances (than only synonymy). Machine learning techniques will be used for the identification of 226 Stefan Trausan-Matu et al. discourse patterns. New rules for the identification of implicit links are now under development. Acknowledgments. The authors wish to express their appreciation to the members of the Virtual Math Teams research project at Drexel University, whose voices are present in different ways in the paper. The research presented here has been partially performed under a Fulbright Scholar post-doc grant (awarded to Stefan TrausanMatu), the EU-NCIT EU excellence centre and the CNCSIS project K-Teams. Any opinions, findings, or recommendations expressed are those of the authors and do not necessarily reflect the views of the sponsors. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. Bakhtin, M.M., Problems of Dostoevsky’s Poetics, Ardis, (1973) Bakhtin, M.M., The Dialogic Imagination: Four Essays, University of Texas Press, (1981) Holmer, T., Kienle, A., Wessner, M. “Explicit Referencing in Learning Chats: Needs and Acceptance,” in Innovative Approaches for Learning and Knowledge Sharing, First European Conference on Technology Enhanced Learning, EC-TEL 2006, Nejdl, W., Tochtermann, K., (eds.), Lecture Notes in Computer Science, 4227, Springer, (2006) 170184 Koshmann, T., Toward a Dialogic Theory of Learning: Bahtin’s Contribution to Understanding Learning in Settings of Collaboration, in C.Hoadley and J. Roschelle (eds.), Proceedings of the Computer Support for Collaborative Learning 1999 Conference, Stanford, Laurence Erlbaum Associates, (1999). Schegloff, E., Discourse As An Interactional Achievement: Some Uses Of 'Uh huh' And Other Things That Come Between Sentences, in Tannen, D. (ed.), Georgetown University Roundtable on Languages and Linguistics 1981; Analyzing Discourse: Text and Talk, Georgetown University Press, Washington D.C. (1981) Sfard, A., On reform movement and the limits of mathematical discourse, Mathematical Thinking and Learning, 2(3), (2000) 157-189 Stahl, G., Group Cognition: Computer Support for Building Collaborative Knowledge, MIT Press, (2006) Tannen, D., Talking Voices: Repetition, Dialogue, and Imagery in Conversational Discourse, Cambridge University Press, (1989) Trausan-Matu, S., Chiru, C., Bogdan, R., Identificarea actelor de vorbire în dialogurile purtate pe chat, in Stefan Trausan-Matu, Costin Pribeanu (Eds.), Interactiune OmCalculator 2004, Editura Printech, Bucuresti, (2004) 206-214. Trauşan-Matu, S., Stahl, G., Sarmiento, J., Polyphonic Support for Collaborative Learning, in Y.A. Dimitriadis et al. (Eds.): CRIWG 2006, Lecture Notes in Computer Science 4154, Springer, (2006) 132 – 139 Voloshinov, Marxism and the Philosophy of Language, New York Seminar Press, (1973) Vygotsky, L., Mind in society, Cambridge, MA: Harvard University Press, (1978) Wertsch, J.V., Voices of the Mind, Harvard University Press, (1991) Winograd, T., Flores, F., Understanding Computers and Cognition, Norwood, N.J.: Ablex, (1986) HECACEJ: B-Learning Tool for Static Content Creation in Joomla!1 Angel Mora-Bonilla, Domingo López-Rodríguez, Enrique Mérida-Casermeiro, Salvador Merino-Córdoba Department of Applied Mathematics, University of Málaga, Málaga, Spain {amora,dlopez, merida}@ctima.uma.es,smerino@uma.es Abstract. In this work we have reviewed the concepts of Learning Managementystem (LMS), Learning Content Management System (LCMS) and Content Management System (CMS). Their main features are studied, and a comparison among them is presented, along with a summary of some wellknown integrations between LMS and CMS. Another important point in this work is presenting HECACEJ, a B-learning tool designed to create static content in Joomla! This tool is a specific content manager for publishing the material for a subject, which does not need help from the administration menu of Joomla! (difficult to use by a non-expert professor) and is able to publish these contents in Joomla! main menu. This tool eases the publication of Blearning contents (class notes, past exams, professors' data, etc.) in a static web page that can be accessed from the main menu of the platform Joomla!, which provides facilities for manipulating the contents but it was not developed adhoc for our proposal. Keywords: B-learning, LMS, CMS, Joomla!, integration of tools 1 Introduction The group FERMAT is formed by professors of Mathematics who wanted to join all the subjects relative to Mathematics Applied to Telecommunications in a single point. Since 2002, students in Telecommunications Engineering School have a meeting point in Fermat website (www.fermat.uma.es). Every subject is studied and worked within a B-learning framework. In a first version, FERMAT website [1] was a web page 2 maintained by several professors spending a lot of work and time. It became necessary to use a CMS tool in order to optimize resources maintenance and to take advantage of new learning tools. 1 2 This research was partially supported by Spanish PIE04/030 and UMA N 008 DGI projects. Old FERMAT web (http://campusvirtual.uma.es/fermat/) was translated to six languages and it has been recognized as the second winner of the award for the III Inter-university competition in research and teaching in the web (http://www.campusred.net/certamen). Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 227-236, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 228 Angel Mora-Bonilla et al. It is very well-known that e-learning presents big advantages on classical distance teaching, since besides allowing flexibility of timetable, it allows the students to interact and to instantaneously access to on-line resources and multimedia: text, graphics, videos, audio and animations, among others. We propose a blended learning model [2] for our subjects: normal classes are mixed with contents that we upload to the FERMAT web. The professor, as an element to favour learning, should design a succession of motivational environments in which student's activity is developed in order to obtain the pursued purposes. When we are in the B-learning model, the good design and variety of these environments is much more important than in the classical model of teaching. To design the learning environments in e-learning, the professor has diverse software tools. Among the most used ones we have: • Learning Management System (LMS): also known as Platform or Virtual Campus, it consists on programs that allow to automate the administration of formative actions, it registers users, organizes a catalogue of courses, manipulates user's data, carries out reports, allows to provide and to modify contents, controls and evaluates student's participation, etc. It also offers tools for the interaction professor-student as well as professor-professor and student-student (mail, forum, chat, video-conference, ...). • Content Management System (CMS): This type of programs [3] aims at simplifying the creation and administration of contents. Basically this is achieved by separating the creation of contents and their presentation. The program takes charge of locating titles, text, graphics, links to the material, etc. This type of programs has a wide incidence in periodic publications. • Learning Content Management Systems (LCMS): They have their origin in LMS, but they are focused to the educational environment, also allowing the creation and management of content. This system type allows to create, to publish, to store and to administer educational resources, being their main users the professors. It is the appropriate tool to design learning situations in a B-learning environment. Thus, it can be deduced that in an environment of e-learning, we should have a tool of the type LCMS, or a LMS and a CMS that should also be integrated, so that the designs and contents, created by the latter, can be presented by means of the former. In this paper, we are interested in facilitating the design and management of Blearning tools in an university environment, so, besides the integration necessity, we should facilitate their use for professors that are not expert in computer languages, which not all content managers allow. For this reason, free content managers, allowing this adaptation, are needed. 2 Learning Content Management System In this section we intend to describe and compare three types of management systems: LMS, CMS and LCMS. HECACEJ: B-Learning Tool for Static Content Creation in Joomla! 2.1 229 Learning Management System We have already indicated how this type of programs automates the administration of formative actions (users' registration, creation of a catalogue of courses, management of user's data, etc), gives the professor methods to distribute contents and to observe the participation of the students, as well as it allows the students to access to interaction mechanisms (forums, chats, mail, etc). A LMS is an application residing in a web server in which formative activities are developed. All the LMSs consist of a `learning environment' to which students and professors access and of an `administration environment', where usually only professors and coordinators will access and where one will be able to enable courses, to register the students, to import contents, to enable or to disable services, to obtain statistics of the course, etc. However, a LMS will not include authoring possibilities (creation of contents). In [4] the author remarks the key of the increasing interest in LMS platforms: `The core purpose of the LMS system proposed in this paper is to help learners pursue proactive and self-oriented education by allowing learners to proactively configure their own content'. Free LMSs and commercial LMSs exist. The most used ones are the free ones since this not implies that they have an inferior quality. In addition, they are more complete than the commercial ones. Nevertheless, some institutions prefer commercial ones since they are configured and supervised by the company. Among free LMSs we distinguish the most used ones: Claroline [5], Dokeos [6], Moodle [7]. There exist many other free platforms as: Bazaar, Docebo, Eledge, Ilias, DattLNR, Ganesha, OpenUSS, Sakai, Spaghetti, ..., while among commercial LMSs we can find: WebCT [8] and Blackboard [9]. In general, all of them perform their functions appropriately, highlighting the platform Moodle due to its quick and extensive installation in the educational community, and it is the chosen by the University of Málaga for the installation of the Virtual Campus. Also, Moodle allows to compile certain mathematical expressions written in Latex, what makes it very appropriate for technical subjects. In [10] the author remarks the process involved in the development of this learning management system. The author discusses the process of building a LMS evolving from the first experience in a subject with some basic learning material. The tools that a LMS system provides to the professors, normally is enough to translate the material to a LMS and to immediately develop new learning material for the students. In our experience, we move to Moodle the subjects and develop a lot of learning questionnaires, lessons, forums, cites, glossaries with hundreds of definitions added by the students, wikis, tutorials, books, etc. In [11] the authors explain all these learning tools developed in their subject. 2.2 Learning Content Management System LCMSs [12] are software applications whose objective is to manage educational content. In general, they combine the courses management capacities of a LMS with creation and storage capacities of CMS. This way, LCMSs constitute an unique platform that allows students to personalize the resources and contents. 230 Angel Mora-Bonilla et al. Contents can be re-used and published in several formats, being stored in a centralized database which can be accessed in many different forms. Data can be visualized in different ways, what allows to adapt them to diverse groups of students. In general, contents use XML tags and follow several standards like AICC and SCORM, allowing to export designed materials. A good example of LCMS is Atutor that is easy to use. The demo version can be downloaded from [13]. 2.3 Differences Between LMS and LCMS The fundamental difference between both of them [14] is the objective that they pursue. While LMSs focus their attention on the administrative management and the student's performance (manages students' data and qualifications, programs events), LCMS platforms focus in contents management, allowing their adaptability and controlling the browsing among them. In the first case, the student tends to feel like observed and controlled, while in the second he/she feels that the important thing is the process of learning. However, Brooks et al. [15] remarks that `Most popular learning content management systems provide poor collaboration support for learners'. 2.4 Content Management Systems CMSs [3] are a group of tools, supported by a database located in a web server and, optionally, one series of client programs that allow the access to those tools. These tools allow to manage in an uniform, accessible and comfortable way a dynamic website with periodic upgrades and on which several people can act, each one with a certain purpose. In [16] Sehring and Schmidt summarize the use of CMS: `Innovative information systems such as content management systems and information brokers are designed to organize a complex mixture of media content -- texts, images, maps, videos, etc'. From the client's point of view, it is a dynamic website, with an uniform appearance, with design centered in the user and which, therefore, allows to easily carry out the tasks it has been designed for. In summary, a CMS has two main functions: • To facilitate contents creation: To create content becomes as easy as to fill in a form. There exists a single source for all them. • To manage the presentation of those contents: It facilitates the publication of contents in several formats starting from an unique source, adds meta-data to contents in order to facilitate the browsing among them in multiple facets (timeline, categories, author, ...). It also manages the contents and their maintenance (creation, presentation, upgrade ...). Their use is relatively recent and we can say that, at user level, they have extended since 1999. HECACEJ: B-Learning Tool for Static Content Creation in Joomla! 231 There exist CMSs of different types, according to the contents to manage: Forums, Blogs, Wikis, Websites or Portal type systems. Those managing websites are the most interesting due to their modularity. We could also classify CMSs according to the license type: in some cases the source code is protected and it cannot be modified, while in other cases free access is allowed. Using a CMS, users can elaborate and manage their webs obtaining dynamic pages with plenty of functionality. Thus, a CMS is useful due to: • Inclusion of new functionalities in the WEB. With a CMS, new modules can be included without supposing many changes. So the web is able to grow and to adapt to future necessities. • Maintenance of a great number of pages. Mainly, if they are maintained by several people with access permissions to different areas. • Re-use of objects and components. • Interactive pages. The static pages arrive to the user just as they are stored. However, the dynamic ones are generated according to the user's petitions. The CMS achieves this by connecting with a central database where all the data of the website is stored. • Visual changes of the WEB. CMSs facilitate style changes, by using the existing standard CSS to design the style of presentation of the contents. • Consistency of the WEB. The WEB presents an uniform aspect according to the style selected. • Control of accesses to the WEB. A CMS allows to manage the access to different areas of the WEB to diverse groups of users. In general, open code CMSs are more appropriate since they allow to adapt and to modify themselves to our necessities. A great quantity of free CMSs exist (Action Applications, Apache Lenya, ASP Nuke, DoorSoft, ...) but we would like to highlight Mambo, Joomla! and Xoops, which are the most extended. We have chosen Joomla! [17] due to its great diffusion, good integration with Moodle and its `online' management, that is, via Internet and by means of the HTTP protocol. Joomla! works in Linux platforms as well as under Windows. The user does not need to know the HTML language or to access a WebMaster to upgrade something. Joomla! manages contents by means of the relational database MySQL. In [18] the authors pose some problems of the use of CMS: `However, learners' experiences of these systems are almost invariably static, with information being delivered regardless of their background or knowledge. Due to variation between learners, it is suggested that these web-based distance-learning systems would benefit from the capability of adapting their content to meet individual needs'. In the same sense, the use of Joomla! by a FERMAT professor, even with computer science knowledge at user level, is complicated and justifies the development of a specific application that facilitates the use of Joomla! in our framework. Actually, FERMAT joins ten subjects and was developed using Joomla! but the process of maintenance of the information contained in each subject was excessively increasing. Joomla! provides facilities for manipulating the contents but it was not developed ad-hoc for our proposal. 232 Angel Mora-Bonilla et al. 2.5 Integrations of CMSs with the Platform Moodle Once Moodle is chosen as the LMS to use, we should elect a CMS that can be integrated with it. The integration of a CMS with Moodle implies that Moodle can be accessed from the CMS (without necessity of identifying us if we have already made in CMS, and if we have an account in Moodle). The official page of Moodle [7] informs us that Moodle integrates officially with Mambo, Xoops and many other CMSs. Also, Joomla! work-team has carried out the integration Moodle-Joomla! which can be downloaded from [19]. (b) (a) Fig. 1. (a) HECACEJ integrated in the main menu of Joomla!, (b) Past exams table created by HECACEJ, an example of static content created by HECACEJ and associated to one of the subjects which were also created by HECACEJ. 3 Our Context The group FERMAT3 is formed by several professors of the Department of Applied Mathematics in the University of Málaga in Spain. These professors formed the group with the aim of making some resources of Applied and Theoretical Mathematics available to the students of this University. Thus, in order to achieve better results, a web-page (www.fermat.uma.es) was created and all available resources were included in it. This website was developed with the help of the CMS Joomla! The contents of this page are all necessary resources to study several subjects (Calculus, Algebra...) in the Engineering School of Informatics in the University of Málaga: professors' data, tutorships, exams, class notes... for each of the subjects. 3 ‘Foro Experimental de Recursos de Matemáticas Aplicadas a las Telecomunicaciones’, in English: Experimental Forum of Resources of Mathematics Applied to Telecommunications. HECACEJ: B-Learning Tool for Static Content Creation in Joomla! 233 Information is shown in different web pages, which are called `Static Content' by Joomla!, meaning that they do not require constant updates. The information shown in those pages is conceived to last for a long time. The current problem, which has motivated the development of the tool presented in this work, is the difficulty that carries the creation of a subject (page with the program of the subject, tutorships and listings of the professors, etc.) in Joomla!, which, as mentioned before, is the educational platform used by FERMAT to maintain the communication with the students. Every time a professor wants to create a subject page, he/she had to enter into the complicated (for a non-expert user) administration menu of Joomla! and insert `by hand', using a Web editor, every content being a part of the subject, including designs, that is, colors, font shapes and sizes, tables which are used to represent the contents. Obviously, it is not useful to create a page design every time for each of the subjects, when, actually, this design is going to be always the same, or very similar. But, perhaps, the most time-wasting process is the creation of the static contents, since the professor has to repeat some listings of the corresponding professors' data, create links associated to the documents uploaded to the server, build tutorship tables... for every subject. These drawbacks motivated the development of a Contents Meta-Manager, able to collect all the information only once, save it in the database and show it in a Web page every time that it is necessary. Fig. 2. Main menu of HECACEJ: Subjects, Professors and Passwords. 4 HECACEJ: A Meta-Manager for Static Content Creation As commented before, the platform Joomla! has a drawback for our proposal: its complicated administration menu for a non-expert user. The professors must browse the administration menus of Joomla! to introduce a news for the subject or search a previous news related a subject, etc. For instance, if a professor must change the timetable of the subject it is necessary to search in a lot of news stored for all the subjects. We wanted to create a tool that could substitute the administration menu of Joomla!, and that could automatically create (with a more friendly interface) the 234 Angel Mora-Bonilla et al. different subjects, that is, the contents of each subject. HECACEJ4 is a tool with an educational purpose, associated to the platform Joomla!, used to create static web contents. This tool is a Contents Meta-Manager, which means that is associated to a main CMS and operates `above' it. In our case, the main Contents Manager is the administration menu of Joomla!. In other words, HECACEJ is a tool that manages the contents of the platform Joomla! directly, without needing to use the administration menu, since it works in a higher level. The integration between the platform and HECACEJ can be observed in Fig.1. HECACEJ has a very intuitive and easy-to-use interface (see Fig. 2.) which, step by step, leads us by several menus to collect all necessary information about the subject we are creating, and finally updates automatically Joomla! internal data base with all contents and therefore updates the external view of these data, making it public in the web page of FERMAT (see Fig. 1. (a)). By entering in the subjects menu, we will be able to create a new subject or to choose one of those already existent, see Fig. 4. By selecting one of them, we will have a blackboard where to locate important news related with the subject (evaluation method, suspension of classes, schedule modification, practical classes, etc.). After it, we can give the description of all the aspects of the subject with an uniform format: program and class notes, professors, exams, presentations, available questionnaires, problems, practical classes, suggested bibliography (see Figure 5). Fig. 3. Tutorships table created by HECACEJ. Fig. 4. Subjects menu used by HECACEJ to create new or modify existing subjects. 4 HECACEJ - `Herramienta Educativa para la Creación Automática de Contenidos Estáticos en Joomla', which is the translation in spanish of B-learning tool for automatic static content creation in Joomla!. A full demo of HECACEJ is available at www.satd.uma.es/amora/hecacej HECACEJ: B-Learning Tool for Static Content Creation in Joomla! 235 In Fig. 1. (b), we can observe an example of static content created by this tool, a table representing a repository of past exams. Each cell of the table is a link to the corresponding exam. If the professor has released the solved exam, the student will be able to open it with one click. (a) (b) Fig. 5. (a) Dialog where to introduce the program of the subject and possible class notes, (b) dialog to modify or create questionnaires. But not only subjects can be created. In addition, this tool has a menu to collect and store professors' information, see Fig. 6. Every authorized professor, can insert his/her own data (including tutorships...), which will be stored and later published in every subject page he/she belongs to, as shown in Fig. 3. The integration of HECACEJ in Joomla! is rather easy. By using the administration menu of Joomla!, a link to the tool main page has to be created. Once this is done, we return to Joomla! main page and will find the link to the tool. Just clicking on it takes us to the main window of HECACEJ, Fig. 2. Fig. 6. Professor menu of HECACEJ. 236 5 Angel Mora-Bonilla et al. Conclusions In this work, the concepts of Learning Management System (LMS), Learning Content Management System (LCMS) and Content Management System (CMS) have been reviewed. The main features of each one of these systems are studied, and a comparison among them is presented, along with a summary of some well-known integrations between LMS and CMS (e. g., Moodle - Joomla!). One important contribution of this work is to present HECACEJ, a B-learning tool designed to create static content in Joomla!. This tool provides a way to manage contents without the help of the administration menu of Joomla!, and is able to publish these contents in this main menu. Thus, this tool eases the publication of Blearning contents (class notes, past exams, professors' data...) in a static web page that can be accessed from the main menu of the platform Joomla! References 1. Ángel Mora, Mérida, E., et al.: Fermat: E–learning of mathematics. Current developments in technology assisted education - m-ICTE 2006 - http://www.formatex.org/micte2006sp/ (2006) 1263 – 1267 2. Martin Oliver, K.T.: Can blended learning be redeemed? E–Learning 2 (2005) 17 – 26 3. William H. Dutton, P.H., Park, N.: Social and institutional factors shaping e-learning in higher education: case study of a university-wide course management system. (European Conference on E Learning). 4. Kim, S.: Web based learning application for lms. LNCS 4113 (2006) 1164 – 1169 5. Claroline. (url: http://www.claroline.net/) 6. Dokeos. (url: http://www.dokeos.net/) 7. Moodle. (url: http://www.moodle.org/) 8. WebCT. (url: http://www.webct.com/) 9. Blackboard. (url: http://blackboard.com/) 10. Richardson, J.: Building an effective LMS. LNCS 2783 (2003) 27 – 36. 11. Ángel Mora, Mérida, E., López, D.: Development of a virtual learning community for the subject numerical methods under Moodle. Current developments in technology assisted education- m-ICTE 2006 - http://www.formatex.org/micte2006sp/ (2006) 361– 365 12. Duc, P.T., Haddawy, P.: A modular approach to e–learning content creation and maintenance. LNCS 3143 (2004) 217 – 224 13. ATutor. (url: http://www.atutor.ca/atutor/demo.php/) 14. Greenberg, L.: LMS and LCMS: What’s the difference? (Learning Circuits. http://www.learningcircuits.org/2002/dec2002/greenberg.htm) 15. Christopher Brooks, R.P., Greer, J.: Awareness and collaboration in the ihelp courses content management system. LNCS 4227 (2006) 34 – 44 16. Sehring, H.W., Schmidt, J.W.: Beyond databases: An asset language for conceptual content management. LNCS 3255 (2004) 99 – 112 17. Joomla! (url: http://www.joomla.org/) 18. Zakaria, M.R., Moore, A., Ashman, H., Stewart, C., Brailsford, T.: The hybrid model for adaptive educational hypermedia. LNCS 2347 (2002) 580 – 585 19. Integration Moodle-Joomla! (url: http://forge.joomla.org/sf/frs/do/viewSummary/projects.joomoodle/frs). MAAS – Mobile Administrative and Assessment System Apple W P Fok, Horace H S Ip, Chilli C K Chan Centre for Innovative Applications of Internet and Multimedia Technologies (AIMtech) City University of Hong Kong Email: {applefok@cs.cityu.edu.hk, cship@cityu.edu.hk, ckcchan@cs.cityu.edu.hk} Abstract. Researches focus on studies of using mobile technologies to strengthen teaching and learning by offering free, boundless access. Less attention has been paid to non or less educational activities which may also affect the teaching and learning effectiveness. Consider key technical, managerial and organizational issues of the deployment of mobile devices for school administration and assessments, in this paper we will introduce the design and development of a Mobile Administrative and Assessment System (MAAS) to facilitate school teachers performing routine administrative tasks and collecting valuable formative assessments records. Keywords: Mobile System, Formative Assessments, School Administration 1. Introduction Understanding students’ learning behaviors and needs is the most important stage before we decide what and how to select and organize instructional materials. To collect and analysis students’ behaviors and performances continuous to bother teachers not only simply because of the assessments process, but more importantly, some of the behaviors and performances can hardly be collected and recorded in a systematic and effective way. Mobile technologies can make a significant impact in teaching and learning [1, 2]. High and dynamic interactions between the learners, teachers and educational resources in an indoor or outdoor environment are achieved and benefited from the development of short-range wireless technologies. Mobility has provided an emerge capability to learning technology. A great demand for an ‘anytime’ and ‘anywhere’ learning environment by combining mobility, is resulted by information explosion in the 21st century. Wireless technologies provide the interactive communication between students and/or teachers, access to highly informative contents from internet, look for knowledge with keyword, and take part in an activity anytime and anywhere. A mobile learning environment can be constructed by wireless equipments such as PDA, that may improve performance over that come from traditional learning. New learning activities or models for active learning can be also created under these wireless equipments in mobile learning environment [3]. As children view mobile devices as another version of their favourite Gameboytm or Nintendotm, educators believed that mobile learning helped to promote the participation of students in Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 237-256, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 238 Apple W P Fok, Horace H S Ip, Chilli C K Chan learning. They believed that mobile learning could help to change the attitude and behavior of the students, and could provide them with a greater motivation for further learning. [7] even believed that mobile technology helped family learning. Mobile devices are personal, accessible, flexible and allow for collaborative solutions. They can help to realise collaboration. They allow sharing, exchanging, organizing and managing information easily. Everybody does not need to be together or fix in one place can communicate with each other and receive and response quickly. This indicated that mobile technology could provide the opportunities for collaborative learning and improve communication between learners and teachers. Collaboration between teachers and non-teaching staff in school environment where educational events occur in close relations to less educational events is described as a team working process where one exchanges information, expands on them, changes, modifies or discards them together. Schools can make use of this mobile environment not only improving the learning but also benefiting to other operations at schools, such as administrative work and assessments. Taken the advantages of using handy size-wise mobile device like PDAs in this study, teachers can perform not only daily routine school administrative works effectively and also conduct formative assessments and record keeping and analysis efficiently. Consider key technical, managerial and organizational issues of the deployment of mobile devices for school administration and assessments, in this paper we will introduce the design and development of a Mobile Administrative and Assessment System (MAAS) to facilitate school teachers performing routine administrative tasks and collecting valuable formative assessments records. 2. MAAS System Architecture Mobile Administrative and Assessment System (MAAS) is an information management system for student assessments, attendance and discipline records. MAAS consists of two components, a front-end PDA application and a backend database management system. Figure 1 illustrated the MAAS system architecture. On the server side, MAAS provides web services including user management, student management, assessment management, assessment result, attendance checking, assign and check homework and student discipline records. Front-end PDA application is divided into two parts. The core part of the application runs on PDA devices and the web service interface is held on a server. PDA devices connect to the server through WiFi wireless network and exchange data with database by the web service interface created on MS IIS (Microsoft Internet Information Server). Attendance lists, assignment checklists, assessments rubrics and administrative records can be uploaded/downloaded through wireless connection to/from the server. MAAS – Mobile Administrative and Assessment System 239 Fig. 1. MAAS System Architecture Front-end PDA application includes two programs: i) the Assessment program that Create, save or import student grouping (specialized to contents in group assessment), and record students’ performances; ii) the Administrative program that facilitates checking and monitoring of students general behaviors in school include attendance (i.e. records of student attendance in school and activities), and discipline (i.e. records of homework submission, misbehaviors, and good behaviors). Figure 2 shows the MAAS web services and Figure 3 shows the MAAS PDA applications. Taken the advantages of the mobility of PDA devices, these tasks can be accomplished effectively through wireless connection anywhere, anytime in the school. During the assessments, all assessment data is stored in the database on PDA in file format. Teachers can check and update the records easily. Fig. 2. MAAS Web services 240 Apple W P Fok, Horace H S Ip, Chilli C K Chan Fig. 3. MAAS Front-end services 3. MAAS Multi-facet functionalities To facilitate the school administrative tasks through collaboration, MAAS provides distinctive services for three distinct user groups: school system administrator, school administrative staff and teachers. Although they share the same underlying platform and basis functionalities, like rubric design and display, view rubric results and analysis, check and retrieve reports and report summary, different group touches different sets of data and have multi-level user access rights to the data and the analysis results. To facilitate different user groups, MAAS provides different sets of features. Figure 4 illustrated the functionalities of different user groups and the data in relations to the access rights. To make the administrative work more efficient for teachers and reduce their workload from these administrative tasks, MASS provides useful features in six categories related to the administrative work in schools. These six categories are User Management, Student Management, Assessments and, Attendance, Discipline and Behavior Records. Through these features provides by MAAS, teachers can perform assessments using a small mobile device, e.g. PDA or smart-phone, in a convenient way, with high accuracy, no location limitation and instantly. It enhances the way of assessment from the traditional paper-based process and makes the process more effective. Moreover MAAS analyses the statistics of assessments and, attendance, discipline and behavior records to gives out meaningful and useful results for school operations. Administrative staff can get a clearer picture about students to make an accurate and appropriate school planning and change the daily school operations in an efficient and effective way such as attendance taking and student report generating. MAAS – Mobile Administrative and Assessment System 241 Fig. 4. User Management System Admin Admin Staff Teachers User Management System admin, admin staff and teacher information management √ √ × Student Management Class, Student, Subject management Assessments Form Management Form browsing, Result Reporting, Result Statistics Attendance and, Discipline and Behavior Records Attendance records management Homework records management Stationary records management Behavior records management Statistics Reporting √ √ × √ √ √ √ √ √ × × × × √ √ √ √ √ √ √ √ √ √ × √ √ √ Fig. 5. MAAS Functionalities of different user groups and the data in relations to access rights. User Management MAAS includes User Management feature which is simple and effective. School administrators and administrative staff can easily manage user accounts through this feature. In User Management, user account information can be added, displayed 242 Apple W P Fok, Horace H S Ip, Chilli C K Chan clearly, modified with well organised form or removed by a single click then confirm. This feature releases the burden from the heavy workload of the annoying management of user accounts. Figure 5 shows the screen layout of User Management in MAAS. Fig. 6. User Management in MAAS MAAS – Mobile Administrative and Assessment System 243 Student Management To support system administrators and administrative staff to manage student accounts and related information, e.g. class and programme information, Student Management provides efficient tools for adding, editing, and locking different user accounts. These features reduce the stress of administrators and administrative staff for managing over thousand student accounts in a school. The details of student accounts can be create shown, changed and deleted by a few simple steps without any pre-requested technique. Also MAAS can manipulate class and programme, e.g. physics or biology stream programmes, with the same advantages. Student Management from MAAS smooth out the complicated tasks of managing student accounts then administrators and administrative staff can stay away from these repetitive jobs. Figure 6, 7 and 8 show Student, Class and Program Management respectively. Fig. 7. Student Management in MAAS Fig. 8. Class Management in MAAS 244 Apple W P Fok, Horace H S Ip, Chilli C K Chan Fig. 9. Program Management in MAAS In addition to the analysis results in form, class, MAAS provides the analysis result in student level. Senior admin staff can obtain statistical overviews in form and class levels. It helps them to set the goal of school year plan, make the direction of school development and design the school year plan. On the other hand, MAAS enhances the work and reduces the workload for general admin staff. The analysis results in student level from MAAS gives the general admin staff a hand with a complex and heavy workload task, student reporting. On the other hand, strengthen the communication with teachers, admin staff can inform and update teachers ad hoc or latest information received when accidental events happened. Updated records and alerts will be sent to teachers through MAAS. Through three levels quantitative analysis, the class level, student level, and strand level, MAAS provides student performance analysis for teachers. Student performance status can be obtained from different sources, including the assessment scores, participation in activities, and even students’ general behaviors. It uses data mining techniques off-line to cluster and understand students’ behaviors so as to predicate the trends of each student. To develop diverse assessment types and formats to better align with the curriculum, theories and practice of learning and teaching, teachers can design and share different types of assessment rubrics as formative assessment tools to collect and evaluate students’ learning progress. A teacher is allowed to check and update the information and view the analysis of student(s) in his/her class, and also the summary records of other classes. Summative Assessments and Comparison with other classes or a grade level can give teachers clearer observations on their students, so as to adjust their teaching plans and strategies accordingly. MAAS – Mobile Administrative and Assessment System 245 To create a pleasant and collaborative working environment that strengthens the communication between teachers and school administrative staff, MAAS not only supports the recording services such as assessment records, attendance records, homework submission records and, discipline and behaviors records, but more significantly, provides alerts or notifications of records updated, continuous misbehaviors of one particular students and accidental events. While teachers are attending a class, s/he will also receive relevant information as if s/he needs. For instance, a class teacher needs to be informed if one of his or her class student is required an early leave because of instant sickness. 4. MAAS Supports/Responses to Different Types of Assessments In order to enhance the effectiveness of schools, the Education and Manpower Bureau (EMB) of Hong Kong introduced the School Management Initiative Scheme in 1991 [4]. This scheme offered a school-based management framework for improving school management and performance for the provision of quality school education. School-based management involves decentralization of decision-making from the Education and Manpower Bureau to schools. The devolution of responsibilities enables schools to streamline their administrative procedures for their goals and endvision. To support efficient retrieval of students’ data for internal as well as external review Apart from the academic performance, another key aspect of student learning emphasises student’s proper behaviours namely “student discipline”. Schools have to be strictly observed in handling students with discipline problems. Student discipline has to be concerned by all teachers at schools. Teachers have to play a role in reinforcing positive behaviour (e.g. politeness, punctual and responsible etc) and handling misbehaviour (e.g. running, smoking, fighting, foul languages etc) of students [5]. Outcome-based approaches continuously dominant the world wide education system and stress the importance of assessments. Empirical studies stated that “assessment for learning” is one of the principles of facilitating learning and teaching. It is an integral part of the learning/teaching cycle. To improve the student’s learning, a diversity of assessment modes has to be used [6]. Formative assessments should be used to provide effective formal or informal feedback (e.g. corrective, diagnostic) that motivates and improves learning. In-class-behaviours Assessment accesses qualitative work based on the purpose and process of learning, not academic outcomes. For instance, observation may be used to assess participation in discussion, performance in laboratory session, and project work for interdisciplinary thinking [4]. And schoolbased assessments will address the diversified needs of students and evaluate students’ competency levels from multiple perspectives such as arithmetic, linguistic, and musical etc. Through Assessment Form Management in MAAS, school administrators and teachers can add, browse and delete assessment forms and subject information required for assessments. Four rubrics design templates support the assessments 246 Apple W P Fok, Horace H S Ip, Chilli C K Chan record in Grading or exact Marking with one or two levels descriptions (Figure 9). MAAS provides the ability to school administrators and teachers for managing this information in a single and simple menu. Assessment form details can be reviewed by school administrators, administrative staff and teachers through MAAS. MAAS collects the assessment results of students to generate the statistics. This statistics given by MAAS provides administrative staff supportive evidences for planning the school development. On the other hand, MAAS not only gives the statistical assessment results of students but also the student performance ranking from the assessment results. This information from MAAS helps teachers to have a clear overview to student performance and let them design their teaching plans according to different abilities of classes or even individual students. These teaching plans are customized for corresponding classes or students; enhance the teaching process and benefit students most. Nevertheless MAAS generates reports for teachers and administrative staff from the assessment results to let them have knowledge about student performance and report generation is also a heavy load and repetitive task. With MAAS, teachers and administrative staff can reduce their workload from this task. Figure 9 and 10 show the rubric templates and an example as an assessment rubric. The results of these rubrics will be recorded and analysis as Class, Student Assessment Statistics and Student Performance Ranking respectively (Figure 11). Fig. 10. Four Rubric Assessment Templates MAAS – Mobile Administrative and Assessment System Fig. 11. Example of an Assessment Rubric Class Assessment Statistics 247 248 Apple W P Fok, Horace H S Ip, Chilli C K Chan Student Assessment Statistics Student Performance Ranking Fig. 12. Class, Student Assessment Statistics and Student Performance Ranking To make the grading process more efficient, grouping feature is designed. Teachers can form and adjust different groups according to either performances or discipline reasons. Figure 12 shows the grouping features. MAAS – Mobile Administrative and Assessment System 249 Fig. 13. Grouping Features of MAAS Behavior Records and Analysis To observe and understand students’ daily non-academic performances, several behavioral records are used including the Attendance Records, Homework Submission Records, and Discipline Records. 250 Apple W P Fok, Horace H S Ip, Chilli C K Chan MAAS lets teachers perform student attendance taking anytime they need through mobile devices like PDAs. Collaboration among teachers to student attendance taking can be achieved by MAAS. For late students, teachers can instantly update their attendance record during the lesson through MAAS. Therefore MAAS provides teachers an up-to-dated record of student attendance and helps teachers and administrative staff to monitor students effectively. Also MAAS records student attendance records and generates attendance statistics from these records. These records from MAAS can give teachers a summary on the attendance of students and have a facility to find out students with attendance problems. Then by the summary reporting function in MAAS, corresponding arrangements can be quickly taken for students with attendance problems. The student attendance statistics reports in class level included in MAAS can helps administrative staff to adjust the school operations and change the design of school plan according to this supporting statistical data. It makes the administration in schools more efficient and effective. Figure 13 shows Student Attendance Record and Student Attendance Summary respectively. In behavior record feature in MAAS, homework submission of students can be recorded. Homework submission record is a daily time-consuming task. MAAS can simplify and enhance the efficiency of this task by that teachers assign homework through MAAS and record the submission of students with PDAs anytime and anywhere. Then the student submission information is uploaded to MAAS. Accurate and up-to-dated homework submission records can be kept by the anytime and anywhere update from MAAS. In the global view of homework submission, administrative staff can use this statistics for analysis and find out student problems. This useful statistics then helps the design an appropriate strategy to tackle these problems in schools. Figure 14 shows the features that support the assign homework cycle including Homework assigned, Homework Submission Record and Homework Submission Statistics. Student Discipline Control is an important task in School Management. Teachers are always under stress from student discipline problems. MAAS provides teachers a convenience method to record student discipline records through PDAs. These records are instantly updated to MAAS and the system often keeps the most up-todated records. The mobile nature supported by MAAS creates a collaborative environment for teachers to perform student discipline record task. The stress from student discipline can be shared among teachers through MAAS. Individual student summary record in MAAS can point out students with discipline problems to teachers and alert teachers to take appropriate actions. Also MAAS gets student discipline records and produces a summary of student discipline records. It is very useful information to administrative staff for the modification to the current school planning and operations. According to MAAS student discipline statistics, customized school plan and operations can enhance the school development and benefit to schools significantly. Figure 15 and show Student Discipline Record, Student Discipline Statistics in MAAS respectively. MAAS – Mobile Administrative and Assessment System Fig. 14. Student Attendance Record and Student Attendance Summary 251 252 Apple W P Fok, Horace H S Ip, Chilli C K Chan ↓ ↓ Fig. 15. Features support the assign Homework cycle MAAS – Mobile Administrative and Assessment System 5. 253 Evaluation and Discussion Taking the advantages of advanced technologies that provide any where, any time access through the web, incorporate a portable mobile device (i.e. PDA), the MAAS acts as i) a convenience data collector ; ii) an effective data transmitter; and iii) an interactive communication channel. To strengthen the proof of the advantages from MAAS, an implementation study has been conducted in one Hong Kong Secondary School in January 2007. Using two different sets of rubrics in English and Chinese languages for the subjects Physics and Chinese Culture, teachers are required not only to try out the system functionalities, but also compare the differences and evaluate the effectiveness of using such a device. All system features are tested and used for two different classes for two different assessments, science laboratory test and Chinese Oral Presentation. Both assessments taken in this evaluation have been completed smoothly and teachers can familiarize with the operations of MAAS. By the mobile nature of MAAS, teachers can walk around the classroom to inspect student work and access student performance at the same time (Figure 16). All the students observable learning behaviors are measured and records through MAAS. The time spends on students’ learning is increased. With the support from the rubric description, the assessments can be done more effectively and precisely. In addition, the mobile nature of MASS, gives an opportunity for collaboration between teachers. The centralized administrative and assessment work can be distributed to and completed by teachers at any parts in schools instantly. Therefore teachers can share these administrative and assessment jobs. The workload of teachers can be reduced. MAAS faces the same problem that a limited screen size on PDA limits/constrains mobile PDA applications nowadays. It is really challenging to design a rich yet concise interface. The selection of essential elements to be shown on the limited screen that can not only provide sufficient information, but also serve the functionalities that enhance the effectiveness of accomplishing a task required thoughtful plan and sacrifice. To minimize the navigation path, the general operation flow would be changed. For instance, a teacher used a class list for attendance checking. Names of students are shown on one page. However, on the PDA, not only affected by the font size, but also the check-box or text box for information input, it is not pleasant to read all the information at once. Alternatively, default values such as “Present” can be set beforehand and teachers can simply change the stage to absent or an “Absent” page can be designed for simply class number inputs. 254 Apple W P Fok, Horace H S Ip, Chilli C K Chan Fig. 16. Student Discipline Record and Student Discipline Statistics in MAAS MAAS – Mobile Administrative and Assessment System 255 Most mobile devices are difficult to handle with, especially during the inputting of characters. Thus in MAAS, we overcome this problem by serving predefined items that users can simply click onto interact with the system, and utilizing some skills such as enlarging buttons when they are about to be clicked. On the other hand, to highlight and discriminate the usage of rubrics and different rubrics items, special effects are required. The marked/graded rubrics will be highlighted and a summary of the rubric usage will also be provided. Fig. 17. MAAS in classes (Physic and Chinese) 6. Conclusion and Future Work In this paper, we have introduced the design and development of a Mobile Administrative and Assessments System (MAAS) that helps to reduce the workload of teachers from the administrative and assessment. Furthermore, MASS helps create a collaboration environment for teachers to complete school administrative and assessment work successfully. Recently, Mobile Administrative and Assessment System (MAAS) has been setup at two Hong Kong schools, one Secondary School and one Primary School (Fig. 17). To fully optimize the use of MAAS, analysis reports and report summaries will be designed. To further evaluate MAAS performance and effectiveness, different use cases studies including different assessment rubric designs that support multiple subjects and operations in class, and enhance the readabilities and extensibilities of different components of the system will be conducted. In response to the current mobile devices trend, the interface design would be adjusted to 320 x 240 that requires not only item representation changes, but also the navigation flow should be adjusted and changed. This in turns may affect users’ usage pattern and styles. Apart from the evaluation of the system performances and features provided, we also emphasis the applicable use of mobile devices in school environment. What and how mobile devices can be use to facilitate and enhance teaching and learning effectiveness? Pedagogical uses of mobile devices to strengthen e-education are our on-going research work. 256 Apple W P Fok, Horace H S Ip, Chilli C K Chan References 1. 2. 3. 4. 5. 6. 7. Perry D. (2003) Handheld Computers (PDAs) in School. Coventry, U.K.: Becta (for DfES) Zurita, G., & Nussbaum, M. (2004). A constructivist mobile learning environment supported by a wireless handheld network. Journal of Computer Assisted Learning, 20, pp 235-243. C.Y. Chang, J.P. Sheu and T.W. Chan, “Concept and design of Ad Hoc and Mobile classrooms,” Journal of Computer Assisted Learning (2003) 19, 336-346 EMB – School Administration Guide. http://www.emb.gov.hk/index.aspx?nodeID=681&langno=1 accessed 21st January, 2007 Student discipline. http://www.emb.gov.hk/index.aspx?nodeID=1740&langno=1 accessed 21st January, 2007 Principles of facilitating learning and teaching. http://www.emb.gov.hk/index.aspx?nodeID=1690&langno=1 accessed 21st January, 2007 DfES. (2005) The e-Strategy ‘Harnessing Technology: Transforming learning and children’s ervices’. Retrieved from http://www.dfes.gov.uk/publications/e-strategy/ Large-scale Computer-Assisted Assessment in Computer Science Education New Possibilities, New Questions Mario Amelung, Michael Piotrowski, and Dietmar Rösner Otto-von-Guericke-Universität P.O. Box 4120, 39016 Magdeburg, Germany {amelung,mxp,roesner}@iws.cs.uni-magdeburg.de Abstract. Since 2003 we have successively introduced the use of e-learning and computer-assisted assessment (CAA) components into all of our courses, namely online multiple-choice tests, electronic submission of assignments, and automatic testing of programs. We originally did not intend to make major changes to the courses; our primary motivation was just to make them more efficient and more effective by freeing teachers from administrative burdens and by offering more flexibility and interactivity for students. After several semesters of usage we have noticed, however, that the courses have changed much more radically than originally envisaged. The electronic support of faceto-face courses offers many new possibilities, but it also opens up new questions. This paper describes our system and our experience, and discusses some of the questions we have encountered. 1 Introduction In computer science education, lectures are typically accompanied by exercise courses or tutorials. These courses are essential for the learning effect since they provide opportunities for students to solidify the knowledge acquired in lectures and to apply it to practical problems. We were, however, dissatisfied with a number of aspects of the traditional way of teaching, practicing and assessing in undergraduate computer science courses at our university: They offered only relatively little motivation for students and allowed only restricted conclusions about the students’ performance during the course, whereas the administration of the courses required a lot of work. Especially unsatisfactory was the handling of programming assignments: The traditional way of handing in programs on paper and discussing them on the blackboard is only viable for very small programs, and some problems are hard to detect without actually running a program and testing it. Students thus only gained little practical programming experience. In fact, there were serious complaints that even some of our second and third-year students still had insufficient programming skills. These problems prompted us to develop a number of e-learning components (described below) to support the teaching of face-to-face courses. We did not want to Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 257-266, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 258 Mario Amelung, Michael Piotrowski, and Dietmar Rösner abolish face-to-face courses, but rather make them more efficient and, hopefully, also more effective. Since winter semester 2003/2004 we have successively introduced the use of elearning components into all of our courses. This includes online multiple-choice tests, electronic submission of assignments, and automatic testing of programs. After several semesters of usage we have noticed, however, that the processes within the exercise courses have changed much more radically than we expected, and we strongly feel that now, with many technical aspects solved, with a mature implementation, and after several semesters of usage, it is time to reflect on our experiences. The structure of this paper is as follows: Section 2 gives an overview of our elearning environment, briefly explaining some technical details of the automatic program testing module to provide the necessary background for the subsequent discussion. In section 3 we describe the structure of our exercise courses, before and after the introduction of electronic support. Section 4 discusses the new possibilities offered by the electronic support of face-to-face courses and the questions raised by some of these possibilities. Section 5 summarizes the discussion. 2 The Learning Environment Instead of using a separate learning management system (LMS), which would require additional training and administration, our approach is based on modules that extend a general-purpose content management system (CMS) with e-learning functionality, in our case the open-source CMS Plone1. We designed, implemented and deployed a number of Plone modules. These modules—collectively called eduComponents— provide specialized content types offering the following functions (see also [1,2]): − ECLecture: Course information and registration. − ECQuiz: Electronic multiple-choice tests. − ECAssignmentBox: Electronic submissions for assignments and support for manual assessment and grading. − ECAutoAssessmentBox: A version of ECAssignmentBox with automatic checking and assessment of programming assignments with immediate feedback. These components can be used separately or in concert, and they can be combined with other Plone products (e.g., discussion boards, glossaries, and bibliographies) to create tailor-made learning environments. We are using the eduComponents modules to support face-to-face courses, but they can also be used in pure online courses. ECQuiz supports the creation and delivery of multiple-choice tests (see also [3]). Multiple-choice tests are especially useful as formative tests to quickly assess the performance of all students of a class without the need for extra grading work. The assessment of essay-like student submissions offered by ECAssignmentBox is semi-automated, meaning that the teacher does the assessing but is aided by the tool 1 http://plone.org Large-scale Computer-Assisted Assessment 259 during the entire process of grading students’ work and giving feedback. Therefore ECAssignmentBox defines a specialized workflow for student submissions. Because automatic testing of programming assignments is especially interesting in computer science education, we will describe our approach in more detail: ECAutoAssessmentBox allows students to submit solutions for programming assignments via the Web at any time during the submission period. Submitted programs are automatically tested and students get immediate feedback on whether their programs are syntactically correct and—more importantly—then yield the expected results. Fig. 1. ECAutoAssessmentBox automatically checks submissions to programming exercises and immediately offers feedback (student view). When a student submits a program, it is first sent to ECSpooler, a Web service which manages a submission queue and several backends. Backends provide testing for a specific programming language, usually in conjunction with the corresponding compiler or interpreter. Currently implemented are backends for Haskell, Scheme, Erlang, Prolog, Python, and Java. Backends can also be used to implement different approaches for testing: For example, we have implemented one backend for Haskell which compares the output of the student solution with the output of a model solution 260 Mario Amelung, Michael Piotrowski, and Dietmar Rösner for a set of test data, and, as an alternative, we have implemented another backend for Haskell which uses QuickCheck [4] for testing based on formal specifications of properties required for a correct solution. With the appropriate backends, the system can also be used to check submissions in other formal notations or to analyze naturallanguage assignments (we have already experimented with style checking and keyword spotting). The results of the tests performed by the backend are immediately returned and are displayed by ECAutoAssessmentBox (see Fig. 1). There are quite a number of systems which provide facilities for automatic testing of programs (e.g., TRAKLA [5], SchemeRobo [6], CourseMarker [7], AT(x) [8], or Praktomat [9]). Our approach differs from most other systems in its architecture, which clearly separates frontend, spooler, and backend, offering a high degree of flexibility and enabling a variety of frontends/backends to be used. Our system also concentrates on testing and does intentionally not offer advanced tutoring functions. The eduComponents modules are freely available as open-source software licensed under the terms of the GNU Public License.2 3 Course Structure Since winter semester 2003/2004 we have been gathering experiences with eduComponents for online multiple-choice tests, electronic submission of assignments and automatic testing of programs in our exercise courses. During the last semester (winter 2006/2007) our learning environment was actively used by over 200 students at our institution. Before we introduced CAA—and in particular the automatic testing of programming assignments—into our courses, the typical procedure was as follows: Once a week students were given assignments; they then worked through the assignments and prepared their solutions for the classroom session of their exercise group (each group comprising about 12 to 24 students) on paper. For each assignment, the tutor called on a student to present his or her solution. The presentation was given at the blackboard and typically involved writing the solution onto the board. This mode of writing and copying was time-consuming and errorprone. Given the time constraints, only a limited number of solutions could be presented and discussed. It was also not clear whether the presented programs would actually work. Now, the exercise courses use the following process: Students get access to online exercise sheets (see Fig. 2) and submit their assignments using ECAssignmentBox or ECAutoAssessmentBox during the submission period, which typically ends several hours before the classroom session. Submissions for programming assignments are automatically tested. Depending on the tests defined by the teacher for the assignment, submissions must at least be syntactically correct; typically they must also yield the expected results or comply with a formal specification. 2 Available from: http://wwwai.cs.uni-magdeburg.de/software/. The automatic testing components are not yet publicly released, but are available on request. Large-scale Computer-Assisted Assessment 261 Fig. 2. View of a typical online exercise sheet The automatic testing of programs is not intended to replace the testing of programs by the student with the appropriate compiler or interpreter. The motivation is primarily didactic: Our experience shows that quite a number of students tries to avoid writing and testing programs and would rather submit non-working sketches. The automatic testing of programs enforces the requirement that programming submissions must be running programs. But the student program can also be tested with input unknown to the student, thus uncovering unhandled edge cases; depending on the settings, the student may then submit a corrected version. For teachers, automatic testing provides help in judging the acceptability of programs, since they can easily see which programs run, and which do not. To prepare for the course, teachers then review the submissions, looking for recurring problems or outstanding solutions. Based on this information and on previous student performance they also select the students who will have to present their solutions in class. 262 Mario Amelung, Michael Piotrowski, and Dietmar Rösner Students have to present their solutions as before, but during the classroom session, the assignments and the presented solutions are displayed using a data projector. This removes the need to copy solutions to the blackboard. If the solution and the student’s presentation have been satisfactory, the submission is moved to the corresponding workflow state (e.g., graded). 4 Experiences 4.1 Effects on Students and Teachers The paper-based system allowed students to get points for assignments they had not actually completed, whereas they now have to submit written solutions for the assignments electronically before the classroom session. For programming assignments with automatic testing the demands for students’ solutions are much more explicit and rigid with respect to correctness, quality, and clarity: Sketchy notes and vague ideas on a piece of paper are no longer sufficient.3 Students thus also have to try to make sure that their solution is working correctly. Consequently the intensity of work needed for the exercises has effectively increased with the new system. For teachers using automatic testing of programs, the most significant effect is that the effort for designing assignments has increased. This is something that other users of automated program testing systems have also reported (e.g., [9]). Once there is a collection of assignments to choose from, this problem is alleviated, but automatic testing always requires problems and tasks to be formulated much more formally and precisely. This is necessary to enable automatic testing and in order to avoid misunderstandings, which could result in students trying to solve a different problem than the one the teacher had in mind and then getting puzzled about the reactions of the automatic testing system. 4.2 New Opportunities The fact that all assignments and tests are now managed as electronic documents in a CMS results in substantial improvements of the learning environment and offers many new opportunities. Major aspects include: − Student solutions of assignments are available online and can easily be presented, discussed, or modified during classroom sessions. − Alternative solutions of assignments can easily be made accessible as a portfolio for further self-study or for exam preparation. − New types of assignment are feasible, for example, peer reviewing of programs of other students (cf. below). − Personalized assignments with additional tasks are possible; these could be designed to help with problems observed in prior assignments. 3 While it would have theoretically been possible to enforce these requirements in the paperbased system, the workload for teachers would have been unacceptable. Large-scale Computer-Assisted Assessment 263 Additional advantages result from the data collected, for example the number of tries needed before a correct solution is found, or the number of submissions for an assignment. Cumulative statistics could help to detect a variety of problems; they could also be used to assess the quality and the difficulty of assignments. From the teachers’ point of view, for both programming and essay-like assignments, reviewing larger numbers of student submissions has proven to be feasible because the submissions are collected at a central location, so that they can be browsed and inspected before the classroom session. Specific problems observed in the submissions can then be addressed. Since all submissions are now also available online in the classroom session, solutions can easily be presented and compared; faulty solutions to programming assignments can be corrected and immediately tested. The time spent formerly to write sketchy solutions onto the blackboard is now free for discussion. 4.3 New Questions Not all of the new possibilities are completely novel, but they are much easier to realize and easier to actually use with the help of the new technical capabilities. Some of the questions are inherent in many other computer applications as well. Having records of information (in our case: assignments and log data) raises questions of access, privacy, and prevention of misuse. Some other questions have more to do with pedagogical considerations. These often involve the need for decisions about clear rules for all participants in the learning processes. An example could be the issue of personal attendance to classroom sessions: Some of our students felt that there was no need for personal attendance to the classroom session after they had submitted working solutions to the programming assignments. However, for pedagogical reasons, we insisted on personal attendance and participation in discussions—but other decisions could be made. Technically, using ECAutoAssessmentBox and ECAssignmentBox in a pure elearning context would, of course, also be possible. New possibilities often come with new demands: For example, ECQuiz makes it easy to set up multiple-choice tests and to get an automatic evaluation for a large number of students. The challenge for the teacher is, however, to design tests that require more than the recall of previously memorized knowledge and instead explore more demanding cognitive dimensions in Bloom’s taxonomy [10], such as application, analysis, or evaluation. Open vs. Testable Assignments Obviously, to enable automatic testing, programming assignments have to fulfill certain requirements. For example, the students’ solutions have to contain a top-level function with a certain name and a certain signature, i.e., the order of the arguments and their types and type of the result are specified in the assignment. However, sometimes the task consists in finding a good representation for the problem and its solution, so that, for example, the return type should be left open. Interactive programs are another class of tasks that have more degrees of freedom and that are harder to test automatically. 264 Mario Amelung, Michael Piotrowski, and Dietmar Rösner In our experience, teachers get quickly used to the automatic testing facilities and their requirements and then habitually consider automatic testability when designing programming assignments. This presents the danger that assignments are discarded if they cannot be tested automatically. It is therefore necessary to consider the learning objectives and to critically evaluate the appropriateness of assignments for certain objectives. For an assignment with an interactive Prolog program for playing tic-tac-toe we therefore decided to not use automatic testing but to assign the submissions anonymously to other students for critical evaluation and peer review. This ensures that the students can be creative in designing their solutions, and students get more actively involved; cf. [9] for a report of a successful use of peer review in computer science education. The possibility to create this type of assignment is of course also based on the availability of all submissions as electronic documents. Policy Issues The new learning environment available with eduComponents not only allows, but somehow demands decisions about policy issues. An example is the issue of plagiarism: Since all student submissions are available electronically, it is easily possible to automatically check for suspected plagiarism. However, this raises a serious policy issue: How much effort should be invested in plagiarism detection? When do we run the risk of wasting energy for an “arms race” between plagiarism detection and more sophisticated techniques of hiding plagiarism? Is there a chance to instead suppress plagiarism by stimulating a spirit among students that mastering challenges is rewarding and that handing complete solution to a peer for copying is counterproductive and not an act of solidarity? A related policy issue is team work. On one hand, being able to cooperate is essential for success in the job, e.g., as a project member in the software industry. On the other hand, students must also be able to successfully work on problems on their own, so free riding in a team should be avoided. How can this conflict be resolved? Is it possible to support both goals? Shall individual work on assignments be enforced (running the risk of plagiarism) or shall group submissions be allowed if declared as such? What are the consequences? Do we rely on students to cope with attempts in free riding on their own? Can we rely on the final exams, which have to be passed individually, as “hour of truth”? Control and Privacy The fact that all students submissions and log data about the number of attempts, submission times, etc., are available and storable for a long time as electronic documents, allows for much more control of students’ working habits, results and overall performance. But do we really want to exercise such a type of control? Do we really want glass box students? Or, in other words: What is more effective in the long run, a teaching system that rewards positive outcomes and that stimulates engagement, creativity and striving for excellence, or a teaching system that invests much energy in exercising control and in the punishment of unwanted behaviors? And an even more fundamental question could be: What is our general conception of students? The logging in ECAssignmentBox and ECAutoAssessmentBox was intended to allow evaluation of the effects of this system for the learning process and investigation into related questions. Astonishingly, there were hardly any concerns Large-scale Computer-Assisted Assessment 265 articulated from our students that they would feel uncomfortable with this kind of data collection. The only exception was the question posed by some students why a record of the previous submissions is stored when they submit an improved solution. This feature had been introduced to investigate the usefulness of the automatic feedback. 4.4 Unexpected and Unintended Usage ECAssignmentBox and ECAutoAssessmentBox have been designed and implemented as lightweight solutions. They support either direct typing of answers into a text field and the uploading of assignments from a file, but they intentionally do not offer sophisticated editor functionality. Nevertheless, there were unanticipated usages of the system. Some students used it as a kind of “online storage” to work on essay-like assignments: They started to work on an assignment from one computer, used the submission feature to store an intermediate version, and later continued to work on the same assignment from a different computer. This resulted in a large number of spurious superseded submissions. Other students abused ECAutoAssessmentBox as a Web-based interpreter to solve programming assignments. We therefore introduced a parameter for teachers to restrict the number of possible re-submissions for automatically tested programming assignments. This limitation also enforces a secondary learning objective: We expect that our students are able to use the native programming environments and interpreters for the various programming languages and to leverage them instead of submitting untested programs. 5 Conclusions At the end of each semester we ask our students to complete a questionnaire on their experience with the learning environment. The questions cover three areas: The use of electronic submissions in general, their effect on the students’ working habits, and the usability of the eduComponents. The results in all three areas have consistently been very positive. Students especially value the reporting and statistics features, which help them to track their learning progress. Furthermore, students find it helpful that their assignments are stored centrally, and can quickly be accessed for discussion in the course or for later review. Students also report that they work more diligently on their assignments because teachers can now easily access and review all of their submissions. A seemingly minor change in the organization and technical basis of exercises— i.e., introducing that all assignments and all solutions of students are electronic documents in a CMS—resulted in significant changes in the learning environment and changed learning processes much more fundamentally than expected in the beginning of the transition to the new system. When we started using CAA and other e-learning components we had the primary motivation to relief teachers and students from administrative burdens by automating certain processes and supporting others. Our experience is, however, that the change in the way how assignments are submitted has 266 Mario Amelung, Michael Piotrowski, and Dietmar Rösner lead to many other changes due to the new possibilities offered by the system. But the new opportunities also pose new demands for both teachers and students. Although the workload for students has increased there is a broad acceptance of the new system and students would welcome its use in other lectures as well. We interpret this as a positive reaction on the new opportunities and as an indication that students accept the higher intensity of their own engagement because they experience and appreciate an improved return on investment for their learning outcomes. References 1. Amelung, M., Piotrowski, M., Rösner, D.: EduComponents: Experiences in e-assessment in computer science education. In: ITiCSE ’06: Proceedings of the 11th annual conference on Innovation and technology in computer science education, New York, NY, USA, ACM Press (2006) 88–92 2. Rösner, D., Amelung, M., Piotrowski, M.: E-Learning-Komponenten zur Intensivierung der Übungen in der Informatik-Lehre – ein Erfahrungsbericht. In Forbrig, P., Siegel, G., Schneider, M., eds.: 2. GI-Fachtagung Hochschuldidaktik der Informatik. Bonn, Germany, GI-Verlag (2006) 89–102 3. Piotrowski, M., Rösner, D.: Integration von E-Assessment und Content-Management. In Haake, J.M., Lucke, U., Tavangarian, D., eds.: DeLFI2005: 3. Deutsche e-Learning Fachtagung Informatik der Gesellschaft für Informatik. Bonn, Germany, GI-Verlag (2005) 129–140 4. Claessen, K., Hughes, J.: QuickCheck: a lightweight tool for random testing of Haskell programs. In: ICFP ’00: Proceedings of the fifth ACM SIGPLAN international conference on Functional programming, New York, NY, USA, ACM Press (2000) 268–279 5. Laakso, M., Salakoski, T., Korhonen, A., Malmi, L.: Automatic assessment of exercises for algorithms and data structures – a case study with TRAKLA2. In: Proceedings of the 4th Finnish/Baltic Sea Conference on Computer Science Education, October 1-3, 2004, Koli, Finland, Helsinki, Finland, Helsinki University of Technology (2004) 28–36 6. Saikkonen, R., Malmi, L., Korhonen, A.: Fully automatic assessment of programming exercises. In: ITiCSE ’01: Proceedings of the 6th annual conference on Innovation and technology in computer science education, New York, NY, USA, ACM Press (2001) 133– 136 7. Higgins, C., Hegazy, T., Symeonidis, P., Tsintsifas, A.: The CourseMarker CBA system: Improvements over Ceilidh. Education and Information Technologies 8(3) (September 2003) 287–304 8. Beierle, C., Kulaš, M., Widera, M.: A pragmatic approach to pre-testing Prolog programs. In Seipel, D., Hanus, M., Geske, U., Breitenstein, O., eds.: Applications of Declarative Programming and Knowledge Management. 15th International Conference on Applications of Declarative Programming and Knowledge Management, INAP 2004, and 18th Workshop on Logic Programming, WLP 2004, Potsdam, Germany, March 4-6, 2004, Revised Selected Papers. Heidelberg, Springer (2005) 294–308 9. Zeller, A.: Making students read and review code. In: ITiCSE ’00: Proceedings of the 5th annual SIGCSE/SIGCUE ITiCSE conference on Innovation and technology in computer science education, New York, NY, USA, ACM Press (July 2000) 89–92 10. Bloom, B.S., Engelhart, M.D., Furst, E.J., Hill, W.H., eds.: Taxonomy of Educational Objectives, Handbook 1: Cognitive Domain. Longman, White Plains, NY, USA (1956) Exploratory Research on an Affective e-Learning Model Liping Shen1, Enrique Leon2, Victor Callaghan2, Ruimin Shen1 1 – Computer Science Department, Shanghai Jiaotong University, China 2 - Computer Science Department, University of Essex, UK lpshen@sjtu.edu.cn, {eeleon, vic}@essex.ac.uk, rmshen@sjtu.edu.cn Abstract. This paper explores how emotion evolves during the learning process with the longer term aim of developing learning systems that are able to recognize and respond appropriately to emotions exhibited by learners. We undertook this research by designing and building an experimental prototype of an emotion aware learning system conducting experiments and studying the relationship between emotion and learning. We report on our initial results which not only indicate there is a usable relationship between affect and learning, but by using the emotion states in Russell’s affective model, we have been able to make some significant progress towards experimental validation of Kort’s learning spiral model, which has not been empirically validated to-date. Keywords: e-Learning, affective computing, emotion-aware. 1. INTRODUCTION Background Technology is changing our lives at a breathtaking rate, no more so than in the world of education and e-Learning. The evolution of e-Learning can be traced from its roots in Computer Aided Instruction, through Intelligent Tutor System and Web-based Learning, to the Smart Classroom, Mobile Learning, Pervasive Learning and Personalized Learning technologies of today. To date, in these developments, there has been a bias towards the cognitive and relative neglect of the affective. Of course nobody denies the role of ‘affect’ or emotion in learning. Certainly teachers know that it plays a crucial role in motivation, interest, and attention. Research has demonstrated, for example, that a slight positive mood does not just make you feel a little better but also induces a different kind of thinking, characterized by a tendency towards greater creativity and flexibility in problem solving, as well as more efficiency and thoroughness in decision making[7]. These findings suggest emotion may be an important factor in learning and point to new advances in understanding the human brain not just as a purely cognitive information processing system, but as a system in which both affective functions and cognitive functions are inextricably integrated with one another. Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 267-278, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 268 Liping Shen et al. Related Work The extension of cognitive theory to explain and exploit the role of affect in learning is, at best, in its infancy [12]. Kort [9] has proposed a four quadrant learning spiral model in which emotions change while the learner moves through quadrants and up the spiral, yet it has not been empirically validated.. He also proposed 5 sets of emotion that may be relevant to learning, but, no follow-on studies into these basic emotion sets for learning was reported. The Affective Computing Group at MIT’s Media Lab is investigating the interplay of emotion, cognition, and learning as part of its “Learning Companion” project. This project is developing an ‘affective companion’ prototype that will provide emotional support to students in the learning process, assisting them by helping to alleviate frustration and self-doubt [1]. Studies carried out by the AutoTutor Group have provided evidence for a link between learning and the affective states of confusion, flow and boredom [4]. For user emotion modeling, Russell’s two-dimension ‘circumplex model of affect’ [14], where emotions are seen as combinations of arousal and valence, is widely referenced. The OCC [11] model has established itself as the standard appraisal model. This model specifies 22 emotion categories based on valenced reactions to situations constructed either as being goals of relevant events, as actions of an accountable agent, or as attitudes of attractive or unattractive objects. Conati and Zhou are using the OCC theory explicitly for recognizing user emotions in their educational game Prime Climb [3]. Katsionis and Virvou have adapted OCC theory to model students’ emotions while they learn in an educational game [8]. Beyond education applications, there is also relevant work underway such as that by Hanjalic and Xu who represent and model video content (in their case, movies) with emotion tags to support personalization that can be used for applications such as the automatic generation of ‘video highlights’ or personalized recommendations for video films [6]. 2. TECHNOLOGY PLATFORMS The work reported in this paper is based on the integration of an emotion detection system used to augment the operation of a cutting-edge intelligent environment testbed in Colchester known as the iDorm (intelligent Dormitory), with a massive eLearning test bed in Shanghai. The pervasive e-Learning platform (Fig1) was developed by the School of Network Education of Shanghai Jiao Tong University [17]. It delivers fully interactive lectures to PCs, laptops, PDA, IPTV and mobile phones. It also includes a number of, what are called, "smart classrooms". The lecture material can be accessed by students both in real-time (i.e. live) or from an archive (within minutes of the lecture finishing). There are more than 15000 Students in Network Education College, most being part-time students. They have different backgrounds with dynamic knowledge structures. Given such diversity, it is important to provide personalized learning services and to create learner profiles for students the system has harnessed data mining technologies [16]. The intelligent Dormitory (iDorm) [2] is a cutting edge test-bed, based at Essex University, for pervasive computing taking the form of a digital home The operation of the iDorm is orchestrated by intelligent agents and the ooccupants of the iDorm Exploratory Research on an Affective e-Learning Model 269 utilize a variety of networked services, including e-Learning (the iDorm is University based, and occupants are frequently learners). Thus iDorm and the Smart Classroom share much in common. As part of the iDorm work, Leon et al. have developed a realtime emotion detection system, which achieved an 85.2% correct recognition rate in experiments involving three emotional categories, (neutral, positive, and negative), on 8 subjects [10]. This approach comprises an eXperimental Vital-sign-based Emotional State Transmitter (X-Vest), a finger clip with built-in sensors providing physiological signals (heart rate - HR, skin resistance -SR, blood volume pressure -BVP, gradient of skin resistance –GSR, and speed of the changes in the data -CS). It recognizes affective changes using a combination of Auto Associative Neural Networks (AANNs) and sequential analysis (UK Patent 0611762.6). Figure 1. Pervasive eLearning Platform in Shanghai 3. AFFECTIVE LEARNING MODEL 3.1 Rational & R&D Strategy Russell and Kort’s models share a common axis: the emotional state. If, during learning, emotion is found to change in a consistent manner then this would provide a means to study how learning behaviors relate to emotion (and vice-versa). At a simple level this might be employed to provide teachers with feedback on a learner’s emotional state (useful for remote learning where there are no visual cues). Moreover if, during learning, the transition between emotional states on Kort’s model displays some kinds of loops then this would indicate a tighter coupling between Russel and 270 Liping Shen et al. Kort’s models, opening the possibility for the theory associated with these well established models (e.g. Kort’s affective learning spiral) to be applied to emotionaware e-Learning systems. Thus our experimentation focused on gathering data to explore the affective evolution during learning and the relationship between Russell and Korts Models. 3.2 Affective Learning Models As Picard [13] stated, “Theories of affect in learning need to be tested and evolved. However, there is still very little understanding as to which emotions are most important in learning, and how they influence learning. To date there is no comprehensive, empirically validated, theory of emotion that addresses learning”, so as a first step, we will use our prototype to fix the user emotion space using Russell’s ‘circumplex model’. We will then use the emotion states (personalized to the user) detected during learning process to empirically validates Kort’s ‘Learning Spiral Model’. The following is the description of these models and our rationale for exploring the relationship between these two models. 3.3 Russell’s Circumplex Model of Affect Figure 2. Russell’s circumplex model of affect In Russell’s circumplex model of affect (Fig2), emotions are distributed in a system of coordinates where the y-axis is the degree of arousal and the x-axis measures the valence, from negative to positive emotions [13]. This model focuses on subjective experience which means emotions within these dimensions might not be placed exactly the same for all people. In fact, Figure 3 is the author Russell’s own dimensional model of emotion. Exploratory Research on an Affective e-Learning Model 271 Whilst Russel provides a comprehensive set of emotions, these are not well matched to our more focused application of learning, and are too numerous for selfassessment tests; therefore we have chosen a carefully selected subset and additions to explore a basic emotions for learning, namely, interest/curiosity, engagement, confusion/comprehension, frustration, boredom and hopefulness/optimism. At this stage it is not clear that we have the optimum set for our needs, rather this is a starting point and undoubtedly this may evolve or take many investigations before it is well established. 3.4 Kort’s Learning Spiral Model Kort [9] has proposed a four quadrant learning spiral model in which emotions change while the learner moves through quadrants and up the spiral (Fig3). In quadrant I the learner is experiencing positive affect and constructing knowledge. At this point, the learner is working through the material with ease and has not experienced anything overly puzzling. Once discrepancies start to arise between the information and the learner’s knowledge structure, they move to quadrant II, which consists of constructive learning and negative affect. Here they experience affective states such as confusion. As the learner try to sort out the puzzle but fails, he might move into quadrant III. This is the quadrant of unlearning and negative affect, when the learner is experiencing states such as frustration. After the misconceptions are discarded, the learner moves into quadrant IV, marked by unlearning and positive affect. While in this quadrant the learner is still not sure exactly how to go forward. However, they do acquire new insights and search for new ideas. Once they develop new ideas, they are propelled back into quadrant I; thus, concluding one cycle around the learning spiral of Kort et al. As learners move up the spiral, they become more competent and acquire more domain knowledge. Figure 3. Kort’s Learning Spiral Model 272 Liping Shen et al. 3.5 Rationale for Exploring Relationship between Russell and Korts Models Russell and Kort’s models share a common axis: the emotional state. If, during learning, emotion is found to change in a consistent manner then this would provide a means to study how learning behaviors relate to emotion (and vice-versa). At a simple level this might be employed to provide teachers with feedback on a learner’s emotional state (especially useful for remote learning where there are no visual cues). Moreover if, during learning, the transition between emotional states on Kort’s model displays some kinds of loops, then this would indicate a tighter coupling between Russel and Kort’s models, opening the possibility for the theory associated with these well established models (e.g. the affective learning spiral) to be applied to emotionaware e-Learning systems. Thus our initial experimentation has been focused on gathering data to explore the affective evolution during learning and the relationship between Russell and Korts Models. 4. PRELIMINARY EXPERIMENTS AND RESULTS The preliminary experiment was carried out in the intelligent inhabited environment, iDorm2. 4.1 Methods The participant was a female visiting scholar who lived and worked in the iDorm2. During the experiment, she wore the X-Vest which provided the valence value and raw data from 5 biosensors. Data from the X-Vest was collected every 2 seconds. As Skin Resistance (SR) is a very good indicator of arousal [12], we used the raw SR data to linearly evaluate and track, in real-time, the arousal value. A low level of SR denotes high arousal and vice versa. We observed that the subject’s skin resistance can vary by as much as a factor of ten between morning and evening. To settle this diurnal SR variation problem, we introduced a dynamic normalization (averaged over the previous 5 minutes). The participant was asked to conduct the experiment twice a day for 5 days, wearing the X-Vest and collecting arousal and valence data while she was learning. In this preliminary experiment, the learning process and learning materials are not pre-designed, i.e. the subject and learning materials are selected by the participant herself. Each learning session lasted at least 30 minutes. The arousalvalence data was displayed, in real time on a colored four quadrant diagram. Each time the system detected a change of emotion, a multi-choice dialog was triggered, listing six basic emotions, from which the participant had to select the nearest match to her current emotion. All the raw data, arousal, valence, and self reports were recorded together with time tag in a data file for further study and analysis. 273 Figure 4. Skin Resistances Experimental Data Exploratory Research on an Affective e-Learning Model 274 Liping Shen et al. 4.2 Results Experimental data was gathered from 9 learning sessions and 1 TV session (unstructured entertainment). Each session lasted at least 40 minutes with 4 sessions including a self-reporting function. Arousal Results From the data, we observed the following interesting characteristics about how arousal evolved during learning process: 1. During a single learning session, the arousal remains relatively stable. The standard deviation was found to be around 100 K-Ohms (Fig 4b, c). 2. During the TV session, the arousal varied greatly and the standard deviation was as large as 846 K-Ohms (Fig4a). This is consistent with the unstructured nature of the material. 3. Arousal was not only the result of learning, but was influenced by other factors such as physical exertion. For example, the participant reported that she was more aroused to learn when she walked to and fro (which she usually does when she feels tired or sleepy). The recorded SR data revealed this phenomenon (Fig 4d). 4. From Figure 4b, c, d, it can be seen that when the participant was learning, the arousal was usually moderate, i.e. not too high or low. Self-reporting Results Russell’s two-dimension model of affect focuses on subjective experience; as such, emotions within these dimensions might not be placed in exactly the same for all people as it relies on personalities and the diversities of language (i.e. understanding and expression of words differs greatly on culture and self-experience) and what’s more, one emotion does not have a single fixed value in the Russell’s space [15]. Thus, to work on Russell’s model, we need to locate the experimenter emotions within this space. As explained earlier, we adopted a set of 6 basic learning emotions to locate the participant within the Russell’s space. From the self-report data (Table 1), for this participant, we observed confusion and engagement were the two most frequent emotions her learning, whereas the frustration and boredom rarely occur. The standard deviations are all very large, so we have chosen to use an 80% confidence interval for each emotion. Figure 5 gives the emotional valence-arousal space of the participant. Table 1. The means, standard deviation and confidence intervals of 6 basic emotions Emotion Interest Engagement Confusion Frustration Boredom Hopefulness valence arousal valence arousal valence arousal valence arousal valence arousal valence arousal Mean 0.655 208.276 0.918 171.265 -0.569 74.845 -0.667 -199.667 -1 -333 0.917 -122.250 Standard deviation 0.484 252.428 0.277 216.681 0.5 334.884 0.577 70.002 0 76 0.289 208.722 80% Confidence intervals (0.537, 0.773) (146.751, 269.8) (0.867, 0.97) (131.042, 211.489) (-0.654, -0.484) (17.831,131.859) (-1, -0.038) (-275.876, -123.458) -1 (-409, -333) (0.803, 1) (-204.400, -40.1) Probability (%) 19.1 32.2 38.1 2.0 0.7 7.9 Exploratory Research on an Affective e-Learning Model 275 Figure 5. Participant’s Emotional Space Affective Loop Results Kort has suggested that learning behavior would manifest itself in a spiral-like form i.e. a series of linked cycles separated in time. From our data we observed three loops across the 4 quadrants (the red, green and purple loops) during a 15 minutes learning process (Fig6). In addition, like all real-life processes, they are not idealized forms, rather a noisier (e.g. our recognition rate is around 85%) and less smoothly formed geometry. Even at this early stage of our work, these results suggest that there is an approximately spiral nature to this data, although clearly we need more data and better visualization to confirm this. We believe that learning loop depends on the learning material and learning activity but, again, these need further study to validate. However, we hope these initial results will prove encouraging to others who have speculated on this relationship and hopefully will motivate more detailed work on this aspect. Figure 6. Affective Loop during Learning Process 276 Liping Shen et al. 5. DISCUSSION Whilst our research is still ‘work in progress’, even at this early stage we have uncovered some interesting results, such as: • During a single learning session (up to 40 minutes), the arousal is relatively stable • People usually learn best in a state of moderate arousal. • Arousal is not only the result of learning, but is effected by other factors. • The participant’s emotional space was compatible to that of Russell’s model. • Our experimental data reveals some kinds of learning loops which, to some extent, validates Kort’s model (although more experiments and analysis are needed) The results we have reported in this paper are of preliminary experiments that, whilst very encouraging, are still very coarse and need further refinement. In particular we flag the following issues for additional research: • Kort’s learning spiral model is restricted to a constructive approach and it needs to be broadened out to include other ‘types’ of learning process, for example, conceptualization and identification [5]. • The learning material used to evaluate this model needs to be more formally designed to reveal learning behaviours, be more diverse and representative. • There are factors, other than learning, that could influence emotion; for example, who people are learning with; what they are learning; where they are learning and so on. It may be that combining these variables at the right degree is the key to a better affective learning model. • To simply use skin conductivity as the sole indication of arousal is too crude. There needs to be some investigation as to how more reliable arousal can be obtained from physiological signals. Likewise, ideally valence would be continuous than discrete. • Our current experiments are based only on one participant; clearly, to make the results more reliable, we would need to have a bigger and more controlled sample. As should be clear from our discussion, this paper is work-in-progress and we are reporting results from the first phase of a much longer term research program. Our immediate aims are to refine the arousal analysis, design structured learning material, and gather data from more participants. After that we plan to develop the affective learning model combining affective information with wider learner profiles and the existing Shanghai architecture. Finally we aim to deploy it in the Shanghai e-Learning platform and evaluate it with real learners. We will look forward to report on this work as it moves from research to real deployment over the coming years. 6. ACHNOWLEDGEMENTS This is a collaborative research project conducted between the Digital Lifestyles Center, University of Essex, UK (http://digital-lifestyles.essex.ac.uk) and e-Learning Center, Shanghai Jiaotong University, China (www.dlc.sjtu.edu.cn ). During the Exploratory Research on an Affective e-Learning Model 277 formation of the research ideas and models, we have enjoyed many useful discussions with our colleagues. We would especially like to thank Chris Fowler for his valuable guidance on the role of arousal in learning, Graham S. Clarke for his inspiring ideas, Malcolm Lear for the biosensor technical design and support, my student Chenping Lu for his programming work, and Michael Gardner for his strong support which has been important in making this collaboration possible. REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. Burleson W., Picard R. W. et al, A Platform for Affective Agent Research, Workshop on Empathetic Agents, International Conference on Autonomous Agents and Multiagent Systems, New York, NY, July 2004. Callaghan V, Clark G, Colley M etc. ,Inhabited intelligent environments, BT Technology Journal – Kluwer,Vol.22, No.3, July 2004 Conati, C. & Zhou, X. (2002) “Modeling students’ emotions from cognitive appraisal in educational games”. In S.A. Cerri, G. Gouarderes and F. Paraguacu (Eds.) : Intelligent Tutoring Systems 2002, LNCS, 2363, pp. 944-954, 2002. Craig S. D. , Graesser A. C., Sullins J. etc., Affect and learning: an exploratory look into the role of affect in learning with AutoTutor, Journal of Educational Media, Vol. 29, No. 3, October 2004 Fowler C.J.H., Mayes J.T., Learning Relationships from Theory to Design, Association for Learning Technology Journal, 1999, Vol. 7 No.3, 6-16. Hanjalic A., Xu L.Q., Affective Video Content Representation and Modeling, IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 7, NO. 1, 2005 Isen A M: ‘Positive affect and decision making’, in Lewis M and Haviland J (Eds): ‘Handbook of emotions’, Guilford, New York, (2000). Katsionis G., Virvou M., Adapting OCC theory for affect perception in educational software, Proceedings of the International Conference on Human-Computer Interaction 2005 (HCI 2005), Las Vegas, USA. Kort B,Reilly R,Pcard R.An Affective Model of Interplay between Emotions and Learning: Reengineering Educational Pedagogy-building a Learning Companion. Proceedings of IEEE International Conference on Advanced Learning Technologies,2001. Leon E., Clarke G., Callaghan V., A User-independent Real-time Emotion Recognition System for Software Agents in Domestic Environments, to appear in: Eng. App. of Artificial Intelligence, Int’l Journal of Intelligent Real-Time Automation, summer 2006. Ortony A., Clore, G. L. and Collins, A., the Cognitive Structure of Emotions, Cambridge University Press, Cambridge, UK, 1988. Picard R W and Scheirer J: ‘The Galvactivator: a glove that senses and communicates skin conductivity’, Proceedings from the 9th International Conference on HumanComputer Interaction, New Orleans (August 2001). Picard R W, Papert S, Bender W, etc., Affective learning — a manifesto, BT Technology Journal • Vol 22 No 4 • October 2004 Russell, J.A. (1980) A Circumplex Model of Affect, Journal of Personality and Social Psychology, 39:6, pp. 1161-1178, American Psychological Association. Russell J. A., Core affect and the psychological construction of emotion, Psychological Review, 110, 145–172. 2003 278 16. 17. Liping Shen et al. Shen L.P., Shen R.M., Ontology-based Intelligent Learning Content Recommendation Service, International Journal of Continuing Engineering Education and Life-Long Learning, v 15, n 3-6, 2005, p 308-317 Shen R.M., Yang F., Han P., A Dynamic Self-organizing E-Learner Communities with improved Multi-Agent Matchmaking Algorithm IN:AI2003, Dec.3-5, 2003,Perth, Australia Design and Development of Blended Learning through LMS K. P. Hewagamage, S.C. Premaratne, K.H.R.A. Peiris e-Learning Centre, University of Colombo School of Computing, Email: {kph, scp, rap @ucsc.cmb.ac.lk} Abstract. Design and Development of Blended Learning is a challenging task than it is anticipated when we consider features and facilities in a Learning Management System (LMS). In this paper, we present what those challenges and how we have solved both technical as well as social problems in implementing a blended learning environment for the University of Colombo School of Computing (UCSC). It is very hard to achieve what we plan without all stakeholders participation with positive attitude. ICT awareness is very important for the success of blended learning but it is not everything for its success. More learning happens when students start engaging in activities which overlap both the classroom as well as the virtual learning environment (VLE). We also present our experience in activity mapping as a way to extend the classroom environment Keywords: blended learning, VLE, LMS, activity mapping, learning activities 1. Introduction In a self-evaluation of undergraduate programs, UCSC (University of Colombo School of Computing) discovered several problems which are affecting negatively for the internal student courses. In the analysis of these problems, the faculty decided to use e-learning based methodology as a part of enhancing learning environment of internal students. The e-Learning Centre of UCSC was given the responsibility of carrying out this initiative of setting up the blended learning environment. However, this project faced so many challenges than anticipated in the very beginning. In this paper, we are presenting our experience in developing a blended learning environment for undergraduates’ courses conducted by the University of Colombo School of Computing (UCSC). The faculty and students are ICT literate community since their main discipline is computer science. Therefore, the background of faculty and students is enough to undertake the responsibility in participating e-learning based activities. Technical infrastructure of the UCSC and technical knowledge of users (staff and students) in the organization are not enough to achieve objectives of the e-learning framework. Those two factors are like front wheels of a vehicle and we need 4 wheels with good set of tires for a smooth and steady journey. Irrespective of power and capabilities, obstacles are unavoidable in this journey and we should be prepared to face these challenges. Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 279-291, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 280 K. P. Hewagamage, S.C. Premaratne, K.H.R.A. Peiris We first describe on our vision of improving the relevance and quality of undergraduate education, which we have quoted as IRQUE project of the organization. In this project, we have carried out several evaluations with respect to education process, initial capabilities of students including their soft skills, final outcome and output of the degree program, and staff capabilities and weaknesses. Before building a blended learning environment, such evaluation is very important to identify the background and foundation. e-Learning Centre (eLC) was established in order to assign the responsibility of carrying out e-learning integration of all the academic and professional services of UCSC. A specialized staff was recruited in planning and development of virtual learning environment (VLE) for all internal undergraduates. However, many of issues faced in this project didn’t have ready made solutions and we had to carry out research and development work in finding suitable solutions. In the section 3 of the paper, we will summarize initial research and development work related to the blended learning environment. We identified the VLE is the heart of a blended learning environment and its proper integration with the classroom teaching is the key to initiate the change. It is important to consider the VLE as an extension of classroom activities rather than the supplementary system to access learning resources. At the same time, VLE should provide facilities to implement collaborative learning activities among students in the classroom. Some of these activities will be initiated and carried out independent of the teacher/lecture of the classroom. In this paper, we describe challenging issues we have encountered in this endeavor together with our solutions so far in building blended learning environment in Section 5. Finally we conclude the paper by summarizing the work presented together with our future plan in enhancing our blended learning environment in the Section 7. 2. Improving Relevance and Quality (IRQUE) What is the quality and relevance in higher education study programs? There are different interpretations about them but simply they are some sort of indicators about the study programs and its applicability in the industry. The quality can be measured using student performance in the subject evaluation and their ability to apply subjective knowledge in problem solving. The relevance is usually measured with respect to their ability provide solutions at very beginning of first employment. Generally, there is a gap between employer’s requirement and graduates capabilities. If graduates have enough soft skills, then they can reduce the gap through a simple training program. However, if graduates cannot demonstrate their capabilities to reduce the gap, they will be unemployed or underemployed. At the same time, these two attributes are not completely independent ones and they are dependable on each other. The most important process which is required to enhance the education process in which teachers and students interact each other is the transformation of the knowledge. However, we identified under the IRQUE evaluation that teacher student ratio has adversely affected due to the increase of intake by 400% in recent batches of Design and Development of Blended Learning through LMS 281 under progress. Teachers are delivering lectures to large number of students (around 120 – 160 students) at once and there is no dialog between teachers and students. Both parties find difficulties to involve in any sort of dialog in front of all these students. All lecture materials (usually power point slides) are given in advance of lectures. There is no feedback about lecture notes from students to teachers. Reading those lecture notes is not interactive and students are helpless in their learning process. In this evaluation, we observed the failure rate and drop out rate are also increasing. The lecturer and students ratio is not so bad but the interaction between two parties seems to be poor. Although students have enough computer lab access facilities together with Internet, electronic learning resources are not provided through this infrastructure. Library does not have enough copies of recommended reading materials and no digital versions of those recommended reading materials are provided to access through the local area network of the organization. It was also not possible to give attention to different levels of students when there are around 160 students in a lecture hall. These students cannot be grouped according to their skills and provide different lecturing due to administrative and resource constraints in the organization. Most of students have language difficulties and they need a special attention due to poor understanding contents/factors taught in the lectures. 3. Establishment of e-Learning Centre UCSC is a computing faculty of University of Colombo (UoC) and its faculty wanted to find out IT solutions to minimize bad effects in this process. The use of information and other technologies are not a proper integration of IT and education. Sometimes, it could diversely affect the education process. For example, the use of power point slides in the lecture halls reduces student active role in the process. Students are very keen and enthusiastic to have more and more notes and nice illustrations on visually projected screens but many students fell in sleep and may not properly absorb the required knowledge at the end of teaching sessions. When the UCSC is formed as an autonomous faculty of UoC, the importance of eLearning and its future role is identified [4,7] and a separate centre was established to provide the service of e-Learning to students and staff while promoting R&D work in this area. A coordinator was appointed among academic staff members who have experience of practical difficulties and challenges faced in the academic education process. A separate staff was recruited for the system administration as well as content development in e-learning centre. Initially the focus was towards the distance learning initiative of the UCSC. However, as mentioned above, after IRQUE evaluations, the importance of proper integration of technology with the education process is highlighted specially due to sudden increase in the intake of students (from 40 to 160 students). 282 K. P. Hewagamage, S.C. Premaratne, K.H.R.A. Peiris 4. Learning Management System (LMS) The importance of learning management system was identified as a significant requirement. Several systems both commercial and open source products were evaluated [2] considering various factors such as network resources and infrastructure, resource availability for sustainability (i.e. funds allocated from the institute budget), customization of the product for specific requirement of the organization, external acceptance of similar academic organization, etc. We first evaluated several commercial products WebCT, Blackboard and Theducation (a Swedish product localized to English), but they were not selected specially due to two reasons, the customization of the product and the cost of product which was based on the number of users of the system. However, when we evaluated Open Source Products, those two critical issues in the commercial products were not major issues. When we evaluated the community around these products, Moodle (Modular ObjectOriented Dynamic Learning Environment) seems to be acceptable FOSS product with a word wide user group. At the same time, the computing resources required for the deployment of product are something which is commonly used in a web server (i.e. Apachie, PHP, MySQL). After several testing, eLC obtained the approval from the academic board of study to use the Moodle LMS as the learning management system to implement the blended learning environment of UCSC. 5. Social Resistance for the Blended Learning When the term e-Learning was used among the academic community as a way to enhance education process, there was some resistance among some faculty members just like the way the computerization or ICT enable environment is introduced for any working environment. At the beginning, it was a surprise for us since UCSC is a computing faculty which advocates its theoretical & engineering development as well as integration of computing with other working environment. However, every human teacher will have the common fear of ICT, when he/she did not properly understand the potential of the ICT in education. Hence, the very first challenge of eLC in its blended learning initiative was a social constraint due to the various interpretation of e-Learning and its applicability of e-Learning. Negative Arguments against the Blended Learning Initiative: • Technology adversely affects students’ active status in the classroom [this argument was based on the use of power point slides in a classroom teaching. When students are not taking down notes, student attention may vary and some of them will start engaging other activities like chatting, sleeping etc.] • Accessibility of technology during classroom [all students do not possess laptop or any portable devices when lectures are delivered] • Imbalance of Number of Computers: Students ratio and its adverse affect in allocating enough time to access LMS or Internet. • Increase in the teachers’ workload Design and Development of Blended Learning through LMS 283 • Teachers’ unawareness of facilities and methodologies [Many of senior staff have highly academically qualified researchers but they do not have a special training about teaching methods] Positive argument for the Blended Learning Initiative; • Possible solution being tested [6,5] • Technology infrastructure has already been established • ICT-awareness is above the required level (UCSC is a computing faculty) Action taken to promote the blended learning: • Special demonstrations are held about LMS to senior staff. • Several workshops were carried out to educate the academic staff (Workshops are carried out with respect to small groups since it was difficult to get all staff committed to a single time slot). • A LMS Administrator was appointed to provide on demand help for academic staff • All research assistances at eLC are advised to work closely with respective teachers (senior academic) in setting up relevant courses in the LMS. These actions didn’t give 100% solution to social issues in implementing a blended learning environment. However, it was a satisfactory beginning. Student training and evaluation of the service: Documentation doesn’t solve all problems faced when a virtual learning environment is introduced for the blended learning. A demonstration followed by a proper training is very crucial even before the starting formal academic courses. Training is usually conducted as hands on workshops for small groups at a time. In this training, LMS features are introduced illustrating how students can participate in the blended learning activities. 6. Technical Challenges to introduce LMS When a new system is installed, it is natural to face many technical problems which are not unforeseen during the testing stage. Especially when a system is being used more than 100 students at the same time. Therefore, it takes some period of time for the system to become stabilize. At the same time, we are using an open source product, it is very important to keep a good eye on possible bugs in the system and carry out regular update with the system. At the same time, it is not advisable to experiment with an online running system which has real users. Proper software engineering approach must be utilized in the maintenance of the system. Therefore we sat up an identical offline copy of the same system, for testing when system updates are posted. The LMS admin evaluates updates/patches of the system with the offline system, before updating them with an online real system. Taking the back up is a critical requirement since it is hard to justify when what will be happened. It could be a technical failure of hardware or software but eLC must have to accept the damage if something is happened. Taking the backup is not enough and it doesn’t give any validation or verification that the backups can be restored. We found the offline testing version is very useful to verify these backup taken. 284 K. P. Hewagamage, S.C. Premaratne, K.H.R.A. Peiris Taking a backup once in a month was also not enough; we had to develop a policy for taking backups. Our policy was based on following factors: • Time between backups is fixed to one day [Incremental approach for the backup taking was implemented] • Backups are supposed to take when LMS activities are low. • Teachers were instructed to maintain their course backup [Many teachers were not happy with this request but we wanted to convince it is a responsibility of teacher] One of the big threats is Internal hacking of LMS by students. The direct hack through server as well as hacking through the LMS is another issue. Sometimes, friendly hacking is done by their friends. We usually ask students to update their profile with the personal email address but students are usually very careless. They also do not get any official email address and it also affected the official communication. Simply, LMS login and other system accesses must be linked through a central system of the organization in order to avoid some of technical failures. Network congestion A computer is multi-tasking equipment and students (as well as other users) usually use a computer for several activities at the same time. Many of these applications use network/Internet bandwidth in addition to the processor and memory of the machine. A typical example is that the user may open a web browser with several windows to access different web sites, a messenger application, streaming application for audio or video content or an online game. When a student logs into the LMS with all these active applications, accessibility is not quick as they want. Depending on the resources, the users must select relevant applications without disturbing to their own as well as others in the laboratory (for example, when a student starts downloading a big file, it affects everybody in his/her network). In addition to all these things, many student machines in computer labs are infected by viruses and it creates unnecessary traffic in the network system. Therefore, it is very important to establish a resource access policy when the blended learning environment is being introduced. This policy could be a guideline to reduce the problems that users face when they access the virtual learning environment. It is not very difficult to introduce a policy but people hardly follow these policies and keep complaining the system and the network. 7. Activity Design for Blended Learning: This is the hardest part when the virtual learning environment (VLE) is introduced to enhance the classroom environment. It is not possible to implement all classroom activities through the VLE but some activities can be enhanced by overcoming limitations in the classroom environment. At the same time, a new set of activities can be designed and developed using the features of VLE, and if properly used, the efficiency and effectiveness of the whole education process can be enhanced significantly. We discuss mapping of some selected classroom activities for the blended learning based on our experience in this section. Design and Development of Blended Learning through LMS 285 i. Classroom Activity: Power point based lectures Description: A lecturer usually conducts one or two hour lectures in a lecture room for 40 – 160 students at a time. (Depending on the classroom, some lectures use a microphone otherwise students find very difficulty to hear the lecturer’s narration). Lecturer may ask some questions but students hardly answer when there is a large number. Students can ask clarification from a teacher but it is not commonly practiced in the classroom environment. Students may get a printed note set in advance and he/she may take down some notes based on the explanation in the lecture session. If a student looses his attention, he may find difficult to understand what is presented. A teacher may find very boring exercise when he/she has to repeat basic details. Sometimes, the lecturer may late and he may not give the full description due to time constraints in the schedule of classrooms and/or the lecturer has to complete more content with respect to syllabus. Blended Design: Lectures are recorded based on the curriculum without any time constraints. Based on the topics and sub-topics in the syllabus, recorded videos (lecturer’s face and power point slides synchronized) are segmented for short durations. A segment size may be 15 – 30 minutes (It was recommended to keep segment size 15 minutes if possible). However, it was not possible to distribute these video materials through LMS due to their size and available bandwidth in the network. Instead of normal lecturer, these video presentations were used in the classrooms and after every 15 minutes, the lecturer or tutor, stops playing the video presentation for a discussion or questions. Effectiveness is high but time management became a problem to cover the syllabus. Therefore, these resources are made available from the library. ii. Classroom Activity: Question and Answer ( Q&A) Session Description: A lecturer usually spends last 5 -10 minutes in every lecture to discuss difficult things that students cannot understand and answer some of their questions related to the session. All students do not get this opportunity and many of them do not want to ask questions in front of others since they are not sure about the question. Blended Design: The course page, in the LMS, is structured with respect to syllabus and a discussion forum is added at the end of every section. In the classroom, when a lecturer completes a section, he/she reminds students to ask relevant questions under this discussion forum. If there are no questions in a section, the teacher will ask a random question to initiate the dialog. In the classroom environment, when a student asks a question, only a teacher/tutor answers this question. However, in the discussion forums, other students also started answer questions by promoting collaborative learning environment which was not easy to establish in a normal classroom. 286 K. P. Hewagamage, S.C. Premaratne, K.H.R.A. Peiris Fig. 1. Course Page in the Moodle LMS - Based on the sections in the syllabus of the course, lessons and other resources are added iii. Classroom Activity: Giving References for Topics Description: It is a common practice that a lecturer gives references during a session. This could be a reference of recommended reading materials, a web resource or a personal note. In reality, the students find various difficulties to follow up these references and there will be no dialog between teacher and students about these references. Blended Design: The teacher can add all these references as resources under the relevant topic in the course page of LMS. He/she can scan relevant pages of recommended reading pages and upload as resources in this course page. Students can download them for offline reading without waiting to access materials from the library. iv. Classroom Activity: Revision Exercises (MCQ practice questions) Description: During lecturers or after lecturers, students usually get exercises as practice questions. Sometimes they may get answers to verify themselves. However, there is no feedback Design and Development of Blended Learning through LMS 287 from lecturers or tutors directly. Question type also vary from the multiple choice questions (MCQ) to structured or essay type questions. Blended Design: At each section of course page, there is a practice question covering most important key questions. However, the question type is limited to MCQ and closed type questions. It was possible to provide essay type questions but the feedback was not possible to automate. v. Classroom Activity: Open Discussion of selected topics of interest Description: In reality, open matters related to a particular subject is rarely discussed during a lecture or after a lecture. Occasionally, some students meet a lecturer to discuss these types of issues outside the classroom. However, they are never heard by other students in the same class/group except the lecturer mentioned them in the next session when he meets all students. Fig. 2. Open Discussion among Teachers and Students Blended design: If a student has an open issue or question, he/she can post it in a discussion forum of relevant section or news forum of the course. Posting to a news forum bring the attention of all students and teachers in the course. Teachers can also initiate interesting dialog by posting open issues of the subject (See Fig. 2). 288 K. P. Hewagamage, S.C. Premaratne, K.H.R.A. Peiris vi. Classroom Activity: Formative Assessment Description: According to curriculum, all courses offer around 40% of marks from the formative assessment and the rest of marks from summative assessment at the end of semester. This formative assessment is usually based on the assignments given during the course period. Some of these assignments are individual ones and others are group work. In reality, there will be several administrative difficulties teachers as well as students faces when they are supposed to submit assignment work. For example, printers in the lab may be difficult resource to use near the deadlines, and some students later complain they submit on time but no administrative assistants to accept the submission just before the deadline. Blended Design: The distribution of assignments and submission of assignments were easy to manage or handle when those activities are channeled through the LMS. It saves lots of resources (when 160 students have to submit soft copies only) and makes it efficient since the teacher and assistants do not want to keep the manual record of submissions. Students can directly submit the report from the LMS. The deadline can be easily implemented without causing administrative problems. vii. Classroom Activity: Tutoring Description: When lecturers are difficult to understand, tutorial classes are conducted for small groups by tutors. More exercises are usually discussed in these classes and their success depends on the skills of tutors. Students have mix opinions on these classes and attendance is poor or irregular. Blended Design: Interactive e-learning lessons are designed as additional learning resources for all students. They are usually developed by instructional designers at the e-Learning Centre according to teachers’ (Subject Mater Experts) explanation of learning content. Instruction designer will use different presentation to simplify the learning content. Figure 3 shows a sample image of such interactive lesson. viii. Classroom Activity: Group Work Description: When a group work is given, it is important that all members discuss the given work and distribute among all members the work. In reality, all members are not active and sometimes some members complain that they do not get a good opportunity to participate in these activities. At the same time, there are passengers in groups, who get full marks without doing anything. Lecturers usually use lot of adhoc techniques to allocate marks among group members but they are not very successful. Design and Development of Blended Learning through LMS 289 Blended Design: When the group work is given, students are categorized as groups within the course of LMS. All students are supposed to participate in the discussion only through group forums which are not visible to other students. When they have classroom meetings, the group leader has to write minutes of meeting in the group wiki where others can make comments. Teachers use this information when they allocate marks among group members. Fig. 3. Interactive Learning to support tutoring in the blended learning ix. Classroom Activity: Collaborative Construction of Artifact Description: Many assignments are group work and each group must submit the group report on or before the given deadline. Sometimes these reports are copied from past reports in the library. The collaboration of group work is not visible among participants. Teachers find some practical difficulties when they correct these reports and allocate marks among group members. Blended Design: Group reports are required in a form of a wiki. Teacher can see its development progress and can also find out their collaboration. Interestingly, teacher can give his comments before the final deadline which is not an option in the normal classroom environment. 290 K. P. Hewagamage, S.C. Premaratne, K.H.R.A. Peiris x. Classroom Activity: Sharing personal notes Description: Some talented students prepare their own notes based on the lectures and syllabus. They are not shared and sometimes not visible for teachers. If teacher can identify these talented role players, it is very easy to promote collaborative learning model in a classroom environment. Blended Design: All students in the course are allowed to maintain their own blog or journal as their personal notes. Teacher may examine such blogs if they get more attention from other students (based on the LMS statistics). When a teacher wants to give a recommendation or additional marks, he/she will consider these notes. 8. Summary and Future work In this paper, we have descried our experience in developing a blended learning environment for internal undergraduates of University of Colombo School of Computing. The requirement of blended learning was clear when the batch size of student intake was increased by 400%. When e-learning is introduced to enhance the classroom learning, it is possible to get several resistances from both teachers and students. Some of resistances are due to lack of knowledge about potential capabilities and others are due to visibility of activities. These social resistances from the environment can be controlled by assisting and training teachers and students. The virtual learning environment (VLE) given through the learning management system (LMS) is the backbone in introducing blended learning environment. Open source LMSes are feasible products to develop VLE and our experience given in this paper is based on the Moodle LMS. There are several technical challenges which must be carefully and properly handle when any LMS is introduced for the blended learning. Some of challenges are due to other factors such as network congestion as stated in this paper. In order for the success of blended learning, it is very important to use VLE to extend classroom activities rather than supplementary solutions. Therefore, classroom activities must be properly mapped to a blended design. In this paper, we have listed 10 such selected activities to describe our experience. However, those mapping could be further improved to enhance the effectiveness and efficiency in the blended learning environment. According to feedback from teachers and students, we are planning to enhance the activity mapping from classroom environment to VLE. Our immediate plan is to develop a catalog of activities which can be mapped from the real classroom environment to a blended learning environment. The activity mapping will be developed like design patterns [3] for teachers and their assistants when they develop want to develop blended learning environment. We hope this mechanism will be useful in extending classroom activities to integrate the virtual learning environment. Design and Development of Blended Learning through LMS 291 References: 1. 2. 3. 4. 5. 6. 7. K. P. Hewagamage, V. K. Samaranayake, A. R. Weerasinghe, and G. I. Gamage, “Facing Challenges of an External Degree Program using ICT” A Case study of University of Colombo School of Computing”, in the proceedings of Digital Learning 2005 (DL2005), K. P. Hewagamage, K.H.R.A Peiris, W.A.U.C. Weerakoon, “Evaluation and Adaptation of Learning Management System (LMS) for Higher Education”, in the proceedings of Digital Learning 2006 (DL2006), Bangkok, Thailand, May 2006 “Web services for blended learning patterns” Derntl, M.; Mangler, J., Advanced Learning Technologies, 2004. Proceedings. IEEE International Conference 30 Aug.-1 Sept. 2004, “Software Engineering Education Applying a Blended Learning Strategy”, Bunse, C.; Grutzner, I.; Ochs, M.; Peper, C.; Steinbach-Nordmann, S. Software Engineering Education and Training, 2005. CSEE&T 2005. Proceedings. 18th Conference on, Vol., Iss., 18-20 April 2005, “Designing for the changing role of the instructor in blended learning” Danchak, M.M.; Huguet, M.-P. Professional Communication, IEEE Transactions on, Vol.47, Iss.3, Sept. 2004, “The instructional effects of on-line tests on the large-scale IT courses” Nakano, H. etc. Information Technology Based Higher Education and Training, 2005. ITHET 2005. July 2005. “Enhancing the student project team experience with blended learning techniques”, Reichlmayr, T. Frontiers in Education, 2005. FIE '05. Proceedings 35th Annual Conference, Oct. 2005. An Access Monitoring Tool Based on Cookies for Course Management Systems Raquel Hijón-Neira, Ignacio López-López, Ángel Velázquez-Iturbide, Francisco Domínguez-Mateos Universidad Rey Juan Carlos 28933 Móstoles, Madrid, Spain {raquel.hijon,ignacio.lopez,angel.velazquez, francisco.dominguez }@urjc.es Abstract. In this paper, we describe an application developed in our university in order to gain insight of students’ interactions with an e-learning platform. Access to this information can be very useful to teachers, who can rethink the way they present their material based on students’ usage patterns. The application uses cookies as the basis to identify different users. It offers a very wide range of queries as well as diagrams to represent students’ interactions with the platform. The application is portable and can be easily installed at any other type of website that we need to analyze. 1 Introduction It can be expected that the widespread use of Course Management Systems (CMS), both for on-line learning and for blended learning, will affect the way teachers evaluate students. On the one hand, different ways of evaluation can be developed. On the other hand, teachers may use information about the on-line activity of students to evaluate their work. We are interested in the design and implementation of tools and applications to gather information about students’ activity for both purposes. This information may also lead to discovery of learning and usability patterns [1] that may help us to rethink our teaching, or maybe the way we offer the information to the students [2, 5]. We are working and exploding the different ways that the information of students’ accesses to CMSs can be obtained. In [3] we explained the steps that were needed, and its technical requirements. We also found out that there are a wide range of tools that could generate some of this information but none were specifically designed for education and e-learning. In order to fill this gap, we are addressing several alternative ways of obtaining information about students’ activity. Firstly, we developed an ad-hoc application that collects information about students’ accesses to the web site in a dedicated database. Secondly, we analyzed log files using a standard application working on the server. Thirdly, more subjective studies were made by means of survey analysis [4] and equestionnaires [6]. Finally, our most recent approach has been the development of an Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 292-302, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. An Access Monitoring Tool Based on Cookies for Course Management Systems 293 application that uses cookies to analyze students’ interactions with our CMS, visually providing several graphical representations of it (Fig.1 shows the architecture). Fig.1. Tracking Tools assessing URJC CMS In the rest of the paper we describe the application based on cookies. In section 2 we explain our objectives. In section 3 we include an evaluation of existing tools, their pros and cons. In section 4 we give an overview of the functionality, user interface and technologies of the application. In section 5, we describe in more detail its underlying Data Base Description and Cookies Functionality. In section 6, the analysis of the students’ interaction with our tool can be seen. Finally, we describe our conclusions and future work. 2 Objectives of the Application Since we had developed another tool that visualizes nominal accesses, we did not need the personal identification of students entering into the CMS, but a lively and compact tool like this one that allowed us to treat the information from the accesses quickly, online and on real time like it does. The access monitoring tool based on cookies that we are describing implements the following requirements: • Our necessity was the acquisition of information from the students interacting with our CMS, making a distinction between its different pages and parts, such us forums, assignments, theory, etc. • It should provide factual information (in a table format) as well as statistical (in a graphical format) in real time and from any computer connected to the Internet. • It should be shared under a GPL license, and therefore, distributed and upgraded by the user to meet future requirements for free. 294 • • • • • • Raquel Hijón-Neira et al. It should store non personal data from the students accessing the site (operating system, browser, screen resolution, IP and country of origin). It should be able to identify visitors of the CMS in each visit they do to the site. It should track the route followed by each user in each visit. It should have user-friendly interfaces that allow teachers to easily monitor interactions with the CMS. It should provide printable versions of the queries made, selected and grouped under different criteria. It should allow the selection of a time period by a calendar option, and combinations of it, such us: complete days, months or academic years. 3 Evaluation of Existing Applications A wide range of tools that apparently cover the functionality required was analyzed and tested. In the following table (see Table 1), a comparative analysis is shown, where files of the chart refer to the following products: Nedstat [14] European leader in web analysis, it also allows the creation of customized reports. B. Onestat [15] World leader in statistics of the web sites. C. Google Analytics [12] offers information on how the user that found the site, interact with it. D. Estadisticasgratis [10] offers traffic analysis based on the users profile. E. Estadisticas-gratis [11] complete statistical service for free, real time service and 24 hours a day. F. Developers4web [9] Detailed statistics very easy to use. G. Akstat [7] uses markers to measure the web site. H. Logdy [13] very new service, very intuitive and easy to use. I. CQ Counter [8] has an easy to use interface and an extraordinary service. J. XITI [18] offer basic indicators of the web site traffic. K. Weboscope [19] allows multiple web site analyzers. L. Contaweb [16] contains a little element that inserts itself in the web pages that are going to be analyzed. In columns, it is evaluated the most relevant aspects for us, namely: 1. The group of applications that serve our CMS have been developed under the same technology: PHP, MySQL, JavaScript and HTML. So, it is important, that at least, the new application uses the same database, since they have to share tables. 2. Does it use cookies? We studied if they were using this technology, sometimes it was hard to know, since they do not provide us with the source code. 3. Does it do the tracking from analysis of log files? 4. Does it offer detailed time visit analysis by different parameters such us by hour, day, week, month, year, etc? 5. Does it show the kind of operative system the visitors use? 6. Does if offer the type of browser the visitors use to connect to the CMS? A. An Access Monitoring Tool Based on Cookies for Course Management Systems 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 295 Does it offer the information about the screen resolution the visitors have? Does it show the color deepness of users? Does the application bring the plug-ins installed? Does it offer visits´ geographical location? Does it show sources of accesses to the Web? More Common Routes are shown in detail? Does it support local hosting? Does it support multi-language? Does it have a GPL license? Is it a free version? If not, what is the price (in €) per year for an average website of 20 pages? (When their product offers a free and a commercial version it appears both data). 17. Does the system include publicity? 18. Has the application been designed to deal with e-learning and/or CMS? Table 1. Comparative chart of similar products 3 – The application has the characteristic 2 – The application does not have the characteristic ? – By the information offered we can not evaluated it 296 Raquel Hijón-Neira et al. As we can see, none of them has been created to deal with CMS. Furthermore, the free versions some offer are a very weak representation of the features the commercial product has, moreover, these versions support a whole lot of publicity. Thus, either we could not know or they did not use cookies but two of them (D and E), therefore three were not suitable since we wanted to use them in our approach. So, even when paying a fortune, all the requirements are not supported by any of them. 4 Description of the Tool The user interface that the teachers use to query access statistics is shown in Fig. 2. 1 7 2 4 3 5 6 Fig.2. Our Application at the Rey Juan Carlos University In the area 1, a menu allows selecting general facilities, such us: summary of visits, change of password, create another (teacher) user, pages analyzed, obtain PHP code (to allow each new page added into the site to be tracked), summary of the visits, statistics which is the one selected in the previous figure, help about how to use the application, information about it, suggestions and log off. An Access Monitoring Tool Based on Cookies for Course Management Systems 297 If we focus on the part named ‘Statistics by Number of Visits’ (area 2), we can query statistics for a range of dates, select the pages or subjects we want to examine, and summarize the info for different criteria. The graphical format of the results (either a class of graph or a table) can also be selected (area 3) for this query as well as queries in areas 5 and 6. Currently, the following graphs are supported: pie graph, bar chart, line graph, and table. If we click in ‘Most Common Routes’ (area 4), it is shown the most visited sequence of pages. As a parameter of the query, the user is required to give the number of clicks of the sequence. If we click in ‘Number of Visits per page’ (area 5), it is shown the number of visits for a range of dates. If we select ‘Statistics by User’ (area 6), it is shown the user information for a previously selected period of time, grouped by one of the following options: operating system, browser, screen resolution, and country of origin. It is mandatory the selection of a range of dates (area 7). There are options to select a week, month, academic year, as well as a specific date from a calendar that is displayed to enter dates. The application uses cookies to identify the students accessing the site. It was developed with PHP and database is in MySQL. Finally, some JavaScript code allows implementing some special features. The application is available under a GPL license, so it can be distributed and installed in any website. It has a self-installation feature that automatically identifies the different parts of the CMS to be studied and creates the database to store accesses. 6 Data Base Description and Cookies Functionality 6.1 Data Base Description The Data Base has been developed in MySQL, due to the compatibility with the rest of data bases that form the CMS. The structure is formed by five tables that fulfill the data storage requirements (see Fig. 3). In the table 'Users', there are all the data related to the users that have visited at least once our CMS. The attributes are: Users identification, ip address, Operating system, browser, screen resolution and country of origin. In the table 'Pages' are stored all the pages that are visited in the CMS. The page tuple is included the first time a student enters a page. The attributes are: page identification and page name. In the table 'Visits' are stored all the visits that have occurred on the CMS. Each time a student enters a new tuple is created. The attributes are: visit identification, user identification, page identification, previous visit identification (if it exits), day, month, year, week and time. In the table 'Administrator' are stored the information related to the users of the Statistics tool, which are the teachers. The attributes are: user name, password, cookie 298 Raquel Hijón-Neira et al. value, state (identify if the user is authenticated or not, that is to say, if it is a teacher or a student). Fig.3. Entity-Relationship model The table 'Countries' contains the ip sub rank corresponding to each country. The attributes are: identification of register (primary key), integer value that correspond to the beginning of the ip sub rank, integer value that correspond to the end of the ip sub rank, two fields for different codifications of the country (with two and three characters each) and the country name. 6.2 Cookies Functionality As we have been seeing, the users’ identification is being made by cookies. In Fig. 4 there is a description of the process. When a visitor enters into any of the web pages the CMS has, the application validates if the user present the user cookie, in case he/she does, the visit is stored in the corresponding table since the user has been authenticated. In case the user does not present this cookie, the application checks if he/she presents the accept cookie (which is the cookie that is used to check if the user accepts cookies), if this cookie is presented the user information is stored, as well as the user cookie and the visit information. An Access Monitoring Tool Based on Cookies for Course Management Systems 299 Fig.4. Cookies Functionality If the user does not present the accept cookie, the application checks if the user has been redirected , in case she/he has already been redirected, it has been checked that the user does not accept cookies; Finally, if the user has not have been redirected, the accept cookie is saved and he/she is redirected to the home page. 7 An Example of Analysis of Student’s Interaction with the Tool We are going to place here a set of examples of the variety of information the teachers can obtain from the website. We have asked for the number of visits the Software Engineering subject in the fourth course of Computer Science Engineering has had in November of 2006, see the results in Fig. 5. We have selected the information representation in a horizontal bar chart and also in a table form. It can be corroborated that the two days that students most connected to the site where November 10th and 17th which were those Fridays after practice class when they most needed to make downloads and check information in the CMS. In Fig. 6 we have asked for the country of origin of visits accessing the subject pages in our CMS. We have chosen this time a pie chart representation as well as a table form. It can be observed that 98% of the visits are from Spain, but the teacher can check that there are a 2% from 6 different countries in this month, which corroborates that our 2 students that this semester are in Finland and Sweden respectively are working on the subject. In Fig. 7 we have asked the tool for the route most followed in the CMS for three clicks, and we have found out that it is the path that leads to the theoretical material of the subject Programming Methodology in the second course of Computer Science Engineering at our University. 300 Raquel Hijón-Neira et al. Fig.5. Subject Engineering visits in November 2006 Fig.7. Route most followed for 3 clicks Fig.6. Country of origin of visits to subject pages Fig.8. Summary of visits to the CMS An Access Monitoring Tool Based on Cookies for Course Management Systems 301 Moreover, if a summary of the traffic web site want to be seen, a screen with the general information of the interactions can be obtained (see Fig. 8). In this chart it can be seen when the first and last visit occurred, number of different visitors, total number of visits (visits to any page of the site, counted separately), unique visits (is considered when a user enters, goes to different pages and disconnect; it counts a unique visit even he/she watches many pages in the same 'session'), average of visits in the period, average of unique visits and average of different visitors per day. 8 Conclusions and Future Work We implemented a tool to track students’ access to our CMS from a different point of view, the use of cookies. The tool allows teachers to control access to different parts of the CMS, such as forum, theory, assignments, etc. Moreover, all these characteristics can be tracked by course and a certain period of time. On the other hand, the information gathered can be modified and presented to the user (teacher) as needed; this is to say that there is a wide range of visualization possible, such as pie graph, bar chart, and line graph or table format. Furthermore, the information can be easy and nicely printed. Regarding the technical aspects of the CMS site and in order to improve them and be up to date, we needed the information our tool provides about: operative system, screen resolution, browser and so on. We also needed the identification of different visitors and to be able to identify which paths they follow when entering the site. Furthermore, in how many clicks, that we can also get to know when a certain page is difficult to find. Finally, we wanted to be able to share this application with the community of university teachers that have the same needs of getting to know what is happening with the interactions of their students into the CMS they share and use. We have informally used the tool as teachers and, most importantly, we made a first usability study to assess it [20]. The teachers’ reactions were positive and the study provided us with a list of specifications to improve its functionalities. After these improvements are implemented, we plan to use in real situations by teachers using our CMS. 9 Acknowledgements This work is supported by the research project TIN2004-07568 of the Spanish Ministry of Education and Science. This work was supported by projects TIN2004-07568 of the Spanish Ministry of Education and Science and S-0505/TIC/000230 of the Autonomous Region of Madrid. 302 Raquel Hijón-Neira et al. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. Brusilovsky, P.: ‘Adaptive hypermedia. User Modeling and User-Adapted Interaction’, Ten Year Anniversary Issue (1/2) (2001) 87–110. Castro, F., Vellido, A., Nebot, A., Minguillón, J.: ‘Detecting atypical student behaviour on an e-learning system’. In: I National Symposium of Technologies of the Information and the Communications in the Education, SINTICE’2005, Granada, Spain (2005) 153–160. Hijón, R., Velázquez, A. ‘Steps Towards the Analysis of Students’ Activity in E-Learning Platforms’. Proceedings of E-Learn 2005 World Conference on Corporate, Government, Healthcare & Higher Education. Vancouver, Canada, October 24-28, 2005. Association for the Advancement in Computing in Education. AACE Press. 738–743 Hijón, R., Velázquez, A. Web, Log Analysis and Surveys for Tracking University Students. International Conference Applied Computing 2006. International Association for Development of the Information Society. 561–564. Sheard, J., Albrecht, D.W., Butbul, E.: ViSION: Visualization student interactions online. In Treloar, A., Ellis, A., eds.: Proc. of the Eleventh Australasian World Wide Web Conference Gold Coast, QLD, Australia, Southern Cross University, Lismore, NSW, Australia (2005) 48–58. Urquiza, J., Calvo, R., Hijón, R., Velázquez, A. Generation, Management and Publication of XML-Based E-Questionnaires. International Conference Applied Computing 2006. International Association for Development of the Information Society. 153–160. http://www.akstat.com/website/index.php?lang=es_ES (accessed March 21, 2007) http://cqcounter.com/ (accessed March 21, 2007) http://www.developers4web.com (accessed March 21, 2007) http://www.estadisticasgratis.com (accessed March 21, 2007) http://www.estadisticas-gratis.com (accessed March 21, 2007) http://www.google.com/analytics/es-ES (accessed March 21, 2007) http://logdy.com (accessed March 21, 2007) http://www.nedstat.es/web/nedstates.nsf/pages/index. (accessed March 21, 2007) http://www.onestat.com/es/index.html (accessed March 21, 2007) http://sourceforge.net/projects/contaweb/ (accessed March 21, 2007) http://trancos.escet.urjc.es/urjc/ (accessed March 21, 2007) http://www.xiti.com/es/ (accessed March 21, 2007) http://www.weboscope.com/ (accessed March 21, 2007) Hijón, R., Urquiza, F., Dominguez, F., Velazquez, A. Assessing the Usability of a Cookies-Based Access Monitoring Tool for a CMS. The 7th IEEE International Conference on Advance Learning Technologies (ICALT’07). Accepted. Learning News Writing Using Emergent Collaborative Writing Technology Wiki Will Wai-kit Ma Department of Journalism & Communication, Hong Kong Shue Yan University, Hong Kong SAR, PR China wkma@hksyu.edu Allan Hoi-kau Yuen Faculty of Education, The University of Hong Kong, Hong Kong SAR, PR China hkyuen@hkucc.hku.hk Abstract. There is always a call for studies on blended media, which is one of the effective or pragmatic approaches to be used in particular instructional contexts. In the learning of writing, a review of prior studies suggest that revision as one of the key process to quality writing. However, current approaches do not provide relevant processing capability to such an area. Until recently, Wikis have been an emergent technology to effectively capture collective intelligence of individual users through writing, for example, as shown in the successful case of Wikipedia. In turn, this study attempted to explore if Wikis technology would mediate individual learning experience, in particular, in the process of journalistic writing. In a field study of a studentwritten Wiki developed by student journalists for an online shared resources of original news reporting, to complement traditional classroom teaching, it was found that quality of student written work was significantly correlated to revision behvaior, supported to prior claims. Three modes of implementation strategies were suggested in order to boost learners’ motivation to make relevant and quality revision to their work. Keywords: Wiki, Writing Processes, Pedadgogy, Instructional Design 1. Introduction In many disciplines, educational practitioners have always been finding ways on how to motivate students to write more and to write better. Although many approaches, including most e-learning systems, have been proposed to support teaching and learning, they do not change much from the instructor lead design. Systems are basically designed for course management (e.g., list of enrolment details), one-way delivery of learning material (e.g., downloading of presentations), or dissemination of information (e.g., announcement or calendar). On the other hand, two-way interaction communication tools, such as chat room or discussion forums, are always low in usage; even with administrative interventions (e.g., giving scores for participation). It is wonder if these tools could help provide a better learning experience for the learners. Current approaches ignore the importance of learners’ participation in the Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 303-314, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 304 Will Wai-kit Ma & Allan Hoi-kau Yuen learning process. Although they do achieve some promising results, instructors find they spend too much time and effort to manage a course while student learners only come to download required files. The use of these approaches need more pedagogical guidance before satisfactory results will be achieved. Most importantly, these current approaches are irrelevant to the pedagogical issues in writing. Recently, we have found that the emergent technology Wiki, which has a unique feature to collaborative writing and community building, is worth more study to investigate its potential to complement teaching and learning. Wiki is defined and described as “a type of website that allows users to add, remove, or otherwise edit and change all content very quickly and easily.”([1]). A Wiki is basically a set of linked web pages designed as an online database, created through the incremental development by a group of collaborating users, and Wiki is also the software in itself used to manage the set of all the web pages created ([2]). Wikis provide a fast, permanent and searchable platform for knowledge record that facilitates end-users management by them. Wikis provides knowledge at a centralized access point while decentralizes the knowledge production process to all users located far apart. Would the employment of Wikis provide learners with a new learning experience during the learning writing process? Therefore, the objective of this paper is to explore Wikis as a learning medium to learning writing, in the context of news writing. The arrangement of the paper is as follows: In Section 2, we review the effects of learning with media. In Section 3, we review the news writing processes. In Section 4, we evaluate the pedagogical relevance of Wikis in learning writing. In Section 5, we suggest a model framework to investigate the effects of Wikis as a learning medium towards learning news writing. Then, we present the method we use in this study. The analysis of the empirical result findings are shown in Section 7. Discussions are presented in Section 8. 2. Learning with Media The key to use media in learning is “to focus on the pedagogical potential of the media to enhance broader learning and teaching aims”, rather than using it to develop a limited range of skills [3]. Supported by other empirical studies (e.g., [4]), media is as separate or as “add-on” in schooling in recent years. The increase of quantity of media use in the classroom in the absence of effective pedagogies is not well embedded in the curriculum. In a review of learning with media, Kozma ([5]) distinguished media by cognitively relevant characteristics of their technologies, symbol systems and processing capabilities (p.180). However, the most obvious characteristic of a medium is its “technology and the primary effect of a medium’s technology is to enable and constrain its other two capabilities: the symbol systems it can employ and the process that can be performed with it” (p.181). As media characteristics, and the instructional design that employed them, interacted with learner and task characteristics, they influenced the structure of mental representations and cognitive processes. Therefore, to assess the main effect of the medium, learner and task should be fixed. Or else, the medium might have different effects on Learning News Writing Using Emergent Collaborative Writing Technology Wiki 305 different learners, or different tasks characteristics. Specifically, media characteristics affected the structure, formation, and modification of learners’ mental models (see Figure 1). Task Learner Media -Cognitive relevant characteristics -Symbol systems -Processing capabilities Mental Models -Structure -Formation -Modification Figure 1. Effects of Learning Media on Mental Models 3. Journalistic Writing and the Importance of Revising Specifically, in journalistic writing, Ward ([6]) suggests several generic steps, including: (1) To identify and find news and/or information which will attract and interest the key audience/readers; (2) To collect all the materials needed to tell the story/provide the information; (3) To select from the collection the best material; and (4) To present that material as effectively as possible (p.30). Blundell ([7]) also pointed out some similar steps in featured story writing, including gathering relevant raw material, refining the main theme, shaping and advancing story ideas, keeping related material together, and handling key story elements. They show that journalistic writing has its own generally accepted procedures, which needs to be observed and adopted, while not like those leisure writing. Writing quality emphasizes on purpose, focus, information, and structure ([8]). To improve quality of writing, prior studies of writing instruction recommended process writing as an instructional intervention, where text can be developed through planning, drafting and revising in form of a problem solving processes (e.g., [9]). Revising is a core process in writing and has been a concern in many prior studies. For example, Pogner ([10]) studied the text production processes including text production (drafting a document), feedback (reader commentary) and revision (revising the text) from a socio-cognitive approach where peer interactions in the social context affect the text production processes in the discourse community of engineering. That is, revising may not only a cognitive process of the author, but also a social process involving the interactions from the peers or readers. Moreover, to 306 Will Wai-kit Ma & Allan Hoi-kau Yuen examine an effective model in writing, Roundy and Thralls ([11]) suggested writing to include two sections: section one encourages reflection on all aspects of the message, leading revision while section two informs error-analysis and further revision. Therefore, revision is suggested again, as the key to effective writing. To further understand best practices in writing, Halpern ([12]) employs an informal survey of 125 writers in business, industry and government and identifies six key processes: invention (to generate ideas and information for any assignments), adaptation for audience (to adapt the writing for different audiences), clarification of purpose (to clarify purpose of the writing), organization (to organize the material effectively), control of voice or persona (to control the tone or voice of the communication), and polishing (to polish the drafts of different forms). Polishing here does not only refer to the use of better language, but also may include the revision of any of the previous steps in purpose, organization, tone, etc. On the other hand, it is also important to be informed on how to make revision, for example, through enough feedback. Empirical studies found that providing feedback constitutes an integral part of the learning process in writing (e.g., [13]) and suggested that low learners were those who did not receive any teacher feedback and felt general uncertainty in their work. 4. Wikis and Their Relevance to Pedagogy What are Wikis? How are they relevant to improving journalistic writing? Wikis, hence, have the unique design features that complement traditional teaching on the writing processes: (1.) Learner-centre system design: Unlike most e-learning systems that instructors are the only persons to delivery teaching material, Wikis allow all users to add content; (2.) Facilitating the drafting process: Wikis require only Internet connection and a browser, no additional applications are required. Wikis adopt only very simple and limited (mundane design principle) tag language to make adding Wiki pages an efficient process; (3.) A complete support to revision in the writing process: The core concept of Wikis is to encourage users adding content while enriching the content in the long run (organic design principle). Wikis allow anyone to edit (revise) any pages without restriction (flat structure design principle). Wikis record down all the edits so that anyone can check for prior changes in order to further improving the content (open design principle). Meeks ([14]) referred to pedagogy as, “the ways in which an instructor designs the material and social aspects that she, her students, and their tools inhibit as they accomplish a curriculum” (p.1). In the particular context of journalistic writing, Wikis seems very relevant to complement traditional teaching. It is therefore worthwhile to further study the medium to find out the most effective way of use of the medium to support teaching and learning. In fact, the media industry is one of the first disciplines to grasp the full potential of Wikis. Former reporter has implemented Wikis since 2004 for a social networking Learning News Writing Using Emergent Collaborative Writing Technology Wiki 307 web site to allow its users to submit, choose and rank content on news stories (www.digg.com) ([15]). Wikis have also helped collected a multi-layered body of knowledge and linked together disparate bits of information in one place, in an example of advertising application ([16]). Examples show that news reporting could be an appropriate application and promising area of implementing Wikis to manage the relevant knowledge ([17]), for example, Wikinews (http://www.wikinews.org) and USC Online Journalism Review Wikis (http://www.ojr.org/ojr/wiki/). Wikis have been found employed in a number of mainstream media news sites, for example, Los Angeles Times and its Wikitorial – invite visitors to rewrite the newspaper’s editorials using Wikis. 5. Model Framework and Hypotheses Development We suggest a model framework to investigate the effects of Wikis as a learning medium towards learning writing, specifically in the context of news writing (see Figure 2). Task (News Writing) Learner (Writing Self-efficacy) Media (Wiki Usage; Editing Behavior) Mental Models (Writing Performance) Figure 2. Model Framework of Writing Performance Mediated by Wikis According to Kozma’s review on learning with media, task, learner, and the medium are the three core factors affecting the formation, structure and modification of learners’ mental models. Here in this study, we define our learning task as news writing. Hence, it is the same for all learners in the study. Moreover, we measure the writing performance as a proxy for the learning effects on learners’ mental models. For the learners, they may have different background, domain knowledge and confidence that may affect their final performance (e.g., [18], [19], [20]). In news reporting process, it involves the interview of eyewitnesses, the collection of background information, the selection and organization of evidence and material, the drafting of the news reporting, and the editorial review on the accuracy, reliability and completeness of the news story. In this complicated journalistic process, it is expected 308 Will Wai-kit Ma & Allan Hoi-kau Yuen that the one with a higher self-confidence in writing may have a higher chance to accept the task as a challenge and to complete the difficult task with determination. On the other hand, the one with a lower self-confidence in writing may more hesitate to accept even an ordinary writing task. They will try to avoid the problem, not to face the problem directly, spend less time on the task, finally, may not be able to complete the task. Even if they complete the task, the quality of work does not truly reflect their competence in the area. Therefore, we postulate, H1: Individual learner’s writing self-efficacy would have a direct and positive relationship to his or her writing performance. Revision has been suggested to be an important step in the writing process. In view of the journalistic process, including interview eyewitnesses, collecting background information, organizing key story elements, etc., journalistic writing is no easy task. It is logical to expect that a piece of quality news reporting cannot be made at the first time. It probably requires many more rounds of revision to finally achieve the reporting goals. It is therefore, H2: Individual learner’s revision behaviour would have a direct and positive relationship to his or her writing performance. Although the core processing capability of Wikis is the support of the editing process, there may also be other benefits of using Wikis. Learners add content to Wikis. Regular usage of learners and the organic grow of content in Wikis gradually form a virtual community of Wikis users. They login the platform, they read news headlines, news articles, they add their own, they revise their own work, and they add links for key terms while linking independent articles together. Therefore, all these social activities may also have effects on individual mental representations and cognitive processes. We postulate, H3: Individual learner’s usage behaviour would have a direct and positive relationship to his or her writing performance. 6. Methodology 6.1 Background A student-written Wiki (named, HKNews at http://hknews.syc.edu.hk) has been setup for the Department of Journalism & Communication at a private local university in Hong Kong. HKNews Wiki is open to the public; however, its primary aim is to support teaching and learning by allowing student reporters to contribute original Learning News Writing Using Emergent Collaborative Writing Technology Wiki 309 news reporting. Up till February 27, 2007, there were 3720 news reporting articles in HKNews Wiki site, and a total of 47,675 edits (12.82 edits per article), with a total of 1,938,340 view counts in the whole site. There are 1,480 registered users, of which 3 (or 0.20%) are System Operators (Sysops). 6.2 Data Collection Every Tuesday afternoon, the department arranges industrial leaders from society to offer seminars to all the students and staff in the department. We targeted our study in Year One students who enrolled in the course JOUR100, a compulsory introductory course to all Year One students and they have not used HKNews Wiki before. Stage 1: Survey on writing self-efficacy - On October 9, at a class, 75 of them completed and returned at class (One of them was absent from class). Stage 2: Content analysis on news reporting on HKNews Wiki site – We chose October 10 seminar event and the corresponding news reporting for analysis. This event was chosen because this was the third seminar that student reporters already had two times of basic hands-on experience on using Wikis and had solved most of the technical problems. 6.3 Measure Stage 1 - Survey: For the survey, writing self-efficacy measurement scale was adopted from Jacobs et al. (2005, ibid) and wordings were modified to reflect news reporting tasks. Stage 2 – Content analysis: For the content analysis, we analyzed the personal profile page, the article page and the history page. For the personal page, student reporters included mainly the headlines of their written news articles. Some might inserted some graphics on the page. The article was graded by their instructor, who was a senior faculty member in the department and has more than thirty years working experience in TV broadcasting, radio and publication industry. The score became the final grade, ranged from 0 to 100 marks. 7. Data Analysis Data analysis started with a review on the background and general Wikis’ usage behaviour of the subjects. Then, the student-written Wiki pages would be analyzed to give an understanding of how learners made use of the Wikis to write and present their work. Lastly, the three hypotheses were examined using regression analysis. 7.1 Analysis of Subjects 75 student reporters completed the survey and returned at class where one was absent from the class. Analysis of subjects is shown below (see Table 1). 310 Will Wai-kit Ma & Allan Hoi-kau Yuen Items Gender Age Computer Knowledge (very little to expert, 1-4) Familiarity of Wikis (very little to expert, 1-4) Table 1. Descriptive Analysis of Subjects Statistics Items Male: 19 (25.3%) Wiki’s Usage Female: 56 (74.7%) Frequency (never to always, 1-7) Mean: 19.39 Wiki’s Usage (Last Week Total) Mean: 2.03 Wiki’s Usage (SD: 0.66) (on average, every time) Statistics Mean: 4.24 (SD: 1.03) Mean: 98.05 min. (SD: 55) Mean: 31.76 min. (SD: 31.39) Mean: 2.84 (SD:0.72) Internal consistency of the 26-item writing self-efficacy scale is examined using Cronbach’s alpha value. The scale shows a Cronbach’s alpha value of 0.96, exhibited a valid internal consistency, with above the common threshold for exploratory research ([21]). The results of the 26-item writing self-efficacy survey are shown below (see Table 2). Table 2. Descriptive Analysis of Writing Self-efficacy Writing Self-efficacy (Cronbach’s α=0.96) 26-items: Mean: 56.76 (SD: 12.987) to 72.25 (SD: 16.206) Composite: Mean: 63.82 (SD: 9.81); Min.: 13.85, Max.: 81.92 7.2 Analysis of Student-written Wiki Pages Out of the 75 student reporters, 57 of them had included news headline and the news reporting on the Oct 10, 2006 event. An analysis of these 57 articles is listed below (see Table 3). Table 3. Descriptive Analysis of News Reporting of Oct 10 Event on HKNews Wiki Formatting Analysis Formatting Analysis Author’s Name No: 30 (52.6%); Bulleted list Nil Yes: 27 (47.4%) Length (no. of Min.: 160; Max. 1040 One idea per No: 8 (14%) words) Mean: 423.86 (SD: 148.15) paragraph Yes: 49 (86%) Highlighted Nil Inverted No: 11 (19.3%) keywords pyramid style Yes: 46 (80.7%) Sub-headings No: 23 (40.4%) Photos, Nil Yes: 34 (59.6%) Reference sources Learning News Writing Using Emergent Collaborative Writing Technology Wiki 311 7.3 Analysis of Editing Log of Oct 10 News Reporting Analysis of the news reports found that, 14 (24.6%) student reporters uploaded the news reporting onto the HKNews Wiki site without any further edits while the others edited ranged from 1 time to 10 times. On average, there were 2 times of editing updates for each news report (SD: 2.10); (2) Edits time span: Moreover, there is also a time span between the first published version to the final edited version. Two-third (N=38, 66.7%) of the news reports finished all the editing process on the same day, while the remaining 19 (33.3%) ranged from 1 to 21 days. On average, it takes 2.11 days (SD: 4.17) for student reporters to feel comfortable about the report and not to edit any more; (3) Contributors’ identity: Wiki users may register and login the system to both browse around and making changes in the Wiki environment. They might also not login but were capable to do all the steps; however, the IP address of the computer would be listed instead. It was found that the mean number of account login in editing the news reports was 1.95 (SD: 2.15); while IP address editing was 1.04 (SD: 1.55); (4) Timeliness of news reports publication: The timeliness of the news reports can be measured by the time span between the date of the event and the date of the report published on Wiki. Analysis of the time span of the 58 news reports found that it takes an average of 3.46 days (SD: 4.13), which means that there is a lagged time for the news appeared for the readers. One third of the reports appeared on the same day or 1 day after the event (N=21, 36.8%); another one quarter published two days after the event (N=15, 26.3%); the remaining (N=21, 36.8%) published from 3 days to 23 days after the event. An analysis of this editing log is listed below (see Table 4). Table 4. Descriptive Analysis of Editing Log of Oct 10 News Reporting on HKNews Wiki Editing Log Analysis (1) Number of edits No edits: 14 (24.6%) N.B. Total number of revision, except first posting Max.: 10, Min.: 1 Mean: 2 (SD: 2.10) (2) Edits time span 0 days: 38 (66.7%) N.B. Time span between the first edit through the final Max.: 21 days, Min.: 1 day edit Mean: 2.11 days (SD: 4.17) (3) Contributors’ identity Account login: 1.95 times (SD: 2.15) N.B. Registered and use account login vs. no login but IP: 1.04 (SD: 1.55) shown IP address of editor’s computer (4) Timeliness of news reports publication 0-1 day: 21 (36.8%) N.B. time span between the date of the event and the 2 days: 15 (26.3%) date of the report published on Wiki 3-23 days: 21 (36.8%) Mean: 3.46 days (SD: 4.13) 7.4 Regression Analysis The writing performance was examined by the grades. Linear regression analysis found that total number of edits (β=0.462, p<0.001) was significant factor that predicted the score of the news reporting (supporting H2). The model explained 22.6 per cent of the variance of the score, significant at p<0.005 (Adjusted R-square: 312 Will Wai-kit Ma & Allan Hoi-kau Yuen 0.178). However, Wiki’s knowledge and writing self-efficacy was not found significant to predicting writing performance (H1, H3, not supported). 8. Discussions In this paper, we analyzed the shortcomings of current approaches to support teaching and learning: instructor lead design and irrelevant to learning writing. We then examine the potential of Wikis as a learning medium in the context of journalistic writing. Supported by prior studies in writing literature, the unique design feature of Wikis in supporting revision seem relevant and important to providing learners a unique experience in practicing and learning journalistic writing. Empirical results of this study found that number of editing of a news reporting is significantly correlated to the writing performance, supporting the theoretical assumption. On the other hand, it is important to discuss more of the various instructional designs to encourage more revision and discussion of the issues under different circumstances where revision takes place. We hereby suggest the following framework in the implementation of Wikis to support learning writing (see Figure 3). Individual Authorship Group Authorship Large Scale Collaboration Figure 3. Fusion Modes of Implementation Strategies for Wikis While revising is the key, it is suggested that the three writing modes: (1) Individual authorship; (2) Group authorship; and (3) Large scale collaboration, be introduced in the curriculum, either at the same time or throughout different semesters, in order for a learner to fully benefit from using Wikis to create a unique learning experience. (1.) Individual Authorship: This is important to keep a learner the responsibility to his or her own learning as he or she owns the writing piece. We need to design a mechanism to motivate the learner to self-reflection on his or her writing and do the editing on his or her own. For example, for a writing task, the instructor may suggest a schedule for all learners so that they draft the outline the first week, they finish the first draft the second week, and they make three additional revisions on the three following weeks. This is actually a practice in composition writing and is the basis for process writing. The key is to develop a regular pattern so that learners will continue the revision process. (2.) Group Authorship: Learners work in groups may have a chance for an intense interaction among peers. Form learners into small groups, request them to complete a writing task and develop a mechanism for them peer review each Learning News Writing Using Emergent Collaborative Writing Technology Wiki 313 others work. Wikis provide a platform transparent to all. Anyone can edit any page. Therefore, Wikis make it an efficient tool for a group of people working together. In this instructional design, there is a diffuse responsibility on who owns a writing piece. Learners work together and hence, learners have a chance to learn from a different perspective on how to edit a writing piece. Wikis provide a history log so that anyone can compare any two prior versions. This social interaction expands the cognitive process of any individual learner alone. (3.) Large Scale Collaboration: This design does not care any more about authorship. The focus is on the writing piece. Anyone can create a new writing piece while any other one can edit and enrich the content. Therefore, learners do not write because of the ownership of the writing piece, but are based on the topic that they are interested. A learner can learn as much as he or she wishes by his or her commitment to a specific topic. On the one hand, a learner learns by his or her own study and writing practice in the topic; on the other hand, he or she also learns from other learners interested in the topic. There are several limitations in the study. The domain knowledge about journalistic writing is contextualized and specific; the results may not directly be generalized into other disciplines. Further studies in other writing context may add better understanding of the issue. Moreover, the sample size is small and the time frame is limited, these are also the limitations of the study. In future research, we can study a bigger sample size of subjects, choosing several instead of one writing piece over a period of time that would probably provide us with a rich explanation to the learning process. This is especially true for a learner to fully reflect his or her mental representation into measurable learning output. 9. Conclusion and Future Works In this paper, we analyzed the design features of Wikis, specifically the editing capability, in relation to the writing process using a field research on a group of student journalists on learning journalistic writing. We examined the usage pattern and found significant relationship between revision and writing performance. This result is supported by prior studies in writing while showing that Wikis as an effective learning medium relevant to the learning of writing. However, some problems still need to be solved, such as how to motivate learners from revising their work, how to improve peer interactions to provide basis for quality revision, or how individuals react differently within the medium. We will continue our research on these problems and provide implementation guidance for educational practitioners fully utilize Wikis as an effective learning medium. References [1] Wikipedia:Wiki, Retrieved March 18, 2007, from http://en.wikipedia.org/wiki/Wiki, 2007 (March 18). 314 [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] Will Wai-kit Ma & Allan Hoi-kau Yuen Leuf, B. and Cunningham, W. The Wiki Way: Collaboration and Sharing on the Internet, Addison-Wesley, Reading, MA, 2001. Gaskell, A. "Editorial," Open Learning (20:2), 2005, pp. 109-111. Condie, R. "ICT and Learning: Scottish Schools and Pupils," Proceedings of the Heads of E-learning Forum (HELF), University of Strathclyde, 2005. Kozma, R.B. "Learning with Media," Review of Educational Research (61:2), 1991, pp. 179-221. Ward, M. Journalism Online, Focal Press, Oxford, UK, 2002. Blundell, W.E. The Art and Craft of Feature Writing, Plume, New York, 1988. Davis, K.W. The McGraw-Hill 36-Hour Course in Business Writing and Communication: Manage Your Writing, McGraw-Hill, New York, 2005. Gersten, R. and Baker, S. "Teaching Expressive Writing to Students with Learning Disabilities: A Meta-analysis," Elementary School Journal (101), 2001, pp. 251-272. Pogner, K.-H. "Writing and Interacting in the Discourse Community of Engineering," Journal of Pragmatics (35:6), 2003, pp. 855-867. Roundy, N. and Thralls, C. "Modelling the Communication Context: A Procedure for Revision and Evaluation in Business Writing," Journal of Business Communication (20:3), 1983, pp. 27-46. Halpern, J.W. "What Should We Be Teaching Students in Business Writing?" Journal of Business Communication (18:3), 1981, pp. 39-53. Jacobs, G., Opdenacker, L. and Van Waes, L. "A Multilanguage Online Writing Centre for Professional Communication: Development and Testing," Business Communication Quarterly (68:1), 2005, pp. 8-22. Meeks, M.G. "Wireless Laptop Classrooms: Sketching Social and Material Spaces, Kairos, 9(1)," (20 07: March 12), 2004. Bulik, B.S. "Media Morph: Digg.com," Advertising Age (77:8), 2006, pp. 19. Oser, K. and Kerwin, A.M. "Media Morph: Wiki," Advertising Age (76:48), 2005, pp. 38. Dorroh, J. "Wiki: Don't Lost That Number," American Journalism Review (27:4), 2005, pp. 50-51. Bazerman, C. "Physicists Reading Physics," Written Communication (2:1), 1985, pp. 323. Bandura, A. "Self-efficacy: Toward a Unifying Theory of Behavioural Change," Psychological Review (84:2), 1977, pp. 191-215. Bandura, A. Social Foundations of Thought and Action, Prentice-Hall, Englewood Cliffs, NJ, 1986. Nunnally, J.C. and Bernstein, I.H. Psychometric Theory, McGraw-Hill, NY, 1994. Description Patterns in Learning Design Authoring Systems Lei Xu1, Li Zheng1, Jing Liu1, Yintao Liu1, Fang Yang2 1 Department of Computer Science and Technology, Tsinghua University, Beijing, China 2 Department of Foreign Languages, Tsinghua University, Beijing, China Abstract. Learning design authoring is one of the core functions of a learning management system. As the types of elements to edit inside learning design vary, it can be time-consuming to supply the environment for editing each type of element individually. This paper explores the similarities among the editing of these different types of elements, and proposes an approach to accelerate the development of such a complicated system and to achieve better adaptation for changes of concept model or requirements. The basic idea is to unify the editing environments for different types of elements, by introducing description patterns. A demonstrative implementation utilizing AJAX technology is presented and its potential in authoring systems for other forms of e-learning material, such as content package proves to be feasible and practical. Keywords: Learning Design, LMS, XML, Description Pattern. 1 Introduction 1.1 Emerging of learning design within e-learning research In the development of e-learning industry, the concept of learning design has become increasingly a popular issue. Similar to content package and simple sequencing, learning design can be an approach to organize various educational resources. But it does more than that. Besides physical materials, it takes activities and roles participating in the learning process into account. Therefore it concerns not just simplex activities associated with reading materials but also group work like discussion, role plays and etc. Being capable of organizing learning scenarios like the ones within collaborative learning, it gets learners effectively involved in the learning process. Although the idea of ‘Design of learning’ has already existed implicitly in the traditional teaching context, when it comes to the digital and web environment, new challenges occur in the application of an idea which is not so fresh. In reality, the design of learning lies in the lecture notes of an instructor, and maybe partially in his mind. But e-learning systems cannot utilize such a form of design. It is thus necessary to transform the teaching plan to a form which can be Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 315-326, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 316 Lei Xu et al. easily understood by computers. That is why methods of binding the design of learning to digital content are proposed, and one of them is the IMS 1 -LD 2 specification, which has been commonly accepted by the e-learning research community. 1.2 Digital binding of learning design To some extent, the IMS-LD specification[5] can be recognized as a language which describes how different roles perform activities within environments(including learning objects and services), and how these three categories of building blocks are coordinated into a learning flow, within the method element which contains both play, act, and role-parts[6]. Such an approach to separate building block repositories and the flow control mechanism, make the consisting elements both reusable and sharable. E-learning systems that supporting the IMS-LD specification can easily exchange learning designs in different levels. The undergoing project of the national standards on learning design also takes the IMS-LD specification as the major reference. During our research, we also keep the conformance to IMS-LD as the top agenda. The IMS-LD way of binding is tree-structured, preferably using XML. So the essential part of job of a typical learning design authoring system is to enable the editing of such a tree structure, which will be analyzed in details in subsequent sections. The complexity of such a task is mainly determined by the following factors: 1. number of element types within the tree; 2. numbers of attributes possessed by each type of element; 3. numbers of attribute types possessed by each type of element; 4. number of dependency links among attributes and other elements. As the four factors above increases, the entire job can drive the developers crazy, because there can be numerous types of attributes to take care of, and countless property sheets to build up, let alone the potential changes in the concept model, which can result in major changes in the authoring system which is even harder to accomplish. To avoid such inconveniences, we propose the description patterns to be used which can save most of the trouble. 1.3 What is Description Pattern A description pattern generally supplies information on one of the attributes of its target. By obtaining a set of necessary description patterns of an element, an authoring 1 2 Abbr. for Instructional Management Systems, now commonly mentioned as the IMS Global Learning Consortium is a non-profit standards organization concerned with establishing interoperability for learning systems and learning content and the enterprise integration of these capabilities. Official site: http://www.imsglobal.org/ IMS Learning Design is a specification for a metalanguage which enables the modeling of learning processes. The specification is maintained by IMS Global Learning Consortium. Description Patterns in Learning Design Authoring Systems 317 system will know how to present the property sheet for it, allowing the author to view and alter the value of it. For example, for a Role-part element inside a learning design we may find three description patterns telling us that 1. it has an attribute named ‘title’, of string type, and is optional, and 2. it references a role inside the learning design, named ‘role-reference’, and is a must, and 3. it must refer to a component, which can be a learning activity, or a support activity, or an activity structure, or an environment. So a description pattern is simply a way of describing versatile attributes of an element. As the types of attributes of the target element vary, there should be various patterns to describe them. Elements inside a learning design may be quite different and complicated in different aspects, but they do share a large collection of common types of attributes, which is why it is possible to simplify and accelerate the development of the authoring system using description patterns. 2 Related Work As for now, there are several Learning Design authoring tools, like RELOAD3 Editor, LAMS4. Reload Editor provides users an appropriate user interface model to allow easy creation of learning designs [7], and is quite a convenient LD packaging tool that runs as a Eclipse based local application. LAMS includes innovative design features that put it at the forefront of current tools for activity management [7], utilizing Adobe® Flash® technology. These projects focus on how to build an application that provides better support for IMS-LD and an intuitive and friendly user interface. The purpose of our work is not just develop yet another Learning Design authoring tool, which would appears unnecessary, instead, we explore ways to accelerate the developing process of such tools as it is a commonly accepted fact that “it is not trivial to provide full support” [9] for IMS Learning Design. A Learning Design authoring system acts as the test environment for the approaches we have proposed and the potentials of those approaches are not just limited to Learning Design authoring systems. 3 Use Case Analysis with respect to Description Pattern This section explores the essential functions of a learning design authoring system, in the context of which the description patterns are utilized. The basic use cases of a 3 Reusable eLearning Object Authoring and Delivery, a project that develops tools to support the learning technology interoperability specifications such as IMS and SCORM, more details can be found at http://www.reload.ac.uk. 4 Learning Activity Management System, more information can be found at www.lamsinternational.com. 318 Lei Xu et al. typical learning design authoring environment can be roughly divided into the following 3 stages: 3.1 Tree structure navigation This means the expanding and collapsing of tree nodes. So the required description should indicate if a requested tree node, namely a concerned learning design element is expandable and if so, what the child nodes are. Therefore, a type of description pattern is necessary to give the exact information about descendants of the target. And for that matter, we call it Descendants Description Pattern. 3.2 Tree structure editing Normally a learning design authoring environment is expected to allow users to modify the structure of the tree, which includes four major actions: creating new child nodes under a parent node, moving an existing node elsewhere, duplicating a node, and deleting a node. Therefore, to supply for the information we propose four types of description patterns: 1. Descendant Creation Description Pattern This pattern is expected to tell the authoring system what types of elements can be created as the child node for the target node. So the author can only select from a certain collection of types to create new nodes. 2. Relocation Description Pattern This pattern is expected to tell if the target node can be moved up or down, and if it can be detached from its current parent and attached to a new parent. It determines if the author’s moving action can be performed successfully. 3. Duplication Description Pattern This pattern is expected to tell the system if the target can be duplicated, and if there is a unique identification issue to address, usually by altering certain attribute of the target to distinguish it from the original one. 4. Deletion Description Pattern This pattern is expected to tell the system if the target node can be deleted. If so, how the deletion might affect other remaining elements. For example, if a learning activity is deleted from the Activities repository; all its references go invalid. So this type of description should be able to perform certain post-deletion actions to assure the consistency of the learning design structure. 3.3 Tree node attributes editing In this context, the required descriptions can get much more complicated as the types of attributes of all the elements vary. In spite of this, there are still some common fields that all these patterns share: 1. The target element it describes; Description Patterns in Learning Design Authoring Systems 319 2. The method to get the attribute value; 3. The method to set the attribute value; 4. Whether the attribute is read-only; 5. What is the default value for the attribute; 6. The title of the attribute, which is displayed within the authoring interface. Then we explore the frequently used patterns in a simple-to-complex manner. 3.3.1 Simple field description This category includes the following attribute description patterns: Number (Integer, float point, or percentage) This pattern indicates the corresponding attribute is a number, and its value should fit certain constraints, like minimum value, maximum value, precision and etc. So such a pattern is preferred to have value validation capabilities. Format-free text or string This pattern doesn’t have many constraints, if any, it could be no more than one like ‘maximum characters’ and is easy to ensure. One example is the attribute ‘title’. Special format string Like phone number, URI, date and duration, text should conform to certain regular expression, so such patterns are also expected to have validation capabilities. Besides, it should be able to convert string to an appropriate object, for example, converting a date string to a Date object, as the target element object may require. 3.3.2 Static enumeration This category includes: Boolean This is almost the same as numbers, as only two values are possible (true or false), but may require different presentation approach, which could be radio button while the ones above normally lies in a text field. Vocabulary This is some special enumeration specified for certain type of element attribute. For example, the ‘structure type’ attribute of an activity structure can be ‘select’ or ‘sequence’ only. Such a pattern should be assigned the valid value set (vocabulary) according to the target type at runtime. 3.3.3 Dynamic collection This is the most complicated attribute description pattern which is meant to handle the component reference situation. As mentioned above, the mechanism of reference is widely used within the learning design information model. It works like this: the actual component is stored in a certain repository and once needed, it is only necessary to supply its identity (usually a unique identifier), not a clone of the component itself. So what component can be referenced depends on the accepted types and the contents of the repository, which is determined by the learning design itself, not by some statically predefined vocabulary. In this case, the pattern should be able to supply the information on: 1. The types of elements can be referenced, and 2. The possible value set for a chosen type, which is determined at runtime. 320 Lei Xu et al. Take the role-part element for instance: a first pattern tells that the role-part references a role, the corresponding description pattern should tell the authoring system only one type of element can be references - role, and the value set for the ‘role’ category is the elements stored under the ‘roles’ element, with a tag of ‘Learner’ or ‘Staff’; a second one may tells that it refers to a component that represents the behavior of the role, the accepted types are learning-activity, supportactivity, activity structure, unit-of-learning and environment, and for each type the pattern is expected to supply the possible value set. 4 Interpreting and Presenting Description Patterns This section introduces how the classification of patterns will be used to build the authoring environment, especially the property sheets for various elements inside a learning design. All the description patterns are supposed to be stored in the form of configurations, and will be loaded to the system for runtime retrieval. Using the Descendants Description Pattern, we can easily enable the tree-structure navigation, which is the premise of all the editing actions. With Descendant Creation Description Pattern, Relocation Description Pattern, Duplication Description Pattern and Deletion Description Pattern, the authoring system can guide the author throughout the process of adding child nodes for a parent, moving, duplicating or deleting an existing node. The attribute description patterns, which contain a lot of subcategories, are intended for the authoring system to build the attribute sheet for a learning design element which the author wishes to view and edit. The process is shown in figure 1. Figure 1 Generating attribute sheet for an element Description Patterns in Learning Design Authoring Systems 321 For a requested element, the system first obtains the description patterns for it (arrown), and then it builds the input field for each pattern, and assembles all the fields to form the entire property sheet (arrowo). The most difficult job is to create an appropriate input field for each description pattern. For example, to present a text attribute like ‘identifier’ to the author, the system may dynamically create a text filed in the property sheet with the title illustrating what the attribute is. And for a Boolean description pattern, it creates a group of two radio buttons or a selection box with two options. In the case of a component reference, it shows the current referenced value, which is followed by a set of controls to alter the value: a selection box holding all the acceptable types to reference, and a query button. After the author specifies one category and presses the query button, the system retrieves all the possible components fitting into the requested category, and displays them in a list for selection. In this way, the author is able to change the component references. After the author modifies some of the attributes, the system passes each modified attribute value to the corresponding description pattern (arrowp) and tells them to set the new values (arrowq). Data flow in this process is reversed in contrast to the rendering process. And if some patterns refuse the new values for validation or consistency reasons, the system will explain to the author why the updating has failed (arrowr). 5 Benefits of Description Patterns By introducing description patterns, the learning design authoring system is not concerned about what type the requested element belongs to, instead all it needs to do is retrieve all the necessary description patterns, render these patterns to the interface and pass the input from the author to these patterns to update the attributes of the element. To the authoring system, all elements become unified and separated processing for each type of element is no longer necessary. Furthermore, the description pattern approach has more advantages, two of which are as follows: 5.1 Better Adaptation for changes As the description patterns wraps up all the attributes of elements, the influence that potential changes in the future can produce on the editing environment is greatly reduced. Here we take two major possible changes to consideration. 5.1.1 Information Model Adjustment It is possible that the learning design concept may change in the future, or the specification itself get improved or extended, which would result in the changes in the binding tree structure: new types of element added, attributes of existing element changed, etc. To assure the authoring system still function normally, it is only necessary to add description configuration for new type of element, or modify the configuration for changed attributes. 322 Lei Xu et al. 5.1.2 Multiple Authoring Policy Control Although the information model of learning design is relatively stable, the authoring policies required by users may vary a lot. For example, some system administrators may think that it is not a good idea to allow authors to modify the identifier of an element, so he requires the identifier attributes to be removed from view. To satisfy this, we just need modify the description pattern configuration so that all descriptions about identifiers are removed. Then the identifier attribute would disappear from the authoring interface, without modifying the authoring environment code. An author may argue that the selection box holding the type of role element a rolepart can refer to should be ‘Learner’ and ‘Staff’ rather than one whole category ‘role’. To make it work, we just need to modify the description configuration for the ‘role reference’ attribute to accept two categories: Learner and Staff, and the value set for the ‘Learner’ category is the roles with the ‘Learner’ tag stored inside the underlying learning design, and for the ‘Staff’ category, the ones tagged ‘Staff’. Still the authoring environment code is left untouched. As we can see, using description patterns make it easy to achieve customizable authoring environment per author that best fits his habits. And if the author does not wish his learning design to be modified by others, we can assign a complete read-only description pattern configuration to his works once viewed by others. The description patterns provide us a flexible and easy way to control multiple authoring policies. 5.2 Reusability The mechanism of description patterns is not just intended for learning design authoring; instead it provides us with a way of modeling an authoring framework for XML-bound educational resources. Because the use cases we have analyzed above is not only dedicated to learning design itself, other forms of educational resources that has a conceptual tree structure can also take most of the convenience offered by description patterns The only extra effort needed is that some new types of attribute description patterns may need to be defined, and the authoring system needs to be extended to be aware of how to render these new patterns. But the basic logic of description and presentation totally fits. As the follow-up implementation proves, it is easy to be ported to content package or LOM editing environment. 6 Implementation and Application This section focuses on how we implemented a learning design authoring system using description patterns and how it goes after the actual system is put into use. 6.1 System Framework Overview The learning design authoring environment is designed as part of a LMS, which adopts a layered architecture [1](figure 2). Description Patterns in Learning Design Authoring Systems 323 Figure 2 Generating attribute sheet for an element The first part that the learning design authoring system concerns is how the design described by XML is parsed and converted to a series of objects that represent different element within the learning design information model. Such a mapping process happens between the Public Service Layer and the Educational Service Layer, or more precisely, the XML and the Learning Design module. Though we do not care how it works, the objects obtained are the targets that our description patterns working on. After the learning design object are ready to use, the authoring environment which lies in the Educational Application Layer, get all the description patterns for each object the author may view or edit. These patterns, act as the bridge between the presentation interface and the background objects. Information on the tree structure and the attribute set of an element pass on in the form of description pattern to tell the interface how the learning design environment is rendered; and after the author makes adjustment to the design, the patterns fetch the changes and apply them to the corresponding objects so that the author’s thoughts can be persistent in the system after the objects are formatted and written back into XML, as shown in figure 2. The following two major modules accomplish this job. 6.2 Java back-end The task is mainly produce proper description patterns for different types of learning design elements. For the sake of flexibility and convenience, configurations for these patterns and elements are stored in XML format. Figure 3 shows a simple example of how a description pattern is specified by means of XML. In this case, the class 324 Lei Xu et al. LearningDesign has an attribute called identifier, which belongs to the type 'identifiertext' and shall be randomly generated at runtime. Figure 3 Simple attribute configuration specified in XML The dynamic-object case is shown in figure 4, as for a role-part, a role need to be referenced and the role should be selected out of all the roles stored inside the containing learning design. The options here specify the possible categories where the dynamic objects come from, and for the role reference, they are learner and staff. Figure 4 Dynamic-object attribute by XML configuration Thanks to the Java reflection mechanism, the configuration can be well understood by the system, and the required description pattern is easily generated by the name of the Class the actual object belongs to, and the descriptions can easily get and set fields inside the objects. The job of parsing the configuration and forming corresponding Java objects is mainly done by Caster XML, which can marshal5 almost any "beanlike" Java Object to and from XML [8]. 6.3 AJAX front-end As the interface presented to the author is determined by the active description pattern, and all controls for input and output are dynamically created, instead of interpreting a pile of server pages with static fields in them. Once there is an action performed in the front-end, the back-end receives the request and generates all necessary description patterns and sends them back in a certain data format rather than web pages. After receiving and parsing the data, the front-end dynamically changes the user interface according to the description. The back-end is not involved at all. As the user agent is a web browser, this is just a good use case of the AJAX technology, which features in 5 It is the action of converting a stream (sequence of bytes) of data to and from an Object [8]. Description Patterns in Learning Design Authoring Systems 325 asynchronous client-server communication and dynamic web interface control by JavaScript and CSS. XML is used to transfer information on description patterns, and that is the common language both the back-end and front-end speaks. An example of such XML documents is shown in figure 5. Figure 5 The content of XML on description patterns for a role On receiving the responding XML, the front-end will render the user interface (figure 6) to allow the author modify the attributes. Figure 6 Interface rendered for Role editing according to description response The case for dynamic object is a bit more complicated, first the interface displays all the valid categories, and then query on a certain category should be enabled. Figure 7 shows how it works when selecting environment reference for an activity. Figure 7 Dynamic object interface for environment reference selection 6.4 Implanting into other authoring systems As we expected, after the learning design authoring system is implemented, a similar one for editing Content Package is implemented by means of description patterns. As most of jobs are accomplished in the learning design system development stage, the latter one was built and set up rapidly, which shows the amazing power of the 326 Lei Xu et al. description patterns in building similar authoring systems for XML-bound educational resources. The case remains while applying it to LOM editing environment. 6.5 Application Currently, our LMS with support for LOM, Content Package and Learning Design, is serving as a on-line teaching platform for our department, including C++/C programming, Webpage Design, and is hopefully to be integrated to the whole on line learning platform of Tsinghua University, as the system provides the teachers with more effective ways to guide the students in the learning process, and enables more interactions outside the physical classroom. Conclusion and Future Work As the elements within a learning design vary greatly, it is difficult to supply an authoring system for them. But by means of description patterns, we can efficiently perform repetitive tasks such as attribute retrieval and modification. These patterns encapsulate the differences between various types of elements and make them look the same to the rendering front-end. This approach of design pattern proves to be feasible in learning management system. However, it can hardly describe complicated dependencies among elements, or attributes containing logical conditions. In such cases, specialized authoring environment should be considered to balance commonality abstraction and difference handling. This issue will be addressed in our follow-up studies. References [1] Wang, X., Zheng, L., Yang, F.(2004), “An Implementation of Learning Objects Management System”, Advances in Web-Based Learning – ICWL 2004 Lecture Notes in Computer Science, Vol. 3143, (2004)393-399. [2] Liu Jing, Zheng Li, Yang Fang (2005), “Information management in e-learning system”, Advances in Web-Age Information Management. 6th International Conference, WAIM 2005. Proceedings (Lecture Notes in Computer Science Vol.3739) 275-83 2005 [3] Alistair Inglis (2005), “Using conceptual mapping as a tool in the process of engineering education program design”, Journal of Learning Design, Vol. 1 No. 1, 45-55 [4] James Dalziel(2003), “IMPLEMENTING LEARNING DESIGN: THE LEARNING ACTIVITY MANAGEMENT SYSTEM (LAMS)”, ASCILITE 2003 [5] IMS (2003). IMS Learning Design v1.0 Final Specification, retrieved March 28th, 2006 from http://www.imsglobal.org/learningdesign [6] Ann Jeffery and Sarah Currier (2003), “What Is IMS Learning Design?” retrieved January 14th, 2007, from http://www.cetis.ac.uk/lib/media/WhatIsLD_web.pdf [7] Sandy Britain (2004), “A Review of Learning Design: Concept, Specifications and Tools”, retrieved February 12th, 2007, from http://www.jisc.ac.uk/uploaded_documents/ACF83C.doc [8] Castor XML documentation: retrieved September 15th, 2006 from http://www.castor.org/ [9] CoperCore documentation: retrieved April 9th, 2007 from http://coppercore.sourceforge.net/ M-Learning: A Pedagogical and Technological Model for Language Learning on Mobile Phones Yannick Jolliet HEC-INFORGE, University of Lausanne, Switzerland yannick.jolliet@unil.ch Abstract. The mobile learning (m-Learning) is a rapidly evolving area which remains to be further explored, in particular about the possibilities offered by mobile phones which have now become commodities. Is it possible to leverage this fantastic base of 2 billion mini-computers people are regularly carrying with them to allow a mobile learning experience, anywhere, anytime? We answer the question on the basis of a “use case” about learning of a language (that is particularly well suited to this medium), and we develop an approach for such a service. We highlight not only the feasibility but mainly the pertinence and the added-value of such an approach, this from a pedagogical as well as from a technical standpoint. Finally, the development perspectives of mobile technologies are impressive and encourage us further studying and designing such personalized m-Learning solutions. Keywords: mobile learning, m-Learning, e-Learning, LMS, IVR, mobile phone, pedagogy, language learning, use case. INTRODUCTION The objective of this article is to present a pedagogical and technological approach to support language learning via a mobile phone. To the opposite of other studies using advanced devices such as « PDAs – Personal Digital Assistants » or « Tablet PCs », we will focus here on technologies widely available throughout the world (specifically second generation of mobile phones), with the perspective to allow m Learning for “everybody”. We start by positioning the context of mobile-learning (m-Learning) in the general field of flexible learning. We then define typical characteristics of a target market for an m-Learning service focused on language learning. Finally, we will analyse how such a service could be structured and setup, from a pedagogical as well as from a technological stand point. Joseph Fong, Fu Lee Wang (Eds): Blended Learning, pp. 327-339, Pearson, 2007. Workshop on Blended Learning 2007, Edinburgh, United Kingdom. 328 Yannick Jolliet CONTEXT Definition and Characteristics of m-Learning In order to correctly position m-Learning versus other various flexible learning approaches, we refer to the Brown model (figure 1). It positions m-Learning as a distant learning method via an electronic media, highlighting its mobile specificity which distinguishes it from online e-Learning It clearly appears that m-Learning is complementary to other learning approaches, whether they are electronic or not. It is in this perspective that we position our reflexion: how to define an added-valued learning service, complementary – and not substitutive – to other learning approaches. m-Learning distinguishes itself by some characteristics and key advantages, namely its ubiquity and possibilities of learning anywhere/anytime (« just-in-time delivery »). Classical mobile phones (not « PDAs) have some great strength: they are typically common and relatively cheap devices, small, light and with a long autonomy. However, in parallel they do also suffer from some serious weaknesses and limitations: typically very small screens and difficulty to input data for example. Figure 1: Sub-components of flexible learning, Brown (2004) Context and Approach If multiple studies have been conducted about the usage of mobile technologies to support or foster learning, the majority are focused on using complex and expensive devices such as PDAs or Smartphones (phones with PDA features). However, it is known these devices are also perceived as dedicated to professionals and difficult to use (Attewell 2004). On the other side, rare are the studies focusing on learning via standard mobile phones, especially outside of Asia. Device-related constraints highlighted above are often cited to justify the lack of pertinence or viability of such approaches. In addition, studies performed are closer to traditional e-Learning but applied on small screen, which probably explains also the low success rate. M-Learning: A Pedagogical and Technological Model 329 From our perspective, we believe “standard” mobile phones are such widely available devices they represent a fantastic opportunity, and therefore it is important to rethink our pedagogical strategies and our usage of technology with its inherent constraints, in order to offer a new form of m-Learning service. This way, and through time, m-Learning will tend to gradually distinguish itself from conventional e Learning. Our approach will therefore be not to simply adapt pedagogical content to the telephone medium, but rather to use mobile phones’ possibilities -in synergy with other technologies- to develop an effective and innovative mobile learning experience. m-Learning Potential There are today more than 2bio mobile phones throughout the world! Never a technology penetrated the world so rapidly, widely and deeply. Rising speed is increasing, mainly lead now by China and India but also by Brazil. « Informa » predicts 2.7bio mobile phones in 2010, with 50% in Asia-Pacific region (source ZDNet Research, see Reference). The mobile is –and will become more and more– a commodity and a mass-media. Below, we will focus on those rapidly growing markets in Asia-Pacific, but our reflexion applies to more developed markets. We have seen multiple telephony generations (2G, 2.5G, 3G), each offering a higher information flow rate, but also leading to the creation to more and more rich and complex phones. Studies show that by 2010, 50% of users will still be using 2G technology, and only 25% will have upgraded to 3G (source ZDNet Research, cited). Figure 2: Mobile telephony evolution in Asian markets (sources: multiple) Figure 2 represents the evolution of theses multiple dimensions for the fastgrowing Asia-Pacific market: market size (diameter), technology maturity (2G, 3G) and device type. Note that even by 2009, PDAs & Smartphones share remains minimal compared to classical mobiles phones. 330 Yannick Jolliet All these factors drive us to think there is an important technical foundation, rapidly growing, a real platform, for a mass market m-Learning, at the condition of leveraging classical technologies, meaning 2G mobile phones. In parallel of these technical advancements, we see also a profound society mutation related to how we learn. We hear more and more about a « learning society » and real-time learning, through people’s life but also through the day, anywhere, anytime. An important investment in personal development and training has become necessary to remain competitive in the global knowledge economy. m-Learning can naturally bring interesting solutions to answer part of this need, allowing to use “lost” moments and to choose where and when to learn something. A PERTINENT M-LEARNING OFFERING Learning needs obviously differ, but the context of these rapidly growing regions which ambition to become key players in the worldwide economy, encourages students and managers to learn foreign languages, in particular English. Actually, learning English is on one side a key factor for professional success and an important differentiator criterion for individuals, and on the other side a mean for a country to integrate the global economy. To highlight only one example: next Olympic games in Beijing in 2008 have encouraged Chinese government to launch a program to encourage and subsidized English learning, in particular for jobs related to tourism (see internet site in Reference). Similarly, m-Learning techniques obviously apply to multiples learning subjects, but we will focus in this paper on the case of language learning, and this via a 2G mobile phone technology. This specific “subject-technology” couple seems to us to be the most pertinent one to illustrate our point. A language learning service via m-Learning If a learning service via mobile phones offers some distinctive advantages (mobility, choice of time and place, potentially vocal interaction, etc.), it still remains subject to multiple constraints, some having been already mentioned (small screen, difficulty of reading long texts, data storage difficulty when even possible, multimedia limitations on first generation mobile phones, etc.) It is therefore important to set realistic objectives and to structure the learning process in order to take into account mobile phones’ possibilities and limitations. Researches about m-Learning Multiple researches have been conducted these last years in the m-Learning field, and they have served as a basis for our reflexion, especially: • The « m-learning » European project sponsored by the «Information Society Technologies Programme» (see References). • The worldwide MOBIlearn project, lead by Europe, which explores the different mobiles pedagogical approaches (see References). M-Learning: A Pedagogical and Technological Model 331 • The INLET project which attempted to proposed an introduction to the Greek language via mobile phones during the Olympic games of 2004. • Prototypes realised by Regan which have demonstrated that m-Learning could effectively support language learning (Regan 2000). • A mini Italian training via SMS, for foreigners, which was rather successful (see References). • The English CTAD company’s service, exclusively based on a vocal learning via mobile phones. • The basic service of English language training via mobile phones, set up by the BBC in China. Concrete Proposition for a m-Learning Service Taking into account what precedes, our proposition to define an effective learning experience via mobile phones is articulated around the following elements: • A learning of the key 1’000 words of a languages (level: between surviving and autonomy), in roughly 50 modules of 20 new words each, grouped by themes like transportation, food & beverages, health, etc. • Learners can choose anytime any module in the list. • They learn through scenario/role playing, which requires interaction with the system as well as with other learners. • Each learner becomes member of a community and interact with others via text message (SMS) or vocal conversations. • A learning scenario is divided into very small modules (« learning objects »), authorising very short sessions (2-3 minutes) and a great flexibility. • An entire scenario can be played in 2 days. • Learning instructions, including key vocabulary, are sent prior to the course by email (and are also available on an internet site for download or vocal play back). • After each module the scenario and the key points are summarised and available the same way. • An interactive vocal help system is available at any time, completed by a coaching system between learners community members. We immediately highlight some key characteristics which will be detailed further down. On the pedagogical side: the fun & role play approach and the interaction within the community. On the technological side, the complementarities of media, and the value of an interactive vocal service. APPROPRIATE PEDAGOGY FOR M-LEARNING Pedagogical Dimensions, Tools & Approach If it is clear that reading of long text on mobile phones is uneasy if not impossible, the textual dimension is still possible through usage of SMS (or even MMS = multimedia SMS). It is even very effective as messages are pushed to the recipient, who could respond/interact with the system, for example by sending a SMS with the right answer to a question. Finally, this technology is simple to use and largely adopted by phone 332 Yannick Jolliet users. On recent phones, equipped with navigators to read Web or WAP pages (web pages designed for mobile phones), or even equipped with Java client, we could potentially think to go further and send references or multiple paragraphs texts. The speech dimension is also fundamental, and probably evens the corner stone of a good m-Learning service. Phones are designed to transmit voice, and they allow therefore -without any major resistance- the listening of texts of a certain length, or of grammatical rules, or of voice help. Reciprocally, phone allow users to comfortably speak with either a system which can record him, allowing the learner to listen back to himself and compare with an ideal pronunciation; or it could allow an interaction with other learners from the community, or even with a physical professor. As per the writing, it is clear that phones are not so adapted because of their technological limitations. However, they can allow some interesting options: for example, via multiple choices questionnaires on SMS, to choose between different possible spellings, or to input and spell correctly few words. Some pedagogical tools in an m-Learning context offer valuable possibilities: • List of some key words, or the “word of the day”, pushed via SMS to develop vocabulary. • Multiple choices questionnaire “true/false”, “fill-in the blanks” exercises with responses via SMS. • Listening to a text read by vocal service, followed-up by a quiz to practice understanding. Low / Medium Interactivity Listening IVR Speaking IVR Voc & Grammar SMS and IVR (MMS) High Interactivity Writing SMS (WAP) Scenario based learning as we have imagined it (number of participants still need to be tested) with a riddle to solve. Combines IVR and SMS Communication - Usual concepts - Top 1200 words - Top 100 verbs - Advanced sentences - 4 Verb tenses Grammatical rule of the day Text download with News download with Quizz News hearing with Quizz Listen to a sentence, record yourself and listen download (SMS or MMS) or mistakes. Identify mistakes and send how listen to it all with Quizz to your recorded voice many mistakes there are. Receive the corrected text Combines Listening, Speaking, Reading, Voc & Grammar - Usual concepts - Top 600 words - Top 50 verbs - List of 100 basic sentences - 2 verb tenses (present + other) Connect 2 persons wishing to talk English Listen to sounds and identify Text download with Music Text download with Novel hearing (100 words) Listen to a sentence, together record yourself and listen right word mistakes. Identify Quizz with Quizz to your recorded voice mistakes and send how many mistakes there are. Receive the corrected text Survival Autonomy Reading SMS (MMS) Every day simple questions download (e.g. how much does it cost, where is station,...) with Quizz - Basic concepts (my name, etc? Every day simple Listen to a word or a questions listening (e.g. sentence, record how much does it cost, yourself and listen to where is station,...) with your recorded voice Quizz - Top 100 words - Top 10 verbs - List of 25 basic sentences Word download with 3 Connect 2 persons through SMS for English Word / Verb of the day message exchanges download (SMS or MMS) or different ortographs. Indicate whether #1, 2 or or listen to it all with Quizz Connect an English teacher with a customer 3 is the right one for a 10 minutes course Figure 3: m-Learning: matrix of pedagogical tools & technologies, related to the different learning levels (source: Eisenhart, Dufour, Loroch, Jolliet) • Recording of a vocal sequence with a play back to practice accent. • Grammatical explanation via vocal service. • Interactive scenario involving multiple participants for a “real-life” experience (see chapter 4.2). • Reference services with added value, such as dictionary (definition, synonyms, and antonyms) or translation service (word, phrase), etc. These services might be effectively delivered using simple technologies such as SMS or vocal systems (see section « Technology » below). The approach we defend is to rethink the learning sequence, and to structure a totally new course to leverage mobile technologies possibilities. M-Learning: A Pedagogical and Technological Model 333 Figure 3 summarizes different learning possibilities as potential m-Learning services. It positions multiple pedagogical tools & techniques for each learning level, and shows options and multiple and diverse. Scenarios and Role-Play Constructivist learning theories highlight that learners interpret information and the world through their own perception of reality: we learn though observation and interpretation, and we transform the information into personal knowledge (Coopers 1993). Student learns better when they can put learning into context. m-Learning facilities a personalized learning as it permits to learn (and collaborate) in any place at any time, allowing learning to be in context. Josefson’s researches on nurses’ training also show that knowledge is mainly acquired through experience and practice, and not only through mental faculties. In this perspective, m-Learning seems to offer interesting possibilities because of the mobility and freedom it offers, and because of its possible contextualization, especially when comparing to traditional computer e Learning. It is known that fun and games are fundamentals and powerful elements of the learning process. Multiple studies prove this fact, for example Dempsey’s ones which highlight how much simulations, adventure and problem solving contribute to the learning process (Dempsey and al. 2002). 82% of participants to a study on this subject feel that pedagogical mobiles games can help them in the learning of reading and writing (Mlearn 2004 conference). Finally, games stimulate curiosity, willingness to learn and to persevere (Ko, 2002), and allow experimentation with an immediate feedback (Roubidoux and al. 2002). Our approach of m-Learning proposes a situational immersion through role play, and introduces a suspense element with a sequenced scenario. The story allows a better attention and a higher fidelity rate, so a more efficient learning. Concretely, with our approach for a language learning course via m-Learning, courses are structured around mini-scenarios that learners will play with other participants, and with interactions of a vocal computer. « Script », text and vocabulary are shared upfront with students (via a web site or email, or even via fax or mail, depending on people’s technical equipment). Figure 4 shows an example of such a script: it describes the participants-actors, the story with keywords highlighted and a vocabulary list to learn. Anybody can listen the story played by a vocal computer in order to practice listening and understanding, and when participants are ready, they indicate it via SMS. The system assembles then a group of participants and notifies them by SMS in order to initiate the interactive session; the computer could play one or multiple roles depending of people’s availability. 334 Yannick Jolliet Figure 4: example of a role-play script for English lesson via m-Learning (source: Eisenhart, Dufour, Loroch, Jolliet) In our scenario above, Susan calls Tim and plays her role; he eventually correct her, or ask the system to pronounce again the sentence to check her accent, then he continues with his own text; and so on, in a real-time dialog. At any point in time, participants can interact with the system, either via a specific key on the phone or via a vocal command (see section « Technology » below) to obtain help, such as clarification on a word or on the scenario itself. The system can introduce a complicating factor in the form of an unknown participant (played actually by the system) which gives wrong information and complicates the initial scenario (actor in black in the example of script above). This element of surprise forces students to understand what is happening and to solve together an enigma, adding some suspense to the learning experience. The fun aspect allows to move from a sometime fastidious and scholar learning experience, to a stimulating social experience. And we see that phone is an effective medium for such interactivity and for a learning experience in community. M-Learning: A Pedagogical and Technological Model 335 Community and Networks From a long time we know that learning could be reinforced by collaboration with peers (Vygotsky 1978). In fact, the teaching practice evolves largely toward a more collaborative and participative approach. From a directive « teacher-centric » [1-n] model; we progressively move to a « learner-centric » approach (Ally 2004). The professor then becomes a coach and an animator who facilitates a more personalized and community based learning, where each student is a source of knowledge and a support for the others ([n-n] model). With mobile technologies, the learner becomes a true actor who controls the learning process and takes decisions aligned with his cognitive state. The socio-cognitive vision would also add that learning takes place in a social context, and that interactions and communication are as important as content of the learning process. Collaboration and information sharing with peers is a powerful mean to acquire and experiment new learning, and this is an essential element for an e-Learning approach. Phone is by nature a communication medium and without any doubt richer than email or text read on a computer; as such, it authorizes an effective learning experience. And this is why, despite current technical limitations, it is pertinent to use mobile phones in the context of e-Learning (versus PDAs or laptops). A learning community, dynamic and well leveraged, effectively contributes to the learning experience through multiple aspects. For example, support between members as well as the feeling to belong to a group contribute to reassure learners and increase their perseverance and fidelity to the program. Expertise sharing within the community allows a rich learning experience, of the [n-n] type. Anyone can in fact act as an « expert » and share with the network an answer or an explanation, put things in context or give an example (typically via the community site on the Internet, cornerstone of the learning network). Furthermore, interaction with other real persons offers a human dimension to the eLearning, which is often missing with other « Computer Based Training » (CBT) approaches. Finally, emulation between participants is also a success factor: an individual score based on success (e.g. answers to quiz, responses to lessons) and participation (help to others, information sharing) could be published on the web site. Winners of such a « Top100 » would gain access to new modules, or to other services for free (translation, etc.). Won points show progresses made, and are steps toward the final objective (which could be a certification). TECHNOLOGY FOR M-LEARNING We encounter too often resistances against the m-Learning based on standard mobile phones, mainly because of its technical limitations which at a first look seem too important. But in fact, vocal technologies progressed immensely and became very structured, allowing creating m Learning services and vocal applications without “reinventing the wheel” each time. Without entering into technical details, we will briefly describe 336 Yannick Jolliet here how technology could allow the implementation of an m Learning service as described above. Multimedia We mentioned it -and we will come back to this point-: to be really effective, the approach has to be multi-channels and multimedia. The phone becomes the principal mean to interact and to learn, but it has to be supported by other technologies when necessary: namely a web site to allow easy administrative management, but also efficient collaboration among members. Other medium such as emails or even paper allow keeping references of courses (scripts, vocabulary, grammatical rules, etc.). These media can even include hyperlinks to easily jump to an electronic content by entering only a simple code (either on the web site or on the phone via SMS) (see PaperLink technology in reference). Multi-levels Interactive System In order to implement a dynamic m-learning solution, it is important to setup a truly automated and « intelligent » system, a complete platform orchestrating interactions between users and with the system. Such a “server” should allow a rich interaction via multiple communication channels, and a management of vocal dialogs but also text or visual messages. The architecture remains classical, with three logical layers: • A presentation layer to support the interaction with users: a graphical interface (Web, Wap) for web browser, a textual interface for phones (SMS) or computer (email), or even – and mainly- a vocal interface through an « Interactive Voice Response » (IVR). • An application layer to manage information and content logic, as well as to manage the learning process content and to personalise the content: it is the true orchestrator of the system and the heart of the « Learning Management System ». • Finally, a data warehouse, containing all « learning objects » (content, vocabulary, grammatical rules, pre-recorded messages, scenario, etc.) which have to be dynamically assembled by the above layer. Figure 5 describes such a technical architecture, well adapted to an interactive and dynamic m Learning service. All these technologies are existing today and could be assembled to create such an integrated system (e.g. by using architectures and tools such as the ones of VoiceObjects, see references). Interactive Voice Response (IVR) Standards and technologies in the field of voice recently reached a maturity state to allow industrialized development of powerful interactive voice applications. We are far from the old fragmented solutions to be assembled: SMS server, answering machine, basic navigation via phone keys and pre-recorded messages, etc. Today, integrated solutions cover the entire spectrum of rich vocal application development and allow to dynamically creating complete personalized vocal interfaces, similar to what is done on the internet portals (e.g. « My Yahoo »). We speak then about « vocal pages » and of Vocal Content Management System (vCMS). Such systems are based on voice-specific XML standard (« VoiceXML ») which allows describing, coding, managing and playing vocal sequences. « Text to Speech » M-Learning: A Pedagogical and Technological Model 337 technologies are today absolutely Figure 5astonishing; at light-years of artificial play back we used to hear only few years ago. It is now extremely difficult to differentiate the machine from the human, as machine’s voice is now really warm and modulated, and flowing naturally. Figure 5: Suggested rich technical architecture for an Integrated Symmetrically, voice recognition technologies have gone a long way and reach now 99% recognition rate in quiet environments. And this is precisely where we face a bottleneck with mobile technologies: ambient noise, often important, adds up to the poor mobile phones’ microphone and transmission quality: bottom line the voice 338 Yannick Jolliet recognition cannot be really effective in such a case. However, recognition of predefined keywords is absolutely possible and effective, opening an interesting field of possibility to interact with the user who could easily navigate through “vocal pages”, similarly to browsing the web. Of course, interactions and options selection can also be made using phone’s keys. Finally, we should highlight two interesting aspects this type of architecture offers: anonymity and easy payment. It is essential to preserve participants’ anonymity, and a fortiori when these one interact with each others. Here, any relation between participants always go through the system which manages basically a three parts (or more) conversation, and which can directly and dynamically intervene in the pedagogical relationship or the on-going scenario. The payment of such an m-Learning service is largely facilitated by the fact that all interactions and lessons (conversations and SMS) happen via a unique over-taxed phone number, and thus the phone operator can very easily manage the billing of the service and the retribution of the partner. So, such a technical architecture authorizes the truly dynamic generation of personalized vocal pages (taking into account user profile, his level, etc.), a relatively rich interaction with the user (via SMS, keys or vocal commands) and a granular management of the different reusable « learning objects » (voice, word/text, tests, etc.). Theses contents are dynamically assembled depending of the context (number of learners online, personal profiles, course level, etc.) to allow a personalization of the pedagogical scenario, and a rich learning experience. CONCLUSION AND PERSPECTIVES With this article, we tried discussed the viability of an effective learning experience via simple mobile technologies. We demonstrated the pertinence and certain possibilities for learning a language using second generation mobile phones. Adapting pedagogy to the technology is pertinent in this case as it allows capitalising on devices and habits largely adopted, without trying to replace preferred users systems. It is obvious that extremely rapid progresses of communication technology, as well as their wide spread around the world, offer at short term dramatically increased pedagogical possibilities. To cite only few of them, current or available in a very near future, we would highlight: media photo & video or phones equipped for video conference, instant messaging, pushed emails, internet browsing, embarked or downloadable applications (typically in Java)… But even more: content developed in Flash-lite which allows rendering interactive and animated multimedia (see Linsalata 2005), high-speed phones (3G) or geo-localisation via integrated GPS chips… In conclusion, it is clear that numerous of rich perspectives exist down the road, and still remain to be explored in order to position added value m-Learning services. This especially in the mobile learning field, coupled with other teaching forms / media in a truly integrated approach (« blended learning »). Finally, the number of mobile devices will soon be greater than PCs, which promises m-Learning an important role in tomorrow’s teaching. M-Learning: A Pedagogical and Technological Model 339 ACKNOWLEDGMENTS We would like to thank here F. Eisenhart, A. Dufour & V. Loroch for their contribution to our thinking process about some of the aspects developed here. REFERENCES Articles and Books Coopers PA (1993). Paradigm shifts in designing instruction: from behaviourism to cognitive to constructivism. Educational Technology, 33(5), p12-19. Brown (2004), The role of m-learning in the future of e-learning in Africa. In D Murphy, R Carr, J Taylor and W Tatmeng Distance education and technology: issues and practice. Hong Kong: Open University of Hong Kong Press, p197–216. Roubidoux MA, Chapman CM, Piontek ME (2002), Development and evaluation of an interactive web-based breast imaging game for medical students. Academic Radiology, 9(10), p1169–1178. Ko S (2002). An empirical analysis of children’s thinking and learning in a computer game context. Educational Psychology, 22(2), p219–233. Berson MJ (1996). Effectiveness of computer technology in Social Studies: a review of the literature. Journal of Research on Computing in Education, 28(4), p486–499. Dempsey JV, Haynes LL, Lucassen BA, Casey MS (2002), Forty simple computer games and what they could mean to educators, Simulation and Gaming, 33(2), p157–168. Regan, M (2000). Mobile learning, http://sll.stanford. edu/projects/mobile learning. Vygotsky (1978), Mind in Society. Cambridge MA: Harvard University Press. Conferences Ally (2004). Using learning theories to design instruction for mobile learning devices, in A book of papers from MLEARN 2004, published by Jill Attewell and Carol Savill-Smith, p5. Attewell (2004). Mobile learning and social inclusion: focusing on learners and learning, in Learning with mobile devices, research and development, a book of papers, edited by Jill Attewell and Carol Savill-Smith, p3. Hitz (2004). PaperLinks – Linking printouts to mobile devices, in A book of papers from MLEARN 2004, published by Jill Attewell and Carol Savill-Smith, p5. Internet sites ZDNet Research blog, 29.9.2005 12:10AM, http://blogs.zdnet.com/ITFacts/?p=9065. Linsalata (2005), White Paper sponsored by Macromedia, Addressing growing handset complexity with software solutions, in IDC Analyze the Future, www.idc.com. « m-Learning » : European project sponsored by the «Information Society Technologies Programme», www.mlearning.org. « MOBIlearn » : worldwide research & development project, lead by Europeans, www.mobilearn.org. Mini-Italian training course via SMS : realised by the company Albatros in collaboration with the CRMPA, in the context of the cited European m-learning project, www.mlearning.org. VoiceObjects, Voice Application Management Systems, www.VoiceObjects.com. China and English language learning: “Beijing Launches English-Learning Programs” (http://www.china.org.cn/english/2001/Jul/16399.htm), “Olympic Bidding Success Spurs English Language Fever in China” (http://english.peopledaily.com.cn/200107/29/eng20010729_76042.html) Author Index Alberts, Philip P. 53 Alexandru, Catalin 217 Alfonseca, Enrique 186 Amelung, Mario 257 Anguiano, Eloy 186 Barker, Philip 42 Callaghan, Victor 267 Chan, Chilli C. K. 237 Chan, F. T. 79 Chan, Jason K. Y. 101 Chew, Esyin 123 Chong, Melody P.M. 88 Choy, M. 9, 30 Chu, Raymond 136 Dettori, Giuliana 174 Domínguez-Mateos, Francisco 292 Dragan, Alexandru 217 Fang, Jenny 136 Fok, Apple W. P. 237 Fong, Joseph 19, 30 Garcia, Francisco José 206 Griffin, Darren K. 53 Henrich, Andreas 150 Hewagamage, K. P. 279 Hijón-Neira, Raquel 292 Hsueh, Hsiang-Yuan 66 Hua, Jing-Shiuan 66 Huang, Shi-Ming 66 Lam, S. 9 Law, Ken C. K. 101 Lee, Kenneth K.C. 88 Lee, P. W. R. 79 Leon, Enrique 267 Liu, Jing 315 Liu, Yintao 315 López-López, Ignacio 292 López-Rodríguez, Domingo 227 Ma, Will Wai-kit 303 Mérida-Casermeiro, Enrique 227 Merino-Córdoba, Salvador 227 Moebs, Sabine 162 Mora-Bonilla, Angel 227 Moreira, Tiago 206 Murray, Linda A 53 Pascual-Nieto, Ismael 186 Peiris, K.H.R.A. 279 Perez-Marin, Diana 186 Persico, Donatella 174 Piotrowski, Michael 257 Poon, C.K. 9 Premaratne, S.C. 279 Rebedea, Traian 217 Rego, Hugo 206 Rodriguez, Pilar 186 Rösner, Dietmar 257 Shen, Liping 267 Shen, Ruimin 267 Sieber, Stefanie 150 Stephenson, Julia E. 53 Ip, Horace H. S. 237 Jones, Norah 114, 123 Jolliet, Yannick 327 Trausan-Matu, Stefan 217 Turner, David 123 Velázquez-Iturbide, Ángel 292 Kim, Won 1 Author Index Wang, F.L. 9, 30 Weibelzahl, Stephan 162 Wong, Chan Lam 196 Wong, Francis 136 Yang, Fang 315 Yu, Y.T. 9 Yuen, Allan Hoi-kau 303 Yuen, L. 9 Xu, Lei 315 Zhang, Liming 196 Zheng, Li 315 341