Students' Use of Learning Objects
Claire Bradley, London Metropolitan University
Tom Boyle, London Metropolitan University

Abstract
This paper shows how learning objects have been designed and introduced into campus-based higher education modules, and describes the impact they have had on students. This work has stemmed from a large project that had the overriding aim of improving student pass rates and retention. The project has involved over 600 students in the first year, and has been thoroughly evaluated. In this paper we provide examples of some of the award-winning learning objects developed, and the pedagogic principles incorporated into their design. Some of the results from the evaluation are presented, to show how students have used the learning objects, and their opinions of them.

1. Introduction
A project was initiated at London Metropolitan University in March 2002 to solve a real educational problem - to improve pass rates and retention in introductory programming. A set of multimedia learning objects was developed as part of the project. In the university, introductory programming is taught to over 600 students each year on modular courses at HND, BSc, and MSc levels, and a number of problems were being faced. Student numbers had increased dramatically in computing courses, and the adoption of widening access policies in the university had broadened the range of ability and experience within the student population. In addition, programming is a difficult subject to teach to novice students, who need to grasp a number of complex and abstract concepts to become proficient. Other universities were reporting similar problems (e.g. Jenkins and Davy 2001, Jenkins 2002).

The project has introduced a blended learning approach, which incorporates traditional and new elements. Learning objects were incorporated into the modules, designed to improve student learning, and to provide online materials that would support their face-to-face teaching (which was not reduced). More traditional changes included the move to a common base curriculum that would teach Java in all the modules, changes to the assessment strategy, and organisational changes that would provide more effective tutorial support for students.

The multimedia learning objects won a European Academic Software Award (EASA) in September 2004.

2. The learning objects
The learning objects had to be developed in a 6-month period for the start of the next academic year, using the available internal resources. A multi-disciplinary team was created to collaboratively develop the objects, which included specialists from a number of University departments. It was led by a senior researcher from the university's Learning Technology Research Institute, complemented by the module tutors who undertook syllabus creation, addressed and implemented organisational issues, and authored learning objects. Each member of the teaching staff took responsibility for part of the syllabus, and drafted the content for associated learning objects. A multimedia developer from the university's Teaching and Learning Technology Centre produced the multimedia learning objects, and a researcher undertook the project evaluation.

A set of 54 learning objects was created with the aim of helping students to understand the more abstract and complex aspects of Java programming. Many are text-based objects authored as html pages, each focusing on a single topic and providing clear explanations and examples of Java code and how it is constructed. These are complemented by 14 multimedia objects authored in Macromedia Flash that use animations and interactive simulations to provide visual examples of Java concepts and program code. A number of multimedia techniques are employed to engage the students in the subject matter and to help them understand these complex and abstract programming concepts (Boyle 1997). Figure 1 below shows four of the screens from the multimedia learning object on 'While loops'.

Full size image

Full size image

Full size image

The learning object on 'While loops' is typical of the multimedia objects. It seeks to engage the learner through attractive use of visualisation. As the subject matter is about 'repetition', this is illustrated by using repeated actions to move an object across a screen. Familiar objects are chosen and they are animated to illustrate the concept, in this case of a hammer repeatedly knocking in a nail, a car and a submarine moving across the screen. These visual examples are accompanied by the programming code that would be used to re-create each sequence in Java. The user has full control of the running pace and navigation of the object. They can run and re-run the step-by-step sequences, and they can move back and forwards to any screen in the object at any time. 'Scaffolding' is used at the end of the object to enable the learner to engage in building up sequences of programming code in a safe, supportive environment. This takes the form of a short self-assessment quiz. In the first part the student can build up the required Java code to move a horse across the screen by selecting from code fragments provided. In the second part they have to identify errors in the Java code provided. Feedback to the user is provided, and the correct answers are available for users if they are unable to complete the tasks. Throughout the learning object, the student can interact with the visual simulations, becoming actively engaged in the learning process, and they can see the results of their actions.
An external link to the 'While loops' learning object. http://learning.unl.ac.uk/java1/lm_jav1_basrep/lm_jav1_whileloops_x/lm_jav1_whileloops_1.htm
An external link to the learning object 'Changing the attributes of objects'. http://learning.unl.ac.uk/java1/lm_jav1_attributes/lm_jav1_attributes_x/lm_jav1_attributes_1.htm

3. How the learning objects were incorporated into student modules
The learning objects were integrated into a virtual learning environment (VLE) created within WebCT, the system widely used in the university. The VLE contains all the online resource materials that support the module, as well as the online assessments, study tools and communication facilities. The VLE and its contents are designed to support and complement the face-to-face components of the module. Each week students have a 2-hour lecture, and a 2-hour lab session in which they conduct practical programming exercises with on-site guidance from a teaching assistant. Relevant learning objects are linked into the syllabus each week from the lecture notes, which summarises the topics covered in the lecture. The learning objects are introduced to students in lectures and the lab sessions as optional resources that they could access to help them each week with their programming exercises and assessments. Figure 2 shows how the learning aids are viewed from within the WebCT environment.

Figure 2. A text-based and multimedia learning object viewed from within WebCT.
Full size image

The development process we adopted and how the learning objects were integrated into the modules is discussed more fully in the online paper 'The development and deployment of multimedia learning objects', Bradley and Boyle 2003. The design of the objects is discussed in depth in other papers (see Boyle 2003, Bradley et al 2003). The focus of this paper is on the student use and impact of the learning objects developed.

The new modules were delivered to students from September 2002. Student pass rates in the first year of implementation dramatically increased on all modules, compared to the previous year, as shown in Table 1 below. There was an increase of 19 percentage points for the HND module, 15 for the BSc module, and 12 for the MSc module.

Use of the modules has been accompanied by thorough evaluation. The rest of this paper will focus on the evaluation and present some of the results, particularly relating to the use and reception of the learning objects. To students, these are known as ‘learning aids’, and this is the terminology used in the evaluation data that follows. More detailed evaluation data can be found in Bradley and Boyle 2004.

4. The evaluation process
The evaluation process was planned in detail, and incorporated a number of methods to provide both quantitative and qualitative data. It combines 4 student questionnaires, interviews, and the analysis of tracking data from the learning object repository and from WebCT.

A substantial amount of data was gathered during the first semester of use by over 600 students. Relatively large completion rates on the questionnaires provide us with large samples on which the results are based. The questionnaires and interviews with students give us a rich coverage of data relating to their attitudes and views. The tracking data has provided evidence of the access levels of the learning objects and of WebCT.

5. Evaluation results
Access of the learning objects
We have evidence on the amount of use of the learning objects from the data captured from the server log files from the learning object repository. They show that a total of 79,325 page requests for learning aids were made during the semester. We can also see when pages were most frequently accessed. Whilst access was greatest during the day between the hours of 9am and 6pm (74%), 26% were accessed outside these hours, i.e. in the evening and early hours of the morning. The busiest period was between 10.00-12.00 in the morning (22%), which coincided with lab sessions for many students. On a day-to-day basis, 92% of pages were accessed on weekdays and 8% at weekends. The WebCT page tracking data shows an average time per page hit of 5 minutes 35 seconds, indicating that pages are accessed for sufficient time for their content to be absorbed. This data not only shows us that students are accessing the learning aids in large numbers, but also that they are studying in their own time outside of lab sessions, at times of their choosing.

Student opinions of the learning objects
Students were first asked about their opinions of the learning objects and our approach to teaching the modules in the second questionnaire, completed in the middle of the semester by 223 students. We asked students how useful they found each of the major teaching components, and the results are presented in Figure 3 below. They show that students rate the learning aids comparably with the other components in their module. They also demonstrate a preference for the multimedia animated learning aids, which was also confirmed in the interviews.

In the interviews with a random selection of 36 students, they were shown some learning aids and were asked if they had used them and what they thought about them. 81% of the students had used the animation-based aid and 78% the text-based aid. When asked which type of aid they prefer, 50% said animations, 8% said text, but 19% said they liked both, preferring a combination of the two. The majority (78%) said that the animations helped them to learn the concepts being addressed.

The quizzes included in some of the multimedia learning aids received mixed views from students interviewed. They were designed as short, self-assessment exercises, for students to quickly test their understanding of the topic, to reinforce their learning and build up their confidence. Many students said that they did not do them, and many hadn't even seen them. On further questioning, students revealed that if they didn't help them to achieve their weekly goals, then they didn't invest valuable time in doing them. However, the students who usually did these quizzes, found them very helpful in confirming their learning and liked them.

At the end of the semester, we wanted to find out more about the students' learning experience, and the impact that the learning aids had on this (Figure 4). This third questionnaire was completed by 117 students. Approximately 30% of students found the learning aids very useful and 60% useful. Only a small proportion found them not useful or helpful or didn't use them.

Word document: screen shots of the learning aids, and a selection of comments made by students.

An external link to the 'If statement' learning object which has an example quiz at the end. http://learning.unl.ac.uk/java1/lm_jav1_basdec/lm_jav1_if1_x/lm_jav1_if1_1.htm

Figure 4. Helpfulness of the learning aids in learning Java

The main messages coming from students about their opinions of the learning aids overall, is that they are useful, help them to learn, and are a valued component of their module. The students like having support materials online that they can access when they want to, and they particularly like the multimedia aids that provide visual examples of Java code in action.

How students used the learning objects
In the post module questionnaire completed by 47 students on the follow-on programming module, we explored how they were using the learning aids (see Table 2). We found that many students were using the aids they needed to complete their weekly programming exercises (25), but also that 23 students used them for revision, and 22 said they used all the relevant aids each week. 10 students also used them during the exam, which was allowed.

Table 2. How learning aids were used

To reveal their patterns of learning in more detail, the WebCT tracking data was analysed for a random sample of 18 BSc students. It revealed a great variation in activity from one student to another, both in terms of the number of times they logged into WebCT and accessed materials, and in the materials they accessed. The total number of learning aids accessed by them was 1,516, giving a mean average of 84 per student. However, the mean average hides the variation amongst students, which was considerable. 1 student made only 8 accesses to learning aids, whilst another made 206 accesses. This range of access to the learning aids can be seen in Figure 5 (each bar represents the access by one student). One conclusion that can be drawn, is that although our student population is diverse in respect of its range of abilities and previous experiences, it demonstrates a range of use of the available resources, and shows that students are adopting learning patterns to suit their individual needs and goals.


Figure 5. Student access of learning aids (sample of 18 students)

6. Conclusion
This paper has provided an example of how learning objects have been designed and introduced into campus-based higher education modules. Our aim was to improve pass rates and retention, with the introduction of learning objects as part of a blended learning solution to help improve student learning. Pass rates have dramatically improved on all modules. Extensive evaluation of the use of the learning objects with students has also been carried out. One of the issues in evaluating learning objects is that by their nature they will be used as components in larger pedagogical systems. It is difficult therefore to separate out the contribution that the learning objects make to the overall improvements in module results. It is clear, however, that the objects were used extensively by the students, and the students rate them positively. The study indicates that learning objects can have a significant impact in improving perceived pedagogical quality, which is a step towards achieving the vision of using learning objects to improve the quality of learning.

One notable feature of the picture that emerges is the variety across the large student cohort involved. The extent of use varies from practically none to very high levels of access. The general pattern of use is that the student organises the use of the learning objects around the focal concerns of their study. In programming the focal activity is writing programs in assisted laboratory sessions and outside formal contact hours. There is an important lesson here for the use of learning objects. Our experience suggests that the nature and pattern of use will be shaped strongly by the focal concerns in the subject domain. These vary across subjects and this suggests that we might expect strong differentiation in patterns of use across different subject domains.

It is also apparent that students welcome these e-Learning components within their modules. They like having support materials online that they can access where and when they want to, and they particularly like multimedia that provides visual examples that they can understand. Having such materials enables them to take more control over their own learning, allowing more flexible study patterns at times that suit them.

7. References
Boyle, T. (2003). Design principles for authoring dynamic, reusable learning objects. Australian Journal of Educational Technology, 19(1), 46-58. [Online]. Available: http://www.ascilite.org.au/ajet/ajet19/boyle.html
An external link to Boyle 2003. http://www.ascilite.org.au/ajet/ajet19/boyle.html

Boyle, T. (1997). Design for multimedia learning. Prentice Hall.

Bradley, C., & Boyle, T. (2004). Student evaluation of the use of learning objects in introductory programming, in L. Cantoni & C. McLoughlin (eds.), proceedings of ED-MEDIA 2004, World Conference on Educational Multimedia, Hypermedia & Telecommunications, June 21-26, Lugano, Switzerland, AACE, pp. 999-1006, ISBN 1-880094-53-3.

Bradley, C., & Boyle, T. (2003). The development and deployment of multimedia learning objects. Learning Objects Symposium 2003, Educational Multimedia and Hypermedia, June 2003, Honolulu, USA. Available online from: http://www.cs.kuleuven.ac.be/%7Eerikd/PRES/2003/LO2003/Bradley.pdf
An external link to Bradley 2003. [PDF format] http://www.cs.kuleuven.ac.be/%7Eerikd/PRES/2003/LO2003/Bradley.pdf

Bradley, C., Boyle, T., and Haynes, R., (2003). Design and evaluation of multimedia learning objects, in D. Lassner and C. McNaught (eds.), proceedings of ED-MEDIA 2003, World Conference on Educational Multimedia, Hyper-media & Telecommunications, June 23-28, Honolulu, Hawaii, USA, AACE, pp.1239-1245.

European Academic Software Awards (EASA) website. Available: http://www.easa-award.net/
An external link to EASA. http://www.easa-award.net/

Jenkins, T. (2002). On the difficulty of learning to program. In Proceedings of 3rd Annual Conference of the Learning and Teaching Support Network for Information and Computing Science, Loughborough, UK. Loughborough: LTSN-ICS.

Jenkins, T., and Davy J. (2001). Diversity and motivation in introductory programming. Italics, 1(1). [Online]. Available: http://www.ics.ltsn.ac.uk/pub/italics/issue1/tjenkins/tjenkins1.html
An external link to Jenkins 2001. http://www.ics.ltsn.ac.uk/pub/italics/issue1/tjenkins/tjenkins1.html

London Metropolitan University website. Available: http://www.londonmet.ac.uk
An external link to LondonMet. http://www.londonmet.ac.uk

Higher Education Academy Subject Centre for Information and Computer Sciences website. Available: http://www.ics.heacademy.ac.uk/
An external link to HE Academy ICS Centre. http://www.ics.heacademy.ac.uk/

8. Acknowledgements
Our thanks to the following who have also made valuable contributions to this work. Richard Haynes the multimedia developer, Poppy Pickard, Ray Jones, Peter Chalk and Ken Fisher who were members of the core team and authored many of the resources. We would also like to acknowledge the support of the Higher Education Academy Subject Centre for Information and Computer Sciences in the UK for providing funding support for the project.

9. Learning object website
The website listed below includes information about the project and its results, and has more examples of the objects developed.
http://www.londonmet.ac.uk/ltri/learningobjects
An external link to LondonMet learning object website. http://www.londonmet.ac.uk/ltri/learningobjects

********** End of Document **********