Digital Lectures: If You Make Them, Will Students Use Them?
Constraints on Effective Delivery of Flexible Learning Systems
Tim Bell, University of Canterbury
Andy Cockburn, University of Canterbury
Bruce McKenzie, University of Canterbury
John Vargo, University of Canterbury

Abstract
Preparing courses for flexible delivery and distance education is normally a time-consuming and expensive process. This paper describes the design and evaluation of a system that automatically captures and indexes audio and video streams of traditional university lectures without demanding any changes in the style or tools used by teachers. Using a wizard-of-oz technique to simulate the automatic indexing, we ran a four-month trial of the system in a large (746 students) first year Computer Studies course. The results reveal some surprising social implications about making flexible delivery available to students at a residential university. Early in the trial, many students expressed an intention to use the system, but few did. Late in the course, many students stated that they urgently needed the system for revision, but even fewer used it. At the same time, lecture attendance appeared to be lower than normal. We hypothesise that the availability of a flexible alternative to lectures removed the necessity of attending lectures, and that students deceived themselves about their intentions to catch up using the digital medium. A survey of students was conducted to discover the reasons for non-use. Results of that survey showed that other time pressures and logistics problems hampered usage. Recommendations for addressing such challenges are included.

1. Introduction
The rapid development of technology-mediated learning systems has led to a flood of utopian promises involving "flexible delivery," "distance education" and "virtual learning." A range of obstacles hampers the fulfilment of such promises, including technological, financial and social constraints. These constraints limit the practicality of the glamorous visions of virtual learning involving real-time distributed classrooms supported by high-bandwidth multi-party video interaction. High-bandwidth networks will eventually be sufficiently ubiquitous to enable rich virtual learning environments with equitable participation possibilities for proximate and remote students, but current implementations are severely constrained by bandwidth and other considerations.

In 1999 the leaders of our university-the University of Canterbury, New Zealand-announced their intention to equip all lecture theatres on campus (approximately 20) with the necessary equipment to support virtual learning. Unsure what this would entail and at what cost, the authors of this paper proposed a trial research project to investigate practical methods for flexible delivery of lectures.

Without massive institutional change, it was clear that Canterbury academics would not have time to develop extensive course materials for virtual learning. With this constraint in mind, we focused on designing, implementing and evaluating a system that would maximise the pedagogical benefits to students while minimising (to zero if possible) the additional work for the academics who deliver and administer the courses. Having designed and implemented the system, we trialed it as a resource available to the 746 students enrolled in a yearlong first year Computer Studies course.

This paper describes the results of the four-month trial of the Canterbury Digital Lectures system. The system was designed to automatically capture and index traditional lecture content without requiring course teachers to change their presentation styles in any way. Students who were unable to attend lectures at their normal time and place would be able to view them by accessing CD-ROMs held at the university library and in computer labs.

The following section provides the background design rationale for our system by reviewing various possible implementations of "virtual learning." Our description of the system and its objectives is followed by a discussion of the unexpected findings from the evaluation. The final section concludes the paper.

About the authors...

2. Dimensions of Virtual Learning
This section presents four dimensions of 'virtual learning' that critically influence the capabilities of technology for enhancing learning.

2.1 Learning Media
The traditional lecture remains a defining element of most university courses. Although there are many ways to stimulate interaction and small-group work within the lecture environment (for example, see Fischer (1998)) lectures remain a largely one-way channel from the professor to the students. The course notes, readings, and textbook support students in clarifying and elaborating on the 'seeds' of knowledge that are (hopefully) planted during the lecture. Arguably the most valuable part of traditional courses occurs when students actively test, articulate, and manipulate their understanding. This normally occurs in seminars, tutorials, labs, and assignments.

A wide variety of computer-supported media can be used in an attempt to enhance, offer a surrogate for, or replace these traditional media.For example, it is becoming common for students to bring tape-recorders to lectures, but the lack of any indexing into the audio-stream severely inhibits the potential utility of the recordings for a student who understood all of a lecture except for the bit on topic X. Systems such as the Classroom 2000 project (Abowd 2000), the Cornell Lecture Browser (Mukhopadhyay and Smith 1999), and our Digital Lectures project aim to overcome these limitations by automatically capturing and indexing lecture content. A key distinguishing characteristic of new computational learning media is the level of activity required from students. Passive media simply provide a static resource for the students. Recording and indexing lectures and the now standard practice of putting course materials on the web are both examples of relatively passive media. Passive approaches to learning media have been criticised as gift-wrapping (Fischer 1998), but it seems reasonable to expect that they can provide improvements to courses at relatively low costs. For example, web-based course handouts can be updated and improved on-demand, and indexed lecture recordings can aid students who were unable to attend the original lecture.

In contrast to passive media, active media provide an interactive resource that students can use to test and build their understanding. There are three types of active educational systems. Firstly, interactive simulation and exploration environments let students explore the effects of changing properties within an interactive space. For example, animations of algorithm execution (see Brown and Sedgewick (1985) for an early example) allow students to explore the algorithm's behaviour by changing data and parameter values and observing the resultant behaviour. Secondly, intelligent tutoring systems (Sleeman and Brown 1982) attempt to model the student's understanding of a problem domain, and then tailor the information they present appropriately. Thirdly, reflection and discussion spaces, such as bulletin board and Net News systems, allow students to build a repository of information that interlinks contributions by a potentially wide range of participants and sources. Examples include the Dynasites system (Fischer 2000) that was explicitly constructed as an environment to support lifelong learning, and a wide range of design-rationale and FAQ type systems such as the Answer Garden (Ackerman and Malone 1990) and gIBIS (Conklin and Begeman 1998).

We do not advocate either an exclusively passive or active approach to computational media in support of learning. Rather we see that the optimal solution will combine and integrate a variety of media to allow users to choose tools that best suit their needs at a particular time.

2.2 Time and Place of Learning
A key feature of flexible delivery through computer supported learning tools is that it frees students from the need to be in a particular place at a particular time.

There are several important implications of this capability. Firstly, the flexibility gained broadens the potential student base, opening a wide range of educational possibilities for lifelong learning, for the full-time employed, and for those with family commitments. Secondly, the perennial problems of timetabling clashes need not exclude students from particular courses. Thirdly, lecture theatre capacity no longer needs to constrain course sizes. Fourthly, on-line media can be reviewed as many times as the student feels is necessary. Traditional lectures, in contrast, are a once-only medium, and students who missed the lecture or temporarily lost attention have no way to review the real-time explanation. Finally, the freedom from time and place constraints opens the possibility of globally competitive education markets. Commercial Internet-based education resources such as stanford-online.stanford.edu and unext.com are early entrants into this market.

2.3 Support for Collaboration
Traditional courses, taught at residential universities, provide a natural infrastructure for supporting and promoting peer learning. One of the risks of distance education is that the lack of physical proximity will act as a barrier to this valuable resource.

On-line systems can support collaboration through simple text-based bulletin boards, through collaborative simulation and exploration environments that explicitly account for multiple users (for example TurboTurtle's synchronous collaboration (Cockburn and Greenberg 1998) and AgentSheet's asynchronous collaboration (Repenning, Ioannidou, and Phillips 1999)), or through supporting collaboration around a passive medium such as recorded lectures. Of particular interest to us is the notion of supporting collaboration around video recordings of lectures. In an early study on video-based instruction, Gibbons, Kincheloe and Down (1977) showed that engineering students' learning can suffer when students watch lectures individually. However, when students receive support for discussion and collaboration around the video, using the Tutored Video Instruction (TVI) model, their learning improves. This early research has been substantiated by further studies with other content disciplines and in other cultures (Appleton et al., 1989; Murray & Efendioglu, 1999). Subsequent research, such as that reported in Cadiz, Balachandran, Sanocki, Gupta, Grudin, and Jancke, (In Press) has begun to investigate distributed versions of the TVI model, where physically remote students are connected to each other by multiple audio/video feeds.

A final issue is that of equity between students participating remotely and locally. Anecdotal evidence indicates that remote students in live video lectures (one-way video, but two way audio) felt like second-rate participants¹, although it did give them more freedom in the lecture (for example, to arrive late, or to collect a handout from the front).

2.4 Capital Investment, Effort and Social Factors
The final dimension that critically affects the nature of on-line learning support is the degree of capital investment and effort put into developing the resources.

Institutions with a long history of supporting distance education-for example Stanford University in the U.S. and the Open University in the U.K.-make large capital investments in producing high quality media such as broadcast quality TV lectures/documentaries to support their courses. These financial outlays are then ameliorated over thousands of students across the entire nation, and over several years.

Clearly, not all teaching institutions can compete for distance learning students against highly commercialised alternatives. The question then becomes the following: how can traditional residential universities best use their available resources to enhance their support for learning?

Beyond the resources dedicated to on-line courses, many social factors must be addressed in preparing courses for flexible delivery. Systems that depend on lecturers and course administrators changing their teaching habits may promise a wide range of learning benefits to students, but if the lecturers fail to make the necessary changes then no benefits can be realised. Furthermore, many lecturers have concerns about issues such as copyright, liability and job-protection when every action and utterance in the lecture theatre is captured on multiple media (Stein, 2001; Gorman, 1998).

3. Canterbury's Digital Lectures Project
The goal of Canterbury's Digital Lectures project is to develop mechanisms for flexible delivery of conventional lectures that enhance learning while requiring no modifications to the lecturer's teaching methods. Like related work on the Cornell Lecture Browser, our goal is ''to allow a speaker to walk into a lecture hall, press a button, and give a presentation using blackboard, whiteboards, 35mm slides, overheads, or computer projection. An hour later, a structured document based on the presentation will be available [...] for replay on demand'' (Mukhopadhyay and Smith 1999).

The main difference between our work and that of the Cornell Lecture Browser is that we focus on the ways in which students used the end products, while Mukhopadhyay and Smith focus on the technology used to generate the end product.

3.1 The Student's Interface
Figure 1 shows the prototype interface to a digital lecture. In this prototype there are two video streams, represented by the thumbnail images at the top-middle of each window. In Figure 1, the user has chosen to watch a broad video stream showing the front of the lecture theatre, including the lecturer and the overhead projection screen. In Figure 2, the user has clicked on the other thumbnail image to zoom into the text on the overhead material.

Various controls allow the user to navigate freely through the lecture. As well as the normal fast-forward and rewind controls, the user can click on the timeline (beneath the control-buttons) to jump to a particular time in the lecture. Also, the right-hand side of the interface provides a hypertext keyword index to the lecture. Clicking on an item in the list causes the video and audio streams to immediately jump to the associated part of the lecture.

The actual interface that the students used in the trial, reported in the evaluation section, was identical to this interface except that it did not support multiple video streams.

Figure 1. The student's interface.

Web pages demonstrating the system.

A screenshot movie (AVI, 4.9 MB) showing the digital lecture system.

3.2 Capture and Distribution
In investigating automatic capture techniques, we explored a range of research challenges such as optical character recognition from low-resolution video images, and automatic detection of changes in video streams (in order to detect changes in the state of the blackboard or overhead slide, etc.) These technological issues are further described in Adjeroh, Bell, Cockburn, McKenzie, and Vargo (2000).

While continuing to work on methods for automating the creation of digital lectures, we ran a four-month trial of the student's interface to the digital lectures in a large first year Computer Studies course. During this trial we used a wizard-of-oz technique (Gould, Conti, Hovanyecz, 1983) in which human input simulates the automatic capture of the digital lecture. The student's interface, however, was unaffected by the wizard-of-oz data capture. During the lecture, a cameraperson operated the camera. The feasibility of automated camera management is demonstrated by Liu, Rui, Gupta and Cadiz (2000), who show that most people could not tell whether lecture room video was captured by a person or by an automatic system. After the lecture, an operator ran a series of programs that captured and burnt the lecture contents onto a CD-ROM. The programs translated the digital video (DV) tape into a motion-JPEG stream with a synchronised MP3 audio stream. Finally, the operator watched the video and created hypertext links-identifying topics in the lecture-synchronised with the video stream. In doing so, the operator looked for points where the lecturer moved from one slide to the next, and typed the banner text of the slide. Again, the feasibility of automatic techniques is demonstrated by Mukhopadhyay, and Smith (1999) who describes techniques for automatically identifying slide changes and by Wellner (1993a, 1993b) who discusses techniques for retrieving text from video images.

The CD-ROMs (one for each lecture throughout the course) were available for students in the university library and in the laboratories where the students attended weekly lab classes. Using CD-ROMs as a distribution medium allowed us to closely monitor use. We required students to complete a questionnaire each time they used a CD-ROM.

4. Evaluation and Results
In the second semester of the 2000 academic year (July to October), digital versions of each lecture in a large (746 students) first year Computer Studies course were made available to students. At the start of the study, students who wanted to use the digital lectures were required to sign an ethical consent form. Ninety-four students requested to do so.

Initial use of the system was extremely light, with only four students attempting to use the system throughout July and August. At the same time lecture attendance appeared to be lower than during the previous term.

Throughout September and early October only two people accessed the digital lectures. As a result, the course lecturer sent an email message to the class asking whether it was worthwhile to continue to make digital lectures available. Much to our surprise (given the extremely low use of the system until then), fifty-three students replied, strongly urging us to continue producing the lectures. Comments from the students in their email replies indicate that many had been intending to use the digital medium for some time, but had not gotten around to doing so:

> Do you intend to view the Digital Lectures before the COSC110 exam?

Yes I do intend to use the lecture disks but mainly for the lectures I've missed or need to brush up on (thats probably all of them if I was completely honest)

> When are you likely to want to view them?

I intend doing this over the next few weeks when time permits (I work full time)

> Have you tried to use the Digital Lectures already?

No I haven't used them at all yet, so I'm one of the guilty that signed up and not used them. Despite the appeals for us to continue, very few students subsequently accessed the CD-ROMS.

4.1 Discussion
In the light of the strong urging from students to keep the system operational, and the fact that most of them never used the system, it appears that many of the students deceived themselves about their intentions to catch up on lectures using the digital alternative. We hypothesise that by making lectures available at any time, we removed the need for students to attend the live lecture. Normally, the live lecture is a one-time event, and if students miss it, they cannot benefit from it. Most disconcerting to us was that the students never fulfilled their intention to access the digital media. Gerhard Fischer (private communication) suggested a "local traveller" analogy to explain this effect. It is not uncommon for residents in an area to have never visited some of the local attractions: they intend to, but the attractions are constantly available, and there is no pressing need to visit them immediately. Tourists visiting the area, however, typically have only one opportunity to see the sights, and consequently make concerted efforts to ensure that they exploit it. Because lectures were freely available to the students, they were able to postpone viewing the lectures because ''they'll still be available tomorrow." As it often happens with local residents, these intentions are sometimes postponed until it is too late to take advantage of the opportunity.

However this is not the only possible explanation for the phenomenon. Among other logistical issues, students noted other factors, such as poor video quality and a busy lab, which made access difficult.

5. Recommendations 
The development and use of a prototype Digital Lectures system produced poor results at the University of Canterbury. Based on developer experience and student survey comments the following recommendations are made for future development of similar systems:

• While restricted bandwidth remains an issue in delivering flexible learning systems, the use of streamed audio plus images and text may be a better alternative to video.
• The use of Web delivery rather than CD is recommended where possible to increase accessibility.
• Consideration of student motivation is crucial if any learning benefits are to be garnered. Motivational aspects may include feedback systems, mandatory usage for quizzes, evidence provided to students that usage will yield better understanding and higher grades.
• Good training should be provided for both users of the system and for support staff who may need to assist students.
• The system should be provided consistently from the beginning of the course.

6. Conclusions
Advances in desktop multimedia and the World Wide Web have brought about a wide range of new possibilities for education. Many researchers are exploring the possibilities of the technology while others are deploying first generation systems. (For example, see Stanford Online stanford-online.stanford.edu.)

This paper investigated some of the possibilities and problems of virtual learning systems. In particular, we reported the somewhat surprising results of a four-month trial of the Canterbury Digital Lectures Project. In essence, the results of our trial suggest that providing too much flexibility to students at a residential university can have negative learning results. By providing a digital surrogate for live lectures, it appears that students made less effort to attend the lectures, and that they never fulfilled their intention to catch up using the digital media.

In retrospect, it is unsurprising that misplaced technological support can have negative effects on learning (or at least on class participation). In our further work we intend to address the questions that identify the conditions under which learning technology is well placed. For instance, we would like to know whether our technology would have had a positive effect if we had used the web to bring the lectures to students who could not otherwise have participated in classes.

An external link to Stanford Online

7. References
Abowd, G. 2000. Classroom 2000: An experiment with the instrumentation of a living educational environment, IBM Systems Journal 38(4), 508-530.

Ackerman, M. and Malone, T. 1990. Answer garden: A tool for growing organizational memory, in F. Lochovsky and R. Allen, eds., Proceedings of the Conference on Office Information Systems. April 25-27 1990. Cambridge Mass., pp.31-39.

Adjeroh, D., Bell, T., Cockburn, A., McKenzie, B. and Vargo, J. 2000. Technological challenges in automating capture of digital lectures. Working paper in preparation.

Appleton, A., Dekkers, J., and Sharma, R., 1989. Improved teaching excellence by using Tutored Video Instruction: An Australian Case Study, Proceedings of the 11^th EAIR Forum, Trier, Germany.

Brown, M. and Sedgewick, R. 1985. Techniques for algorithm animation, IEEE Computer 2(1), 28-39.

Cadiz, J., Balachandran, A., Sanocki, E., Gupta, A., Grudin, J. and Jancke, G. In Press. Distance learning through distributed collaborative video viewing, in Proceedings of CSCW2000 ACM Conference on Computer Supported Cooperative Work, December 2-6. Philadelphia, Pennsylvania.

Cockburn, A. and Greenberg, S. 1998. The Design and Evolution of TurboTurtle, a Collaborative Microworld for Exploring Newtonian Physics, International Journal of Human-Computer Studies 48(6), 777-801.

Conklin, J. and Begeman, M. 1988. gIBIS: A hypertext tool for exploratory policy discussion, ACM Transactions on Office Information Systems 6(4), 303-331.

Fischer, G. 1998. Making learning a part of life-beyond the "gift-wrapping" approach of technology, in P. Alheit and E. Kammler, eds, Lifelong Learning and Its Impact on Social and Regional Development, Donat Verlag, pp. 435-462.

Fischer, G. 2000. Symmetry of ignorance, social creativity, and meta-design, Knowledge-Based Systems. In press.

Gibbons, J., Kincheloe, W. and Down, K. 1977. Tutored videotape instruction: A new use of electronic media in education, Science 195, 1139-1146.

Gorman, R.A., 1998. Intellectual property: The rights of faculty as creators and users, Academe, Washington; May/Jun 1998; Vol. 84, Issue 3; 14-19.

Gould, J., Conti, J. and Hovanyecz, T. 1983. Composing letters with a simulated listening typewriter, Communications of the ACM 26(4), 295-308.

Liu, Q., Rui, Y., Gupta, A. and Cadiz, J. 2000. Automating camera management for lecture room environments, Technical Report 00-90, Microsoft Research, Redmond. http://www.research.microsoft.com/research/coet/Camera/TRs/00-90.pdf

Mukhopadhyay, S. and Smith, B. 1999. Passive capture and structuring of lectures, in Proceedings of the seventh ACM Multimedia Conference October 30 - November 5. Orlando, Florida, pp. 477-487.

Murray, L., and Efendioglu, A., 1999. Distance education: Delivery systems and student perceptions for business education in China, Proceedings of the Decision Sciences Institute Annual Meeting, 281-3.

Repenning, A., Ioannidou, A. and Phillips, J. 1999. Collaborative use and design of interactive simulations, in ACM Conference on Computer Supported Cooperative Learning (CSCL '99). Stanford.

Sleeman, D. and Brown, J., eds. 1982. Intelligent Tutoring Systems, Academic Press.

Stein, S., 2001. The media production model, EDUCAUSE Review, Jan/Feb 2001, Vol. 36, No. 1, pg. 26. Uchihashi, S., Foote, J., Girgensohn, A. and Boreczky, J. 1999. Video Manga: Generating semantically meaningful video summaries, in Proceedings of the seventh ACM Multimedia Conference October 30 - November 5. Orlando, Florida, pp. 383-392.

Wellner, P. 1993a. Adaptive thresholding for the DigitalDesk, Technical Report EPC-93-110, Rank Xerox EuroPARC.

Wellner, P. 1993b. Self calibration for the DigitalDesk, Technical Report EPC-93-109, Rank Xerox EuroPARC.

********** End of Document **********

IMEJ multimedia team member assigned to this paper	Yue-Ling Wong