Digital Video in Education

Please Cite: Smith, T. L., Ruocco, A., and Jansen, B.J. 1999. Digital video in education, ACM Computer Science Education Conference, New Orleans, LA., 122-126.

See Other Publications

1. ABSTRACT

Digital Video is an exciting new medium with the potential to revolutionize the way organizations train their employees. However, there are questions that must be answered. How practical is video? What is the demand? What is the best use of video? In this paper, we compare the quality and performance of video over a 100 Mbps Switched Ethernet Network using a RealVideo Server, the Oracle Video Server, and a file server. We compare the performance and quality of the RealVideo Stream with MPEG-1 distributed from a File Server using various client and server combinations. We also analyze 50,000 queries from an Excite database to determine the current demand for video. Finally, we present the results of a study that explores the value of digital video in an educational environment.

1.1 Keywords

Introduction

The United States Army has approximately 475,000 soldiers deployed across the United States, Europe, Asia, Central America, and South America. Regardless of their location, soldiers require continual technical and professional training. In fact, one of the primary missions of a peacetime army is to train.

This includes refresher training, new equipment training, and training associated with new personnel, units, environments, or promotions.

In the past, most training has been conducted at the soldier’s duty station. Officers and noncommissioned officer plan and schedule training, with experienced local soldiers providing the instruction. However, expertise does not always exist locally. Either instructors from a central school must travel as a temporary duty (TDY) to the post, or soldiers from a post must travel to a central school. This centralized instruction is very effective, but it is also very expensive.

In 1995, the Army’s Training and Doctrine Command, (TRADOC) initiated a plan to reduce training related travel expenses. TRADOC hopes to leverage technology to provide quality, centralized instruction to soldiers located all over the world. Their plan has two components. The first is Classroom XXI, where soldiers at a training post will be able to access digital materials. The second is the Distance Learning Program, where soldiers not located at training centers will access multimedia-training materials from a digital library through a distributed database.

TRADOC requested the Department of Electrical Engineering and Computer Science at the United States Military Academy (USMA), West Point, New York review current technology and assist with the planning and design for Classroom XXI. This initiative will result in the creation and fielding of over 500 classrooms during the next few years. Wolfe, et al [14] provide an overview of Classroom XXI and Distance Learning.

Central to establishing the TRADOC program is understanding the resource requirements and role associated with digital video. However, these issues have not been fully studied. The major limitation associated with digital video is the resource requirement, i.e., bandwidth and, to a lesser extent, hardware. The role of video has two components, demand and effectiveness. All these issues are intertwined. Demand is heavily influenced by effectiveness, and effectiveness is a dependent on resources. In order to study the resources required we used a CS2 level classroom with switched 100 Mbps Ethernet and workstations that varied from 100 MHz Pentium to 300 MHz PII. Second, we used an Excite® database with 851 770 queries as a metric for current demand. Finally, the Engineering Psychology Department at USMA assisted in a study to determine the effectiveness of digital video using eighty first-year students.

2. Related Work

Research is continuing on the most effective utilization of networks for Digital Libraries [11]. Cox [3] provides a thorough review of current technologies including compression, organization, storage, and retrieval of multimedia signals, searching, and browsing.

Ma, et al [9] describe a video-based hypermedia system for "education-on-demand" using either a networked video server or a local file server. Other current Digital Library programs are using a mixture of removable disks and CD-ROMs to distribute multimedia files to students [5], [6]. The 1.44 MB size restrictions of a floppy disk make it impractical for anything but text files or application files to be distributed. The same is also true of the larger Zip, LS-120, Jazz, and related storage mediums. Likewise, CDs present two challenges to instructional developers. First is the 650-MB capacity limit of the compact disk. With a typical MPEG-1 file averaging 12-18 MB per minute of video, this storage limitation is quickly reached. Second, because of the "write-once, read-many" nature of compact disks, changes to any file on the disk, no matter how slight, require producing a new master with all of its accompanying charges, effectively erasing most of the savings realized through the use of CDs in the first place. DVDs have a much larger capacity (typically 4.7 GB), but copyright considerations have limited availability of writable drives.

Brofferio [2] provides a detailed description of a video-capable, distance education classroom. He had success utilizing video, but notes that the "…major cost of a distance learning system is the telecommunication network…" Davis and Smith [4] detail their experience with audiographics, a "combination of audio conferencing with still frame graphics." Their system operates over ordinary phone lines.

With regard to the impact of video, there have been several studies that suggest that differences in media type do not significantly effect learning [12]. McCleary and Egan [10] found no significant difference between student groups on campus and those using two-way interactive television. Barry and Runyan [1] surveyed distance learning courses in the Army, and found that student achievement was comparable between distant and resident groups.

3. How practical is video?

As always, there were several competing requirements that limited the potential designs of the classroom network. Since portable computers are becoming ubiquitous in Army units, TRADOC wanted us to determine their utility in a classroom environment. This precluded the use of ATM. Funding and personnel were also limiting factors. There exists insufficient funding to hire network administrators for the design and maintenance of the classroom. Instructors installed the network and are responsible for its maintenance in the prototype classroom. Use of personnel in a dual role, as instructor and network maintainer is a constraint that will likely be imposed in an operational setting as the Army implements additional classrooms.

Figure 1 shows the network infrastructure. Each client workstation in the classroom has a switched Ethernet 10/100Base-T connection. All of the machines are connected to a CISCO Catalyst 2924 Ethernet Switch. This switch is then connected to a CISCO Catalyst 2908 Ethernet Switch via Full-Duplex 100BaseT. The Catalyst 2908 is used to support the servers where the web, training database, and video content of the training courses are maintained. We were able to use a T1 line provided through a grant from MCI for Internet access, and the Catalyst 2908 provides an uplink to the T1. This connection is used for distance learning via the Internet. The servers included an nCube M30 Video Server with two processors capable of streaming up to 270 MPEG-1 or MPEG-2 streams to Oracle Video Clients. The file servers consisted of Dual Pentium 200 MHz servers with 160 MB of RAM, a Dual Pentium 90 MHz with 48 RAM, and a Pentium 200 MHz with 32 MB of RAM. These servers provided web, database and video content to the students along with network administration.

Figure 1: Classroom Configuration

We used resource requirements as our measure of practicality. Two measures of resource intensity are CPU utilization and network utilization. Several types of client machines were used. They had 10/100BaseT 3COM Fast Etherlink Network Interface Cards. None of the client machines had decoding hardware, and all produced satisfactory results when playing AVI, MPEG, and RealVideo files. The types of client machines and CPU utilization per file type are shown in Table 1.

Network, that is bandwidth, utilization was compared using the various file formats. Neither the RealVideo streams nor playing a RealVideo file from a file server significantly affected bandwidth availability. The maximum utilization on a 100 Mbps segment using the RealVideo files was 4%. Wolfe and Smith [14] give a detailed account of using RealVideo® over a network.

In addition to network utilization, we watched for degradation in video quality when playing multiple MPEG and AVI files from a file server. We recorded 62% utilization with 19 simultaneous AVI streams. Although, we did not record any collisions or errors, there was a noticeable decrease in video quality. Audio quality remained consistent. We recorded 47% utilization when playing 22 simultaneous MPEG streams. There was no perceptible degradation in either video or audio quality with 22 files. Figures 2 and 3 show the percent utilization Vs. number of files for MPEG and AVI files, respectively, when executed from a file server.

Table 1: CPU Usage from NT Performance Monitor

Additionally, we placed a file server on a 10 Mbps segment and recorded the number of MPEG files Vs. Utilization. We were only able to maintain four simultaneous MPEG videos without a loss of quality.

Figure 2: AVI Files Vs. Bandwidth Utilization

Finally, we installed the nCube M30 Video Server. We needed a minimum 233 MHz PII class machine to view the video and recorded 100% utilization on this processor. Realistically, a 266 MHz PII class machine is required. The Video Server utilizes UDP rather than TCP to deliver the video. The network supported simultaneous streams to each workstation in the room (twenty-two). nCube literature states that this configuration will support at least thirty simultaneous streams. Various configurations can be established to support an increasing number of simultaneous streams. However, there was more overhead involved in setting up, installing, and managing the nCube server.

4. What is the demand for video?

There have been many articles concerning providing multimedia information to users. However, we could find no published studies that analyzed the actual demand for multimedia files from real users. Jansen, Spink, and Saracevic [7] are conducting an on-going analysis of real Web users from a major Internet search engine (Excite®), using real queries, with a real information need. We have pulled the data concerning multimedia files from the transaction log and present them here. This data provides an excellent indication of the current demand for multimedia from a broad cross section of the Web population. An understanding of the user population is a key element in system design.

Table 2: Number of Users, Queries, and Terms.

Table 2 contains overall statistics for the data set. The first column lists the three categories and the second column lists the number within that category. So, the first row of the table indicates that there were 174,866 users. Each user was anonymous; however, each user could be uniquely identified. There were 851,770 queries with a total of 1,797,319 terms.

Table 3: Video Information.

In order to determine the video specific queries, we developed a list of terms relating to video and searched the data set. When a query contained one of these terms, we counted that query as a search for video. The results are presented in Table 3. The first column is the specific term searched for. The second column is the number of queries that contained that term. The third column is the percentage that number represents of all queries. If a query contained more than one video term, it was only counted once. While this list of terms may not be all inclusive of every term related to video, we have reviewed a portion of the data set and are satisfied we have counted the vast majority of video queries.

The single most common search term was "video", its 6,897 queries represent 0.81% of the total queries in the data sample. However, if one combines the queries containing all the various forms of mpeg (e.g., mpeg, *.mpeg, *.mpg, mpg, and mjpeg), both correct and incorrect, there were 10,104 queries with some form of mpeg representing 1.19% of all queries. As a single video format, this is a surprising high percentage and may just that users are most comfortable or most familiar with the mpeg format

There were a total of 21,469 searches for video, which represents 2.52% of all queries. Although this is a small percentage of the total number of queries, it represents one of the largest categories of queries from the data set. In a pilot study, Jansen, Spink, and Sarecervic [8] found that the largest category of Web queries from a 51,000 query data set was only 2.4% of all queries. Video was not included as a category in that study.

5. What is the best use of video?

In an attempt to gain an understanding of the suitability of instruction based on digital video, we conducted a study with USMA's Engineering Psychology Department. Seventy-five freshmen students, male and female, ranging in age from 18-21, enrolled in an introductory psychology class and receiving extra credit for volunteering for our research, participated in the experiment. Our goal was to evaluate the student's ability to understand unfamiliar technical concepts through various media. We anticipated that access to an instructor would produce a higher degree of understanding given the complexity of the domain and various means of delivery. The delivery means included:

• Condition 1: An actual class taught by an instructor using a Power Point slide show. Participants were encouraged to ask questions of the instructor. This is currently the mode used in the centralized schools.
• Condition 2: An MPEG video of the instructor teaching the class with Power Point slides. Interaction was not allowed. The video alternated between slides and the instructor.
• Condition 3: An MPEG video showing only the Power Point Slides with the instructor's audio. Interaction was not allowed
• Condition 4: Power Point slides with student collaboration (Self paced).
• Condition 5: Power Point slides with no student collaboration (Self paced).
• Condition 6: An MPEG video of the speaker without slides (Talking Head video). Interaction was not allowed.

Following the instruction, proctors gave the participants a sheet with five analytical reasoning questions from a Law School Aptitude Test pre-test book to reduce the effects of short-term recall. The proctors then administered an exam of twenty questions based on the digital library material. A committee then graded the exams and compiled the statistics. The results are depicted in Table 4.

Table 4: Results of Digital Video Trials

Though we predicted that interaction with the instructor and collaboration would produce higher scores, there was no significant difference. This is consistent with other studies [12] that suggest no significant difference in the means of delivery. Two factors that may have affected the outcome of the statistical analysis were the high variance in the results of all conditions and the small sample size of each condition group. A follow-on study is planned that will increase the sample size and restrict the domain to a pass/fail task such as programming a mobile communication device. This domain should more closely simulate our planned environment and will reduce the subjective evaluation elements.

6. Conclusion

We were tasked by TRADOC to explore leveraging technology to offset travel costs incurred in training. The underlying functional requirement is to be able to conduct training anytime from any location. Due to the administrative and coordination overhead involved in establishing teleconference sessions, we discounted teleconferencing as the most viable solution. Instead, we focused on digital libraries and digital video. Experiments were conducted to determine the practicality of video, the demand for video, and the best way to incorporate video into courseware. These three areas are intertwined to such an extent that there is no single, simple solution.

RealVideo® required the least resources in terms of CPU utilization and network utilization. However, the quality of the video made it unsuitable for detailed training videos. MPEG-1 video washad suitable quality. It taxed CPUs at 166 MHz and below. CPUs at 233 MHz handled MPEG-1 but were taxed on a proprietary format available for the experiment. With the ever-increasing speed of CPUs, it is unlikely the 233+ MHz CPU speed will be a cost-prohibitive factor.

Video is network intensive. It required a 100 Mbps network segment. Our test environment of 22 simultaneous stream was adequately supported with MPEG-1 format files from a file server. The network adequately supported AVI format streams, but the quality degenerated at 19 streams. The network adequately supported the 22 simultaneous streams from the Oracle® video server. From our experiences, it appears that having the 100 Mbps network is a minimum requirement. In this regard, the costs incurred in establishing such a network should be regarded as a sunken cost. If such costs are prohibitive, then using high-quality video may be moot. There is an open debate on the viability of using the proprietary video server in lieu of a general-purpose file server. The file server was adequate for the 22 simultaneous streams, but the dedicated video server is more reliable and more scaleable. However, it also has an appreciable overhead in costs and required expertise.

Currently, queries do not indicate a great demand for digital video training. The indicator from the commercial world is that video is a frequent request, perhaps even the single largest requested item. However there is no correlation factor between the commercial-based requests and educational-based requests. In time, it is anticipated that video demand in education will increase; though this may be based more on human factors than pedagogical factors.

Many experiments have been conducted on the best way to deliver education. No categorically best method has been developed. So, while use of video may enhance delivery of specific material, an unchecked effort to redesign curriculum is not warranted.

In general, current and readily available technology makes the use of digital video in education practical though there is no inherent demand for it. The use of video should be based on its effectiveness in a particular problem domain.

7. Bibliography

[1] Barry, M. & Runyan, G. A Review of Distance-Learning Studies in the U. S. Military. The American Journal of Distance Education, 1995, Vol. 9 No. 3, P 37-47.

[2] Brofferio, S. C., A University Distance Lesson System: Experiments, Services, and Future Developments, IEEE Transactions on Education, February 1998, Vol. 41, Number 1, P 17-24.

[3] Cox, Richard V., Haskell, Barry, G., Lecun, Yann, Shahraray, Behzad, Rabiner, Lawrence, Scanning the Technology, On the Applications of Multimedia Processing to Communications, Proceedings of the IEEE, Vol. 86, No. 5, May 1998, P 755 - 820.

[4] Davis, James L., and Thomas W. Smith, Comptuer-Assisted Distance Learning, Part I: Audiographic Teleconferencing, Interactive Satellite Broadcasts, and Technical Japanese Instruction from the University of Wisconsin-Madison, IEEE Transactions on Education, VOL. 37, NO. 2, May, 1994, P. 228- 233.

[5] Fox, Edward. Digital Libraries, WWW, and Educational Technology: Lessons Learned. Proceedings of the World Conference on Educational Multimedia and Hypermedia 1996. P. 246 - 251.

[6] Harris, J.A.; Murden, C.; Webster, L.L. The Potential of Interactive Multimedia on CD-ROM to Enhance Laboratory Work in Physical Science and Engineering. 1994 IEEE First International Conference on Multi-Media Engineering Education Proceedings. P. 296-301.

[7] Jansen, Major Bernard J., Amanda Spink, Tefko Saracevic. (1998) Real life information retrieval: A Study of User Queries on the Web. SIGIR Forum, vol. 32, no. 1, P. 5-17

[8] Jansen, Major Bernard J., Amanda Spink, Tefko Saracevic. (1998) Real life, Real User, and Real Needs: An Analysis of 850,000 Web Queries. Unpublished Paper.

[9] Ma, Wei-hsiu, Yen-Jen Lee, David H.C. Du, Mark P. McCahill, Video-based Hypermedia for Educatin-On-Demand, Proceedings of ACM Multimedia 96, Boston MA, P. 449-451.

[10] McCleary, I. D. & Egan, M. W. Program Design and Evaluation: Two-way Interactive Television. Video-based Telecommunication in Distance Education, Pennsylvania State University, Readings in Distance Education, 1995, Number 4.

[11] Palounek, Andrea P. T.,0 Distributed Computing Network for Science and Math Education in Rural New Mexico. Proceedings of the World Conference on Educational Multimedia and Hypermedia 1996. P 557-562.

[12] Russell, Thomas L., The "No Significant Difference" Phenomenon, 4^th Edition, http://www2.ncsu.edu/ncsu/cont_ed/out_ex/oit/nsdsplit.htm, (January 1997), accessed 18 August 1998.

[13] Wei-hsiu Ma, Yen-Jen Lee, David H.C. Du, Video-based Hypermedia for Education-on-Demand, ACM Multimedia, November, 1996, P 449-450.

[14] Wolfe, Doug, Todd L. Smith, and Joe Adams, Active Learning through Streamed Video, Proceedings of IASTED International Conference on Computers and Advanced Technology in Education, 1998, P 51-54.