Experiences from Video Lectures in Software Engineering Education

Published Online May 2017 in MECS

Even though software engineering and programming has been taught for several decades, there are some constants in the computer science education: learning computer science is rather difficult, and the motivational aspects need to be addressed to get the best learning results (for example [1]).

During the last decade, different forms of community support in programming has become increasingly accessible to all different users. For example, programming manuals from libraries are losing popularity, because the students can find answers to virtually any software problem from the online services such as Stack Overflow [2]. In addition to textual answers to a specific issue, these repositories of knowledge can also include complete programming manuals with exercises, even including tutorials and entire video lectures covering various programming languages, techniques, and platforms. The depth and technical level of the works also vary to support student learning in various school levels from an elementary school to the graduate level at an university. The availability of video learning material varies, with some being available only to students participating in a course, and some are available to a global audience in services like YouTube.

Most of the articles regarding the video usage in MOOCs and student attention span suggest that the educational video length should be around six minutes [3]. However, in the case of flipped classroom, the length recommendation is longer, usually around 15-30 minutes [4,5]. Both the MOOC and the flipped classroom are using similar video techniques. This raises the question regarding different recommendations for the video lengths, and as a secondary concern, the issue of how these video lectures perform when compared against traditional classroom lecturing methods? If the course infrastructure in the flipped classroom allows the students to focus more on the material and the topic, would it be a preferable to MOOCs? If the video lectures, in general, perform better than their traditional counterparts, the benefits of aspects such as increased accessibility would make this a significant advancement in teaching infrastructure. These issues lead us to define four main research questions, which are listed as follows.

• 1)    How does the video usage differ between an MOOC and flipped classroom?

• 2)    What are the differences of lecture videos and tutorial videos in the scope of video usage?

• 3)    What general requirements can be identified for the video-based course infrastructures in the development of effective long-distance learning solutions?

• 4)    What are the benefits of widely available teaching videos for teachers and organizations?

• II.    Related Research

In the context of education and especially video-based instructions, students are given a chance to learn in an environment of their choosing. Some researchers in the field of psychology have been developing and studying the effects of the learning environment to the quality of learning, based on the theory of situated learning [6]. Situated learning refers to an idea that learning is unique to the situation and environment where it is learned. In a study by Godden and Baddeley [7] divers were made to learn lists of words either on land or underwater. The study found that what was learned underwater was more effectively recalled underwater and vice versa. The theory of situated learning has been argued against: for instance, Fernandez and Glenberg [8] did a set of experiments, which proved that the environment did not affect the level of learning. They do point out that the situation and environment can affect learning, but some standard learning methods may be ineffective. However, transferring learning materials via videos into the environment does not automatically mean success [9]. The study by Young [10] determines four perspectives of situated learning that should be considered for sufficient transfer of knowledge: 1) selection of situations, 2) provision of an environment with realistic context, 3) teacher’s role as a coach or mentor, and 4) the nature of assessment. These perspectives are often tied to the method of flipped classroom [11,12].

The application of video lectures in different learning platforms is discussed for example by Hansch [13] and is also a component of blended learning [14, 15]. Their study discusses the generation of learning videos in a MOOC context and summarizes that the video material tends to dominate the course content, is expensive to produce, and does not offer any automatically added value. In addition to these concerns, the paper also identifies the lack of media and content production expertise a major issue in the development of learning video material. Based on their interviews with the domain experts, a professionally developed video-based course can be as expensive as 200 000 USD per production [13]. However, the study amends that the impact of production values over the learning outcomes has not been sufficiently studied, and currently it seems that the most important correlation is between the content and the audience. Buckland [14] reported a study, where the university released their lecture videos through YouTube and iTunes free for everyone. While he reached the teaching goals he was aiming with public video sharing, there were also unexpected results, such as external feedback from students, teachers and other interested parties around the globe. Students also use other Youtube videos besides the ones recorded by the lecturer to study the topic further [15].

Video podcasts as a learning tool are also studied by Kay [16], who conducted a literature review on the applications and research work conducted on the topic, covering 53 peer-reviewed papers discussing the video lectures. In most studies, the videos are applied in the undergraduate level courses, with the mean student population of 316 participants. The review identified five main benefits of the video lectures across all of the identified studies:

• •    Learning assistant : the videos had an impact on the understanding, and helping note-taking and class preparation level of the students.

• •    Control tool : students can select the time and pace of learning.

• •    Attitude management : The video lectures seem to motivate and stimulate the student participation.

• •    Behavior benefits : The video lectures offer more tools to assess student behavior via data logging, and allow more independence for the students to participate in the course.

• •    Learning performance : The video lectures seem to enhance the course outcomes when compared to the similar courses without lecture video components.

These five aspects have also been recognized in the study by Manley and Urness [17].

Concerning video lectures and their length, in 2007 Cann [18] suggested that videos meant for computer screens should be under ten minutes and for mobiles under five minutes. The limit of five or ten minutes has been somewhat realized in the MOOCs with the recommended video length for them is around six minutes [3]. However, in flipped classroom, where videos are usually applied the in-class setting, this recommendation is around 15-30 minutes [4,5]. In these different cases, the videos as a tool are treated differently, while they are usually applied in a similar manner to fulfill similar roles. In the article by Chan [15], the survey replies from students showed, that short video length was the least appreciated aspect of a good lecture video.

On the effect of the learning experience, a study by Maniar et al. [19] studied the application of video-based learning in different technical platforms. Their study identified several recommendations, such as that the video-based teaching is most effective learning medium when the topic is divided into several short, focused videos, ranging from thirty seconds with the basic concepts, up to ten minutes for the more complex theories. Maniar et al. also suggest, that the most problematic areas of video-based learning are the maintenance of student attention span, and the ability to provide meaningful demonstrations. Still, the methods have been combined; Bruff [20] introduced a technique known as wrapping an MOOC, which utilizes MOOC in a flipped classroom context and the study by Caviglia-Harris [21] showed improvements in student grades. These studies do not consider the use and engagement of videos and whether there was a difference when MOOC videos were utilized in a classroom. To our knowledge, there has not been a comprehensive study, where the length and engagement of flipped classroom videos are scrutinized.

• III.    Research Process

To assess the usability of video recordings and the video-based learning approaches our team collected feedback and metrics from ten different bachelor’s and master’s level courses from software engineering programs. The collected feedback included both the qualitative open feedback and the results of the end surveys conducted with the students, and the course metrics collected from the participation data. The video usage data and metrics were recorded from the applied video viewing platforms, collecting information on the average session lengths, session count, applied platforms and geographical location of the traffic sources.

The course surveys were sent for the 378 students who actively participated in the courses. From this population, 144 submissions were collected, giving the response rate of 38 percent. The survey instruments are available at

The majority of the data was quantitative, which was collected from course feedback surveys, and from online activity logs and video metrics. This data was analyzed as a comparative case study to assess the strengths and development areas of the individual courses, and to understand how the students used the provided online tools and learning materials. The observations collected from this material were then combined the survey results and further analyzed against the video distribution platform usage metrics. Consistent and most notable results between the different data sources were used as a basis for this study.

• A.    Selection of video service

We identified three different ways to provide learning materials to the students. The method A is to use own infrastructure to share videos, which gives full control over the videos and infrastructure over them, but it might also generate problems: how accessible are the videos with mobile devices or what kind of statistics is available – if any. Additionally, all maintenance costs and issues have to be dealt with locally.

The second method is to use some global video service provider, such as YouTube or Vimeo. This method is probably the fastest way to publish learning materials to the students, and it gives numerous statistical analysis tools. Also, there is no need to worry about the mobile accessibility or broadband capacity. The downside is that one needs trust third party service provider and the viewing tools are not optimized for learning and can include advertisements. Also, timed release of videos might not be possible.

The third method is to use video providing service that is specially designed for long-distance learning. These services can be integrated into virtual learning environments, such as Moodle, and students can make notes for themselves inside the service while they watch videos. The videos can be scheduled to be released, for example, on every Monday at noon and different students can access different videos – if this is required. The downside is that these services usually require authentication, the configuration takes time, and services might not be free of charge.

• B.    Description of the courses

  The course videos were produced between the academic years of 2012-2013 and 2015-2016 with

differing platforms and recording systems. During this time, ten sets of lecture videos were recorded to allow video-based learning and long-distance participation. The course videos were hosted on YouTube ( since at the start of the project no hosting services was locally available. One exception is the dataset J, where a separate closed system (Office Mix, was used to get contrasting data for comparison. Both services are web-based and can serve a wide variety of client devices from desktop to mobile devices. Summary of the courses included in this study is presented in the Table 1.

All of the courses were for the university's degree programs in computer science from the Lappeenranta University of Technology. All of the courses were programming-related, although in the datasets A, B, G, H and J the programming tasks were not the main focus of the course. Datasets A and B focused on the software engineering principles and process models, G and H on the software testing work, test process models and quality assurance, and set J on the databases.

The lectures of datasets A and B were recorded from the traditional lectures from fall 2013 and 2014. The dataset B was also supplemented with a library of 39 shorter tutorial videos focusing on one topic in videos with varying lengths between five and fifteen minutes. This also enabled the dataset B to turn the latter parts of the course into demonstration lectures, where instead of discussing software engineering topics, the students were given examples and cases, which they were required to solve with the appropriate tools, such as UML diagrams or by defining use case scenarios.

Datasets C, D, E are collected from an objected-oriented programming course arranged in three consecutive years. The first iteration in fall 2013 used C++ with a traditional lecture-exercises model. After this, the whole course was completely rebuilt to include modern techniques for both teaching, and applied programming tools [22]. For example, the changes included switching to the Java programming language and applying the flipped classroom teaching paradigm. In 2013, 14 lecture videos were uploaded to YouTube, with the length varying from 45 minutes to 90 minutes depending on the magnitude of the topic. In 2014 the course was flipped completely; it no longer had any physical lectures, but instead the teacher provided 31 shorter tutorial videos on YouTube. In 2015 the video material was the same, except for one video which was added to cover one additional topic. This format was also adapted for dataset I, where the design patterns were introduced with short tutorial videos.

Dataset F followed the example of datasets D and E as the course was rebuild to include most relevant of the modern web technologies, such as HTML5 and responsive design. Also, in this case, the flipped classroom was chosen as the teaching method and the teacher provided short tutorial videos on YouTube. A similar approach was used with the datasets G and H, where the course discussed software testing methods and applied tools.

The course in dataset J is similar to sets D, E, and F in the sense that the course was modernized and the course structure was redesigned to use the flipped classroom method. At the same time lectures were directly produced as videos with occasional interactive content directly into the Office Mix service, where the lecture material was only available to the course participants.

• C.    Course statistics

The recorded videos are divided into three categories: lectures, tutorials, and summaries. The term lectures is used about videos that are directed towards theory-based teaching. This usually means that there are no extensive examples done in the video, such as a large programming demonstrations or equivalent, but they focus more on the abstract concepts, such as program design. The tutorials are a method to show a process or a certain solution that is extensively explained while creating it on screen. This category consists of, for example, programming tutorials or process descriptions. The summaries are retrospectives, which introduce and discuss for example the weekly exercises, course project works or the approaches, which would be useful in their solutions, and replace the weekly exercise sessions for those who are unable to attend the face-to-face events. A summary of videos provided for each course is presented in the Table 2.

There were some differences on the application of the video material archives. The datasets A, B, C, and G were created to supplement the traditional course infrastructure with the possibility for long-distance participation. The other courses used a fully flipped classroom approach, with only the first introductory lessons given as traditional face-to-face lectures. On all of the recorded courses, there were only one, or at maximum a handful of, mandatory face-to-face events such as the introductory lesson, project presentations, exercise task demonstrations or requirement to complete the final exam on a separate, local event.

Table 1. Summary of the Courses.

ID	Dataset/Course	Year the course was arranged	Number of students (started*/finished)	Course teaching language	Main aim of the course
A	Software Engineering Methods	2013	37/27 (73 % pass rate)	English	To learn fundamentals of software process models, software engineering principles, and software design tools.
B	Software Engineering Methods	2014	45/37 (82 % pass rate)	English	To learn fundamentals of software process models, software engineering principles, and software design tools.
C	Object-oriented programming	2013	38/26 (68 % pass rate)	Finnish	To learn object-oriented programming methods (with C++)
D	Object-oriented programming	2014	42/26 (62 % pass rate)	Finnish	To learn object-oriented programming methods and basics of graphical user interface (with Java)
E	Object-oriented programming	2015	61/44 (72 % pass rate)	Finnish	To learn object-oriented programming methods and basics of graphical user interface (with Java)
F	Webbed applications	2015	59/40 (68% pass rate)	Finnish	To understand different parts of WWW infrastructure and learn to develop software in a web environment.
G	Fundamentals of Software Testing	2015	22/17 (77% pass rate)	Finnish	To learn fundamentals of different testing methods, application of different testing tools and generally the software test process.
H	Fundamentals of Software Testing	2016	32/25 (78% pass rate)	Finnish	To learn fundamentals of different testing methods, application of different testing tools and generally the software test process.
I	Object-oriented programming techniques	2016	54/40 (74% pass rate)	English	To learn advanced object-oriented programming using design patterns.
J	Databases	2016	64/46 (72% pass rate)	Finnish	To learn the basic knowledge required to design and model a relational database. Practical knowledge of SQL and applying it in different environments.

* Student completed at least one other course assignment besides enrolling to the course

• IV.    Results

The courses collected feedback with several approaches. The main methods were the viewing platform statistics, course feedback surveys, and student performance metrics. In the following sections, the results for each of the methods are summarized, and finally, implications of the overall results are presented.

• A.    Viewing platform metrics

  As observed from the Table 3, the average lengths for views do not correspond with the lengths of the videos, and the number of participants does not correspond to the number of views. For example, one tutorial video from the dataset B has over 11.5k views, totaling to approximately 35k minutes of time played, with only 0.5% of this traffic being generated from Finland. In total, with the datasets A, B and I, the Finland-originating traffic accounts for only 28 percent of the total time played, and 14 percent of the views. Also, contrast can be drawn with the dataset J, where the videos were made in Finnish and only available to the students. The total number of views is lower compared to some other datasets, but the average length of views is higher. Over these datasets, starting from the first day of the academic year of 2013-2014, 84 percent of the traffic has been generated from desktop computer (146 386 min, 45341 views), 8,6 percent from mobile devices (15035 mins, 4748 views) and 5,0 percent from tablets (8779 min, 2061 views). On the rest 2.3 percent, such platforms as smart TVs and game consoles were identified. The relative ratios of viewing platforms are visualized in Fig. 1.

Fig.1. Relative Share of View Platforms

Observing the trends, the platform preference has not shifted significantly between 2013 and 2016. Between the different platforms, the average view time does not fluctuate between three and four minutes on any of the meaningful platforms (computer, mobile, tablets) or between the different datasets.

Table 2. Information on Videos Provided for Each Course.

ID	Dataset/Course	Types of videos	Number of videos	Total length of videos	Average length of videos
A	Software Engineering Methods	Lectures:	7	8h 26min	1h 12min
B	Software Engineering Methods	Lectures:	5	6h 58min	1h 24 min
		Tutorials:	39	6h 5min	9min 21s
		Summaries:	6	3h 14min	32min 16s
C	Object-oriented programming	Lectures:	13	14h 05min	1h 5min
		Summaries:	1	46min	46min
D	Object-oriented programming	Tutorials:	31	12h 14min	24min
E	Object-oriented programming	Tutorials:	32	12h 17min	23min
F	Webbed applications	Lectures:	3	48min	16min
		Tutorials:	22	10h 4min	27min
G	Fundamentals of Software Testing	Lectures:	12	16h 15min	1 h 21min
		Tutorials:	18	2h 23min	7 min 59s
		Summaries:	10	2h 47min	16min 45s
H	Fundamentals of Software Testing	Lectures:	13	17h 53 min	1 h 22min
		Tutorials:	18	2h 23min	7 min 59s
		Summaries:	10	2h 47min	16min 45s
I	Object-oriented programming techniques	Tutorials:	10	2 h 32min	15min 15s
J	Databases	Lectures:	14	6h 19min	27min
		Tutorials:	7	1h 36min	14min
		Summaries:	3	52min	17min

• B.    Collected Feedback

The qualitative analysis of the collected feedback from the dataset G indicates that the overall attitude towards lecture videos was overwhelmingly positive, although unlike dataset B, the tutorial videos were not considered as important, or even very relevant for the course outcome. The feedback given from the course B indicates, that the "video archive, lecture recordings" was the best-received feature of the course structure (scale 1-5, 5 best grade) with the overall grade of 4.68, whereas the average over all components was 4.27 and the overall grade for the course was 4.45. Also, open feedback included several positive remarks for the video archive system. Similar observations were made on the dataset G, where again the video recordings were the most liked feature 4.47 with the course receiving generally favorable grade 4.12. However, this feedback also included questions about the usability and the amount of application for the different course components. In this comparison, the course tutorials were the third least used component (2.18 on scale 1-5, 1 ‘did not use at all or very little, ' and 5 ‘used constantly'), being applied only a bit more than course social media services (2.0) and additional literature sources (2.06). In both categories, the recorded sessions were considered more usable than their face-to-face counterparts (3.06 vs. 3.41 in lectures, 2.53 vs. 2.65 in exercises). The open feedback lauded the course for its long-distance support, as “there was the general feel that long-distance participation was not punished in any way for example by withholding some parts of the study materials." Overall, all the qualitative feedback from this dataset was positive towards the lecture recordings. The collected metrics are also summarized in the Table 4.

Table 3. Video View Statistics.

ID	Dataset/Course	Number of active students/participants	Type of video	Average length of videos	Average length of views	Total views (in May 2016)
A	Software Engineering Methods	37	Lectures	1h 12min	2 min 54s	970
B	Software Engineering Methods	45	Lectures	1h 24 min	4 min 53s	3287
			Tutorials	9min 21s	2 min 39s	40286
			Summaries	32min 16s	5 min 32s	181
C	Object-oriented programming	26	Lectures:	1h 5min	10min 3s	1760
			Summaries	46min	7min 7s	86
D, E	Object-oriented programming	26 and 44	Tutorials	23min	6min 31s	9281
F	Webbed applications	40	Lectures	16min	4min 40s	221
			Tutorials	27min	8min 1s	1956
G, H	Fundamentals of Software Testing	22 and 32	Lectures	1 h 22min	13 min 55s	440
			Tutorials	7 min 59s	2 min 35s	1882
			Summaries	16min 45s	3min 56s	276
I	Object-oriented programming techniques	54	Tutorials	15min 15s	5 min	1439
J	Databases	52	Lectures	27min	14min	980
			Tutorials	14min	12min	666
			Summaries	17min	9min	129

* Datasets D and E, and G and H are combined as the same video material was used during both course iterations.

Table 4. Statistics Collected from Course Feedback Forms.

ID	Number of active students	Number of respondents	Usefulness of the videos	The rank of the videos in course components	The overall grade of the course
A	37	-	-	-	-
B	45	20	4.68 / 5	1st	4.27 / 5
C	26	15	4.36 / 5	2nd most useful	4.36 / 5
D	26	19	4.58 / 5	4th	4.47 / 5
E	44	16	4.56 / 5	1st (shared)	3.81 / 5
F	40	18	4.53 / 5	2nd (shared)	4.44 / 5
G	22	17	4.47 / 5	1st	4.12 / 5
H	32	-	-	-	-
I	54	10	4.24 / 5	1st	4.1 / 5
J	52	29	4.45 / 5	1st	3.67 / 5

• C.    Implications

Considering the results and the collected feedback there are some general observations and implications, which can be used to summarize the lessons learned and the main points of this study:

• •    To maintain motivation, it is important that the long-distance students feel that they are treated fairly, especially in a course configuration where there are also local traditional teaching events. In this sense, it would seem advisable to either put every aspect of the course online and eradicate every unnecessary local event, or offer something that only extends the local teaching events, like record archive, with no formal support for the longdistance students.

• •    The low average of view lengths implies that structurally it would be feasible to begin the video with a short summary of everything that will be covered during the recording.

• •    With the computer science courses observed, the most common platform for viewing the video content was a full-sized computer system, with over four-fifths majority of the generated traffic. Based on our observations, this trend has not shifted significantly during the period between 2013 and 2016, or between different types of videos. Because of this, further development and tool selections could concentrate more on optimizing their lecture videos to computer screens instead of mobiles.

• •    The production quality of the videos does not seem to be very critical; a recording system capturing the presentation screen and lecturer’s voice was considered sufficient by majority of the students.

• •    As demonstrated by the datasets G, H, and J, a library of short tutorial videos (10-15 minutes) has better length-watch ration than the recorded lecture videos. The experienced usefulness of the videos did not change significantly between the different course data sets, except for the short tutorial videos of dataset G, which were not used by the students to a large degree, even though the videos themselves were well-received. The datasets G and H were also the only datasets, where short tutorial videos and full lecture recordings were both available, covering all the course’s main topics.

• •    There was no correlation between student satisfaction and video length duration. For example, in data J the videos were split into separate topic segments with bookmarks and the rated usefulness did not deviate from the mean.

• •    A non-public set of videos (J) worked equally well from a teaching perspective, but the peripheral benefits from publicly releasing the videos, as mentioned by Buckland [16], were not achieved. This also slightly shifted the results, since the public viewings did not affect the data.

In addition to learning outcome and usability implications, further observations were also made on the marketing and promotion implications. Although no marketing aims were planned when the videos were decided to be uploaded to YouTube, we could see that our videos were also watched by other people than the students of our university. Although the videos in Finnish are only useful for a handful of people, the videos in English were gaining views around the globe. Although we cannot tell exactly why they are gaining many views from several dedicated places, we can guess that some other educational institutions are using them too. As the material is published under a creative commons license, produced by the lecturer, or otherwise is a recording of a public teaching event, this is not considered a problem, and it also generates added value to the university, with global marketing coverage and visibility.

• V.    Discussion

Overall, the experiences with the application of the video-based instruction have proven to be a positive step from the viewpoint of learning outcomes and the course metrics. There are some issues with the collected data and the metrics, especially from the noise in the view metrics that is generated by the random foreign traffic originating from various sources, for example, Google searches and YouTube's similar-video playlists. Moreover, the students using mobile platforms or several different devices to watch the videos might be counted several times, so the count of views does not fully correlate with the number of viewers. Another important point is that our system is not connected to any MOOC, nor is the produced video material intended for MOOC environment, but to supplement traditional undergraduate level courses that apply the flipped classroom method. The impact of the difference and the requirements between the videos intended for full self-study such as MOOC and courses applying flipped classroom are not very well studied. In our case, the production value issue identified by Hansch et al. [13] was not observed, since our courses also offered local, face-to-face teaching events and the students did not have to rely solely on the video-based materials. Thus, as an answer to our first research question How does the video usage differ between an MOOC and flipped classroom? provides a separation in learning where MOOC has not physical contact teaching and it has to provide everything online. Flipped classroom gives an option to serve massive number of students with video lectures and examples, but it also requires teachers to do in-class teaching.

The applied infrastructure has both limitations and benefits. The most obvious limitation of the presented approach is that by using a third party-supplied platform such as YouTube, the shared material has copyright restrictions, and allows an external party to gain information on the users and access to the materials. Additionally, features which are common in more sophisticated learning environments, such as chat features and note-taking, are not supported, at least to a full degree. In short, this approach offers very limited ways of customization for teachers or students. However, the platform offers several benefits, which were found to outweigh the disadvantages. First and foremost was the accessibility and compatibility with the different viewing platforms. YouTube has extensive support for several different platforms, covering even the marginal systems such as Smart TV -devices and game consoles, so the material is universally available to all users. In addition, YouTube as a viewing platform does not require identification and allows easy distribution of links to the materials. This could also be a drawback, if the course materials are not allowed to be distributed openly, or hosted on a third-party platform. The accessibility and service availability are superior to many other solutions and does not cause extra costs in the form of hosting or service maintenance, which can prevent video instruction in otherwise willing institutions.

Considering the research question What general requirements can be identified for the video-based course infrastructures in the development of cost-efficient longdistance learning solutions? It seems that the technical aspects can be simplified into two aspects: the video quality is not a major concern and even basic recorded presentations are sufficient, and the video distribution system should be designed for full-sized workstation platforms since most of the traffic comes from them. Overall, accessibility also seems a key factor; a service that is not available everywhere or for every user does not provide any benefits. From the teacher's point of view, the research question What benefits widely available teaching videos give to teachers and organizations? can be summarized in a sense that the video-based learning allows the same benefits as a flipped classroom-approach; the teacher's focus can shift from serving as a "biological playback device" to providing practical demonstrations and tutoring exercises. This would also enable deeper learning experiences [23], and indirectly lead to better learning outcomes. Other benefits, of course, are that the courses are no longer as location or schedule-oriented since the students and lecturers do not need to be in the same place at the same time for learning to happen.

Also, interestingly, the answer to the research question What is the difference of lecture videos and tutorial videos in the scope of video usage? provided curious observations. On the courses, where both tutorial videos and lecture recordings were available, the tutorial videos were not widely used. However, all of the videos which generated large amounts of external traffic were tutorial videos. It seems that the lecture recordings are watched mainly by the students participating in the course as a part of their study program, but the external participants who are simply looking for information regarding some specific topic prefer tutorial videos. It could be argued that short tutorial videos provide more distilled information, but the lengthier lecture recordings provide a better overview of the topic.

• VI.    Conclusion

In this paper, we present the results of our experiments with video-based learning approaches when applied in the flipped classroom method. The observations are based on usage metrics, and student feedback gathered from ten courses. The courses cover several topics of the software engineering discipline and were held between 2012 and 2016. The video instruction either supplemented traditional course lectures or enabled flipped classroom teaching principles. The usage, applicability, and impact were evaluated from viewpoints of student results and course outcomes, and as a delivery method for a learning experience.

Based on our observations, the applied video-based learning infrastructure was sufficient, although several areas of improvement were discovered. As for the research questions, one clear benefit of the video system over an MOOC was the platform independence, transferability and accessibility of the material. With all datasets, the most common viewing platform was a full workstation system, although different mobile devices were a sizable minority. In this sense, the most usage of learning videos happens in conjunction with the actual working tools, so support systems are useful, but not mandatory to enable learning experiences.

Overall the experiences with the video-based learning and the associated courses indicate that the results are very positive. The problems identified from the prior research, especially added effort and costs of video production, were not considered a major concern. Similarly, the positive course outcomes indicate that the attention span issues or problems understanding demonstrations were not a problem. In most of the measured courses the video lectures were the most useful component, and even if the videos were only recorded screen captures from slides, they received a very positive evaluation from students for perceived usefulness. Unlike in the previous literature on flipped classroom [4,5], the video duration was not as essential in these results. Student satisfaction and survey results were equal in both long and short lectures.

In future, these results can be applied in the design of more applicable and accessible course infrastructures. To assess the usability and to validate these results, they should be applied to a wider variety of different courses and learning environments. Another interesting area that warrants further research is comparing these results with a course infrastructure that uses a full MOOC system as its online platform.

Acknowledgment

This study was partially funded by the by the European Union Regional Development Fund grant number A70554, "Kyberturvallisuusosaamisen ja liiketoiminnan kehittäminen," administrated by the Council of Kymenlaakso.