Too Scared to Ask? How We Solved the #1 Problem with Large University Lectures

Table of Links

1 Introduction

2 Course Structure and Conditions

2.1 Conditions

2.2 Syllabus

3 Lectures

4 Practical Part

4.1 Engagement Mechanisms

4.2 Technical Setup and Automated Assessment

4.3 Selected Exercises and Tools

5 Check Your Proof by Example

6 Exams

7 Related Work

8 Conclusion, Acknowledgements, and References

3 Lectures

We used a mix of slides, live coding, and whiteboard proofs for the lectures. Each topic came with small case studies and examples. The latter were accompanied by suitable QuickCheck tests and inductive correctness proofs when appropriate. The proofs stayed close to the format accepted by the proof checker that was used in the practical part of the course (see Section 5). The students particularly enjoyed the live coding sections, which received 16 positive and no negative comments in the course evaluation form. This is in line with prior studies, which moreover confirm the effectiveness of live coding [33, 31].

In both iterations, students were allowed to interact and ask questions at any time. However, synchronous interaction with hundreds of students is challenging: While the lecturer cannot answer all questions due to time constraints, many students are also too reserved to ask questions given the large audience. As the semester progresses, interaction tends to degrade to questions posed by a small community of motivated students; questions shared by a majority, on the other hand, often go unheard.

In WS19, we partly addressed this problem by offering an asynchronous Q&A forum[6] where students could post anonymously or using their real name. The forum contained separate sections for the theoretical part (including the lectures) and the practical part of the course. Questions posted in the former were answered by the lecturer to increase interaction between students and the lecturer – the lack of which was often criticised by students in our department. Questions in the latter were also answered by teaching assistants and other students. Answers by students were explicitly encouraged by us and outstanding contributions awarded a special prize at the end of the semester.

However, while the forum was a great success with more than 3800 posts per semester, engagement in the theoretical section stayed far behind its practical counterpart (≤ 2%). Moreover, it does not fully address the live interaction problem since questions are answered asynchronously. As such, students stuck with conceptional problems may not be able to keep up with the rest of the lecture, leading to frustration.

For the second iteration of the course – the online semester – we thus added a new interaction method. The lectures were livestreamed and interaction was made possible by means of a live Q&A board[7]. The board was moderated by a PhD student sitting in the same room as the lecturer. Questions could be answered and voted on by students as well as the moderator. The moderator approved and answered very specific and simple questions directly while forwarding questions of general interest to the lecturer in order to increase engagement between the students and the lecturer.

We can report that this format increased engagement when compared to the first iteration: students were less reluctant to submit questions because 1) they had the chance to ask questions anonymously, 2) they were not afraid to “interrupt” the lecture, and 3) they were able to ask new kinds of questions. Examples of the third include discussion of alternative solutions by students, organisational questions, and slightly off-topic discussions that nevertheless increase engagement and curiosity. On average, 20–30 students submitted at least one message per lecture – a significant improvement when compared to the number of interactions during typical offline lectures. The effectiveness of synchronous interactions during lectures has been indicated in prior studies [10, 38]. We hence recommend to offer a moderated live Q&A board even for lectures taking place on campus or running in a hybrid format.

Following the livestream, the recordings were uploaded for asynchronous consumption. Students watching the lectures asynchronously still had the chance to submit questions to the forum.

[6] We used a Zulip instance hosted by our department: https://zulip.com/case-studies/tum/

[7] We used tweedback https://tweedback.de/