Table of Links 1 Introduction 1 Introduction 2 Course Structure and Conditions 2.1 Conditions 2.1 Conditions 2.2 Syllabus 2.2 Syllabus 3 Lectures 3 Lectures 4 Practical Part 4.1 Engagement Mechanisms 4.1 Engagement Mechanisms 4.2 Technical Setup and Automated Assessment 4.2 Technical Setup and Automated Assessment 4.3 Selected Exercises and Tools 4.3 Selected Exercises and Tools 5 Check Your Proof by Example 5 Check Your Proof by Example 6 Exams 6 Exams 7 Related Work 7 Related Work 8 Conclusion, Acknowledgements, and References 8 Conclusion, Acknowledgements, and References 4.2 Technical Setup and Automated Assessment Automated Assessment In WS19, we used an improved version of the testing infrastructure introduced in [4]. However, this system could neither manage online exams nor mark non-programming tasks. We thus switched to a newly written open-source tool developed at TUM called ArTEMiS [22]. ArTEMis is a highly scalable, automated assessment management system and is programming language independent – it only expects test runners to produce tests results adhering to the Apache Ant JUnit XML schema. It already offered support for a few imperative programming languages, and we added support for Haskell.[10] Automated Assessment As ArTEMis takes care of most things, including automated test execution and score management, and offers an exam mode and good support for grading non-programming tasks, the only thing that was left to do was writing the testing code. For the most part, we verified the results computed by a student’s submission by comparing them to those computed by a sample solution written by us. In some cases, we also tested for efficiency using timeouts. Our tests were powered by the following libraries: QuickCheck [7]: Although QuickCheck can automatically generate test data (using the typeclass Arbitrary), most tests and benchmarks used custom input generators. This was necessary to increase coverage and eliminate non-applicable inputs for tests with preconditions. We also used custom shrinkers to provide better feedback to students in case of a failure. In both cases, the flexible combinators provided by QuickCheck made this a straightforward task.
SmallCheck [34]: The exhaustive testing facilities provided by SmallCheck mainly served as a complementary tool that provided small counterexamples for, in many cases, obvious deficiencies.
Tasty[11]: We used Tasty to put QuickCheck, SmallCheck, unit tests, and the checking of “Check Your Proof” proofs (see Section 5) into one common framework that is capable of generating results interpretable by ArTEMiS. We used the unit testing facilities of Tasty to complement our tests with corner cases. Integration of proof checking was pleasantly straightforward: one only has to provide a suitable instance for Tasty’s IsTest typeclass.[12] Moreover, Tasty supports timeouts for individual test cases. solving the issue of truncated test reports mentioned in [4]. QuickCheck [7]: Although QuickCheck can automatically generate test data (using the typeclass Arbitrary), most tests and benchmarks used custom input generators. This was necessary to increase coverage and eliminate non-applicable inputs for tests with preconditions. We also used custom shrinkers to provide better feedback to students in case of a failure. In both cases, the flexible combinators provided by QuickCheck made this a straightforward task. SmallCheck [34]: The exhaustive testing facilities provided by SmallCheck mainly served as a complementary tool that provided small counterexamples for, in many cases, obvious deficiencies. Tasty[11]: We used Tasty to put QuickCheck, SmallCheck, unit tests, and the checking of “Check Your Proof” proofs (see Section 5) into one common framework that is capable of generating results interpretable by ArTEMiS. We used the unit testing facilities of Tasty to complement our tests with corner cases. Integration of proof checking was pleasantly straightforward: one only has to provide a suitable instance for Tasty’s IsTest typeclass.[12] Moreover, Tasty supports timeouts for individual test cases. solving the issue of truncated test reports mentioned in [4]. Development Environment and Online Tutorials In previous iterations, there were no recommendations for students regarding the development environment they should use for the practical part of the course. However, due to the COVID-19 pandemic, this was no longer an option: we needed a way for students to share their code in read and write mode to other peers (pair-programming) and teaching assistants (for feedback purposes) during online tutorials. Moreover, as explained in Section 4.1, we wanted students to use a linter. Finally, we had negative experiences with students installing no compiler at all and instead (mis)using our submission server as a compiler backend. We thus introduced a strict policy for the technical setup to be used during the tutorials. Detailed installation instructions can be found online[13]. Here we briefly list the key components and our experiences: Development Environment and Online IDE: We used VSCodium[14] due to its cross-platform support, rich library of extensions, widespread adoption, and free open-source software philosophy. We did not receive any negative feedback by students and, besides few exceptions, no major installation problems were reported.
Build and dependency manager: We used Stack[15] for these purposes. Since Stack provides curated sets of packages and compiler versions that are checked for compatibility, deterministic builds are guaranteed. Students hence showed little struggle to compile and execute their programs. IDE: We used VSCodium[14] due to its cross-platform support, rich library of extensions, widespread adoption, and free open-source software philosophy. We did not receive any negative feedback by students and, besides few exceptions, no major installation problems were reported. Build and dependency manager: We used Stack[15] for these purposes. Since Stack provides curated sets of packages and compiler versions that are checked for compatibility, deterministic builds are guaranteed. Students hence showed little struggle to compile and execute their programs. Linter: We used HLint[16], which, among other things, provides suggestions for alternative functions and simplified code and spots redundancies. Students reacted curiously and positively to these suggestions and discussed them vividly during pair-programming sessions.


API search engine: Due to Haskell’s strong type system, searching its API by type signature often returns better results than searching for function names. For this purpose, we introduced Hoogle[17] to our students and let them install an extension that integrates Hoogle searches into their IDE. Unfortunately, we have no data to report on whether this enriched students’ programming experiences.


Real-time collaboration: We used VSLiveShare[18] to run our pair-programming sessions (groups of 3–4 students). We can report positively on its connection stability and usability. Unfortunately, the plugin requires a Microsoft or GitHub account, but since almost all students already signed up to the latter before, this requirement passed uncontroversially. Nevertheless, for future iterations, we plan to investigate open-source alternatives. Linter: We used HLint[16], which, among other things, provides suggestions for alternative functions and simplified code and spots redundancies. Students reacted curiously and positively to these suggestions and discussed them vividly during pair-programming sessions. Linter: We used HLint[16], which, among other things, provides suggestions for alternative functions and simplified code and spots redundancies. Students reacted curiously and positively to these suggestions and discussed them vividly during pair-programming sessions. API search engine: Due to Haskell’s strong type system, searching its API by type signature often returns better results than searching for function names. For this purpose, we introduced Hoogle[17] to our students and let them install an extension that integrates Hoogle searches into their IDE. Unfortunately, we have no data to report on whether this enriched students’ programming experiences. API search engine: Due to Haskell’s strong type system, searching its API by type signature often returns better results than searching for function names. For this purpose, we introduced Hoogle[17] to our students and let them install an extension that integrates Hoogle searches into their IDE. Unfortunately, we have no data to report on whether this enriched students’ programming experiences. Real-time collaboration: We used VSLiveShare[18] to run our pair-programming sessions (groups of 3–4 students). We can report positively on its connection stability and usability. Unfortunately, the plugin requires a Microsoft or GitHub account, but since almost all students already signed up to the latter before, this requirement passed uncontroversially. Nevertheless, for future iterations, we plan to investigate open-source alternatives. Real-time collaboration: We used VSLiveShare[18] to run our pair-programming sessions (groups of 3–4 students). We can report positively on its connection stability and usability. Unfortunately, the plugin requires a Microsoft or GitHub account, but since almost all students already signed up to the latter before, this requirement passed uncontroversially. Nevertheless, for future iterations, we plan to investigate open-source alternatives. All in all, we can report positively on the setup. Students were more knowledgeable about their programming environment (compilation, dependency management, etc.) than in previous iterations. Misappropriation of the submission server as a compiler backend also stopped. Authors:
(1) Kevin Kappelmann, Department of Informatics, Technical University of Munich, Germany (kevin.kappelmann@tum.de);
(2) Jonas Radle, Department of Informatics, Technical University of Munich, Germany (raedle@in.tum.de);
(3) Lukas Stevens, Department of Informatics, Technical University of Munich, Germany (stevensl@in.tum.de). Authors: Authors: (1) Kevin Kappelmann, Department of Informatics, Technical University of Munich, Germany (kevin.kappelmann@tum.de); (2) Jonas Radle, Department of Informatics, Technical University of Munich, Germany (raedle@in.tum.de); (3) Lukas Stevens, Department of Informatics, Technical University of Munich, Germany (stevensl@in.tum.de). This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv [10] It now also supports OCaml. [11] https://hackage.haskell.org/package/tasty [12] The code can be found in the repository in resources/cyp integration/test/hs/Test.hs [13] https://www21.in.tum.de/teaching/fpv/WS20/installation.html [14] VSCodium provides free open-source software binaries of VSCode https://vscodium.com/ [15] https://www.haskellstack.org/ [16] https://github.com/ndmitchell/hlint [17] https://hoogle.haskell.org/ [18] https://visualstudio.microsoft.com/services/live-share/

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Apache

From Chaos to Control: Building a Standardized Dev Environment for 1,000+ Students

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

11 Proven Solutions to Common GitHub Copilot Problems

Automated Feedback Systems for Enhanced Tutor Training

Sequence Labeling for Automated Feedback in Tutor Training

GPT Models for Sequence Labeling: Prompt Engineering & Fine-tuning

Discussing GPT Models for Automated Explanatory Feedback in Tutor Training

Future Research: Explanatory Feedback Systems for Tutor Training

11 Proven Solutions to Common GitHub Copilot Problems

Automated Feedback Systems for Enhanced Tutor Training

Sequence Labeling for Automated Feedback in Tutor Training

GPT Models for Sequence Labeling: Prompt Engineering & Fine-tuning

Discussing GPT Models for Automated Explanatory Feedback in Tutor Training

Future Research: Explanatory Feedback Systems for Tutor Training

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps