The Autograder project: improving software engineering skills through automated feedback on programming exercises

Abstract

Many students find it difficult to learn programming skills. One reason for this difficulty is that feedback from teaching staff is often slow. The Autograder aims to improve student learning through rapid feedback and to stimulate self-learning. The Autograder project provides a web-frontend and a server back-end that has been developed for automatically correcting and evaluating solutions to programming exercises submitted by students. Correcting and evaluating student submissions are based on teacher written test cases, which the submitted solutions should pass. From this the students get rapid feedback and a score on the programming assignments. Autograder depends on a custom built continuous integration service, test-driven development and a version control system to deliver its services.

The Autograder has been successfully used in a master-level course at the University in Stavanger. Students and teaching staff was monitored and interviewed through their assignment work. Autograder provided an efficient way for the students to reach their potential, through rapid feedback on submitted exercises. The teaching staff got a better overview of the students progress, which made it easier to follow up each student. They was relieved from the burden of manually correcting assignments and could more easily identify pain points in the exercises. Together with oral examination in the lab, the test results obtained through the Autograder was used as the basis for grading the lab assignments. The lab submissions made it much easier for the teaching staff to prepare for the oral examination. Although we cannot draw strong conclusions at this point in time, we have some data points that seem to indicate that student learning has improved also on the final written exam.