Investigating the influence of Automated Programming Assessment Tools on self-reported understanding of threshold programming concepts in Key Stage 4 GCSE computer science students

Abstract

This thesis investigates the influence of Automated Programming Assessment Tools (APATs) on Key Stage 4 students' self-reported understanding of threshold programming concepts within the GCSE Computer Science curriculum. The study mainly evaluates how factors such as motivation, exposure, and inherent student ability interact with using APATs to affect educational outcomes in programming. The research question was framed to address a significant gap in the existing educational literature concerning the efficacy of automated tools in enhancing student learning in programming. Despite the widespread adoption of APATs in academic settings, more needs to be understood about their impact on student learning, particularly in the context of students' diverse educational and personal backgrounds. Methodologically, the study adopted a mixed-methods approach, combining quantitative data analysis with qualitative insights to provide a comprehensive view of the impacts of APATs. Quantitative data were collected through pre- and post-use assessments that measured students' understanding of specific programming concepts, including variables, selection, iteration, string manipulation, arrays, and subprograms. The qualitative component involved surveys and interviews with students and teachers to capture detailed feedback on using APATs and their perceived effectiveness. The analysis revealed that APATs positively impact the learning outcomes of motivated students who actively engage with the tools. These students reported a better understanding of the threshold concepts after using APATs. However, the study found no substantial evidence to suggest that APATs consistently enhance selfreported understanding across all student groups. Surprisingly, factors traditionally thought to influence educational outcomes, such as student gender, ability, and time spent using APATs, did not significantly impact the effectiveness of APATs in this study. These findings suggest that while APATs can be beneficial, their effectiveness is likely contingent on individual student characteristics and the educational environment. This highlights the need for teachers to consider these factors when integrating APATs into their teaching strategies. The research contributes to educational technology by providing empirical evidence on the utility of APATs in programming education at the secondary school level. It also offers practical insights for educators on the potential and limitations of APATs, suggesting that while they can enhance learning for particular student groups, they are not a universal solution for all educational challenges. Future research is recommended to explore the differential impacts of APATs among various student demographics and learning environments. Such studies could help design more targeted and effective educational strategies that leverage the benefits of APATs while mitigating their limitations.

Date of Award	18 Jun 2025
Original language	English
Awarding Institution	Abertay University
Supervisor	Liz Bacon (Supervisor), Suzanne Prior (Supervisor) & James Bown (Supervisor)