George Kinnear has published a paper with four former undergraduates

George Kinnear has published a paper with four former undergraduates, Max Bennett, Rachel Binnie, Róisín Bolt and Yinglan Zheng, who completed a project with him last year.

The paper, titled 'Reliable application of the MATH taxonomy sheds light on assessment practices', came out of a project investigating the different types of assessment questions used in school and university exams. The MATH taxonomy was designed to categorise questions based on the skills needed to solve them, and has already been used by researchers to examine the mix of assessment tasks used in school exams (at A-Level and SQA Advanced Higher level). This project was the first work to investigate whether multiple coders can reliably apply the taxonomy, i.e. whether different people categorising the questions make the same decisions.

The research was conducted by the students coding a range of school and university assessments, with each mark in each question being assigned to a specific category. In order to establish a common understanding of the taxonomy the students first completed rounds of semi-independent coding followed by group discussion where disagreements could be discussed.

The results showed that the MATH taxonomy can be applied reliably after this calibration process. They also confirmed previous findings that there are differences in the types of questions asked in school and university exams. The students also analysed the different assessments in our first year course, Introduction to Linear Algebra, and found differences between the types of tasks in the online homework, the written homework, and the exam - however, these differences were because the course lecturers had designed the different assessments to test a full range of skills.

The paper can be read in full here: https://academic.oup.com/teamat/advance-article/doi/10.1093/teamat/hrz017/5709768