Background / Rationale 

gAI presents an opportunity for HE as the technology could support students’ learning and/or skill development. However, gAI also poses challenges to current learning environments, particularly those that use essay-based assessments to encourage students to develop discipline-specific academic writing skills.  

 In the biological sciences, Level 1 students need to develop skills in: (i) identifying / citing sources, (ii) selecting / referring to figures, and (iii) developing an argument in an accurate, concise writing style. As such, assessment criteria often focus on these elements of academic literacies, however these are areas where misuse of gAI (e.g. ChatGPT) may prevent students from developing these skills. This project will explore how gAI can contribute to skill development, by designing learning environment(s) that encourage students to adopt an informed, reflective approach to these tools in supporting academic writing. 

 Project aims

This project aims to encourage students’ use of gAI in academic writing in the context of a large cohort (n=~680), Level 1 summative ‘scientific essay’ assessment (Yr 1. BSc Biosciences Common Year One. Academic skills module). This is an ideal point to discuss gAI use and combine this with approaches to (i) academic writing, and (ii) ‘assessment literacy’ at the start of their UG programme. The project will develop over two years:  

(i) Project Year 1: Semester 1, 2023. 

We will design workshops and/or podcasts to discuss how students can use gAI to prepare for a ‘scientific essay’ assessment. We will use the PAIR framework (Acar, 2022) to discuss effective approaches to gAI use (e.g. prompt design, validation etc.), and combine this with ‘traditional’ approaches to research to synthesise the findings etc.   

(ii) Project Year 2: Semester 1, 2024 

The following year we will amend the workshop(s) in response to student feedback, [and] redesign the assessment criteria to encourage students to engage with the process of academic writing. These criteria are likely to focus on elements of research that contribute to essay ideation and structure, for example: (i) ‘mind maps’ of essay themes (ii) lists of proposed academic sources / justification(s) for their use, and (iii) single page ‘bullet point’ essay plans. We propose to design assessment components in collaboration with Y2 students which have progressed from CYO using a co-creation process.   

We will evaluate these intervention(s) using a mixed methods approach to survey students’ use and perceptions of gAI before, and after the ‘workshop – assessment’, and will ‘triangulate’ these with the views of staff / assessors to build a rich picture of the impact of this on students’ writing skills. 

 Project outputs. 

We aim to develop an integrated approach to guiding students’ use of AI in written assessments, with two components: 

(1) Design effective ‘Using gAI in assessment’ guidance workshop. We will use students to inform the format and content(s) of ‘AI & Assessment’ guidance. This may take the form of workshops, podcasts and/or text-based learning materials, which will be optimized using student feedback. 

(2) Redesign an essay-based assessment. We will use a ‘process-based’ approach to scientific writing, with aligned assessment criteria to encourage students to (i) structure their writing logically (ii) identify / justify their sources, (ii) synthesise findings from gAI-based and ‘traditional’ research, and (iii) develop a critical evaluative approach to their findings.  

Finally, we aim to generate a robust evidence base that this approach is effective. Our use of a ‘pre-’ and ‘post-intervention’ study design, and mixed methods evaluation using different stakeholders has the potential to generate publication quality data.   

 Research Team. 

KPN is an established educational researcher with a track record of using action research to evaluate the impact of educational innovation(s). RR and GK are module / programme convenors, and lead the target module or the broader programme curriculum. BW is the Assessment lead for IoPPN and had expertise on gAI / assessment practice. JP is the King’s Academy Lead for Assessment & Feedback with expertise on process-based assessment. Prof Oz Acar (Business School) and Dr Anna Verges Bausili (King’s Academy), have agreed to act as consultants on Phase 1 of the study.