Within-person variability in performance across school subjects

Highlights

• Students perform differently across school subject areas and years, giving rise to within-person grade variability.

• Only 1% of individual differences in within-person grade variability were stable from age 7 through 16.

• Within-person grade variability was not associated with factors that predict between-person differences in school grades.

• Within-person grade variability did not emerge as a meaningful psychological construct.

Abstract

Although thought to be substantial, within-person variability in performance across school subjects has not been systematically studied. Here we analysed data from the Twins Early Development Study (TEDS; N_max = 5919) to describe within-person variability across grades in English, maths, and science from age 7 to 16 years. We found that within-person grade variability was largely unstable across subjects and ages. Within-person grade variability at age 16 was not associated with any of 15 variables that typically explain between-person differences in school performance (e.g. IQ, socioeconomic status, and personality traits). Also, within-person grade variability did not predict later educational outcomes at ages 18 and 21. Within-person grade variability is observable, but did not emerge in this study as a meaningful psychological construct. We conclude that understanding the causes and consequences of within-person grade variability is possibly of limited epistemological value.

Introduction

School performance differs between individuals, and these differences have pervasive long-term effects on academic achievement and educational and occupational outcomes (e.g. Ayorech et al., 2019; Kuncel et al., 2001). Children who achieve better grades in school obtain on average more and higher educational qualifications compared to those who perform poorly in school. However, school performance also differs within individuals: pupils often vary in their performance between subjects (e.g., Jansen et al., 2016). Both within-person and between-person differences are thought to affect school performance (e.g., Galla et al., 2014), but less is known about the causes, consequences, and stability of within-person differences.

Many studies on school performance use between-person measures such as grade averages or single subject grades to capture how well students are mastering the curriculum (e.g., McKee & Caldarella, 2016; Rimfeld et al., 2016; Kuncel et al., 2001). This approach ignores any within-person variability in grades across different subject domains, thus obscuring potentially meaningful differences in school performance at the individual level.

Within-person variability in performance across school subjects – that is, the variability of a student's grades across school subjects at a given time and herein referred to as within-person grade variability – may influence later educational and occupational choices, which are driven partly by average grades, but also by domain-specific performance and interests. If students excel at one school subject above others (e.g., Jansen et al., 2016), they may feel more inclined to pursue that subject further, for example as a university degree. By comparison, if students perform equally well across subjects, they may struggle to decide which degree or profession to choose. Here, within-person grade variability may explain some of the relation between school performance and later educational and occupational outcomes.

Only two prior studies have been published on within-person grade variability. Bakan (1971) assessed within-person grade variability using within-person standard deviations of grades over five years in a sample of 112 junior high school students, who were divided into four variability level groups across three subjects (mathematics, English, and social science) and gender. Bakan (1971) observed no significant differences in IQ or socio-economic status (SES) between those with high and low within-person grade variability, and found that academic achievement and self-concept reduced more rapidly over time in the high-variability group compared with the low-variability group. Because these analyses were based on many sub-groups, the results afforded low statistical power, calling into question their validity.

More recently, Wingate and Tomes (2017) examined within-person grade variability across modules in a group of 487 undergraduate psychology students and tested their associations with cognitive and personality factors. Within-person grade variability was based on each individual's standard deviation across module grades (n = ≤5). Wingate and Tomes (2017) found that those with high within-person grade variability tended to score lower on conscientiousness (r = −0.24), were more anxious (r = 0.34), and felt less motivated (r = −0.36) than those with lower within-person grade variability. These findings suggest that within-person grade variability is meaningfully associated with between-person differences in personality and motivation. However, the university students in this study had selected themselves into a specific subject area or degree (i.e. psychology), whose modules revolve around the same topic area. Their within-person grade variability is therefore likely to be less than that of school students, who study many diverse subjects.

Within-person grade variability for accuracy measures, such as school subject grades, can be operationalized using intraindividual standard deviation (iSD; Golay et al., 2013). However, the iSD can become confounded by the mean when measurements are bounded (Kalokerinos et al., 2020). This is the case for school grades, which typically range between fixed values, for example from A for outstanding performance to U for a fail. Students with a high grade average must have obtained grades close to the scale ceiling across all subjects (i.e., A grades); as a result, the range of their grades is restricted and their within-person grade variability as measured by iSD is low. Confounding by the mean biases the interpretation of associations with within-person variability. To overcome this limitation, within-person grade variability must be estimated independent of the mean, by adjusting the within-person standard deviation for the within-person mean either by using regression or by applying the relative variability index (Mestdagh et al., 2018). The relative variability index takes into account the maximum possible variance given an observed mean, to avoid confounding by the mean due to the boundedness of the measure (Mestdagh et al., 2018). However, if the mean is exactly equal to the upper or lower bound, the resulting variability will be zero; such cases are typically omitted from further analyses when using the relative variability index (Mestdagh et al., 2018). Because the ability to learn is relatively stable and study skills are transferrable across subject domains (Wendt & Kasper, 2016), a good proportion of school students are likely to have low or no within-person grade variability when operationalized by the relative variability index. Here, we measure within-person grade variability using the relative variability index as well as a measure of grade standard deviation that we adjusted for the mean through regression models. More detail on the measures of within-person grade variability can be found in the Supplementary materials.

To identify factors that may predict within-person grade variability, we look to the large body of empirical research which substantiates that between-person differences in school performance are influenced by: (a) family background (e.g., socioeconomic status; SES (e.g., Parker et al., 2012)); (b) intelligence (e.g., Roth et al., 2015); (c) personality traits including the Big Five (e.g., O'Connor & Paunonen, 2007), anxiety (e.g., Pantoja et al., 2020), optimism (e.g., Tetzner & Becker, 2017), ambition (e.g., Jerrim et al., 2020), and grit (e.g., Lai Lam & Zhou, 2019); (d) academic self-concept (e.g., Guo et al., 2016); (e) learning environment (Lizzio et al., 2002); and (f) school engagement (Wang & Holcombe, 2010). However, measurements of between-person differences do not necessarily apply to the level of the individual (Borsboom et al., 2003), and it is unknown whether these factors also predict within-person grade variability.

Here, we report a comprehensive study of within-person grade variability over time in English, maths and science grades from age 7 to 16 in children from the Twins Early Development Study (TEDS), a UK representative cohort study (Rimfeld et al., 2019). We preregistered our study on the Open Science Framework (https://osf.io/vu9zc/). We focused on within-person grade variability across English, maths, and science, which are the core subjects in compulsory schooling in the UK. We aimed to (1) estimate within-person grade variability independent of the grade mean; (2) describe within-person grade variability over time from age 7 to 16 years, including testing for gender differences; and (3) examine the influence of traditional predictors of between-person differences in school performance, including SES, IQ, personality traits, school environment, and engagement with school on grade variability at age 16, when students completed the General Certificate of Secondary Education (GCSE) that marked the end of compulsory education and that guided their subsequent educational choices. GCSE grades at age 16 have been shown to have predictive validity for later life outcomes such as degree attainment at University (Department for Education, 2013). Finally, we (4) examined the predictive validity of within-person grade variability at age 16 on later educational outcomes such as university, work, or NEET (Not in Education, Employment or Training), and whether within-person grade variability influences university subject choices. Educational choices are not solely based on average grades, but involve one's interests and strengths in particular domains. It seems plausible that students with low within-person grade variability, independent of their average performance, may struggle to identify their preferred subject area and could, for example, delay university entry.

Based on the previous literature (Bakan, 1971; Wingate & Tomes, 2017), we predicted that within-person grade variability would be observable. Elsewise, our analyses are largely exploratory and not driven by specific hypotheses.