Adult numeracy randomised controlled trials: Contextualised approach

This report sets out findings from a pilot randomised controlled trial (RCT) of the Contextualised Approach (CA) for Functional Skills Qualification (FSQ) Level 1 maths. This ran as part of a 3-arm trial with the Adapted Mastery Approach (AMA) to FSQ Level 1 maths. Both were part of the Adult Numeracy Trials that were delivered as part of the government’s Multiply programme which ran from 2022 to March 2025. They aimed to test whether these approaches could improve adult learners’ maths attainment and confidence. The trial compared these interventions to a control group, where providers delivered their usual FSQ Level 1 maths courses without change or adaption. As a pilot, the trial provided a small-scale investigation into the feasibility of implementing a trial in this setting, but it was not powered to detect a small effect size.

Key findings

The main goal (or primary outcome) for the impact evaluation was to improve maths attainment as measured by the pass rate for FSQ Level 1 maths qualifications. The trial found no statistically significant evidence at the 95% confidence level[1] that the CA improved pass rates for FSQ Level 1 maths qualifications in the treatment group compared to the control group. While the treatment group saw a 1 percentage point (ppt) lower pass rate than the control group. However, this difference is not statistically significant, meaning it is not possible to be certain that it was caused by the intervention rather than by chance or a sampling error.

A further goal (or secondary outcome) for the impact evaluation was to improve maths confidence. The evaluation found no statistically significant evidence at the 95% confidence level that the CA improved learners’ overall confidence in maths in the treatment group compared to the control group. These maths confidence analyses were limited by low survey response rates (45% for treatment learners and 33% for control group learners). Two confidence measures were used: ‘overall confidence level in maths at the end of the trial’ and a composite of confidence scores across specific everyday financial scenarios requiring maths (e.g., understanding interest rates on bank statements). Notably, these 2 confidence measures produced estimates in opposite directions. The treatment group’s overall confidence score was 0.17 points lower than the control group, while their composite confidence score was 0.24 points higher, confidence being measured on a Likert scale where 1 meant not confident at all and 5 meant very confident.

As would be expected since this trial was not powered to detect small effects, no evidence was found of different impacts across learner demographic characteristics, including sex, age, ethnicity, deprivation and health/disability status.

The qualitative evidence showed that both tutors who delivered the CA and learners who took part were very engaged with the intervention. Treatment learners expressed strong satisfaction with the programme, particularly the delivery style and real-life contexts which they felt increased their problem-solving capabilities. Learners who experienced the CA were more likely to report being “very confident” in maths than the control group, but when “fairly” and “very” confidence categories were combined, there was no discernible difference between the two groups.

In business as usual (BAU) practices, tutors typically explained maths theories and concepts before offering real-life contexts for applying them. By contrast, CA begins with real-world examples to provide learners with tangible applications for sense-making. As such there was no evidence of overlap between trial arms.

Two fifths (42%) of providers dropped out following randomisation (9 treatment, 12 control). However, tutors who delivered the intervention did so according to the intervention guidance. Evidence suggested that the CA is feasible to deliver with minor adaptations and development.

Implementation barriers included late notice of tutor training dates, the intensity of tutor training and incomplete coverage of the FSQ curriculum. Concerns were also raised about the intervention’s suitability for different learner groups, and these are areas requiring consideration for in any future roll out.

Overall, while the impact evaluation did not find statistically significant impacts, the pilot provides promising signs about the feasibility of a conducting a future RCT in adult learning settings. It demonstrates that providers and learners can be recruited successfully, that randomisation at the provider level is workable, and that outcome data can be collected.

[1] Differences that are statistically significant at the 95% confidence level are greater than would be expected by chance alone. This means we can be 95% certain that the difference is real and not due to sampling error. The 95% confidence level is widely accepted as the standard threshold for determining statistical significance in research and evaluation.