Consultation on changes to the College Performance Indicators

4. The following table summarises our proposed changes to the publication:

5. We produce statistics covering further education (FE) and higher education (HE) in Scotland to support our work with colleges and universities to measure and improve performance across the sector.
6. Our College Performance Indicators (CPIs) publication seeks to inform stakeholders about the performance of the sector, concerns the performance of institutions, not individuals, and helps colleges benchmark their performance against other colleges in Scotland.
7. The SFC’s CPIs provide an annual analysis of enrolled student outcomes, with data available by age, sex, subject, full-time/part-time, and key groups. The current outcome measures are:

• Completed Successful.
• Partial Success.
• Withdrawal.

8. SFC also collects information from colleges on early withdrawals and further withdrawals for the purposes of funding. Early withdrawal has historically been defined as a student departing from a course before the funding qualifying date of the course has elapsed. This dovetails with further withdrawal, which indicates that the student attended past that funding qualifying date cut-off but withdrew from their studies before the course ended.

9. Early withdrawals are not funded. The factors influencing student and institutional decisions, as well as the impact of early versus further withdrawals, therefore differ. While both are important metrics to monitor, we believe it is appropriate to separate consideration of early withdrawals from further withdrawals.
10. During an inquiry into College Regionalisation in 2022, Education, Children and Young People Committee members noted that just under a third of students were failing to complete their courses. The Committee also heard evidence regarding completion rates that, amongst other scenarios, a student who enrols and attends for one learning episode during the year would be recorded by SFC as a withdrawal, and that there is difficulty in benchmarking college performance indicators against the other UK nations.
11. In 2023, the Committees’ College Regionalisation report, noted that colleges need to better understand why students are withdrawing (noting that lack of clarity on if a withdrawal is due to employment, a more suitable course, or the inability to cope with coursework).

12. Early withdrawals have historically been defined for funding purposes as a student departing from a course before the funding qualifying date of the course has elapsed. From the 2023-24 academic year onwards, the funding threshold for study on full-time courses over 20 weeks long has changed to 5 weeks and 1 day after the course start date, aligning more closely with definitions for university study. For all other courses, the qualifying funding date will be the day on which 25% of the total calendar days between the course start and end days have passed. Non-completion is defined as withdrawals occurring after this time.
13. It is important to collect and report data on early withdrawals as students who withdraw early still represent a financial and time investment for both the individual and the college, particularly during the stages of recruitment and preparation for the course.
14. To date the SFC has published a single (overall) withdrawal percentage, as a possible outcome alongside completed successful, and partial success as noted in paragraph 19.
15. University data submitted in the annual HESA student return excludes students who withdrew within two weeks of starting without completing. The statistics published for universities thereby exclude students who withdrew from study early as a withdrawal.
16. The statistics published for college study in England, Wales and Northern Ireland include separate consideration of early and further withdrawal.
17. Furthermore, due to various cohort and structural differences, the current CPIs published by SFC are not directly comparable to the rest of the UK:

• Education, Children and Young People Committee members have previously cited the 89% completion figure in England, which is based on the March 2020 Adult (19+) FE and Skills and Participation and Achievements.
• This differs first, in that this is not a completion figure, it is a measure of achievement, demonstrating: “the number of achieved learning aims as a percentage of the total number of learning aims in the cohort that ended.” Second, it is not comparable to SFC data, which includes data from those under 18 and upwards.
• Northern Ireland data published by the Department for the Economy on FE activity, records a retention rate, achievement rate and a success rate, but importantly only presents analysis for final year enrolments whereas the SFC CPIs report on all years of enrolments.

18. For further information on metrics across the rest of the UK please see Appendix B.

19. Having reviewed the CPI data and comparative measures, SFC is proposing to amend the published withdrawal figure to exclude early withdrawals.
20. This will bring us into line with FE data publications from the rest of the UK, which will enable a greater degree of comparability, and acknowledge the difference between the nature of early and further withdrawals.
21. Modelling has indicated that this definition change would result in an improvement in the CPI success rate of around four percentage points for the sector overall, though the magnitude of improvement varies by institution. No colleges will see worse results.
22. Taking the examples raised in evidence sessions held by the Committee, the impact of our recommendation would be that:
    • A student who enrols and attends for one learning episode in the year would not feature in the CPIs but would be noted within the early withdrawals data.
    • A student who is enrolled on a programme but then gains employment and withdraws past the early withdrawal deadline would still be reported as a withdrawal. This is in line with how similar withdrawals would be treated in university performance indicators and the college success metrics in other UK nations.
    • A student who transfers to an apprenticeship programme (delivered by the same college) would still be reported as a withdrawal unless they transferred before the early withdrawal deadline. However, it should be noted that if the student then successfully completed their apprenticeship programme, their second enrolment would be treated as a successful outcome in the measures.
23. Minimising early withdrawals, while representing a different challenge for colleges, is also important for ensuring successful outcomes for students. As previously mentioned, students who withdraw early still represent a financial and time investment for both the individual and the college. While we propose that early withdrawal be removed from the definition of the CPI, we intend that the data on early withdrawal be published alongside the CPI as a separate metric on student success.
24. Early withdrawals will be disaggregated and reported with trend data in a similar manner to the success PI, though any future data published would not be directly comparable with that published in previous years due to the separation of early withdrawals from non-completions.
25. Colleges also submit data on those who withdraw from the course to study in a Higher Education Institution (HEI), transfer to another course within the college, and those who are studying at another institution other than a HEI.
26. Whilst these could be considered ‘successful’ individual outcomes in some cases, to incorporate these outcomes into the statistic would conflate completion and successful study of college courses with onward outcomes. Additionally, we note that these transfers between courses are currently excluded from the Department for Education’s Achievement Rate statistics[1].
27. We therefore do not propose to consider withdrawal to take up study or employment in the metric directly, however this is addressed in more detail in the following section.

[1] https://explore-education-statistics.service.gov.uk/find-statistics/further-education-and-skills

28. SFC has initiated the collection of information on students’ reasons for withdrawal from colleges. The first year of collection of the data in 2021-22 was during the Covid-19 pandemic.
29. This collection will allow stakeholders to better understand high-level reasons, at the point of departure, for student withdrawal, enabling colleges to observe trends and understand the reasons why students do not remain at college, so that the college can identify appropriate actions to reduce future withdrawals where appropriate.
30. To help stakeholders understand the information presented and the nature of complex outcomes, such as the incomparability across the UK nations, information from the withdrawal reasons will be published alongside the CPI.

College performance indicators (college sector only)

Overview

The purpose of this publication is to provide stakeholders with baseline statistics and indicators describing the performance of the college sector, and to help colleges benchmark their activities against other similar colleges thereby supporting a wider quality improvement agenda. The statistics and indicators cover areas such as student activity, student retention and achievement, staff qualifications, student satisfaction and outcomes of HMIE (now Education Scotland) inspection and are reported at sector and individual college level.

The college performance indicator publication can be viewed at:

https://www.sfc.ac.uk/publications/college-performance-indicators-2022-23/

Usage

Reliable website data is not available because of the impact of the rules governing use of ‘cookies’ on websites.

Related information

All of the material provided is also made available to colleges and Education Scotland through a spreadsheet-based Course PI tool that allows key users to analyse and manipulate the data in a way that suits their own requirements.

Commentary

The detail of this report is primarily used by expert stakeholders, who have alternative ways of accessing the same information. However, it is very important that key data, particularly at sector level, is made available to other stakeholders. It is also worth noting that a by-product of the electronic submission process from colleges to the SFC of their student data, is that each college has its own performance indicators available to it immediately.

England

Within the Department for Education’s publication on further education and skills, the national achievement rates tables report on:

• Retention Rates.
• Pass Rates.
• Achievement Rates.

The published data can be found at:

Explore our statistics and data – Explore education statistics – GOV.UK (explore-education-statistics.service.gov.uk)

More details on the definitions can be found at:

Further education and skills statistics: methodology, Methodology – Explore education statistics – GOV.UK (explore-education-statistics.service.gov.uk)

Northern Ireland

As part of the Department for the Economy’s publication on Further education college statistics, Section 4.3 on performance outlines measures on:

• Retention Rates.
• Achievement Rates.
• Success Rates.

The published data, with further breakdowns on these measures provided in the accompanying Excel tables to the publication, can be found at:

Further education college statistics | Department for the Economy (economy-ni.gov.uk)

Wales

The Welsh Government’s report on consistent performance measures for post-16 learning (achievement) contains several different measures on the achievements of Welsh post-16 learners.

Further information can be found at:

Consistent performance measures for post-16 learning (achievement) | GOV.WALES