CCEA identified an issue with the Northern Ireland Numeracy Assessment (NINA) outcomes. The issue was raised with the supplier of the assessment, who confirmed a computer error on 14 December 2016. The error caused the calculation of incorrect scores for a number of children taking the assessment in the autumn term of this academic year.
The error has been rectified and the correct scores have been released.
Children take this assessment using computers in schools, their responses are marked electronically and calculations are carried out to produce a number of scores and reports. The reports are made available to schools soon after the assessments are completed. Outcomes are not collated by CCEA.
A different provider is responsible for the Northern Ireland Literacy Assessment (NILA). The literacy assessment is unaffected.
Schools elect to use this assessment service. They are taken by children in years 4- 7 throughout the autumn term. 267 primary schools elected to use the NINA assessment in 2016 and 24,076 children took the assessment.
Justin Edwards, CCEA CEO commented:
“Computer based assessments, like NINA, are designed to help inform teaching and learning in schools. Using computers for assessment, rather than pencil and paper, they provide immediate feedback to teachers. Not all schools choose to use the service.
For this assessment service we use a third party provider. The assessment provider holds all data relating to the assessments securely. During the autumn term CCEA is provided with information, by the provider, about how the assessment system is performing. However, when the number of schools completing the assessment was sufficiently large, CCEA noted that the outcomes provided demonstrated lower than expected scores. We took immediate action.
CCEA asked for a detailed review of the assessments and systems. We were provided assurances that the assessments were performing as designed. Not satisfied, we continued to raise queries with the provider, until it was reported that they had identified an error within their systems. It was confirmed that the error was in the software used to calculate final scores for a number of children taking the assessment.
The error has been rectified and the correct scores have now been generated. These are now available to schools. In all cases the correct scores will show higher performance than previously reported.
Children do not need to re-take assessments for correct scores to be calculated. All schools that elected to take the assessments are being contacted and supported at this time.
CCEA’s priority has been the children that took the assessments and to ensure that their scores are correct. CCEA instructed the provider to take urgent action to rectify the error and asked for assurances that newly calculated outcomes are accurate. These assurances have been given to CCEA by the company involved.
CCEA is disappointed that the quality of service has fallen short of expectations. We apologise unreservedly for any inconvenience or distress this error has caused.”
- NINA (Northern Ireland Numeracy Assessment) and NILA (Northern Ireland Literacy Assessment) are computer based assessments (CBA) that are taken by pupils in Years 4-7 in primary schools at the beginning of the school year.
- The assessments are not mandatory. 267 schools elected to take the NINA assessment in 2016.
- CBAs are assessments designed to help teachers identify strengths and areas for improvement and to help inform teaching and learning throughout the school year.
- The CBA in numeracy is developed and administered by a third party under contract to CCEA. The NINA assessment is provided by Ardluce, a sub contractor of Hachette. CCEA’s contract is with Hachette.
- 24,076 pupils have taken the NINA assessment to date (the assessment remained open until 16 December 2016)
- The number of pupils affected is 16,808.
- NINA outcomes calculated after 15.12.16 are accurate.
- The Northern Ireland Literacy Assessment (NILA) is unaffected by this issue.
- A CBA helpdesk is available for teachers on 028 90 261274.
CCEA has worked with a number of schools to identify how Computer Based Assessments are being utilised. In agreement with the schools involved, we have put together case studies to share and provide examples of how schools in Northern Ireland have used CBA for a variety of purposes.
CCEA would like to thank the schools involved for their participation in this study.
The case study booklet can be accessed via the link below:
This year’s launch of the CBA Programme is now fully underway and initial findings and feedback from schools are extremely positive.
As we approach half term, almost two thirds of primary schools registered for the CBA Programme have already started their CBAs, and over 30,000 pupil assessments have been completed.
CCEA was pleased to introduce two new developments to the CBAs this year:
- Transfer of CBA data into SIMS/Assessment Manager; and
- Diagnostic Class Reports.
In order to best support schools avail of these new facilities CCEA is providing teacher training through face-to-face events and online Collaborate Training. We are approximately half way through the training sessions and feedback from schools is very positive.
As with previous years, there is a wide range of resources available to support schools with CBA, all of which is available here:
The CBA Helpdesk continues to be available throughout the autumn term and is contactable by both telephone 028 9026 1274 and email firstname.lastname@example.org.