Computer Based Assessment Numeracy Assessment Error

CCEA identified an issue with the Northern Ireland Numeracy Assessment (NINA) outcomes. The issue was raised with the supplier of the assessment, who confirmed a computer error on 14 December 2016. The error caused the calculation of incorrect scores for a number of children taking the assessment in the autumn term of this academic year. 

The error has been rectified and the correct scores have been released.

Children take this assessment using computers in schools, their responses are marked electronically and calculations are carried out to produce a number of scores and reports. The reports are made available to schools soon after the assessments are completed. Outcomes are not collated by CCEA.

A different provider is responsible for the Northern Ireland Literacy Assessment (NILA). The literacy assessment is unaffected.

Schools elect to use this assessment service. They are taken by children in years 4- 7 throughout the autumn term. 267 primary schools elected to use the NINA assessment in 2016 and 24,076 children took the assessment.

Justin Edwards, CCEA CEO commented:

“Computer based assessments, like NINA, are designed to help inform teaching and learning in schools. Using computers for assessment, rather than pencil and paper, they provide immediate feedback to teachers. Not all schools choose to use the service.

For this assessment service we use a third party provider. The assessment provider holds all data relating to the assessments securely. During the autumn term CCEA is provided with information, by the provider, about how the assessment system is performing. However, when the number of schools completing the assessment was sufficiently large, CCEA noted that the outcomes provided demonstrated lower than expected scores. We took immediate action.

CCEA asked for a detailed review of the assessments and systems. We were provided assurances that the assessments were performing as designed. Not satisfied, we continued to raise queries with the provider, until it was reported that they had identified an error within their systems. It was confirmed that the error was in the software used to calculate final scores for a number of children taking the assessment.

The error has been rectified and the correct scores have now been generated. These are now available to schools. In all cases the correct scores will show higher performance than previously reported.

Children do not need to re-take assessments for correct scores to be calculated. All schools that elected to take the assessments are being contacted and supported at this time.

CCEA’s priority has been the children that took the assessments and to ensure that their scores are correct. CCEA instructed the provider to take urgent action to rectify the error and asked for assurances that newly calculated outcomes are accurate. These assurances have been given to CCEA by the company involved.

CCEA is disappointed that the quality of service has fallen short of expectations. We apologise unreservedly for any inconvenience or distress this error has caused.”

Additional Information

  • NINA (Northern Ireland Numeracy Assessment) and NILA (Northern Ireland Literacy Assessment) are computer based assessments (CBA) that are taken by pupils in Years 4-7 in primary schools at the beginning of the school year.
  • The assessments are not mandatory. 267 schools elected to take the NINA assessment in 2016.
  • CBAs are assessments designed to help teachers identify strengths and areas for improvement and to help inform teaching and learning throughout the school year.
  • The CBA in numeracy is developed and administered by a third party under contract to CCEA. The NINA assessment is provided by Ardluce, a sub contractor of Hachette. CCEA’s contract is with Hachette.
  • 24,076 pupils have taken the NINA assessment to date (the assessment remained open until 16 December 2016)
  • The number of pupils affected is 16,808.
  • NINA outcomes calculated after 15.12.16 are accurate.
  • The Northern Ireland Literacy Assessment (NILA) is unaffected by this issue.
  • A CBA helpdesk is available for teachers on 028 90 261274.

Note to Editors

Media enquiries to Ruth Hobson on (028) 9026 1200 ext. 2216, Mobile 07718 424 373 , Email rhobson@ccea.org.uk