Explainer Education, Teaching & Learning Sciences · Educational Measurement and Assessment · Issue 2026-W46

Before ranking schools, decide what assessment data should change

A new Northern Ireland policy paper raises a wider question: do assessment data support learners or describe school performance? The distinction changes when information is needed and how it should be interpreted. We offer a practical test of purpose and an example showing why a high final score cannot, by itself, describe how much learning occurred.

By ARPS Editorial Team Last reviewed 11 November 2026 About 5 min read (estimate)

Source date 2026-11-09

An unwritten workbook and varied stepping stones leading to a lit doorway; a generated metaphor for learning progress.

A new Northern Ireland policy paper raises a wider question: do assessment data support learners or describe school performance? The distinction changes when information is needed and how it should be interpreted. We offer a practical test of purpose and an example showing why a high final score cannot, by itself, describe how much learning occurred.

A policy development, rather than an effectiveness trial

On 9 November 2026, the Organisation for Economic Co-operation and Development published a paper on literacy, numeracy and digital skills for ages 14–16 in Northern Ireland. It draws on five comparison education systems. Its assessment chapter considers how information about starting points and progress could guide support. [1] This is a review of policies and contexts, not a randomised trial establishing that a particular test will improve attainment.

For school leaders and assessment managers, the opportunity is to ask a question before buying another tool: if the results arrived tomorrow, who would make a different decision, with what authority and resources? Without a specific answer, collecting more data may increase workload without establishing a useful route into teaching. That is our management inference, not a causal finding attributed to the paper.

Write the intended decision before designing the report

A school team can give each report a short statement of purpose: we will use this information to change this part of teaching for these learners. If the information arrives after a teaching unit has finished, it cannot help staff adapt that same unit while it is being taught. It may still support planning for a later cohort. That is a different purpose and needs a different interpretation.

We propose separating three questions at a data meeting. Which skill needs support? What resources would make that support possible? What can reasonably be said about the school's performance? Their answers do not have to become a single indicator. Comparing institutions requires attention to different circumstances. A teacher needs information that can inform instruction, rather than simply the school's position in a ranking.

Section 5.3.2 reveals a real tension: contextualised indicators need baseline data, yet Northern Ireland has stated that statutory assessment data will not be used to evaluate schools. [1] We therefore do not propose automatically converting assessment for support into public rankings. Purpose, authority and governance must be settled first; keeping the two uses separate may be the appropriate decision.

The team should also specify who can access individual results, how errors are corrected and how a result is explained to a family without labelling the learner. These are design questions, not evidence that particular schools are failing. If the assessment does not support reliable interpretation at the level of a skill, a general total should not be converted into a detailed diagnosis through intuition.

Two descriptions hidden inside a final score

Consider a hypothetical illustration. One group starts with 60% of answers correct and finishes with 70%; another starts at 80% and finishes at 85%. The arithmetic difference is ten percentage points for the first group and five for the second. The second group has the higher final result, while the first has the larger raw change. A final score alone cannot show both descriptions.

The example does not establish that the first school is better. We assume only that the percentages were calculated in the same way. Interpreting a change in correct answers as a comparable amount of learning requires suitable measurement evidence and equivalent instruments. Test difficulty, group composition and administration conditions can alter the meaning. A before-and-after difference does not become a school effect merely because both measurements are available.

Displaying the starting result, final result and raw difference separately can make interpretation more disciplined. Add a statement of what the measurement supports and what it does not. This is less convenient than one prominent number, but it prevents a simple display from concealing a change in the question being answered. Leaders can use it to request an explanation before attaching rewards or penalties to the result.

Connect information to a change that staff can inspect

In a hypothetical school trial, teachers of one subject select a specific skill about which the assessment offers reasonable evidence. They review examples of responses, identify a possible explanation for an error and change one teaching activity. They then retain later work to examine whether learners can use the skill in a different task. Repeating the original question could instead reward remembering its answer.

The trial should begin only where time for teaching and support is available. Its first implementation signal is whether the team can connect a specific error to a clear instructional adjustment, rather than whether the average immediately rises. If teachers cannot agree on what responses mean, reviewing the instrument and its interpretation takes priority over expanding data collection. Needs beyond their competence remain subject to the school's established support procedures.

This is a limited implementation comparison. It does not isolate the causes of subsequent change or establish the success of a new curriculum. Nor should a learner be blamed for an intervention whose necessary conditions were unavailable. Removing a duplicate report might release time for examining student work, but the team must also identify the purpose that would be lost by removing it. Less collection is useful only when the remaining information can still answer the decisions that matter.

What can travel to another setting?

Questions about purpose, timing and interpretation can travel, subject to local review of curriculum, language, privacy and accountability requirements. The paper does not assess Saudi schools or turn policy options for Northern Ireland into requirements for the Gulf. International comparisons help test assumptions; they do not replace evidence that an instrument is appropriate for its intended learners.

Start with one report that teachers actually receive and connect each field to a feasible decision. If a field changes neither support, teaching nor a legitimate accountability question, reconsider its necessity. Which information does your school currently collect only to receive it after the opportunity to use it has passed?

For continuing professional learning, explore ARPS communities. The public page describes free ordinary membership and saving articles, without requiring paid membership for that benefit.

References

  1. Building core literacy, numeracy and digital skills in the first phase of upper secondary education in Northern Ireland — OECD · 2026-11-09 · DOI 10.1787/1ee825b5-en

References open in a new tab.

Discuss this with practitioners

Create a free account to save articles and join the Trainers community discussion.

Create a free account

Discussion 0 published comments

No published comments yet. Be the first to share your experience.