Why We Publish Our AI Scoring Criteria
If you are choosing an English test for hiring, admissions, or training, one question comes up quickly: can a machine be trusted to score speaking and writing? Multiple-choice answers can be marked against a key, but open-ended answers need judgement, and judgement is where an unexplained system can go wrong without anyone noticing.
Our answer starts with publishing what the AI is asked to judge, and how its scores become a level. This article sets out what we publish, why the testing standards ask for it, and what a buyer can and cannot check with it.
QUICK ANSWER
We publish the criteria our AI models use: five for speaking (fluency, coherence, grammar, vocabulary, pronunciation) and four for writing (grammar, vocabulary, content, organisation). Each criterion is scored from 0 to 100, the scores are averaged with equal weight, and the level comes from a fixed table. Publishing shows what is judged; it is not by itself proof of fairness.
What we publish
The "How we score" page on our site sets out the method. In short:
| Part of the test | How it is scored |
|---|---|
| Grammar, vocabulary, reading, listening | Multiple-choice, matched against a verified answer key. No AI. |
| Speaking | AI language models (Google Gemini, Anthropic Claude) score five criteria: fluency, coherence, grammar, vocabulary, pronunciation. The recorded audio is evaluated directly. |
| Writing | The same models score four criteria: grammar, vocabulary, content, organisation. |
| Score to level | Each criterion is scored 0 to 100, averaged with equal weight, and placed on a fixed table: below 35% A1, 35 to 49% A2, 50 to 68% B1, 69 to 78% B2, 79 to 94% C1, 95% and above C2. |
The AI model returns a score for each criterion. It does not choose the level; the level is calculated in code. For the full walk-through, see inside our CEFR scoring model.
What the testing standards say
The Standards for Educational and Psychological Testing, published jointly by AERA, APA and NCME and available open access, are a widely used reference for test quality. Two of them apply directly here:
- Standard 4.18: procedures for scoring and, where relevant, scoring criteria "should be presented by the test developer with sufficient detail and clarity to maximize the accuracy of scoring". The comment adds that this is especially important for extended responses such as essays.
- Standard 7.1: the rationale for a test, its recommended uses and the information needed to interpret scores "should be documented".
Publishing the criteria and the score-to-level rule is how we apply those two standards to the AI-scored part of the test.
Other providers publish criteria too
Published criteria are common among established tests. IELTS, for example, publishes the four criteria used for Writing (task achievement or task response, coherence and cohesion, lexical resource, grammatical range and accuracy) and for Speaking (fluency and coherence, lexical resource, grammatical range and accuracy, pronunciation), and states that the criteria are weighted equally (IELTS scoring in detail). What differs between tests is who applies the criteria: trained examiners, AI models, or both.
What a buyer can do with published criteria
| What you want to do | What the published method gives you |
|---|---|
| Check what is being measured | The list of criteria for speaking and writing |
| Explain a result to a candidate | Which criteria the result is built from, and the fixed level table |
| Compare with your own requirements | A CEFR level you can set against a role or course requirement |
| Re-check multiple-choice marking | A deterministic key: the same answers always give the same score |
What published criteria do not show
Publishing criteria has real limits, and naming them is part of the job:
- Criteria names are not the full rubric. They tell you what is judged, not every detail of how the model weighs a particular answer.
- Publishing is not proof of fairness. Knowing the criteria makes it possible to ask precise questions about bias. Getting answers requires evidence, such as comparing scores across groups of candidates.
- Human review sits with the question bank. In the published method, items that need human judgement are flagged for editorial review; speaking and writing answers are scored by the AI models against the criteria above.
Who stands behind the method
International English Test is listed as an Associate Member on the ALTE associate members page. ALTE is the Association of Language Testers in Europe. We describe that status as associate membership, not accreditation.
For what makes a result hold up when someone challenges it, see what makes an English test result defensible. If you are considering the test for an organisation, see English assessment tests for companies. To try the AI-scored part yourself, see the speaking test.
Key points
- We publish the AI criteria: five for speaking, four for writing.
- We also publish how criterion scores become a level: 0 to 100 per criterion, equal weights, a fixed threshold table.
- The Standards for Educational and Psychological Testing ask test developers to present scoring criteria clearly (Standard 4.18) and to document how scores should be interpreted (Standard 7.1).
- Published criteria show what is judged; they do not on their own prove fairness.
Frequently Asked Questions
International English Test Editorial Team
ALTE Associate Member · UK English assessment provider · Est. 2023
Found this helpful? Share it:
