Research behind DSI v1
From decision-space collapse to measurement integrity.
This research programme is built around two linked questions. The first: can an AI response narrow the set of options visible to a user without becoming obviously wrong? The second: how do we know the instrument measuring that phenomenon still means the same thing when its own population, reference, sensors or authority change? The original Decision-Space Collapse research addressed the first question. Developing DSI as a reproducible measurement and assurance architecture exposed the second, which is now an active and open line of work, not a completed one.
Research publications · archived on Zenodo
Latest research publications.
Which options stay visible? Decision-Space Collapse and preference narrowing
Research note · synthesis · no new experiment
Published 7 October 2026 · DOI 10.5281/zenodo.23224092
Decision-Space Collapse in Advisory Language Models
Configured trajectory omission · framing sensitivity · recovery
Published 8 June 2026 · expanded v1.2 published 4 October 2026
Original v1.1: OSF DOI 10.17605/OSF.IO/KW25A
R-EGA: a governed intervention did not beat a one-line prompt
Negative result · analysis plan frozen before unblinding · closed 2026-08-30
Published 23 September 2026 · DOI 10.5281/zenodo.22919859
The ruler is part of the result
Methods paper · constructed demonstrations
Published 23 September 2026 · DOI 10.5281/zenodo.22919868
The research story so far
Observe, measure, challenge, govern.
The original research investigated trajectory omission, framing sensitivity and recovery in genuinely multi-path advisory prompts — the ways an answer can read well while quietly narrowing what the user gets to weigh. This produced the preprint and the public replication materials.
The work then moved from describing the phenomenon to defining a repeatable measurement: configured expected maps, surfaced and omitted evidence, coverage accounting, recovery, and provenance designed to make reported measurements reproducible and bounded.
Developing and challenging the instrument exposed cases where a legitimate evaluator change could alter evidence strength without necessarily moving a headline metric; where population and admission rules could affect what was actually being measured; and where an authority change could matter to the meaning of a measurement even though downstream scoring logic was unchanged. These are findings from instrument development and challenge, not universal empirical laws.
That led to the current research direction: separately governing population authority, reference authority, sensor authority, measurement identity and cross-version comparability. The research question is no longer only what did the model omit? but also what measurement produced that conclusion, and are two such measurements authorised to be compared?
Measurement engineering
When the instrument changes, the number may not tell you.
DSI is a research measurement and assurance architecture for governed structural comparison. In building it, we encountered a broader measurement-engineering problem: changes to admission, reference construction, sensors, or authority can alter what a metric means without necessarily altering the metric value.
The general contribution is a claim-relative architecture for behavioural measurement in which population authority, reference authority, sensor authority, measurement identity and cross-version comparability are separately governed.
- Population authority — what enters the measured population.
- Reference authority — what expected/reference structure defines the measurement.
- Sensor authority — what evidence is permitted to count.
- Measurement identity — which governed instrument produced the result.
- Cross-version comparability — whether two measurements are authorised to support a comparative interpretation.
This describes the research architecture and direction of the work, not a claim that every layer is present in the private-evaluation DSI Audit v0.2.1 build or that the behavioural instrument has been independently human-validated.
The method
Compare, report, fingerprint.
DSI Audit compares an AI's advisory response against a configured map of reasonable option paths for the domain, reports which were surfaced and which were missing, and records the result as reproducible, fingerprinted evidence. The measurement is configured-path visibility — defined by a named domain configuration, classifier, and scorer — not a judgement of advice quality or safety. The same response audited under the same configuration yields the same numbers.
Phase I · public empirical foundation
The original empirical foundation.
DSI originated from research into decision-space collapse — the tendency of advisory language models to narrow the visible set of options presented to users. That research produced a preprint, replication materials, and a public evaluation repository. It remains the public empirical foundation of the programme.
Decision-Space Collapse in Advisory Language Models
Measuring Trajectory Omission, Framing Sensitivity, and Recovery Through Decision-Space Integrity.
Andrew J. Cousins
The preprint introduces the decision-space collapse framing and the DSI measurement framework. It reports configured expected-path visibility, framing sensitivity, and recovery analyses across advisory model outputs.
This work measures visibility of configured expected paths in model outputs. It does not measure advice quality, factual correctness, user outcomes, or regulatory compliance.
OSF · DOI 10.17605/OSF.IO/KW25A
Latest evaluation package
Evaluation Package v1.2 — validation and transparency update.
Same findings, better evidence package. v1.2 introduces no new findings, no new model evaluations, and no new headline claims; it improves transparency, validation, provenance, and reproducibility.
This release adds:
- Validation documentation and reviewer guidance
- Scientific-scope clarification and provenance reporting
- A second-judge classifier-reliability note — an independent LLM-reviewer cross-check, reported as a limitation, not human validation
The core Decision-Space Collapse findings, evaluation corpus, evaluated models, and primary conclusions remain unchanged.
Claim boundary
What this research does — and does not — claim.
This work measures visibility of configured expected paths in model outputs. It does not measure advice quality, factual correctness, user outcomes, or regulatory compliance.
Research timeline
How the work has unfolded.
This is the programme record, not the product. Work described here — including the wider target architecture and the authority and controlled-execution strand — is preserved in the governed repositories and excluded from DSI v1. Nothing on this page is a v1 capability claim.
2026
-
May–June 2026
Decision-space collapse framing, empirical study, preprint and public replication package.
-
June 2026
DSI Audit v0.2.1 made available for private evaluation; Evaluation Package v1.2 improved validation and transparency without changing findings.
-
July–August 2026
Internal challenge and measurement-engineering work expanded the programme from behavioural omission measurement into instrument integrity, measurement authority and comparability.
-
23 September 2026
Two research publications archived on Zenodo: R-EGA, a negative result (DOI 10.5281/zenodo.22919859), and The Ruler Is Part of the Result, a methods paper (DOI 10.5281/zenodo.22919868). A revision notice was added to the historical preprint.
-
4 October 2026
The expanded version of the original preprint archived on Zenodo as v1.2 (DOI 10.5281/zenodo.23145725). The original v1.1 study is preserved unchanged and was not rescored.
-
7 October 2026
A research note archived on Zenodo: Decision-Space Collapse and Preference Narrowing in Personalized Language Models (DOI 10.5281/zenodo.23224092). It is a synthesis with no new experiment and reports no new data.
-
Current
Independent human validation remains outstanding; semantic measurement, expected-map validity and cross-version measurement governance remain active research questions.
Research and replication status
What's available, and what's pending.
- Public: preprint, replication package and published evaluation materials
- 6,480-output empirical study completed (part of the public replication package)
- DSI Audit v0.2.1 available for private evaluation (verified to build and run from a clean install)
- Evaluation Package v1.2 — validation and transparency release published
- Published 23 September 2026 and citable: R-EGA (DOI 10.5281/zenodo.22919859) and The Ruler Is Part of the Result (DOI 10.5281/zenodo.22919868). Citable is not the same as independently rerunnable: the raw generations, experiment code and held-out materials behind them are withheld.
- Archived 4 October 2026 and citable: the expanded preprint v1.2 (DOI 10.5281/zenodo.23145725), alongside the original v1.1 (DOI 10.17605/OSF.IO/KW25A), which is preserved unchanged and was not rescored.
- Published 7 October 2026 and citable: the research note Decision-Space Collapse and Preference Narrowing in Personalized Language Models (DOI 10.5281/zenodo.23224092). It is a synthesis and reports no new data; the Decision-Space Collapse figures it discusses come from the original study and have not been rescored.
- Internal and unpublished: subsequent challenge, engineering and measurement-integrity studies, including internally frozen reproduction and intervention-recovery reports
- Independent human validation has not yet been conducted. Internal annotation, adjudication and calibration informed instrument development, but the lexical instruments remain self-validated. These activities do not establish independent agreement or accuracy.
Replication
What you can reproduce, and what is held back.
The public replication repository is the place to scrutinise the method. It carries the materials needed to reproduce the reported measurements — and we are explicit about what is not published.
- Public replication repository
- Prompt matrix
- Expected-map artifacts
- Reproduction instructions
- Private product code
- Unpublished research
- Proprietary datasets
Evidence status — honest
What is established, and what is not.
The phenomenon and the audit/recovery measurement have been reproduced internally and frozen as internal reports. Internal only — not yet externally replicated.
Independent human validation has not yet been conducted. Internal annotation, adjudication and calibration informed instrument development, but the lexical instruments remain self-validated. These activities do not establish independent agreement or accuracy.
Every audit is bound to fingerprints and version identifiers, so a reviewer can reproduce and bound any number the product reports.
We would rather state this plainly than overclaim. DSI Audit is available for private evaluation; the DSI method and research record are open to external review, and we welcome scrutiny of the method and the numbers.
Research FAQ
Common questions, answered plainly.
What is decision-space collapse?
Decision-space collapse is a failure mode in which an advisory language model answers a genuinely multi-path question by making some reasonable option paths visible while leaving others out. The answer can read well — and even be factually fine — while quietly narrowing the set of options the user gets to weigh.
What is DSI?
Decision-Space Integrity (DSI) is the measurement and assurance architecture for comparing a governed reference structure against what an AI output actually made visible, and for governing that comparison so it keeps meaning the same thing over time. It is not a product or a package. DSI Audit is the current local implementation of it.
How is this different from existing AI evaluation?
Most AI evaluation scores the answer that was produced — its quality, helpfulness, or safety. DSI asks a different question: of the reasonable option paths a good answer could have kept in view, which are actually visible in this response? It measures configured expected-path visibility and omission, which answer-level scores are not designed to surface. DSI is meant to complement those methods, not replace them.
Does DSI measure correctness?
No. DSI does not measure advice quality, factual correctness, user outcomes, or regulatory compliance. It measures the visibility of configured expected paths in a supplied response.
Does DSI certify AI systems?
No. DSI does not certify AI systems. It can support governance documentation by providing decision-space evidence, but it issues no certification of governance, safety, or compliance.
Does DSI guarantee safety?
No. DSI is not a safety guarantee and not a safety review. It reports configured expected-path visibility — one input a reviewer might consider, not an assurance of safe outcomes.
Can DSI Audit tell users what decision to make?
No. DSI Audit never writes the advice and never recommends a decision. It audits a response your system produced and, where configured, reports which missing paths could be made visible — leaving the generation to your system and the decision to the user.
Author
Who is behind this work?
Intellectual property
Applications filed, not granted.
The work is the subject of patent applications filed in the UK, subject to prosecution — not yet granted. Nothing here should be read as a grant or a defined claim scope.
For reviewers
We'd value your challenge.
If you evaluate advisory AI, audit its outputs, or research evaluation methodology, we'd value your challenge. The product runs locally with a short setup, the numbers are reproducible from the evidence bundles, and the limitations above are stated up front.