How can I inspect and challenge a consequential decision an automated system made or assisted?
An automated system denies a benefits application. You ask why, and receive only "model output" — no reasoning you can inspect or contest.
The situation
You applied for a benefit. An automated system scored the application and produced a denial. When you asked for the basis, the only answer available was the bare output itself, with no accompanying reasoning, evidence, or path to challenge it.
What the system knows
- The application data submitted.
- A model score and a denial output.
- No attached reasoning trail, evidence citation, or confidence/limitation statement.
What the system wants to do
Treat the denial as final and binding with no further inspection available to you.
Who has authority?
The system may well have authority to score an application. Whether it has authority to make that score final and binding without an inspectable basis is a separate, harder question — and on what's shown here, that authority is not established.
- Principal / subject
- You, the applicant
- Provenance
- NOT ESTABLISHED — no reasoning trail or evidence citation was attached to the output
- Evidence
- NOT ESTABLISHED — as available to you
- Standing
- Score-generation: current. Final binding determination without inspectable basis: NOT ESTABLISHED
- Proposed action
- Finalize the denial as binding
- Requested consequence
- Binding adverse decision, no challenge path presented
What Genesis checks
- Whether provenance, evidence and reasoning inputs remain inspectable before a decision becomes binding.
- Whether "the model said so" is being treated as sufficient on its own.
- Whether a real challenge path exists.
This is a design principle this project applies to its own systems, not a claim about what the law currently requires of any particular system. See Rights in a Machine-Speed World — Sixth Amendment for the civic question this mirrors, and its own limits.
What happens next
Inspect why
See the reasoning
A bare score is not, by itself, an inspectable basis. What's missing is named explicitly — reasoning trail, evidence citation, limitation statement — rather than the gap being smoothed over. The decision stays open, not silently upheld and not silently reversed.
Explore the architecture
Genesis components involved: