Case study · agentic diagnostic system

liteOdyssey

Rare-disease diagnosis with a reasoning process you can inspect.

Project ↗Demo ↗Workbench (request access) ↗Preprint ↗

How it works

  1. 01 / InputPatient phenotype

    HPO-coded features, with or without genetic findings.

  2. 02 / InvestigationPolicy + biomedical tools

    A reasoning model gathers and checks evidence under a written diagnostic policy.

  3. 03 / OutputRanked differential

    Diseases and genes, with the evidence and tool calls behind each rank.

The learning happens in the policy. Expert review gates revisions; model weights stay fixed.

Explore the diagnostic workflow and policy-development loop
A

Diagnose one case

The policy decides which evidence to gather and when.

Patient caseHPO-coded phenotype, with or without genetic findings
Reasoning modelexecutes the diagnostic policy
Ranked differentialdiseases and genes, with the evidence and tool calls behind each rank

Diagnostic policyThe policy is a natural-language document that integrates expert diagnostic reasoning principles, specialized biomedical tools and the rules for using them, organized as an eight-phase workflow. Development updates the policy; model weights stay fixed.natural language · workflow, tools and the rules for using them

Pattern recognitionPhenotype-based candidate genesEvidence triageDisease-level investigationConfidence assessmentCorrective searchReflective adjudicationRanked final output

ToolsThe tools are part of the policy, and each phase specifies which ones it draws on. When a failure traced to the evidence a tool returned, its specification was revised, so the workflow, phase criteria and tools co-evolved.

Main path

dx-monarchphenotype to gene candidates dx-omimdisease and family review

Called when the answer could change the ranking

dx-clingencurated calibration dx-gene-evidenceconstraint tiebreak dx-pubmedtargeted literature
B

How the policy learned

Policy Iteration with Human Feedback

  1. Run the current policy on the 50-case development set
  2. Locate recurrent failures by reasoning stage
  3. Gate revisions clinically
  4. Persist the improved policy

↺ next evaluation round

Model weights stay frozen. The written policy transfers across models.
The harness that automates this loop: PIHF-MCP

Performance

Disease Recall@1, matched baseline vs LiteOdyssey (GPT-5.4)

Matched baselineLiteOdyssey
Public benchmarks1,243 cases · 722 diseases
26.5%
59.3%
LIRICAL
35.1%
58.6%
PhenoPacket Store
22.9%
59.6%
UDN clinical cohort515 patients
16.7%
20.4%

1,193 of the public cases, covering 679 diseases, were never used in policy development. Four reasoning models each improved over their matched baselines (baselines 18.7% to 26.5%). In the UDN cohort, blinded physician review corroborated the gain.

Matched baseline: the same model and clinical features without the policy or tools. Public-benchmark scores use a judge from a different model family than the one evaluated; exact-OMIM sensitivity analyses are in the preprint.

Notes

Contribution. I designed and implemented LiteOdyssey and am first author of its preprint.

Go deeper. liteodyssey.org has the method, the full evidence and a reviewer workspace. The preprint: arXiv 2606.16149.