Case Study — Customer Zero

We built a clinical AI platform.
Then we built the hospital to break it.

CareCompile is a live HL7-native clinical intelligence platform. MediFlow is the synthetic hospital it is tested against — every night, on real infrastructure, with zero real patients. This is what that loop looks like in numbers.

369,402
HL7 messages processed
2,234
Synthetic patients
64,396
Lab results stored
22,391
AI analysis jobs run

The problem every clinical AI team has

You cannot develop against real patients. Real HL7 feeds mean PHI, BAAs, IRB questions, and a hospital partner willing to let you learn on their data — which no early team has and no responsible team wants. The common workaround is a folder of stale sample messages, and it produces software that works in the demo and breaks on the first real ADT quirk.

CareCompile took the other path: build a synthetic hospital first, and make passing through it the price of every release.

How the loop runs

Continuously
MediFlow streams admissions, discharges, lab panels, clinical notes, telemetry, and billing into CareCompile through the same three production channels a hospital would use — HL7 bridge, FHIR API, EKG API. 3,500+ messages in a typical week.
Every night, 2:00
The knowledge index rebuilds from the curated clinical knowledge tree.
Every night, 5:00
Golden-set evaluation: known-answer clinical cases — hyperkalemia, hypoglycemia, ACS, TTP, and a normal panel that must NOT alarm — run through the live AI routing. Structural validity, clinical keywords, and risk level are asserted. A failure emails the team before breakfast.
On demand
Crisis drills (STEMI, sepsis cascade, code blue), load runs, and failure-injection tests: kill a listener mid-batch, verify dead-letter capture and clean replay with no duplicates.
Weekly
An automated engineering review reports pass rates, queue health, and drift — the system reports to its operators, not the other way around.

What it caught — a real week

In its first 48 hours of operation, the nightly evaluation caught three distinct model-layer failure modes — none of which had produced a visible error before:

Reasoning-budget truncation

A cloud reasoning model silently exhausted its token budget mid-analysis and returned empty content. Fixed with automatic budget-doubling retry. Found by a failed hyperkalemia case, not by a clinician.

Peak-hour timeout

Provider latency at 5 a.m. exceeded the client timeout — responses were arriving and being thrown away. Fixed the same morning. Found by three failed cases with identical signatures.

Unparseable output under fallback

When the primary model pool rate-limited, the fallback model answered in a format the parser rejected. The eval distinguished this infrastructure failure from a clinical failure automatically — because valid-but-wrong medicine is never retried, only re-examined.

Every one of these would have surfaced eventually. The point is where they surfaced: on synthetic patients, at night, in an email — not in an interpretation a clinician was relying on.

Why this matters if you're evaluating either product

If you're evaluating CareCompile: this is the verification story. The platform is exercised by hundreds of synthetic encounters weekly, regression-tested nightly against known answers, and drilled on failure recovery — with the evidence in queryable tables, not slideware.

If you're evaluating MediFlow: this is the realism proof. The synthetic hospital is not a demo generator — it is the daily test harness of a production clinical AI platform. Its scenarios carry declared expected outcomes precisely because a real system's correctness is asserted against them every night.

"We built the hospital to test the AI, and the AI to justify the hospital — and every night they make each other better."— CareCompile Engineering

Run your worst cutover scenario against it

A 30-minute live session: pick a scenario, watch the full loop — generation, delivery, AI analysis, assertion — on real infrastructure.

Request a Pilot Explore MediFlow