Clinical Decision Support AI: Sepsis Early Warning and Readmission Risk Across 12 Hospitals

A High-Performing Health System With a Preventable Death Problem

The hospital system ranked in the top quartile nationally on most quality metrics. Their sepsis bundle compliance rate was 78%, well above the national average of 64%. And yet sepsis remained the leading cause of in-hospital mortality across their network, accounting for 34% of all inpatient deaths annually. The Chief Medical Officer had a precise description of the problem: clinicians were following the protocol correctly once they identified sepsis, but they were identifying it too late.

The average time from sepsis onset to clinical recognition in their system was 6.8 hours. Published clinical research indicates that each hour of delayed treatment increases mortality risk by approximately 7%. At 6.8 hours average delay, even excellent bundle compliance was producing avoidable deaths. The system needed earlier detection, not better protocols.

They had attempted to solve this before. Three years prior, they had deployed a commercial sepsis prediction tool from a major EHR vendor. The tool had achieved 71% sensitivity in post-hoc validation but had a specificity of only 42% in live deployment, meaning it was generating so many false alerts that clinical staff had disabled it in 9 of their 12 hospitals within 6 months. Alert fatigue had destroyed clinician trust in automated clinical AI across the entire system.

The second challenge was readmission risk. The system was facing $18.4M in CMS readmission penalties annually across three high-priority conditions: heart failure, pneumonia, and total hip and knee replacements. Their existing readmission risk tool, a logistic regression model built in 2019, had a C-statistic of 0.64 against a clinical benchmark of 0.75 for meaningful intervention utility.

Alert Fatigue and Clinician Trust: The Real Deployment Problem

Most clinical AI deployments focus on model performance metrics in isolation: sensitivity, specificity, AUC, C-statistic. These matter. But the dominant cause of clinical AI failure is not model performance. It is clinician adoption failure caused by alert fatigue, workflow disruption, and the perception that AI recommendations do not add value beyond what an experienced clinician can identify independently.

The failed commercial sepsis tool had a 42% specificity. That means 58% of its alerts were false alarms. In a busy ICU or medical-surgical unit, a nursing team receiving 12 sepsis alerts per shift, of which 7 are false positives, will disable the system. This is not an unreasonable clinical decision. It is a rational response to a tool that wastes more clinical attention than it saves.

To rebuild clinician trust, we established three non-negotiable requirements before any model development:

• Alert specificity must exceed 80% in live deployment, not in retrospective validation. This was a substantially higher bar than the commercial tool and required a different approach to threshold optimization.
• Every alert must include actionable context, not just a risk score. A clinician receiving an alert that says "Sepsis Risk: High" has no basis to determine whether to act. An alert that says "Rising lactate trend over 4 hours, elevated heart rate pattern, recent urine output decline" gives the clinician the information they need to make a judgment.
• The system must integrate with existing clinical workflow, not add steps. If responding to an alert requires a clinician to leave their primary work interface and log into a separate system, adoption will fail. Every alert needed to surface within the EHR workflow the clinicians were already using.

Two Separate Models, One Workflow, Built for Clinical Reality

Sepsis Early Warning: Temporal Deep Learning with Hard Specificity Constraint

We built the sepsis early warning model as a multi-task temporal neural network trained on 4.2 years of EHR data covering 380,000 inpatient encounters across all 12 hospitals. The model architecture used a bidirectional LSTM processing 22 continuous vital sign and lab value streams in real time, with feature extraction designed to identify deterioration trajectories rather than point-in-time values.

The key architectural decision was training the model against a hard specificity constraint of 82% at the target sensitivity threshold. Most clinical models are trained to maximize AUC, which allows the model to optimize the tradeoff between sensitivity and specificity freely. We specified that we would accept lower sensitivity in exchange for the specificity level required to prevent alert fatigue. The final deployed model achieved 83% sensitivity and 84% specificity at the alert threshold, against a benchmark of the failed commercial tool at 71% sensitivity and 42% specificity.

The "explainability layer" was as important as the model itself. For each alert, the system generated a structured clinical narrative using the top contributing features to the deterioration trajectory. These narratives were reviewed for clinical accuracy by the system's sepsis clinical champions before deployment and refined through three rounds of feedback. The final narratives were written at a level appropriate for both attending physicians and bedside nurses.

Readmission Risk: Gradient Boosted Model with Discharge-Trigger Integration

The readmission risk model was built as a gradient boosted ensemble targeting the three CMS penalty conditions. Rather than a single generalized readmission risk score, we built condition-specific models for heart failure, pneumonia, and orthopedic surgery, each with features calibrated to the clinical risk factors specific to that condition.

The integration decision was to trigger the readmission risk score at two points in the care workflow: 48 hours before anticipated discharge, when care planning interventions were still possible, and at the point of discharge order entry, when the score would influence discharge instructions and follow-up scheduling. This dual-trigger design was developed in collaboration with the system's care management and social work teams, who provided input on which interventions were feasible at each stage.

The final C-statistic for the condition-specific models was 0.79 for heart failure, 0.77 for pneumonia, and 0.76 for orthopedics, all above the 0.75 clinical utility benchmark. More importantly, the models were built to produce actionable risk drivers, not just scores, so that care managers could prioritize specific interventions for each high-risk patient.

Change Management: Clinical Champions and Staged Rollout

We identified a clinical champion at each of the 12 hospitals before deployment began. Champions were senior nurses or attending physicians with credibility among their peers and direct involvement in the model validation process. Their early participation in validating alert outputs and providing feedback on clinical narratives meant they arrived at the rollout as advocates, not skeptics.

The rollout was staged: three pilot hospitals in weeks 14 to 17, with full deployment paused until we had adoption data from the pilots. The pilot data showed 79% alert response rate (alert viewed and action documented within 2 hours) in the first week, rising to 91% by week 4 as clinicians built familiarity with the system. The full network rollout proceeded in weeks 17 to 20, with each subsequent hospital cohort briefed on the pilot outcomes before go-live.