Giving Doctors Back the Hour They Spent Typing
Ambient clinical documentation, imaging triage, ICU demand forecasting and a molecular screening pipeline, built on one HIPAA-compliant data platform for a US academic health system by our AI software development team.
Overview
The client is an academic health system in the US Midwest: four hospitals, about 1,100 beds, a network of outpatient clinics, and a translational research institute that works with pharma partners on early-stage drug programmes. They run Epic. Like most systems their size, they’d tried a handful of AI tools already. Some worked. Most stalled somewhere between the pilot and the second department.
Their CMIO put the real problem better than we could. Clinicians weren’t short of software. They were short of time, and every new tool asked for more of it.
So the brief wasn’t “deploy AI.” It was: take work off clinicians without adding a single new screen they have to babysit. We built four AI services on a shared, HIPAA-compliant data platform, each one landing inside the tools staff already use. It’s the approach we’d take for most healthcare software development work now. Fit the workflow first, then the model.




The Challenge
Physicians were finishing their notes at 10 p.m.
Internal surveys showed outpatient physicians spending close to two hours on documentation for every eight hours of patient time, a lot of it after clinic. “Pajama time,” the staff called it. Burnout scores were climbing and two departments had lost physicians in a year, both citing workload.
Radiology worklists didn’t know which scan was urgent
CT and X-ray studies landed in reading queues in order of arrival. A head CT showing an intracranial bleed could sit behind a dozen routine follow-ups. Overnight, with one radiologist covering three sites, that delay was measured in tens of minutes. In stroke and trauma care, minutes matter a great deal.
ICU beds were managed by phone call
Bed managers learned an ICU admission was coming when a ward nurse called it in, which was usually when the patient was already deteriorating. No forward view meant elective surgeries got cancelled at the last minute and ED patients boarded for hours waiting for a bed.
The research team was screening compounds the slow way
The research institute’s medicinal chemists ran docking simulations on a shared cluster, one target at a time, with queue times of days. They wanted to screen far larger virtual libraries and get to wet-lab testing with better candidates, faster, without buying an entire computational chemistry division.
Our Approach
We spent the first three weeks in clinics, reading rooms and the bed management office, not in a conference room. Shadowing a hospitalist for a full shift tells you more about documentation burden than any requirements workshop.
Two rules came out of that AI consulting phase and held for the whole engagement. First, every output lands inside Epic (a draft note, a worklist flag, a census forecast) through standard FHIR and HL7 interfaces, never in a separate app. Second, the AI suggests and a clinician decides. No model in this build takes a clinical action on its own.
Delivery ran eight months: data platform and governance first, then ambient documentation and imaging triage (the clearest wins for staff), then ICU forecasting and the research pipeline. Every clinical tool went through a silent pilot, running on real data with outputs hidden from clinicians and compared against what actually happened, before anyone saw a single suggestion.
The squad was ten people: a delivery lead, a clinical informaticist, three ML engineers, an LLM engineer, two integration developers, a data engineer and a QA lead. The health system’s clinical AI committee joined sprint reviews every fortnight.
Solution Delivered
Ambient clinical documentation
With the patient’s consent, the physician taps once in the Epic mobile app and the visit is captured. Speech-to-text with speaker separation produces a transcript, and a language model turns it into a structured draft: HPI, exam findings, assessment and plan, plus suggested orders and ICD-10 and CPT codes. The draft drops into the note in Epic within about a minute of the visit ending.
The physician reviews, edits and signs. Nothing files on its own. Audio is deleted after the note is signed, and the model was tuned per specialty because a cardiology note and a dermatology note are honestly different documents. This is the piece clinicians talk about, and it’s built on our generative AI development framework.
Medical imaging triage
A computer vision pipeline reads incoming CT and X-ray studies as they arrive from the PACS. For head CT it looks for intracranial haemorrhage; for chest CT, pulmonary embolism; for X-rays, fractures and pneumothorax. Suspected positives jump to the top of the radiologist’s worklist with a flag and a heatmap showing where the model is looking.
We were careful about scope here. The models prioritise. They don’t diagnose. Every study still gets a full read by a radiologist, and the flag is a reason to read sooner, not a conclusion. Detection models for these findings came from FDA-cleared vendor algorithms where they existed; our work was the orchestration, the integration and the monitoring around them, plus custom models for two site-specific use cases under the health system’s research governance.
Predictive ICU triage and bed forecasting
Every 15 minutes, a model scores each inpatient’s risk of needing ICU care in the next 12 hours, using vitals, labs, nursing assessments and medication data from Epic. High-risk patients surface to the rapid response team with the top contributing factors listed.
The same scores roll up into a 24 and 48-hour census forecast for each ICU. Bed managers see expected admissions and discharges on a capacity dashboard, which lets them hold beds, move elective cases, or call in staff before the crunch rather than during it.
AI-accelerated drug discovery for the research institute
For the research team, we built a virtual screening pipeline. A graph neural network trained on public binding data (ChEMBL, BindingDB) plus the institute’s own assay results predicts binding affinity for candidate compounds against a target protein. Structure predictions feed in where no crystal structure exists. Top-ranked compounds then go through physics-based docking for a second opinion before anything reaches a chemist.
The point isn’t replacing chemists. It’s letting them screen tens of millions of compounds in the time it used to take to screen tens of thousands, and spend wet-lab budget on far better bets.
Inside the Model Layer
One clinical data platform
Epic’s Clarity and Caboodle extracts, real-time HL7 feeds, PACS metadata and research data all land in a single lakehouse on Azure, mapped to the OMOP common data model. Every model reads from the same patient record, defined the same way.
Speech built for clinic rooms
The ambient system runs a medical-vocabulary speech model with speaker diarisation, tuned on de-identified recordings from the health system’s own clinics. General-purpose speech models do surprisingly badly with drug names and accents in a noisy exam room. Ours was benchmarked on that, not on podcasts.
LLMs that stay inside the walls
The note-drafting model runs in the health system’s private Azure tenancy under a BAA. No PHI goes to a public API. Prompts are templated per specialty and per visit type, and outputs are checked against the transcript so the draft can’t include a finding nobody said out loud.
Imaging at the edge of the PACS
A DICOM router sends studies to GPU inference nodes as soon as they’re acquired. Results come back as DICOM structured reports and worklist priority flags, the formats radiology systems already understand.
Monitoring for drift, by site
Model performance is tracked per hospital, per unit and per scanner. When a site swapped a CT scanner mid-project, the haemorrhage model’s calibration shifted measurably within a week. Monitoring caught it before radiologists did.
Clinical Safety and Compliance
In healthcare, the governance is the product. A model clinicians don’t trust is a model they’ll switch off.
HIPAA by design
PHI encrypted at rest and in transit, role-based access, full audit logging, BAAs with every cloud and model provider, and de-identification for anything used in training.
Clinician in the loop, always
Notes are drafts until signed. Imaging flags change reading order, not reads. ICU scores prompt a review, not an admission.
Silent pilots before go-live
Each clinical model ran hidden for four to eight weeks and was compared against real outcomes before clinicians saw it.
Bias and subgroup testing
The ICU model was validated across age, sex, race and payer groups; thresholds were adjusted where performance diverged. Results go to the clinical AI committee every quarter.
Regulatory awareness
Where a function met the definition of Software as a Medical Device, we used FDA-cleared components or kept it within the health system’s research and quality governance. We’d rather say that plainly than overclaim.
Alert fatigue budgets
The rapid response team agreed a maximum number of ICU-risk alerts per shift before go-live. The threshold was tuned to fit it. Otherwise the alerts get ignored, and then they’re worse than useless.
Three Modules Nobody Screenshots
A feedback button that actually goes somewhere
Every AI output in Epic carries a one-click “this was wrong” option with an optional comment. Feedback is triaged weekly by the clinical informaticist and fed into retraining or prompt changes. Clinicians started using it once they saw their complaints fixed within a sprint. Trust, it turns out, is mostly about being listened to.
A consent and audio retention service
Patients can decline recording at check-in or in the room, and the decision is stored against the encounter. Audio lives encrypted for no longer than it takes to sign the note, then it’s deleted, with the deletion logged. Dull to build. Absolutely essential.
A capacity dashboard for bed managers
The ICU forecasts feed a Power BI capacity view showing expected admissions, discharges and staffed beds across all four hospitals, 48 hours ahead. Built on the health system’s existing data analytics stack, so nobody had to learn a new tool at 3 a.m.
Key Features
Ambient visit capture
One-tap recording in Epic mobile, with patient consent.
Specialty-tuned note drafts
Structured HPI, exam, assessment and plan in about a minute.
Order and code suggestions
Suggested orders plus ICD-10 and CPT codes for physician review.
Imaging worklist triage
Suspected bleeds, PEs, fractures and pneumothorax moved to the top.
Visual heatmaps
Shows radiologists where the model is looking.
ICU deterioration score
12-hour ICU risk for every inpatient, refreshed every 15 minutes.
48-hour census forecast
Expected ICU admissions and discharges per unit.
Virtual compound screening
GNN binding-affinity predictions across millions of compounds.
Docking cross-check
Physics-based docking on top-ranked candidates.
Silent pilot mode
Models run hidden against real outcomes before go-live.
Per-site drift monitoring
Tracks performance by hospital, unit and scanner.
HIPAA audit trail
Every access, output and override logged.
Results & Outcomes
Measured six months after go-live across the participating departments.
Documentation time per physician
Down 52 minutes a day on average
64% less
After-hours charting.
81%
Of drafts signed with minor or no edits.
From 38 to 9 minutes overnight
Time to read for flagged critical head CTs overnight.
73%
ICU admissions predicted 12+ hours ahead
29% fewer
Last-minute elective surgery cancellations
From ~40,000 to 25 million+
Compounds screened per target
Delivery
8 months, discovery to full production
The number the CMO kept repeating, though, was a softer one. Physician-reported burnout in the pilot departments dropped for the first time in three years.
Technology Stack
EHR integration
Epic (FHIR R4, HL7 v2, SMART on FHIR), Epic mobile
Data platform
Azure Data Lake, Databricks, OMOP CDM
Speech and language
Medical ASR with diarisation, privately hosted LLM, LangChain
Imaging
DICOM router, MONAI, PyTorch, FDA-cleared vendor models, NVIDIA GPUs
Predictive models
XGBoost, temporal deep learning models, scikit-learn
Drug discovery
PyTorch Geometric, RDKit, AutoDock Vina, structure prediction models
MLOps
MLflow, Azure ML, Evidently for drift
Analytics
Power BI
Security and compliance
Azure (BAA), Key Vault, private endpoints, HIPAA audit logging
The Part People Underestimate: Workflow Beats Accuracy
Our first ICU model was more accurate than the one we shipped. It was also useless, because it fired about 40 alerts per shift and the rapid response team would have muted it inside a week.
That’s the lesson healthcare AI keeps teaching. A model that’s 2% more accurate but lives in a separate tab loses to one that’s slightly less accurate and shows up in the right place at the right moment. Most of our time went into integration, threshold tuning with clinicians, and making sure each output appeared exactly where a nurse or physician already looks.
It’s also why we’d push health systems toward AI integration into the EHR rather than a collection of standalone tools. Every extra login is a reason not to use it.
What’s Queued Next
Inbox management.
An AI agent that drafts replies to patient portal messages and routes refill requests, for physician review.
A patient-facing assistant
for appointment prep, pre-visit questionnaires and post-discharge follow-up, built on our chatbot development stack.
Sepsis early warning
reusing the ICU deterioration pipeline with a separate validated model.
Prior authorisation drafting
pulling the clinical evidence payers ask for straight from the chart.
Patient app integration
bringing visit summaries into the health system’s healthcare app.
Each one plugs into the data platform that already exists. No new foundations needed.
Conclusion
The health system didn’t want AI for its own sake. It wanted its clinicians to go home on time and its sickest patients to be seen sooner.
Eight months, one squad, four services, one data platform. What made it work: shadowing clinicians before writing any code, keeping every output inside Epic, and refusing to put any model in front of staff until it had proved itself in a silent pilot.
If your organisation is weighing where AI fits in care delivery or research, our AI and ML solutions team can help you start in the right place. Or hire dedicated developers as a full squad that stays from discovery through go-live.
Want AI that fits your clinical workflow instead of adding to it?
Let’s talk about where it would make the biggest difference for your teams.
Schedule a Free Consultation
