DrugDoseUnit conversionRoute
Verify
Every verbal order gets ten seconds to be read back. Drug, dose, unit and route are compared, so 0.1 mg and 100 mcg agree while rocuronium and succinylcholine do not.
EveryVerbalOrderintheOR,VerifiedOutLoud
Surgr listens to the operating room, checks each drug order against its read-back in real time, tracks the WHO Surgical Safety Checklist as the team speaks it, and speaks up the moment a loop does not close.
Built on AssemblyAI. Every integration verified against the live services.
StreamingSpeaker LabelsLLM GatewayVoice Agent
// Capabilities
Listens,verifies,speaksup
Sign InTime OutSign Out21 items
Checklist
The WHO Surgical Safety Checklist ticks itself as the team speaks. A phase that ends with gaps, or an incision called without a Time Out, raises an alert.
Voice AgentUnder 1 sEHR exportTimestamps
Speak & record
Alerts are spoken into the room word for word through AssemblyAI's Voice Agent, and every order, item and event lands in an audit-ready record linked to the audio.
// Why it matters
Theloopbreaksquietly
Closed-loop communication is the standard: an order is spoken, repeated back, and confirmed. In a busy room the repeat is skipped, a dose is misheard, a checklist item is assumed. Nothing beeps. Surgr is the thing that notices.
1 in 20
perioperative medication administrations involve an error or adverse drug event
Nanji et al., Anesthesiology, 2016
11% → 7%
major complications after adopting the WHO Surgical Safety Checklist; deaths fell from 1.5% to 0.8%
Haynes et al., NEJM, 2009
~2 in 3
sentinel events involve a breakdown in team communication
The Joint Commission
310M+
major operations are performed worldwide every year
Weiser et al., 2016
// How it works
Listen,understand,intervene,document
01
Listen
Room audio streams to AssemblyAI at 16 kHz. Medical mode gets drug names and doses right; speaker labels tell surgeon, anesthesiologist and nurse apart on one microphone.
02
Understand
A deterministic classifier parses orders, read-backs and checklist statements in under a millisecond. Utterances it cannot settle go to the LLM Gateway under a strict JSON schema.
03
Intervene
Every order gets ten seconds to be read back. Wrong dose, wrong drug, wrong route or silence, and Surgr speaks the alert into the room through the Voice Agent API.
04
Document
An operative communication record where every medication, checklist item and safety event links to the exact second in the audio, with JSON export and a print view.
Watch the loop close
Live enginesame code as the cockpit
Listening to the room…
Verbal orders appear here the moment they are spoken.
Surgr says
Silent until a read-back is wrong, missing, or a checklist item is skipped.
Room mic→Streaming STT→Rules + LLM Gateway→Read-back state machine→Voice Agent · EHR record
// The cockpit
Onescreenforthewholeroom.Nobodyhastolookatit.
The display exists for the circulating nurse, the debrief and the audit. The alerts are spoken. The record writes itself.
Transcript by role
Surgeon, anesthesiologist and nurse in their own colours, every order and read-back tagged as it lands.
Order board
A ten-second countdown per order. A mismatch goes red and stays red until a correct read-back arrives.
Checklist that ticks itself
Sign In, Time Out and Sign Out complete from speech; a phase that ends with gaps raises a spoken alert.
// Built on AssemblyAI
Fourproducts,onesafetyloop
Every integration was verified against the live services: a streaming session with medical mode and speaker labels, a Voice Agent reply spoken verbatim, and JSON-schema outputs from the LLM Gateway.
Streaming STT v3
universal-3-6-pro · medical-v1
Sub-second transcripts from the room microphone. Medical mode formats drug names and doses; keyterms boost 47 operating-room drugs and checklist phrases.
Speaker labels
true diarization
One microphone, three roles. Speaker embeddings separate the team in real time, so a read-back from the wrong person is caught.
LLM Gateway
JSON-schema outputs
Structured classification for the utterances rules cannot settle, and the narrative sections of the operative record. Rate budget tracked from the gateway's headers.
Voice Agent API
reply.create
Surgr's voice in the room. Alerts are spoken verbatim through a persistent agent session, with browser speech as a fallback.
// Try it
Ninewaystobreaktheloop
The simulator plays scripted operating-room conversations through the same pipeline as the live microphone, so you can trigger a wrong dose without an operating room. Turn your sound on.
Correct closed loop
Surgeon orders, anesthesia reads back the same drug, dose and route.
Expect: Order confirmed within seconds. No alert.
Dose mismatch
Anesthesia reads back a tenth of the ordered fentanyl dose, then corrects it.
Expect: Critical mismatch alert is spoken into the room; the corrected read-back clears it.
No read-back
An antibiotic order goes unanswered while the team talks about instruments.
Expect: Timeout alert after 10 seconds, then a late confirmation closes the loop.
Equivalent units
Order in milligrams, read-back in micrograms.
Expect: 0.1 mg and 100 mcg are recognised as the same dose. Confirmed, no alert.
Wrong drug read back
Surgeon orders rocuronium, anesthesia confirms succinylcholine.
Expect: Critical drug mismatch alert.
WHO Time Out
A complete Time Out led by the scrub nurse.
Expect: All eight Time Out items tick off as they are spoken.
Rushed Time Out
The team skips introductions, antibiotics, sterility and imaging and calls for the knife.
Expect: Checklist alert lists the unconfirmed items before incision.
No Time Out at all
The surgeon calls for the knife without any Time Out.
Expect: Critical alert: incision called without a Time Out.
Full case (3 min)
Sign In, induction orders, Time Out, intra-op orders with one mismatch, and Sign Out.
Expect: All three checklist phases complete, four confirmed orders, one mismatch alert that is corrected.
Launch app Or start a live session in Chrome and say “give fifty milligrams of propofol”.
// Details
Straightanswers
Is Surgr a medical device?
No. It is a hackathon prototype built to show what real-time closed-loop verification can look like. It is not cleared for clinical use and does not replace clinical judgement or institutional protocols.
What is real and what is simulated?
Every AssemblyAI integration is live: streaming transcription with medical mode and speaker labels, the LLM Gateway, and the Voice Agent that speaks the alerts. The simulator injects scripted text into the same pipeline so a wrong dose can be triggered without an operating room, and the demo above runs the real engine in your browser.
Does it work without an API key?
The simulator, the rule-based read-back engine, the checklist and the local report all run without any cloud call, and the voice falls back to the browser's speech synthesis. The live microphone, spoken alerts through the Voice Agent and the LLM narrative need a key, which stays on the server.
How quickly does an alert fire?
The rule classifier decides in under a millisecond. A mismatched read-back is spoken within about a second; an unanswered order is called out after the ten-second read-back window.
Where does the audio go?
The browser streams directly to AssemblyAI using a short-lived token minted by the server; the API key never reaches the page. Streaming PII redaction can be enabled for the exported record.
Why is the LLM used so sparingly?
Hackathon accounts get two gateway requests per minute. The deterministic classifier does the real-time work, and the LLM is reserved for drug-related utterances the rules cannot resolve and for the end-of-case narrative. Rate limits are read from the gateway's headers and honoured automatically.