Ivan Kairatov brings a biopharma lens to clinician-AI collaboration, blending R&D rigor with practical know‑how from tech-enabled trials. In recent studies, chatbots not only matched but sometimes outperformed clinicians on management reasoning tasks, especially when teams used deliberate workflows like parallel analysis. With evidence from cohorts of 46 clinicians per arm, five diverse cases, and a 70‑doctor randomized trial, Ivan unpacks why “tool to teammate” is more than a slogan—and how to translate it into safer, faster, and more humane care.
In complex management decisions like stopping blood thinners before surgery or handling prior drug reactions, where do you see AI adding the most value, and where should human judgment clearly take the lead? Can you share a case where this balance made a measurable difference?
AI shines at surfacing the option space and reminding us of edge-case pitfalls—it’s like having a meticulous colleague who never forgets a contraindication list. In the studies with five de‑identified cases, AI consistently ticked more rubric items than clinicians limited to internet references, which tells me it’s catching steps we often skip under pressure. Human judgment must lead on goals-of-care tradeoffs, risk tolerance, and how past adverse reactions actually felt to the patient—that lived nuance isn’t in structured fields. In one perioperative case review, the clinician chose a conservative anticoagulant hold; the AI flagged timing alternatives and a pathway for bridging. The team blended both: they kept the human’s plan but used the AI’s checklist to confirm post‑op monitoring and follow‑up scheduling. The measurable difference was a clean handoff with no missed labs and on-time follow‑up, which mirrored how clinicians paired with a chatbot performed as well as the chatbot alone across the five cases.
When diagnosing is “pinpointing the destination” and management is “choosing the route,” how should teams operationalize that metaphor? What concrete steps, checklists, or handoffs help clinicians translate an AI’s plan into safe, patient-centered actions?
I translate “route choice” into a three‑column board: options, constraints, and commitments. Start with an AI‑generated list of actions, then run a micro‑checklist—patient preferences, follow‑up reliability, and system scheduling capacity—before you commit. Build a templated handoff that captures the top three AI rationale bullets, the clinician’s final pick, and the one contingency plan if conditions change. In trials with five varied cases, what separated stronger plans was explicit articulation of “why now vs later,” and you can encode that in a two‑minute sign‑out ritual. Finally, close the loop: schedule follow‑ups in the same session and attach the AI’s summary to the EHR note so the next clinician sees both plan and reasoning.
In studies where chatbots matched or outperformed clinicians on management reasoning, what specific rubric elements did AI consistently hit? Which items did humans do better on, and how would you train teams to close those gaps?
The AI reliably hit completeness items on the rubric—enumerating differential actions, noting surveillance intervals, and specifying which imaging or labs confirm trajectory. It also did well at identifying potential contraindications and sequencing steps, which is why it outperformed doctors who only had internet references. Humans were stronger at tailoring to context—recognizing a patient’s fear of procedures, anticipated no‑shows, and local bottlenecks in scheduling. To close gaps, I’d run weekly drills on five‑case bundles where teams must state: the AI’s top three actions, the human’s adjustments based on context, and a reconciliation paragraph. Score each plan against the rubric and require a brief reflection: which item the AI saved you from missing, and which human factor you added.
Imagine an incidental upper-lobe lung mass with high metastatic risk. How would you structure a side-by-side plan from clinician and AI, including biopsy timing, imaging choices, and patient preferences? What metrics would you track to judge the plan’s quality?
I’d set up a dual track. The AI plan typically proposes immediate tissue diagnosis or advanced imaging; so it might outline same‑admission biopsy vs short‑interval outpatient biopsy, and a PET‑CT before intervention. The clinician plan would integrate preferences—say the patient is averse to invasive procedures—and local realities, like whether follow‑up scheduling is reliable. We then reconcile: if follow‑up reliability is poor, prioritize inpatient pathways; if strong, sequence high‑yield imaging first, biopsy second. Quality metrics would include time from detection to definitive action, documentation of the patient’s stated preference in the note, and on‑time completion across a 30‑day window. Borrowing from the five‑case approach, I’d also score the plan against a rubric: did we list alternatives, justify timing, and name contingency triggers?
Context often drives management: patient follow-up reliability, health system scheduling, and aversion to invasive procedures. How can AI capture these nuances in real time, and what data inputs, prompts, or workflow checkpoints make those factors explicit?
Start with structured context prompts: “Summarize prior no‑show history, current transportation options, and expressed aversion to procedures in one paragraph.” Feed the AI EHR signals—missed appointments in the last year and documented preference notes—so it can surface risk of loss to follow‑up. Add a checkpoint where the clinician must confirm or correct that context before signing orders; a 30‑second attestation anchors the plan in reality. In the studies with 46 clinicians supported by a chatbot, explicit reasoning improved performance; we operationalize that by forcing the model to state how context changes sequencing. Finally, embed a soft stop: if follow‑up reliability is low, the AI should auto‑propose inpatient completion of critical steps.
A trial found parallel analysis—clinician and AI working independently, then reconciling—outperforms sequential review. How would you implement that on a busy service? What staffing, EHR integration, and time-boxing strategies keep it fast while improving accuracy?
I’d mirror the 70‑doctor randomized trial: require independent drafts first, then a concise reconciliation. In practice, the clinician spends two minutes outlining their plan; in parallel, the AI generates its plan. The system auto‑produces a delta summary—agreements and disagreements—in under 30 seconds, which the clinician can accept or edit. A scribe or nurse coordinator pastes the final plan into the EHR, tagged as “parallel reviewed.” For staffing, keep it within existing roles; for time‑boxing, cap the reconciliation at 90 seconds and use templated fields drawn from those five‑case study rubrics to accelerate scoring of completeness and risks.
When AI evaluates a case after the clinician, it may anchor on the human’s opinion. How do you guard against that bias? What prompt designs, blind reads, or adjudication steps have you found to preserve independent reasoning?
Force a true blind read: hide the clinician’s narrative until the AI locks its reasoning, mirroring the parallel approach that beat sequential workflows in the 70‑doctor trial. Use prompts that prohibit adoption of prior conclusions—“List your plan before reading any human notes; you will be scored on divergence where justified.” After both drafts exist, generate a structured comparison that highlights rationale differences, not just end decisions. If divergence is high on safety‑critical items, route to a quick adjudication huddle with a second clinician, but time‑cap it to two minutes. This preserves independence yet keeps throughput viable.
If a chatbot alone can outperform clinicians using only internet references, how should training and CME evolve? What concrete curricula, simulation drills, or assessment benchmarks would help clinicians learn to interrogate and augment AI outputs?
CME should pivot to “AI‑augmented reasoning” with hands‑on drills. Start with modules based on five‑case sets: clinicians submit a plan, then critique the AI’s version and write a reconciliation paragraph. Benchmark performance against the same rubric used in the studies, tracking how often they catch omissions or unsafe shortcuts. Add simulations where the AI is intentionally wrong on one critical element—trainees must spot it within two minutes. Finally, require annual proficiency where clinicians demonstrate they can work in parallel mode and reach parity with the AI‑alone baseline observed in those 46‑clinician cohorts.
For high-stakes choices—anticoagulant holds, biopsy readiness, escalation vs watchful waiting—what is your step-by-step protocol to validate an AI recommendation before acting? Which safety checks and thresholds trigger human-only overrides?
Step 1: Independent human plan first, even if it’s just bullet points. Step 2: AI generates its plan with explicit contraindication checks and sequencing. Step 3: Reconcile with a focus on red‑flag deltas—procedure timing, monitoring intensity, and follow‑up certainty. Step 4: Apply a safety gate: if the AI’s plan shortens monitoring or defers a critical test without strong rationale, default to the human plan. Step 5: Document the patient’s preference and capacity for follow‑up. Overrides trigger when the AI conflicts with a known adverse reaction history, when follow‑up reliability is low and the AI pushes outpatient, or when there is high divergence without adequate justification per rubric standards.
What failure modes worry you most: over-trust in AI, underuse in time-pressured settings, or poor data quality? Can you share an example where each risk nearly derailed care, and how you built guardrails to prevent recurrence?
Over‑trust: a team almost accepted an AI‑suggested outpatient plan despite a spotty follow‑up history; we caught it in reconciliation and added a rule—if prior no‑shows exceed a set count, inpatient completion is preferred. Underuse: during a surge, clinicians skipped AI review; we instituted a 90‑second parallel requirement modeled on the 70‑doctor study and embedded it in the discharge workflow. Poor data quality: a missing allergy note led the AI to propose a drug class the patient had reacted to; now the system prompts a quick allergy verification before plan generation. Across all three, the fix was the same pattern—parallel analysis, short hard stops, and rubric‑aligned checks inspired by how the five‑case evaluations were scored.
How should hospitals measure ROI beyond accuracy—think throughput, patient-reported outcomes, equity, and clinician burnout? What baseline metrics would you collect now, and what improvements over 3, 6, and 12 months would signal true impact?
Start with a baseline month and collect time to definitive action, on‑time follow‑up rates, and patient‑reported clarity of the plan on discharge. Track equity by comparing these metrics across demographic groups to ensure improvements don’t cluster only in well‑resourced patients. Add clinician metrics—after‑hours documentation time and perceived cognitive load—because the shift from “tool” to “teammate” should ease burnout. At 3 months, look for faster reconciliation cycles and fewer missed steps per rubric itemization; at 6 months, improved on‑time follow‑up; at 12 months, sustained gains in patient clarity scores and narrowing of equity gaps. Use the five‑case rubric quarterly to audit plan completeness, mirroring the study design for continuity.
Patients are told not to skip clinicians despite strong AI performance. How do you counsel patients to use AI safely at home? What guidance, red flags, and shared-decision tools help them separate credible insights from misleading ones?
I tell patients AI is a smart map, not the driver. Use it to learn the terrain—what tests exist, what side effects to watch—but always bring its suggestions to your clinician. Red flags include definitive instructions without context, advice that ignores your preferences or prior reactions, and plans that don’t specify follow‑up. A simple at‑home checklist helps: ask the AI to list two alternative options, the reasons for and against each, and what symptoms should trigger urgent care. That structure mirrors the rubrics used in research, making it easier for clinicians to validate or correct when you come in.
What is your forecast for AI-assisted clinical decision making?
Over the next few years, we’ll see parallel analysis become the default, because the 70‑doctor randomized data showed it reduces anchoring and elevates both parties. Expect every EHR to auto‑generate side‑by‑side plans and a delta summary within seconds, with clinicians finalizing reconciliation in under two minutes. Training will normalize rubric‑based scoring, using five‑case bundles to maintain a shared standard of completeness and safety. Patients will engage more—but through guided tools that emphasize preferences and follow‑up reliability—so AI recommends not just the “right” action, but the right action for this person in this system. The future is not AI replacing clinicians; it’s teams where 46 humans plus one model work as a single, reliable unit.
