It’s 2025, and AI has officially checked into every part of healthcare—from the front desk scheduler to the operating room. It reminds your patient to refill their medication and helps flag cancer on a slide. It doesn’t complain, take breaks or call in sick. It’s fast, tireless and… maybe not enough? Because for all its progress, we’re still seeing AI systems stumble in real clinical settings. It might not handle edge cases as effectively as we’d like. It misses context that experienced clinicians could catch before their first cup of coffee. And no, this isn’t about fearing automation or fighting the march of progress. I’ve spent years building and implementing these systems, and I know firsthand that AI can be an incredibly effective collaborator… but only when it’s built to work with people. with It might be true that in some industries, you can plug AI in and get pretty incredible outcomes, with a few duds here and there. But healthcare doesn’t have much margin for error. What matters is good judgment and context, and that takes human input. Let’s talk about what it means to build AI infrastructure that puts humans in the loop by design. AI Isn’t Failing Because It’s Wrong—It Just Needs Help Starting with data, because it’s where your AI will: AI models want structure. Healthcare gives them chaos. Any clinician is familiar with the digital mess behind the scenes, ranging from poorly scanned PDFs of half-filled forms to different EHRs calling the same condition by different names. An algorithm might see an “abnormal” lab and sound the alarm, not realizing the patient just had surgery last week. Or maybe there’s a coffee stain over a 0. According to ECRI’s 2025 safety report, AI-driven clinical errors, ranging from misdiagnoses to inappropriate treatment recommendations, are now considered one of the top patient risks. And to be clear, the models aren’t malicious. But letting them fly blind and build a foundation on human error is asking for trouble. We need to provide a scaffolding for uncertainty and ambiguity. And coffee stains. ECRI’s 2025 safety report ECRI’s 2025 safety report This is a huge context problem, and wouldn’t you know it: humans are very good at context. AI sees the signal. Humans know the story. The best results happen when AI proposes and humans decide. False Positives Create More Work Let’s follow that same example further. What happens when AI goes off on its own? A radiology AI cries wolf and flags nodules of interest in every single scan, including the ones that are just shadows or scar tissue. Trust falls off, teams start to ignore it. Then usage drops. At some point, the system gets shelved. So much for a decision support system. decision support system This is a classic tech-in-medicine problem. A 2023 NPJ Digital Medicine study flagged this exact pattern. AI without oversight created so many false positives that it added to staff burden, increasing alert fatigue and driving its users to disengage. 2023 NPJ Digital Medicine study 2023 NPJ Digital Medicine study We’ve seen in the field again and again that a model can’t triage itself, and it certainly won’t know whether its prediction is helpful or harmful. That’s the role of a human reviewer: someone needs to be partnered and trained to verify and decide on what action needs to happen next. It’s an important layer of checks and balances that should drive every AI-assisted medical decision. Black-Box Models Don’t Fly in Healthcare This shouldn’t come as a surprise: Even when the model is right, doctors need to know why. If your AI tells a provider that patient X is at high risk of deterioration, they’re going to ask: based on what evidence? Is it recent vitals? A pattern? If the model can’t explain itself, or if it dumps raw scores with no context, it’s not providing meaningful or actionable output. A 2024 American Medical Association survey found that nearly half of physicians rank transparency and human oversight as top criteria for trusting an AI tool. Accuracy is one thing, but someone has to be accountable, and it will always be the practicing physician. 2024 American Medical Association survey 2024 American Medical Association survey That’s where human-in-the-loop design really counts. Reviewers vet the alert, while clinical leads can escalate or override. Everyone sees what changed and why, and knows who to turn to for questions. Introducing HITL turns your AI from a black box into a system with feedback and contextual memory. Regulations Are Catching Up The FDA, WHO, and pretty much every major regulatory body now agrees: AI in healthcare needs oversight, even in day-to-day ops. The FDA’s 2025 guidance emphasizes traceability and documented human oversight at each major decision. The World Health Organization warns against deploying AI without rigorous evaluation and expert supervision. A very recent U.S. Senate investigation found that fully autonomous AI systems in insurance led to care denials 16 times higher than expected, calling for immediate policy intervention. I won’t mince words: If you’re not building with humans in the loop, you’re inviting much more scrutiny in an already audit-heavy industry. But Humans Are Slow... Right? Not If You Design for It There’s that persistent myth that adding people into an AI system slows things down. But that’s only true if the system wasn’t designed to support collaboration in the first place. A few ways to do it without bogging down your ops: Set confidence thresholds. Only flag low-confidence cases, where human input adds value.
Build effective tooling. Reviewers need intuitive and actionable dashboards, not a 50-column CSV.
Track outcomes. Log every review, override and resolution. You’ll be ready for the next audit and simultaneously build a goldmine of data for model improvement. Set confidence thresholds. Only flag low-confidence cases, where human input adds value. Build effective tooling. Reviewers need intuitive and actionable dashboards, not a 50-column CSV. Track outcomes. Log every review, override and resolution. You’ll be ready for the next audit and simultaneously build a goldmine of data for model improvement. It’s easy to start thinking it’s bureaucratic overhead, but a well-designed HITL system becomes a learning system. The more it runs, the better it gets. If You Want to Scale AI in Healthcare, Start With Humans Let’s recap. We don’t need to guess where things go wrong. AI models are left in a vacuum, trusted to succeed with poor inputs, then start eroding trust through noisy outputs. Clinicians tune out and the systems get shut off. But healthcare has never been about hitting 99% accuracy in a sandbox. We focus on making the right call when everything’s messy to begin with. That’s what makes human judgment irreplaceable, and why we have the responsibility to design with the people who still carry the weight of patient care. By all means, let AI handle the routine stuff. Automate the admin, surface insights, reduce friction where it makes sense. But when it comes to escalation and practicing good judgment, build systems that know when to hand it back to a human. In the future, we might start realizing the smartest thing an AI can do is know when to ask for help.

AI in Healthcare Works—But Only When Humans Stay in the Loop

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

5 Emerging Technologies That will Improve Your Operational Efficiency in Healthcare

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

5 Emerging Technologies That will Improve Your Operational Efficiency in Healthcare

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps