Plain-English explanations for understanding AI in medicine
You do not need to become an AI expert. You do need enough understanding to know what a system can do, what it can reach, and what questions to ask when it begins to shape a decision.
A benchmark is simply a test used to compare performance. It can tell us whether a system identifies a diagnosis, predicts an outcome or answers a set of questions more accurately than a comparison group under defined conditions.
But a benchmark measures medicine as it is graded, not medicine as it is lived. It does not tell us how much authority a system should have over a patient, which tradeoffs a patient would choose, or who should answer when the system is wrong.
Performance is evidence about capability. It is not, by itself, a grant of authority.
We use the term “AI” for systems that can do very different things. These are not three official technical categories. They are a useful way to think about what a system can do, how much freedom it has to act, and what kind of oversight it needs.
A tool produces something for a person to examine and use.
It might read an image, find a pattern, draft a note, search a large amount of information or help write code. It may even perform a particular task better than a human can.
A system that flags something on an imaging scan for a clinician to review is this kind of AI.
The important point is that the system is still helping someone do the work. A person can examine what it produces, question it and remain responsible for what happens next.
An agent can do more than give you an answer. It can take a goal and carry out a series of actions to try to accomplish it.
It might open files, use software, run code, communicate with other systems or decide which step to take next without a person directing every move.
In a hospital, for example, an agent might read parts of a patient’s record, prepare discharge paperwork and schedule follow-up care.
That changes the questions we need to ask. It is no longer only: Was the answer accurate?
We also need to know:
A tool gives you something to use. An agent can act.
AI is also starting to take part in the research and engineering used to build the systems that come after it.
Instead of simply helping a researcher write a piece of code, an AI system can propose experiments, run them, evaluate the results and help design the next system.
This is already happening. OpenAI has described reaching an “automated research intern” and is working toward increasingly automated AI research. Anthropic has also discussed the possibility that AI could dramatically accelerate the work of top research teams.
There are still humans involved. But the relationship is changing.
When AI helps design, test and improve the systems that follow it, the feedback loop begins to close.
That is different from a medical-imaging tool, and it is different from an agent carrying out a task on a computer.
The more AI contributes to AI research itself, the more important it becomes to ask whether our ability to understand, test and control each new system is keeping pace.
Sources: OpenAI, “Research acceleration: The view inside OpenAI” (September 6, 2026); OpenAI, “Building standards for the next phase of AI” (September 21, 2026); Anthropic, Frontier Safety Roadmap (updated July 2026).
A medical-imaging classifier, an autonomous software agent and a system helping researchers build the next generation of AI may all be called “AI.”
But they do not have the same abilities. They do not create the same risks. And they should not automatically be governed by the same rules.
The more a system can do on its own, and the more it can reach, the stronger the controls around it should be.
You have probably heard a lot lately about artificial general intelligence, and about the risks it could one day bring. Those concerns are real, and I take them seriously.
But hospitals and insurers are already using AI to write notes, score risk, decide who is seen first, and shape what insurance will cover. None of these systems needs to be superintelligent to change what happens to you or to someone you love.
So I think of AI governance as running on two clocks. The frontier clock asks what AI may become. The bedside clock asks what it is already doing, and who answers when something goes wrong. We need both.
Worrying about AGI, ASI, or whatever name comes next should not leave us powerless to change what is already pulling medicine toward automated decisions, until one day the system quietly becomes the authority.
Imagine you type into a chatbot: I have been getting some tightness in my chest. Yesterday it happened carrying groceries. Sometimes it happens when I walk uphill. I have been really stressed lately. When I sit down, it goes away. Then you ask it to clean this up for your doctor.
It might give you: intermittent chest discomfort, possibly stress-related, improving with rest.
What the patient said
What the summary may say
“Intermittent chest discomfort, possibly stress-related, improving with rest.”
What becomes harder to see
And if the chatbot goes further and says “go see a cardiologist,” it is no longer just summarizing. It is giving medical advice.
Give a good chatbot those four sentences and it may well connect the tightness to effort and the relief to rest. That is not the point.
Patients do not usually tell their stories in four sentences. They may talk for twenty minutes. The groceries come up once in the middle of a story about their daughter. They mention the hill, then say maybe it was the heat. They tell you they are under stress and point you toward the explanation they already believe.
Somewhere in all of that may be the detail that should change the next question.
To summarize the conversation, the system has to decide what to keep, what to emphasize and what to leave out. That is already part of clinical reasoning.
When it gets those choices right, that can be enormously helpful. When it gets them wrong, the summary may still look clean, confident and professional, and the clinician may never know that the detail that mattered was left behind.
So my point is not that AI cannot see the pattern. Often it may see it better than we do. My point is that summarizing a medical history is not neutral.
Doctors have a name for deciding on an explanation too early: premature diagnostic closure. When a chatbot tidies up a patient’s messy story, it can make that easier to do, because the doubts and the connections between the details are harder to see.
The summary can sound finished before the clinical thinking is.
Imagine a doctor seeing an alert: 94 percent probability of septic shock. The patient looks stable, and the system gives her no meaningful way to understand where the number came from.
The problem is not simply that the model might be wrong. The problem is that the alert carries weight while giving the clinician little way to question it. That is where help starts to turn into authority.
A proxy is what we measure when the thing we really care about is hard to measure.
The real question in medicine is who needs more help. But an algorithm can be given a different question: who is likely to cost more. Those sound close. They are not the same.
Picture two people leaving a grocery store. One spent a hundred dollars. The other spent twenty, even though she had been hungry for a long time, because twenty was all she had, or the store was two buses away, or nobody had ever helped her get what she needed. If you used their receipts to decide who was hungrier, you would get it exactly backwards.
Healthcare spending works the same way. It reflects how sick someone is, but also whether they had insurance, whether there was a clinic nearby, and who has been able to get care at all. Some people spend less on healthcare not because they are healthier, but because they could never get through the door.
This is not a thought experiment. Researchers have found a widely used healthcare algorithm making exactly this mistake. The book tells what they found.
The algorithm did not decide that cost should stand in for need. People did. So before asking how accurate a system is, it is worth asking: accurate about what?
Before an AI answer reaches a patient, a whole human and physical system has already made it possible.
There are people who label data and screen what models produce. There are chips and the minerals inside them, data centers, and the electricity and water it takes to keep them running and cool.
On a screen, the answer feels weightless. It is not.
The cloud has an address, and communities live next door.
If the cost falls on people who never set foot in a hospital, the red line cannot stop at the hospital door.
A standardized test used to score AI systems. It measures performance under test conditions, not in a real clinic.
Deciding on an explanation too early and stopping the search before other explanations have been properly considered.
Software that drafts the clinical note from a visit.
Artificial general intelligence. A term generally used for AI capable of performing a broad range of cognitive tasks at a high level rather than one narrow task.
In AI research, artificial superintelligence usually refers to a hypothetical system that would outperform humans across nearly every kind of intellectual work. The current federal use of “Super Intelligence” is different; see the September note.
A system whose output can carry significant weight even when the reasoning behind it cannot be meaningfully inspected by the person expected to act on it.
A measurable stand-in used when the thing we actually care about is difficult to measure directly.
A phrase suggesting that a person remains involved in a process. The important question is not simply whether a human is present, but what that person can actually see, question, override and change.