AI for Medical Diagnosis: How It Works and Where It Is Used

AI for medical diagnosis covers software that detects, classifies or flags disease from medical data: images, physiological signals, lab and pathology results, or a combination. Some of these tools are FDA-authorized medical devices in routine use. Many more are research models that perform well on benchmarks but have not been tested in practice. This guide explains the main types of diagnostic AI, how regulated tools differ from research models, what evidence to look for and how these tools fit into clinical workflows.

How diagnostic AI works

Most diagnostic AI is trained on large sets of labeled examples, such as images marked by specialists as showing or not showing a condition. The model learns patterns that link the input to the label, then applies them to new cases. The output is usually a score, a flag, a highlighted region or a recommendation, which a clinician reviews. A small number of tools are authorized to give a result without a specialist reading the image first.

Types of diagnostic AI

Type	Input data	Typical tasks	Example
Imaging	X-ray, CT, MRI, ultrasound, retinal images	Detect findings, triage urgent cases, measure structures	IDx-DR for diabetic retinopathy screening; Viz.ai ContaCT for suspected large vessel occlusion stroke
Digital pathology	Scanned slide images	Highlight areas likely to contain cancer	Paige Prostate
Physiological signals	ECG, EEG and other waveforms	Detect rhythm abnormalities and other patterns	ECG analysis software on the FDA's AI-enabled device list
Clinical data and risk prediction	EHR data, labs, vital signs	Flag patients at risk of deterioration or a specific condition	Predictive decision support inside the EHR
Multimodal and language models	Text, images and records together	Answer clinical questions, summarize findings	Mostly research, such as Google's Med-PaLM 2

Radiology accounts for the largest share of AI devices on the FDA's AI-enabled medical device list. We cover imaging in more depth in our guide to AI in medical imaging.

Three examples from the FDA

• IDx-DR (2018): FDA described it as the first device authorized to provide a screening decision for diabetic retinopathy without a clinician also interpreting the image, making it usable in primary care (FDA announcement).

• Viz.ai ContaCT (2018): analyzes CT images for signs of a suspected large vessel blockage and alerts a specialist. The radiologist still reviews the full scan (FDA announcement).

• Paige Prostate (2021): highlights the area of a biopsy image most likely to contain cancer. In FDA's study, pathologists' detection on individual slide images rose from 89.5% to 96.8% with the software (FDA announcement).

Regulated tools versus research models

Software intended to diagnose or detect disease is generally a medical device. In the US it needs FDA marketing authorization through 510(k), De Novo or premarket approval, and the authorization covers a specific intended use. Software that analyzes medical images or signals stays under FDA oversight even when it only supports a clinician, according to FDA's Clinical Decision Support software guidance.

Research models, including large language models that score well on medical exams, have not been through that process. Benchmark results show what a model can do on a fixed test set. They do not show how it performs on your patients, in your workflow, or over time. Our article on Med-PaLM 2 shows how wide that gap can be.

Evidence to look for

Question	Why it matters
Is the tool authorized for this exact use and population?	Authorization covers a defined indication, such as a specific scan type and patient group
What were sensitivity and specificity, and on which data?	Results depend on the population, scanners and disease prevalence in the study
Was it tested on data from sites other than where it was built?	External validation is a better guide to real-world performance
How does performance vary by subgroup?	Models can perform worse for groups under-represented in training data
Did it change clinical outcomes or workflow?	Accuracy alone does not prove patients benefit

Fitting diagnostic AI into the workflow

A diagnostic tool is only useful if its output reaches the right person at the right moment. Imaging AI usually connects to the PACS and reading worklist. Risk prediction connects to the EHR, often through FHIR APIs or decision support standards such as CDS Hooks. Plan for local validation before go-live, clear rules on who acts on an alert, and ongoing monitoring for drift. For predictive tools in certified health IT, ONC's HTI-1 rule requires source attributes such as intended use and performance information to be available to users (ONC HTI-1 final rule). Our article on AI in clinical decision support covers the integration side.

Teams building diagnostic software can see how we approach regulated products on our SaMD solutions page.

Conclusion

AI for medical diagnosis is already in clinical use, mostly in imaging, pathology and signal analysis, and mostly as support for clinicians. The difference between an authorized tool and a promising research model is evidence: a defined intended use, external validation, subgroup results and proof that it works in a real workflow. Ask for that evidence before you adopt or build.

If you are planning a diagnostic AI product or integration, book a consultation with Ekipa's HealthTech engineers.

Frequently asked questions

How is AI used in medical diagnosis?

It analyzes images, slides, physiological signals and clinical data to detect findings, triage urgent cases and flag patients at risk, usually for a clinician to review.

Is diagnostic AI regulated by the FDA?

Software intended to diagnose or detect disease is generally a medical device and needs FDA authorization for its specific intended use. Software that analyzes medical images or signals stays under FDA oversight.

Can AI diagnose without a doctor?

A few tools are authorized to do so for narrow tasks. IDx-DR, for example, gives a diabetic retinopathy screening result without a clinician interpreting the image. Most diagnostic AI supports clinicians.

How accurate is AI for medical diagnosis?

It depends on the tool, task and population. Look for sensitivity, specificity and subgroup results from external validation, and check that the study population matches your patients.

Are large language models used for diagnosis?

Mostly in research. Some models score well on medical exam questions, but benchmark results do not prove safe performance in clinical practice, and they are not authorized as diagnostic devices.