Five Things Every Surgeon Should Understand About Generative AI

Large language models (LLMs) such as ChatGPT, Gemini and Claude are now widely available, and many surgeons already use them to draft letters, summarise papers or prepare teaching material. Their fluency can make them seem more capable than they are. The points below summarise what the published literature currently suggests, and where uncertainty remains.

1. They predict text; they do not look facts up

LLMs are built on the transformer architecture described by Vaswani and colleagues in 2017 [1]. They are trained to predict the most likely next word given what has come before. This makes them very good at producing plausible, well-structured prose, but plausibility is not the same as accuracy. Unless a tool is explicitly connected to a verified source, its output reflects statistical patterns in its training data rather than a check against the current literature.

2. Confident errors ("hallucinations") are a known feature

A substantial body of research describes the tendency of language models to generate content that is fluent but unsupported or false, including fabricated citations [2]. In a clinical context, this may take the form of an invented reference, an incorrect drug dose or an outdated classification. Every factual statement, and every reference, should be checked against a primary source before it is relied on.

3. Performance on exams is not performance in practice

Models have achieved high scores on medical licensing-style questions [3], and reviews describe a wide range of possible applications in medicine [4]. However, multiple-choice benchmarks do not capture the uncertainty, incomplete information and accountability of real clinical work. Evidence from prospective evaluation in surgical settings remains limited, and published reporting standards for clinical AI studies exist precisely because early results often do not generalise [5].

4. Patient data needs particular care

Entering identifiable patient information into a public AI service may amount to sharing personal data with a third party. UK GDPR and the Information Commissioner's Office guidance on AI and data protection apply [6]. Before using any tool with patient information, surgeons should check whether it has been approved by their organisation's information governance team.

5. Professional responsibility stays with the clinician

The General Medical Council's Good medical practice (2024) sets out that doctors remain responsible for the decisions they make and the information they provide [7]. AI output can be a useful first draft, but it should be treated as assistive rather than authoritative, with a human reviewing and taking ownership of anything that reaches a patient, a colleague or a publication.

Summary

Generative AI may save time on routine writing and help with learning, provided its limits are understood. Treating every output as an unverified draft, protecting patient data and keeping clear human oversight are sensible starting points while the evidence develops.