Introduction
Effective communication is paramount in surgical practice, with patient history-taking representing a foundational step that shapes diagnosis and management. Traditionally, surgical education has leveraged experiential learning and standardized patient simulations to cultivate these skills. However, recent advancements in artificial intelligence (AI), specifically deep language learning models (DLM) such as ChatGPT developed by OpenAI, have introduced innovative possibilities to enhance training modalities.
This article critically reviews a randomized controlled trial investigating the incorporation of a DLM as a simulated patient (SP) in surgical history-taking training for senior undergraduate medical students. Complementary studies on simulation-based communication skills in consent acquisition and core clinical skill development are also appraised to contextualize findings within broader pedagogic trends.
Study Background and Clinical Context
Strong surgeon–patient communication correlates with improved diagnostic accuracy, patient adherence, and outcomes. However, the traditional apprenticeship model, relying on unstructured clinical exposure, may inadequately ensure uniform competence among trainees.
Simulation-based medical training (SBMT), involving standardized patients and role-play, has emerged as a validated approach to bridge this gap. Currently, AI-driven conversational agents offer scalable, interactive, and potentially unlimited scenarios for trainees to practice communication. Nevertheless, robust evidence regarding their efficacy in medical education remains limited.
The burden of suboptimal communication in surgery is reflected in informed consent challenges, diagnostic inaccuracies, and patient dissatisfaction, underscoring the need for innovative educational interventions that can supplement clinical learning.
Study Design
The primary study (McCarrick et al., 2025) employed a single-center randomized controlled trial (RCT) design to evaluate the impact of DLM simulation on surgical history-taking skills. Ninety senior medical students enrolled in a core surgical module were randomized via cluster sampling into control and intervention groups (n=45 each).
– Control group: Received standard experiential learning comprising routine clinical rotations and usual instructional modalities.
– Intervention group: In addition to standard learning, participated in three structured sessions utilizing ChatGPT as a simulated patient through text interactions. These interactions were later reviewed by tutors for feedback.
All participants completed Objective Structured Clinical Examinations (OSCEs) with live human SPs at baseline and post-intervention phases. Assessments were conducted by blinded evaluators to minimize bias. Post-intervention, intervention group students completed anonymous surveys evaluating their communication confidence and perceptions about the DLM simulation.
Two additional related RCTs were considered for comprehensive understanding:
1. Impact of simulation training on communication skills and informed consent (McCarrick et al., 2025) evaluated communication simulation training (CST) with tutor-led roleplay and peer discussion to improve informed consent competencies.
2. Impact of Simulation Training on Core Skill Competency (McCarrick et al., 2024) assessed the effects of multi-week SBMT on surgical history-taking and physical examination proficiency using actor-based simulations and clinical assessments.
Key Findings
1. Deep Language Model-Based Simulation Trial (McCarrick et al., 2025)
– The content generated during DLM-based interactions was consistently appropriate for surgical history-taking scenarios.
– Baseline OSCE scores were comparable between groups, confirming balanced initial competencies.
– Post-intervention, the intervention group exhibited a statistically significant improvement in OSCE scores (p < 0.001), with an educational effect size (Cohen's d) of 0.37 compared to 0.19 in controls.
– Survey data revealed that 57% of intervention group students reported increased confidence in their communication skills. A substantial majority (72%) appreciated the richness and detail of histories obtained via DLM, and 95% expressed willingness to use the tool again.
2. Communication Skills Training for Informed Consent (McCarrick et al., 2025)
– Among 122 students, 61 received CST. CST participants demonstrated significant post-intervention improvements in consent communication measured by University College Dublin OSCE rubric and Global Communication Rating Scale (GCRS).
– Mean grades improved from C to B+ in the CST group, with an effect size of 0.79, while control group performance remained static.
– Enhancements were noted across communication domains including initiation, verbal articulation, session structuring, and information relay.
– Self-confidence in obtaining consent rose markedly from 11 to 62 students post-training.
3. Simulation Training on Core Skill Competency (McCarrick et al., 2024)
– One hundred senior medical students participated, half of whom completed a 10-week SBMT program involving actor-based simulations of acute abdominal pain cases.
– Baseline Southampton Medical Assessment Tool (SMAT) scores were equivalent in intervention and control groups.
– After training, the intervention group showed significantly higher SMAT scores (p = 0.0006).
– Student feedback was overwhelmingly positive: 94% recognized benefits, 85% reported increased confidence in history-taking, and 78% noted improved abdominal examination skills.
| Study | Intervention | Outcome Measures | Effect Size | Key Results |
|---|---|---|---|---|
| DLM Simulation (McCarrick et al., 2025) | 3 sessions with ChatGPT SP + standard learning | OSCE history-taking scores, student surveys | 0.37 (DLM) vs 0.19 (control) | Significant improvement, increased confidence, high acceptability |
| Communication Skills Training (McCarrick et al., 2025) | Tutor-led roleplay, peer discussion | OSCE consent communication, GCRS, confidence surveys | 0.79 | Marked score increases, confidence, skill gains |
| Simulation Training (McCarrick et al., 2024) | 10-week actor-based SBMT program | SMAT clinical competency scores, feedback survey | Significant improvement (p=0.0006) | Increased competence and confidence |
Expert Commentary
The advent of DLMs like ChatGPT offers unprecedented versatility in simulating realistic patient interactions without logistical constraints inherent in actor-based SP programs. The demonstrated educational effect supports DLM integration as a supplementary tool rather than a replacement for human encounters. Notably, effect sizes for DLM intervention are smaller than more traditional CST but still clinically relevant, reflecting the novelty and early stage of AI integration.
Limitations include the reliance on text-based interaction which might lack nonverbal communication nuances critical in clinical encounters. Furthermore, the generalizability of results warrants caution as the study population comprised senior medical students from a single institution. Future trials could explore hybrid models combining AI with human SPs, multimodal interaction platforms, and longitudinal clinical outcomes.
Current guidelines emphasize the importance of active, deliberate practice in communication training; DLM simulations provide a practical environment for such repetitive learning with instant feedback, making them promising adjuncts in surgical curricula.
Conclusion
The integration of deep language learning model-based simulation significantly enhances surgical history-taking skills and student communication confidence. This technology represents a scalable, feasible adjunct to traditional clinical teaching modalities in undergraduate medical education.
Complementary evidence from broader communication simulation and actor-based SBMT underscores the transformative capacity of simulation in cultivating core surgical competencies.
Future research should prioritize expanding AI-driven interactive training, evaluating multimodal communication dynamics, and assessing real-world patient care outcomes linked to these educational innovations.
References
1. McCarrick CA, McEntee PD, Boland PA, et al. A Randomized Controlled Trial of a Deep Language Learning Model-Based Simulation Tool for Undergraduate Medical Students in Surgery. J Surg Educ. 2025 Sep;82(9):103629. doi: 10.1016/j.jsurg.2025.103629. PMID: 40729832.
2. McCarrick CA, Moynihan A, McEntee PD, et al. Impact of simulation training on communication skills and informed consent practices in medical students— a randomized controlled trial. BMC Med Educ. 2025 Jul 18;25(1):1078. doi: 10.1186/s12909-025-07671-0. PMID: 40682099; PMCID: PMC12273346.
3. McCarrick CA, Moynihan A, Khan MF, et al. Impact of Simulation Training on Core Skill Competency of Undergraduate Medical Students. J Surg Educ. 2024 Sep;81(9):1222-1228. doi: 10.1016/j.jsurg.2024.06.006. Epub 2024 Jul 8. PMID: 38981819.
4. Silverman J, Kurtz S, Draper J. Skills for Communicating with Patients. 3rd edition. CRC Press; 2013.
5. Association of American Medical Colleges. Core Entrustable Professional Activities for Entering Residency. 2014.
