国内大语言模型辅助训练可显著提升医师的麻醉急救能力

Domestic large language model-assisted training can significantly enhance the anesthesia emergency rescue capabilities of physicians

摘要:

目的 通过临床麻醉急救模拟场景对比国内大语言模型（LLM）的准确性，探讨 LLM 在临床麻醉急救 能力提升中的应用。方法 向豆包、文心、元宝、DeepSeek、Kimi、千问共 6种国内主流 LLM 上传临床麻醉急救相关 试题并获取答案，通过分析作答准确率选出高性能LLM。选取40名麻醉科医师为研究对象，随机分为试验组（n = 20）与对照组（n = 20）。训练前采用麻醉急救案例问卷对全体受试者进行测试，并按统一评分细则评分。试验组接 受 LLM 辅助训练，对照组采用传统训练模式，训练周期统一为 1周，每日训练时长为（120 ± 10） min。训练后采用 不同的麻醉急救案例问卷测试并评分，对比两组的成绩来评估LLM辅助训练效果。采用χ2 检验比较LLM在单选题的差异，采用 Mann-Whitney U 检验比较 LLM 在不定项选择题与整体的差异，采用独立样本 t 检验与 MannWhitney U检验分析LLM辅助训练的效果。使用Bonferroni校正控制Ⅰ型错误，检验水准设定为α= 0.05，即校正P 值 < 0.05时认定为差异具有统计学意义。结果 豆包在基础知识、相关专业知识、专业知识三类单选题测试中表现 最优，平均准确率达92.3%；在专业实践知识（不定项选择题）中以77.5分位居首位。试验组成绩提升幅度显著高于 对照组（P值 < 0.05），试验组全员训练后成绩及格且稳定性更高，对照组30%医师的成绩出现负向提升。结论 国内 主流LLM在临床麻醉急救中已具备一定成熟度与稳定性，其中豆包在知识深度、适配度及精准度上形成领先优势。 LLM 辅助训练可显著提升医师的麻醉急救能力，具有提高效果显著、训练效果稳定、少有负面风险等优点。LLM 在麻醉急救领域具有应用价值。

Abstract:

To compare the accuracy of domestic large language models (LLMs) in simulated clinical anesthesia emergency scenarios and to explore their application in improving clinical anesthesia emergency capabilities. Methods Questions related to clinical anesthesia emergencies were submitted to six mainstream domestic LLMs, including Doubao, ERNIE, Yuanbao, DeepSeek, Kimi and Qwen, and corresponding answers were collected. Highperformance LLMs were selected based on the accuracy of responses. A total of 40 anesthesiologists were enrolled and randomly divided into an experimental group (n = 20) and a control group (n = 20). All participants completed a questionnaire based on anesthesia emergency cases before training, and scores were calculated using unified scoring criteria. The experimental group received LLM-assisted training, while the control group adopted a traditional training mode. The training period lasted for one week, with a daily training duration of 120 ± 10 minutes. After training, another questionnaire involving different anesthesia emergency cases was administered for evaluation and scoring. The scores of the two groups were compared to assess the effectiveness of LLM-assisted training. The chi-square test was used to analyze differences in the performance of LLMs in single-choice questions; the Mann-Whitney U test was applied to compare their performance in open multiple-choice questions and overall outcomes. The independent-samples t test and Mann-Whitney U test were used to evaluate the effect of LLM-assisted training. Bonferroni correction was applied to control Type I errors. The significance level was set at α =0.05, and a corrected P < 0.05 was considered statistically significant.Results Doubao demonstrated the best performance in single-choice questions involving basic knowledge, related professional knowledge and specialized knowledge, with an average accuracy of 92.3%, and ranked first with a score of 77.5 in professional practical knowledge (open multiple-choice questions). The experimental group showed significantly greater score improvement than the control group (P < 0.05). All participants in the experimental group passed the post-training test with higher score stability, while 30% of anesthesiologists in the control group showed negative improvement in academic performance. Conclusion Mainstream domestic LLMs have attained desirable maturity and stability in clinical anesthesia emergency practice. Among them, Doubao has leading advantages in knowledge depth, applicability and accuracy. LLM-assisted training can significantly improve anesthesiologists' anesthesia emergency capabilities, with the strengths of prominent efficacy, stable training effects and low adverse risks. LLMs possess favorable application value in the field of anesthesia emergency management.