Gemini模型在医学领域的能力
人工智能
2024-05-02 v2 计算与语言
计算机视觉与模式识别
机器学习
摘要
在医学领域广泛应用的卓越表现，对AI提出了巨大挑战，要求具备先进的推理能力、获取最新医学知识以及理解复杂多模态数据。Gemini模型在多模态与长上下文推理方面展现出强大能力，为医学应用提供了激动人心的可能性。基于Gemini的核心优势，我们引入Med-Gemini，一套高度能力的多模态模型，专为医学领域设计，具备无缝调用网络搜索能力，可通过自定义编码器高效适配新型模态。我们在14个医学基准测试中对Med-Gemini进行评估，在10个基准测试中实现最新SOTA性能，均超越GPT-4模型家族，常常以显著优势获胜。在流行的MedQA（USMLE）基准测试中，最佳Med-Gemini模型实现91.1%的准确率，采用新颖的不确定性引导搜索策略。在7个多模态基准测试中，包括NEJM Image Challenges和MMMU（健康与医学），Med-Gemini相较GPT-4V平均提升44.5%。我们通过在长de-identified健康记录中的信针寻 haystack任务以及医学视频问答中展示了Med-Gemini长上下文能力的有效性，超越了仅依赖原样学习的先前专用方法。最后，Med-Gemini的表现表明其在医学文本摘要、医学对话、医学研究与教育等实际场景中具有实用价值，尽管在此安全关键领域的真实部署仍需更严格的评估。
引用
@article{arxiv.2404.18416,
title = {Capabilities of Gemini Models in Medicine},
author = {Khaled Saab and Tao Tu and Wei-Hung Weng and Ryutaro Tanno and David Stutz and Ellery Wulczyn and Fan Zhang and Tim Strother and Chunjong Park and Elahe Vedadi and Juanma Zambrano Chaves and Szu-Yeu Hu and Mike Schaekermann and Aishwarya Kamath and Yong Cheng and David G. T. Barrett and Cathy Cheung and Basil Mustafa and Anil Palepu and Daniel McDuff and Le Hou and Tomer Golany and Luyang Liu and Jean-baptiste Alayrac and Neil Houlsby and Nenad Tomasev and Jan Freyberg and Charles Lau and Jonas Kemp and Jeremy Lai and Shekoofeh Azizi and Kimberly Kanada and SiWai Man and Kavita Kulkarni and Ruoxi Sun and Siamak Shakeri and Luheng He and Ben Caine and Albert Webson and Natasha Latysheva and Melvin Johnson and Philip Mansfield and Jian Lu and Ehud Rivlin and Jesper Anderson and Bradley Green and Renee Wong and Jonathan Krause and Jonathon Shlens and Ewa Dominowska and S. M. Ali Eslami and Katherine Chou and Claire Cui and Oriol Vinyals and Koray Kavukcuoglu and James Manyika and Jeff Dean and Demis Hassabis and Yossi Matias and Dale Webster and Joelle Barral and Greg Corrado and Christopher Semturs and S. Sara Mahdavi and Juraj Gottweis and Alan Karthikesalingam and Vivek Natarajan},
journal= {arXiv preprint arXiv:2404.18416},
year = {2024}
}