Keynote Speaker
 |
Peter Elkin MD, MACP, FACMI, FNYAM, FAMIA, FIAHSI
Professor and Chair
|
|
|
Abstract: |
|
| Large language models (LLMs) show promise in healthcare, but their clinical accuracy requires enhancement to ensure safe implementation. To address this, we developed Semantic Clinical AI (SCAI), a retrieval-augmented generation (RAG) method that integrates formally represented clinical knowledge into LLMs. We evaluated SCAI’s impact by testing three Llama models (13B, 70B, 405B) on USMLE Step 1–3 exams—a rigorous benchmark for physician competency in the U.S. Our results demonstrated significant performance improvements: the 13B model met the passing threshold (60%) for Step 3 only with SCAI-RAG, while the 70B and 405B models achieved high scores (70B: up to 92%; 405B: up to 95.1%) even without augmentation. Notably, SCAI-RAG consistently boosted accuracy across all models, with the most dramatic gains for smaller models (e.g., 13B on Steps 1 & 3) and targeted benefits for larger ones (e.g., 70B on Step 2). These findings highlight that semantic augmentation can bridge critical gaps in LLMs’ clinical reasoning. This study underscores the importance of domain-specific adaptations to maximize LLMs’ utility in healthcare. | |
Biography: |
|
| Dr. Elkin serves as the UB Distinguished Professor and Chair of the UB Department of Biomedical Informatics. Dr. Elkin has published over 300 peer reviewed publications and book chapters. He received his Bachelors of Science from Union College and his M.D. from New York Medical College. He did his NIH/NLM sponsored fellowship in Medical Informatics at Harvard Medical School and the Massachusetts General Hospital. Dr. Elkin has been working in Biomedical Informatics since 1981 and has been actively researching health big data science since 1987. Dr. Elkin is a Master of the American College of Physicians and a Fellow of the American College of Medical Informatics and a Fellow of the New York Academy of Medicine. He was awarded the Mayo Department of Medicine’s Laureate Award for 2005. Dr. Elkin is the index recipient of the Homer R. Warner award for outstanding contribution to the field of Medical Informatics. Dr. Elkin is an internationally renowned expert in knowledge representation, ontology, AI, natural language processing and health IT standards. Dr. Elkin in 2018 received the Team Science Award from the NCATS and was elected an inaugural fellow of the American Medical Informatics Association for Clinical Informatics excellence and has been elected to the International Academy of Health Sciences Informatics. He serves on the Board of Directors of AMIA and the ACTS, and is the President Elect of the Friends of the NLM. He publishes the Springer series Textbook on Ontology, Terminology and Terminological Systems. | |