Biostatistics Department Seminar: A Scalable, Data-Driven Framework for Identifying Patient Subgroups on Which an AI/ML Model Underperforms

Biostatistics Department Seminar

Title: A Scalable, Data-Driven Framework for Identifying Patient Subgroups on Which an AI/ML Model Underperforms

Abstract: A fundamental goal of evaluating the performance of a clinical model is to ensure it performs well across a diverse intended patient population. A primary challenge is that the data used in model development and testing often consist of many overlapping, heterogeneous patient subgroups that may not be explicitly defined or labeled.  While a model’s average performance on a dataset may be high, the model can have significantly lower performance for certain subgroups, which may be hard to detect.  In this talk I will describe an algorithmic framework for identifying subgroups with potential performance disparities (AFISP), which produces a set of interpretable phenotypes corresponding to subgroups for which the model's performance may be relatively lower. I will discuss an application of AFISP to a patient deterioration model, illustrate the method’s ability to detect significant subgroup performance disparities, and show that AFISP is significantly more scalable than existing algorithmic approaches.

Speaker

Adarsh Subbaswamy, PhD '23, is a Staff Fellow (Regulatory Scientist) at the U.S. Food and Drug Administration in the Division of Imaging Diagnostics and Software Reliability at the Center for Devices and Radiological Health.

Registration

If you would like to join via Zoom, please register here.

You do not need to register to attend in person. The seminar will be held in Room W2008, Wolfe Street Building.

2024 Monday Seminar Series

All seminars are held at 12:05 PM via Zoom and onsite in Room W2008. View all seminar information here.