Federated Learning Framework Aims to Improve Fairness in AI Screening Tools

A research team from the University of Pittsburgh (Pitt) Swanson School of Engineering has been awarded a $1.7 million National Institutes of Health grant to develop a federated learning (FL)-based approach to achieve fairness in artificial intelligence (AI)-assisted medical screening tools.

FL is a privacy-focused method that allows researchers to train machine-learning (ML) algorithms without exchanging datasets. Typically, data is stored in a central repository and shared with research teams for algorithm development, but secure healthcare data sharing can be a challenge. FL addresses this issue by allowing local data samples to be held on decentralized devices or servers.

Pitt’s project, known as “Achieve Fairness in AI-Assisted Mobile Healthcare Apps through Unsupervised Federated Learning,” has both a data component and an algorithm-based component. Using FL allows the researchers to prioritize data privacy and helps with data collection and patient participation.

“Existing and easily accessible data sets are inherently biased. It’s not always easy for people in marginalized communities to participate in data collection and research, and these communities might also lack medical professionals,” said Jingtong Hu, PhD, associate professor and William Kepler Whiteford Faculty Fellow of Electrical and Computer Engineering at Pitt, in the press release. “AI could make critical healthcare more accessible for these communities; but without a dataset that accurately reflects the diversity of the population, AI could misdiagnose people that are under-represented during the data collection stage, thereby increasing healthcare disparities.”

Following data collection, the FL process is focused on algorithm development. Separate models are trained on-site with the local datasets and sent back to a central server, where they are combined to create a master model. When researchers acquire more data, they can download the latest master model from the central server, update the model with the newly acquired data, and then send the updated model back to the server.

The project would address issues related to algorithm development and prevent health disparities using an on-device learning framework, which would learn from users' data while they are engaging with a particular mobile application. The on-device model would then be shared with researchers to help train a shared master model for AI-assisted medical screening.

“By using this method, not only can we improve the global model to be fairer by incorporating more equally represented data, but we can personalize the model for each individual. After all, the most important metrics for each user is the accuracy for him or herself,” said Hu. “A user could use our app to diagnose their skin condition, for example, to see if a skin issue is skin cancer or just normal eczema. Meanwhile, our algorithm will learn from the new images locally. Patients' images will not be uploaded to the server; they will be analyzed on their own cell phones.”

According to the press release, the project’s reliance on smartphones and other mobile devices will allow more people to participate in the research, which can address underrepresentation in AI training datasets.

Throughout the course of the project, which is a four-year collaboration between Pitt’s Schools of Engineering and Medicine, the University of Notre Dame, and George Mason University, researchers will use the framework to consider the fairness of different ML models and develop an ML approach that will automatically search existing learning models and use the best architectures for datasets with diverse data.