Researchers at MIT have created a new method that can speed up a privacy-preserving AI training technique by approximately 81 percent. This development allows a broader range of edge devices, like sensors and smartwatches, to use more accurate AI models while maintaining user data privacy. The team enhanced the efficiency of federated learning, which is a system where connected devices collaborate to train a shared AI model.<\/p>\n
In federated learning, a central server broadcasts the model to wireless devices, which then train the model using local data before sending updates back to the server. This ensures data remains on the device, safeguarding privacy. However, not all devices have the required capacity, computational power, or connectivity to handle the model efficiently, causing delays that hinder training performance.<\/p>\n
The MIT team devised a technique to address these memory and communication limitations. Their approach is tailored for a diverse network of devices with varying constraints. This innovation could facilitate the deployment of AI models in critical sectors like healthcare and finance, which have stringent security and privacy requirements.<\/p>\n
Irene Tenison, a graduate student in electrical engineering and computer science and lead author of the study, highlighted the importance of this work in enabling AI capabilities on small devices. Co-authors include Anna Murphy, a machine-learning engineer at Lincoln Laboratory; Charles Beauville, a visiting student from EPFL and engineer at Flower Labs; and senior author Lalana Kagal from MIT’s CSAIL. The research will be presented at the IEEE International Joint Conference on Neural Networks.<\/p>\n
Many federated learning methods assume devices in the network have sufficient memory and stable connectivity to facilitate quick updates. However, this is not the case with heterogeneous devices like smartwatches, sensors, and phones, which often have limited resources and intermittent connectivity.<\/p>\n
Typically, the server waits for updates from all devices before averaging them to complete a training round. This can slow down or even halt the training process. To address this, the researchers created a framework called FTTE (Federated Tiny Training Engine) that minimizes memory and communication demands on each device.<\/p>\n
FTTE introduces three key innovations. First, it sends only a subset of model parameters to devices, reducing memory needs. It uses a special search process to determine which parameters optimize accuracy within the memory constraints. Second, the server updates the model asynchronously, processing updates as they arrive instead of waiting for all devices. Third, updates are weighted based on their arrival time to prevent older data from slowing the process.<\/p>\n
In tests with hundreds of varied devices and models, FTTE completed training 81 percent faster than traditional methods, with a significant reduction in memory and communication overheads. While there is a tradeoff in accuracy, the speed improvement could offset this in some applications.<\/p>\n
FTTE also showed scalability and better performance with larger device networks. Real-device tests confirmed its effectiveness across different computational levels. Tenison noted the potential benefits for users with less powerful mobile devices, especially in developing regions.<\/p>\n
Looking ahead, the researchers plan to explore personalizing AI model performance on individual devices and conduct larger experiments on real hardware. The project received partial funding from a Takeda PhD Fellowship.<\/p>\n