2023 · 10 citations · 25 references
ComputingComputer ScienceDistributed SystemsKnowledge RepresentationMachine LearningContinual LearningDistributed AlgorithmsDistributed ComputingDistributed LearningFederated LearningGenerative ModelingConcept DriftConcept Drift ModelingData DistributionDecentralized Federated LearningDistribution ShiftExperience ReplayGenerative Adversarial Networks
Federated and continual learning are training paradigms addressing data distribution shift in space and time. More specifically, federated learning tackles non-i.i.d data in space as information is distributed in multiple nodes, while continual learning faces with temporal aspect of training as it deals with continuous streams of data. Distribution shifts over space and time is what it happens in real federated learning scenarios that show multiple challenges. First, the federated model needs to learn sequentially while retaining knowledge from the past training rounds. Second, the model has also to deal with concept drift from the distributed data distributions. To address these complexities, we attempt to combine continual and federated learning strategies by proposing a solution inspired by experience replay and generative adversarial concepts for supporting decentralized distributed training. In particular, our approach relies on using limited memory buffers of synthetic privacy-preserving samples and interleaving training on local data and on buffer data. By translating the CL formulation into the task of integrating distributed knowledge with local knowledge, our method enables models to effectively integrate learned representation from local nodes, providing models the capability to generalize across multiple datasets.We test our integrated strategy on two realistic medical image analysis tasks — tuberculosis and melanoma classification — using multiple datasets in order to simulate realistic non-i.i.d. medical data scenarios. Results show that our approach achieves performance comparable to standard (non-federated) learning and significantly outperforms state-of-the-art federated methods in their centralized (thus, more favourable) formulation.
25
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Richard Zhang, Phillip Isola, Alexei A. Efros et al. · 2018 · 12.4K citations
Conditional Generative Adversarial Nets
Mehdi Mirza, Simon Osindero · arXiv (Cornell University) · 2014 · 8.9K citations · Full text
Advances and Open Problems in Federated Learning
Peter Kairouz, H. Brendan McMahan, Brendan Avent et al. · Foundations and Trends® in Machine Learning · 2020 · 4.7K citations · Full text
Philipp Tschandl, Cliff Rosendahl, Harald Kittler · Queensland's institutional digital repository (The University of Queensland) · 2018 · 2.8K citations