D4L: Decentralized Dynamic Discriminative Dictionary Learning

Abstract

We consider discriminative dictionary learning in a distributed online setting, where a network of agents aims to learn, from sequential observations, statistical model parameters jointly with data-driven signal representations. We formulate this problem as a distributed stochastic program with a nonconvex objective that quantifies the merit of the choice of model parameters and dictionary. We consider the use of a block variant of the Arrow-Hurwicz saddle point algorithm to solve this problem, which exploits factorization properties of the Lagrangian to yield a protocol in that only requires exchange of model information among neighboring nodes. We show that decisions made with this saddle point algorithm asymptotically achieve a first-order stationarity condition on average. The learning rate depends on the signal source, network structure, and discriminative task. We illustrate the algorithm performance for solving a large-scale image classification task on a network of interconnected servers and observe that practical performance is comparable to a centralized approach. We further apply this method to the problem of a robotic team seeking to autonomously navigate in an unknown environment by predicting unexpected maneuvers, demonstrating the proposed algorithm's utility in a field setting.