Decentralized collaborative continual learning: A multi-objective minimization-based technique

Key points

• For a given inference task t (corresponding to a common minimizer shared by the cost functions of all agents), agents collecting data in a distributed and streamed manner are only allowed to perform local computations and to exchange information with neighboring agents over the underlying communication graph.1
• The proposed decentralized continual learning approach is analyzed in the mean square error sense under general assumptions on the individual cost functions and gradient noise processes.1
• The analysis reveals that cooperation among agents improves the performance of continual learning.1
• In particular, by exchanging information with neighboring agents, decentralized collaborative learning can exploit the diversity of locally observed data and memory buffers to improve the network average mean-square deviation (MSD) across tasks.1

Sources (1)

• [1]Decentralized collaborative continual learning: A multi-objective minimization-based technique
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 08:31 PM

  In this work, we formulate decentralized continual learning within a multi-objective optimization framework.

  For a given inference task t (corresponding to a common minimizer shared by the cost functions of all agents), agents collecting data in a distributed and streamed manner are only allowed to perform local computations and to exchange information with neighboring agents over the underlying communication graph.

Extractive summary: sentences quoted from the sources.