This paper is available on arxiv under CC 4.0 license. Authors: (1) Thomas Pethick, EPFL (LIONS) thomas.pethick@epfl.ch; (2) Wanyun Xie, EPFL (LIONS) wanyun.xie@epfl.ch; (3) Volkan Cevher, EPFL (LIONS) volkan.cevher@epfl.ch. Table of Links Abstract & Introduction Related work Setup Inexact Krasnosel’ski˘ı-Mann iterations Approximating the resolvent Last iterate under cohypomonotonicity Analysis of Lookahea Experiments Conclusion & limitations Acknowledgements & References 9 Conclusion & limitations We have precisely characterized the stabilizing effect of linear interpolation by analyzing it under cohypomonotonicity. We proved last iterate convergence rates for our proposed method RAPP. The algorithm is double-looped, which introduces a log factor in the rate as mentioned in Remark E.4. It thus remains open whether last iterate is possible using only τ = 2 inner iterations (for which RAPP reduces to EG+ in the unconstrained case). By replacing the inner solver we subsequently rediscovered and analyzed Lookahead using nonexpansive operators. In that regard, we have only dealt with compositions of operators. It would be interesting to further extend the idea to understanding and developing both Federated Averaging and the meta-learning algorithm Reptile (of which Lookahead can be seen as a single client and single task instance respectively), which we leave for future work.

Stable Nonconvex-Nonconcave Training via Linear Interpolation: Conclusion & limitations

Too Long; Didn't Read

The Interpolation Publication

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...