Abstract

Stochasticoptimization methods, e.g., stochastic gradient descent (SGD), have recentlyfound wide applications in large-scale data analysis, especially in machinelearning. These methods are very attractive to process online streaming data asthey scan through the dataset only once but still generate solutions withacceptable accuracy. However, it is known that classical SGDs are ineffectivein processing streaming data distributed over multi-agent network systems(e.g., sensor and social networks), mainly due to the high communication costsincurred by these methods.

In thistalk, we present a few new classes of SGDs which can significantly reduce theaforementioned communication costs for distributed or decentralized machinelearning. We show that these methods can significantly save inter-nodecommunications when performing SGD iterations. Meanwhile, the total number ofstochastic (sub)gradient computations required by these methods are comparableto those optimal ones achieved by classical centralized SGD type methods.

This talkis based on the following two papers.

1. G. Lanand Y. Zhou, Random gradient extrapolation for distributed and stochasticoptimization, SIAM Journal on Optimization, 28(4), 2753-2782, 2018.

2. G. Lan, S. Lee and Y. Zhou, Communication-efficientAlgorithms for Decentralized and Stochastic Optimization, MathematicalProgramming, to appear, 2018.

Time

12月17日（周一）14:00-15:00

Speaker

Guanghui (George) Lan serves as an associate professor in the H. Milton Stewart School ofIndustrial and Systems Engineering at Georgia Institute of Technology sinceJanuary 2016. Before that he had been a faculty member in the Department ofIndustrial and Systems Engineering at the University of Florida from 2009 to2015, after receiving his Ph.D. degree from Georgia Institute of Technology inAugust, 2009. His main research interests lie in optimization and machinelearning/intelligence. Dr. Lan serves as the associate editor for MathematicalProgramming, SIAM Journal on Optimization, and Computational Optimization andApplications.

Venue

信息管理与工程学院102室

上海财经大学（第三教学楼西侧）

上海市杨浦区武东路100号