题目：Optimization in Alibaba: Beyond Convexity
报告人： 谭剑 博士 阿里巴巴达摩院
摘要： In this talk, we will talk about the recent developments in large-scale optimization that are beyond the conventional wisdom of convex optimization. I will specifically address three challenging problems that have found applications in many Alibaba businesses. In the first application, we study the problem of optimizing truncated loss functions that are of particularly importance when coming to learning from heavily tailed distributions. We show that, despite of its non-convexity, under appropriate condition, a variant of gradient descent could efficiently find the global optimal. In the second application, we study the problem of how to find local optimal in the case of non-convex optimization. We show that with introduction of appropriate random perturbation, we could find the local optimal at the rate of O(1/\gamma^3) where $\gamma$ defines the suboptimality, which significantly improves the results of the existing studies. In the last application, we consider optimizing a continuous function over a discrete space comprised of a huge number of data points. The special instances of this problem include approximate nearest neighbor search and learning a quantized neural network. The most intriguing result from our study is that this optimization problem become relatively easier when the size of discrete space is sufficiently large. We provide results of both theoretical analysis and empirical studies.