Generalized Low Rank Models

What Hacker News says

1. jointpdfSep 2022
   VMLS by Stephen Boyd (also of Stanford) would be great background reading for the vector/matrix stuff (i.e. most of this course): https://news.ycombinator.com/item?id=18678314

   I love this book. As opposed to the way that linear algebra is typically introduced, this book focuses on concrete applications (like text analysis, image/signal processing, finance, ML, etc.) and eschews more arcane concepts (like eigen). To me, building practical intuition is the best way to learn* the subject.

   A more advanced but still accessible manuscript by Boyd et al: Generalized Low Rank Models (establishes connections between PCA/SVD and many other matrix factorization methods, and shows you how to roll your own): https://web.stanford.edu/~boyd/papers/pdf/glrm.pdf

   Show morethread
2. LolWolfNov 2018
   1. > It's theoretically impossible to guarantee a convergence to global optima using gradient descent if the function is non-convex.

   This is false. See, e.g., [0][1].

   2. I'm not really sure what the question is here.

   3. If your loss is bounded from below (it is a square norm) by 0 and you achieve 0 loss, this means that 0 is a global optimum, since, by definition, no other objective value can be smaller than this number.

   ---

   [0] Theorem A.2 in Udell's Generalized Low-Rank models paper https://arxiv.org/pdf/1410.0342.pdf

   [1] B&V Convex Optimization (https://web.stanford.edu/~boyd/cvxbook/), Appendix B.1. In fact, I can't find the reference right now, but you can easily prove that GD with an appropriate step-size converges to a global optimum on this problem when initialized at (0,0), even though the problem is non-convex.

   Show morethread