Name: Convex optimization
ISBN: 9780521833783

Convex optimization

Convex optimization problems arise frequently in many different fields. This book provides a comprehensive introduction to the subject, and shows in detail how such problems can be solved numerically with great efficiency. The book begins with the basic elements of convex sets and functions, and then describes various classes of convex optimization problems. Duality and approximation techniques are then covered, as are statistical estimation techniques. Various geometrical problems are then presented, and there is detailed discussion of unconstrained and constrained minimization problems, and interior-point methods. The focus of the book is on recognizing convex optimization problems and then finding the most appropriate technique for solving them. It contains many worked examples and homework exercises and will appeal to students, researchers and practitioners in fields such as engineering, computer science, mathematics, statistics, finance and economics.

What Hacker News says

txnfSep 2024

convex optimization by boyd vandenberghe should be mentioned https://web.stanford.edu/~boyd/cvxbook/

thread

FullStackDev1Apr 2023

I have been a Java Developer then full stack developer all my career. I want to get started on AI. I am not sure where to start as it is overwhelming to look at the rapid changes coming in. Are there a list of books i can start with? My goal is to be not outdated in 6 months and lose my job to chatGPT or some other version of it. So i asked ChatGPT to give me a detailed plan and here is what it gave me. Could you please let me know if this is a way to go or if there is a better way

Transitioning from a full-stack engineer to the AI field can be a fulfilling and exciting journey. To make the most of this transition, you should focus on building a strong foundation in mathematics, programming, and machine learning concepts. Here's a detailed learning plan, broken down into four stages:

Stage 1: Building a Strong Foundation

Mathematics:

Linear Algebra: "Linear Algebra and Its Applications" by Gilbert Strang

Calculus: "Calculus: Early Transcendentals" by James Stewart

Probability & Statistics: "Probability and Statistics for Engineers and Scientists" by Ronald E. Walpole, Raymond H. Myers, Sharon L. Myers, and Keying E. Ye

Optimization: "Convex Optimization" by Stephen Boyd and Lieven Vandenberghe

Programming:

Python: "Python Crash Course" by Eric Matthes

Git: "Pro Git" by Scott Chacon and Ben Straub

Machine Learning & Data Science Basics:

"Introduction to Data Science" by Laura Igual and Santi Seguí "Python Data Science Handbook" by Jake VanderPlas

Stage 2: Learning Machine Learning and Deep Learning

Online Courses:

Coursera: "Machine Learning" by Andrew Ng

Coursera: "Deep Learning Specialization" by Andrew Ng

Fast.ai: "Practical Deep Learning for Coders"

Books:

"Pattern Recognition and Machine Learning" by Christopher M. Bishop "Deep Learning" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville

Stage 3: Expanding Knowledge of AI Subfields

Natural Language Processing (NLP):

"Speech and Language Processing" by Daniel Jurafsky and James H. Martin

"Natural Language Processing with Python" by Steven Bird, Ewan Klein, and Edward Loper

Computer Vision:

"Computer Vision: Algorithms and Applications" by Richard Szeliski

"Deep Learning for Computer Vision" by Adrian Rosebrock Reinforcement Learning:

"Reinforcement Learning: An Introduction" by Richard S. Sutton and Andrew G. Barto

"Deep Reinforcement Learning Hands-On" by Maxim Lapan

Stage 4: Staying Current and Gaining Practical Experience

Read research papers:

Subscribe to the arXiv mailing list in your areas of interest Regularly read papers from conferences like NeurIPS, ICML, and ACL Participate in online competitions:

Kaggle: Participate in machine learning competitions to improve your skills and build your portfolio AIcrowd: Another platform for AI competitions Contribute to open-source projects:

TensorFlow, PyTorch, or other popular AI frameworks Look for projects on GitHub related to your interests and contribute by fixing bugs, implementing new features, or improving documentation Network with AI professionals:

Attend AI conferences, workshops, and local meetups Join AI-related forums, LinkedIn groups, or online communities like Reddit's r/MachineLearning Remember that the learning plan can be adjusted based on your pace and interests. The key is to stay consistent and always be curious to learn more. Good luck on your journey into the AI field!

thread

chestertnDec 2022

Stephen Boyd’s course on Convex Optimization https://see.stanford.edu/Course/EE364A

thread

zekriocaOct 2022

Not OP, but probably Convex Optimization [1].

[1] https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf

thread

bhoproFeb 2022

Convex Optimization - Boyd

thread

rwilson4Dec 2021

Haven't read the paper yet, but Boyd and Vandenberghe's book, Convex Optimization, is available for free on the authors' website and covers all the foundational material.

Update: I've skimmed the paper and here's the gist as I understand it. The paper alleges that Newton's method can have convergence issues, even when using line search. Their approach resolves these convergence issues. It's 6am where I am which isn't great for processing this stuff, but I write a lot of custom Convex Optimization solvers, and I've never had convergence issues with Newton's method with the default line search parameters suggested by the book mentioned above, even with some bad condition numbers that throttle off-the-shelf solvers (which is why I write my own).

The paper talks about speed of convergence, which initially made me think this was a first order method, not requiring calculating the Hessian (matrix of 2nd order derivatives). For large systems this is really slow and 1st order methods speed this up by working only with the first derivatives. But that's not what this paper is: they still use the Hessian but add regularization, which presumably just improves the condition number. Reminds me of proximal algorithms but haven't explored that connection.

Update on the update: the paper addresses situations where the self-concordance assumption does not hold. TBH I didn't pay much attention in class when we went over self-concordancy and convergence, but the contribution of this paper seems to be extending convergence guarantees even to non-self-concordant systems.

thread