Algorithms for Optimization
A comprehensive introduction to optimization with a focus on practical algorithms for the design of engineering systems. This book offers a comprehensive introduction to optimization with a focus on practical algorithms. The book approaches optimization from an engineering perspective, where the objective is to design a system that optimizes a set of metrics subject to constraints. Readers will learn about computational approaches for a range of challenges, including searching high-dimensional spaces, handling problems where there are multiple competing objectives, and accommodating uncertainty in the metrics. Figures, examples, and exercises convey the intuition behind the mathematical approaches. The text provides concrete implementations in the Julia programming language. Topics covered include derivatives and their generalization to multiple dimensions; local descent and first- and second-order methods that inform local descent; stochastic methods, which introduce randomness into the optimization process; linear constrained optimization, when both the objective function and the constraints are linear; surrogate models, probabilistic surrogate models, and using probabilistic surrogate models to guide optimization; optimization under uncertainty; uncertainty propagation; expression optimization; and multidisciplinary design optimization.
Stats for nerds
5recommend
13mentions
11people
2020–2025years
46% recommend54% neutral
Mentions per year
What Hacker News says
- > The labs on this website are designed to accompany the Foundations of Applied Mathematics textbook series. [1]
It's great that they are making the labs free and putting them on GitHub. Even so, the fact that the textbooks are not free and open source make this much less interesting. In contrast, "ALGORITHMS FOR OPTIMIZATION" is available online [2] as well as code from the book, and two more books by the same authors [3].
Seems like university departments should start funding fully open, downloadable books by their faculty and staff as a marketing tool. The books [2],[3] are great marketing for Stanford.
[1] https://epubs.siam.org/action/doSearch?ContribRaw=Jarvis%2C+...
- I'd recommend getting the 9th or 10th edition of Introduction to Operations Research by Hillier and Lieberman. 9th: https://www.amazon.com/dp/0077298349 You can search for the 10th edition. Both are available used for less than 50 USD in good condition. The book covers a lot more than linear programming. A solution manual for both editions can be found on the internet.
A good "free-pdf" optimization book, to support the above is, Algorithms for Optimization by Kochenderfer & Wheeler ( https://algorithmsbook.com/optimization/ ). It has a chapter on constrained linear optimization with Julia code and is a good secondary resource. Kochenderfer, Wheeler, and colleagues also have two other free optimization books that are a little more advanced. It is exceptionally cool that they make the high quality PDF freely available; more authors in the technical space are making their books freely available as pdf and I applaud them for it.
- There are already good recommendations here for getting into the "standard practice" of ML.
To really understand what is going on though, the path I am having some early success with (as a long time developer / data pipeline guy, but newly into the standard python / ML practice) is to run through Kochenderfer's "Algorithms for optimization" from 2019 (MIT press), including implementing the exercises, as optimization is the cornerstone of the majority of ML methods. Some of the most fun I've had in a long time.
Freely available here:
https://algorithmsbook.com/optimization/
From there on, I'm less sure, but expect I might experiment with implementing my own deep learning methods just for fun, or similar.
- For those who want to learn more, I really like the book "Algorithms for Optimization": https://algorithmsbook.com/optimization/#outline
It's a great intro because it covers a large breadth and gives you a sense of what class of techniques are useful for what (rather than just going super deep into one particular technique).
They also have a github in julia notebooks implementing most of the ideas: https://github.com/sisl/algforopt-notebooks
- The first author (MK) also wrote "Algorithms for Optimization" (MIT Press), which is also implemented in (and is a nice guide to) the Julia language, which is itself a pretty sweet language.
- I like all Strang's books... at least the ones I have. I don't have his Learning from Data book, yet... however.
I also really like the applied linear algebra book by Boyd Vandenberghe: https://web.stanford.edu/~boyd/vmls/ Free PDF is available on their website. There is Julia and Python code companions for the book and lecture slides from both Profs their websites. Also check out their other books, many of which have free PDF's available.
I can also recommend Data-Driven science and engineering by Brunton and Kutz. http://databookuw.com/ There used to be a free preprint PDF of the book but I can't find it now. Book is totally worth picking up... MATLAB and Python code available. Steve Brunton's lectures on YouTube are pretty damn good and compliment the book well: https://www.youtube.com/channel/UCm5mt-A4w61lknZ9lCsZtBw/fea...
Another really cool book is Algorithms for Optimization by Mykel Kochenderfer and Tim Wheeler: https://mitpress.mit.edu/books/algorithms-optimization. Julia code used in book.
Related
Machine learning
15 recommend · 31
Introduction to Algorithms
274 recommend · 880
Neural Networks and Deep Learning
72 recommend · 94
Mathematics for Machine Learning
20 recommend · 21
The algorithm design manual
195 recommend · 321
Optimization by Vector Space Methods
7 recommend · 24
