Name: Foundations of optimal control theory
ISBN: 9780471522638

What Hacker News says

graycatJun 2015

Ordinary differential equations?

Okay:

Just freshman calculus actually is a good start. E.g., for viral growth consider

y'(t) = k y(t) ( b - y(t) )

where t is time, y(t) the size at time t, and the rate of growth y'(t) = d/dt y(t) is proportional to both the current size y(t) and the size

( b - y(t) )

yet to be achieved. So, e.g., the growth rate is proportional to the number of present customers talking y(t) and the number of potential customers listening

( b - y(t) ).

We assume that the present is time t = 0 and we have the eventual size b and the current size y(0).

Then, sure, we have an initial value problem (that is, we know y(0)) for a first order, linear ordinary differential equation.

But all that is needed for a solution is just freshman calculus. It's just a routine exercise.

Trivial? Once that equation kept FedEx from going out of business.

Polished, elegant, insightful, balanced, expert, great first text:

Earl A. Coddington, 'An Introduction to Ordinary Differential Equations', Prentice-Hall, Englewood Cliffs, NJ, 1961.

For when you want to take some really big next steps up past nearly everyone else in mathematics:

Earl A. Coddington and Norman Levinson, 'Theory of Ordinary Differential Equations', McGraw-Hill, New York, 1955.

For one of the larger reasons to be interested in ordinary differential equations:

Michael Athans and Peter L. Falb, 'Optimal Control: An Introduction to the Theory and Its Applications', McGraw-Hill Book Company, New York, 1966.

Also

E. B. Lee and L. Markus, 'Foundations of Optimal Control Theory', ISBN 0471-52263-5, John Wiley & Sons, New York, 1967.

Once in graduate school I got a reading course to give a lecture a week from

Coddington and Levinson, Athans and Falb, and Lee and Markus. The prof didn't show up again after the first lecture.

At one time our current Fed Chair Janet Yellen indicated that she saw some potential in using control theory to help manage the economy.

And a really sweetheart application of modern control theory, doing amazing things with automatic control of quadrocoptors:

https://www.youtube.com/watch?v=w2itwFJCgFQ

A broader view of some of what can be done with ordinary differential equations, and a long, gorgeous desert buffet of applied math:

David G. Luenberger, 'Optimization by Vector Space Methods', John Wiley and Sons, Inc., New York, 1969.

Generally, though, apparently the big glory days of ordinary differential equations were for the US DoD and NASA during the Cold War and the Space Race. Likely there are plenty of people now, with gray hair, who supply the expertise needed for current aerospace, etc.

Numerical solutions? That field is also nicely developed.

Partial differential equations? That's related but quite different.

thread

graycatJul 2025

Yup: And stochastic dynamic programming and stochastic optimal control! Suppose a cargo plane is to fly from Kansas City to St. Louis, Chicago, Indianapolis, Dayton, Nashville, and Memphis and at each city pickup cargo. Suppose the cargo weights are known only in probability distribution from history and otherwise are probabilistically independent. Have weather reports for the winds and temperatures, but the actual values can vary from the reports. And suppose the fuel prices vary significantly among the stops but are known.

Then, how much fuel should the pilot buy at each stop to minimize expected cost for the trip? I.e., in simple terms, buy extra fuel where it's cheap and carry (tanker) it to where it's expensive but in so doing burn extra fuel from the extra weight of the extra fuel.

Solution: Stochastic dynamic programming.

Hint: In Canada, the solution is illegal!

Was meeting with Nemhauser (see references below); my cab back to the airport was waiting; and he gave me a 15 second lecture. On the plane thought about his lecture, ..., and stood for my Ph.D. orals and passed (extra credit for guessing my employer).

Some references (with TeX markup):

Stuart E.\ Dreyfus and Averill M.\ Law, {\it The Art and Theory of Dynamic Programming,\/} ISBN 0-12-221860-4, Academic Press, New York.\ \

Dimitri P.\ Bertsekas, {\it Dynamic Programming: Deterministic and Stochastic Models,\/} ISBN 0-13-221581-0, Prentice-Hall.\ \

George L.\ Nemhauser, {\it Dynamic Programming,\/} ISBN 0-471-63150-7, John Wiley and Sons, New York\ \

E.\ B.\ Dynkin and A.\ A.\ Yushkevich, {\it Controlled Markov Processes,\/} ISBN 0-387-90387-9, Springer-Verlag, Berlin.\ \

Wendell H.\ Fleming and Raymond W.\ Rishel, {\it Deterministic and Stochastic Optimal Control,\/} ISBN 0-387-90155-8, Springer-Verlag, Berlin.\ \

Michael Athans and Peter L.\ Falb, {\it Optimal Control:\ \ An Introduction to the Theory and Its Applications,\/} McGraw-Hill Book Company, New York.\ \

Dimitri P.\ Bertsekas and Steven E.\ Shreve, {\it Stochastic Optimal Control: The Discrete Time Case,\/} ISBN 0-12-093260-1, Academic Press, New York.\ \

E.\ B.\ Lee and L.\ Markus, {\it Foundations of Optimal Control Theory,\/} ISBN 0471-52263-5, John Wiley, New York,

thread

graycatAug 2021

Calculus of variations is, yes, a part of optimization, now regarded as an old part. But subsequent to Bellman's book was the Pontryagin maximum principle. There is an entry at Wikipedia:

https://en.wikipedia.org/wiki/Pontryagin%27s_maximum_princip...

There is some treatment in

David G. Luenberger, Optimization by Vector Space Methods.

which I call fun and profit from the Hahn-Banach theorem.

There may be more in

L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mischenko, The Mathematical Theory of Optimal Processes.

and in

E. B. Lee and L. Markus, Foundations of Optimal Control Theory.

and in

Michael Athans and Peter L. Falb, Optimal Control: An Introduction to the Theory and Its Applications.

At one time I was interested in such math as a way to say how best to climb, cruise, and descend an airplane. So, I chatted with Athans in his MIT office and got a copy of his class notes. And I got accepted to grad school at the Brown University Division of Applied Mathematics where Falb was. Then I applied a little skeptical judgment and didn't go back to Athans and went elsewhere for my Ph.D.

I also visited Brown and Cornell. At Cornell I met with Nemhauser, and he gave me three words of advice that was the real direction for my Ph.D. dissertation. Flying back on the airplane, I figured out what Nemhauser told me.

When I got back, I called a meeting to outline what I'd discovered from Nemhauser, Athans, etc. My manager ordered me not to hold the meeting, but I did anyway. All the C-level people and the founder, COB, CEO showed up. I got a promotion and big office next to the founder. What I presented was the start of two Ph.D. dissertations.

The Fleming reference may be enough. Uh, while I visited Brown, I had lunch with Fleming -- a very bright mathematician.

A good source for prerequisites is the first (real) half of the W. Rudin, Real and Complex Analysis.

Broadly that optimization -- deterministic optimal control -- works with curves where each curve is regarded as one point, vector, in some vector space. The space is short on good assumptions so may be only a Banach space. Actually can still do a lot in Banach space, e.g., Luenberger's book -- one of my favorites. Gee, there can also get a really short treatment of Kalman filtering!

The fraction of mathematicians who know all that math well is tiny. The people who make good money from applications has to be smaller -- I would believe so small there are none. From all I can see, good applications are so rare there almost aren't any.

With the rapidly growing power of computing and the rapidly growing complexity of computer applications, discrete time stochastic optimal control (that can attack problems that first cut intuitively seem hopelessly difficult yet be comparatively simple mathematically) has a chance of valuable applications, so far not a very good chance, but a chance, probability small but still greater than zero.

That spreadsheet connection I outlined may be a seed for some applications.

Microbiology research gets some motivation from applications, saving the lives of people dying of cancer, heart disease, new viruses, etc. Maybe research in mechanical engineering works on how to put up buildings and bridges that won't fall down. Some years ago I concluded that math is being throttled by too much isolation from important, motivating applications. So, if my startup works, that will be n = 1 cases of evidence that math can do well, at least make money, with some new applications outside of math.

thread

NY_USA_HackerMay 2011

If you want something written as well as Halmos, then start writing, and good luck!

"Historical accident": Well, sure, in part nearly all the pure math departments pushed out any such topics!

You are correct: Optimal control has been mostly in advanced parts of electrical engineering.

So, optimal control in EE was an example doing math outside math departments. Of math done outside math departments, control theory is relatively good mathematically.

And, yes, there should be applications elsewhere.

For physics, yes, the deterministic theory of optimal control has been seen as replacing the older calculus of variations which goes back to Newton.

My references are old.

More recent work on stochastic optimal control has been by R. T. Rockafellar at University of Washington.

For

Stuart E. Dreyfus and Averill M. Law, 'The Art and Theory of Dynamic Programming', ISBN 0-12-221860-4, Academic Press, New York.

'dynamic programming' is a big part of the discrete time versions of optimal control. It can be stochastic or deterministic. The linear-quadratic-Gaussian (linear 'plant' or system, quadratic cost to be minimized, and Gaussian exogenous random variables) has 'deterministic equivalence' -- nice -- and this book treats it. The book is a good, elementary start. Apparently Dreyfus was a R. Bellman student.

For

L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mischenko, 'The Mathematical Theory of Optimal Processes', ISBN 0-470-69381-9, Interscience Publishers, John Wiley & Sons, New York.

it was, of course, the main source of the Pontryagin maximum principle, and, thus, a swift kick in the back side for parts of US aerospace in the 1960s.

For

Michael Athans and Peter L. Falb, 'Optimal Control: An Introduction to the Theory and Its Applications', McGraw-Hill Book Company, New York.

Athans was long in EE at MIT and did some military work, e.g., on parts of the C5A airplane. Falb was at Brown's Division of Applied Mathematics.

One Athans story was the 'control' for least time to climb to, say, 100,000 feet for an F-4: Go up to a few thousand feet, go into a dive, get supersonic, get the lower drag of a few hundred knots above Mach 1, and then with the lower drag continue supersonic to the final altitude. I was not able to know if that 'control' idea was just intuitive or directly from computation and the Pontryagin maximum principle which is, after all, just a necessary condition, i.e., local optimality.

Also in that division at Brown, see the papers of Harold Kushner. As I recall, he wrote out a stochastic version of the Pontryagin maximum principle.

For

E. B. Dynkin and A. A. Yushkevich, 'Controlled Markov Processes', ISBN 0-387-90387-9, Springer-Verlag, Berlin.

there are connections with economic planning.

For

Dimitri P. Bertsekas and Steven E. Shreve, 'Stochastic Optimal Control: The Discrete Time Case', ISBN 0-12-093260-1, Academic Press, New York.

the math is done carefully. So there is a lot of attention to measurability. Part of the reason is the issue of 'measurable selection': This can be a deep subject, but often a relatively simple way out is via regular conditional probabilities as in

Leo Breiman, 'Probability', ISBN 0-89871-296-3, SIAM, Philadelphia.

For

David G. Luenberger, 'Optimization by Vector Space Methods', John Wiley and Sons, Inc., New York.

this has likely the easiest mathematical treatment of deterministic optimal control and also Kalman filtering and also the math needed for 'least action' in physics.

For

E. B. Lee and L. Markus, 'Foundations of Optimal Control Theory', ISBN 0471-52263-5, John Wiley & Sons, New York.

this tried to be clean mathematically when it was written. At the beginning, should know some relatively advanced results in ordinary differential equations, e.g., as in

Earl A. Coddington and Norman Levinson, 'Theory of Ordinary Differential Equations', McGraw-Hill, New York.

For more, you can consider non-linear filtering and connections with mathematical finance.

The above is just from my bookshelf. Likely now a better bibliography could be assembled via the Internet which actually is at least a little larger than my bookshelf!

thread