Functions of Several Variables

What Hacker News says

1. LudwigNagasenaJan 2024
   Well, I don't know much about the way calculus is taught in the West, but I remember that Zorich's Mathematical Analysis (ch. 5 Differential Calculus and ch. 8 The Differential Calculus of Functions of Several Variables) was pretty clear about everything.
   thread
2. graycatMay 2022
   Bluntly, with only occasional exceptions, have to learn from the books. Period. No biggie: Once are in grad school or a prof, have to learn from papers, and the good books are typically MUCH easier to read than the papers!

   For first calculus, read one of the well respected books. In my experience and opinion, don't need high school advanced placement calculus -- regard it as a waste of time. Also the last time I looked at Khan Academy, I concluded that it was a waste or worse and by people who didn't have a good background in calculus. Generally the Internet sources are too elementary. Solution: Just use some good books. Protter and Morrey is good (I taught from it), but it is a bit elementary. I learned from Johnson and Kiokmeister; I like it. At the time it was also being used at Harvard. The exercises are especially good. It's old, but calculus hasn't changed much since 1950 or so about when it gave up on teaching infinitesimals.

   But might check out the Maxima software for symbolic indefinite integration -- as I recall, its algorithms are so good they do any indefinite integration that can be done in closed form.

   For linear algebra, sure, Nering (a student of E. Artin at Princeton) and then Halmos (a finite dimensional introduction to Hilbert space and written when Halmos was an assistant to von Neumann, office not far from Einstein), etc. For still more, R. Bellman has a good book -- he went all over active and put in everything including the kitchen sink. Then for numerical work, old but a start, Forsythe and Moler and/or the documentation of LINPAK.

   For multivariate calculus through the divergence theorem and as needed, e.g., for Maxwell's equations, start with the most elementary treatments of vector analysis and f'get about the lack of proofs. Actually notice, can see via a Google search, that the divergence theorem on a box is just a dirt simple application of the fundamental theorem of freshman calculus. Then the difficulty is proving the thing on other shapes. Then do multivariate calculus again via, say, Apostol, Mathematical Analysis: A Modern Approach to Advanced Calculus. That he uses the Jordan curve theorem is a bit much but just go with the flow. Fleming's Functions of Several Variables does it with both measure theory and exterior algebra. Or W. Rudin's, as I recall, third edition, Principles of Mathematical Analysis. For calculus done via Lebesgue's measure theory, Royden's Real Analysis and/or the first half of Rudin's Real and Complex Analysis.

   My favorite author on ordinary differential equations is Coddington, and his elementary An Introduction to Ordinary Differential Equations is nicely done and likely enough for nearly any applications might encounter. Will want some of this if do A/C circuit theory or deterministic optimal control.

   For basic abstract algebra Herstein, Topics in Algebra should usually be enough and is well written. With that will be well prepared for, say, algebraic coding theory which actually has some important applications.

   Once have a background in measure theory, for probability, of course, Loeve or either of his students Breiman or Neveu. Breiman is really nice to read. Neveu is a good candidate for the most polished and elegantly done math book or writing of any kind in all of civilization. Be sure to get to the good stuff on the Radon-Nikodym theorem, conditional expectation, martingales, and ergodic processes.

   For relatively good results, better than hardly any students get just from courses, study the theorems and proofs until can do them yourself and can also explain them to any common man in the street and also explain where they fit in more generally. Also be able to do the exercises. E.g., Rudin's Principles has the inverse and implicit function theorems as exercises! Fleming applies them to Lagrange multipliers. These two theorems are the main prerequisites of differential geometry -- some parts of physics, now popular, touch on this math!

   In my Ph.D. program I led the class in 4 of the five qualifying exams, and for the analysis and linear algebra exams, the books above were the secret. After I did that, the department tried to have the students do better by offering a course from Rudin's Principles; the course didn't work; that book is too hard for a course and needs weeks of quiet time where a student can chew on the text slowly line by line. Rudin has some of the most precise presentations on Fourier theory. So when physics uses Fourier theory or engineers use the fast Fourier transform, you will likely know Fourier theory quite a lot better than they do. Right, the uncertainty principle in quantum mechanics is just a basic result in Fourier theory. With Fourier theory and linearity, Schroedinger's equation starts to make sense. A good exercise in early measure theory is differentiation under the integral sign, Leibniz's rule -- be sure to cover that with a good proof.

   With the background from the books above will be well prepared for a wide range of more narrow topics in pure and applied math.

   Nearly all these books are available used in nearly new condition for low prices. In a move I lost some of my library and rebuilt with used copies.

   Show morethread
3. luhbegMay 2022
   High-school dropout with a phd in a social science here. My grades weren’t good enough to study math during undergrad and I was dealing with a chronic illness that meant I didn’t have my shit together enough to transfer into a useful degree either. I’m a machine learning scientist at a company you’ve heard of and probably spent money on (not FAANG). Math is a big part of my day job.

   Here’s what I did to learn math. I wouldn’t recommend this path. You will have a much easier time getting a job with a quantitative degree. You’re only 26, so you can go back to school without really losing much time. If you must do it this way, do the exercises, do the exercises, do the exercises, build a portfolio, and do the exercises.

   Calculus/Analysis

   Stewart single var calculus, then the multivar book. These are easy starters

   Real analysis: Series, Functions of Several Variables, and Applications. Miklós Laczkovich, Vera T. Sós

   Spivak Calc and Calc on Manifolds books

   Bonus: Advanced Calculus A Geometric View, Callahan. This is what I turn to when I want to punish myself or remind myself how certain analysis proofs go.

   Linear Algebra

   Strang, Intro to LA is great. Start with this one

   Strang, LA and learning from data. Will be tough without the first book

   Stats

   Hogg, Introduction to Mathematical Statistics

   Gelman et al, BDA3 for Bayes

   ML

   Bishop PRML and Elements of statistical learning. Do the exercises. Build the algos in Python.

   Show morethread
4. graycatJul 2020
   Linear algebra, and more than one such book.

   IMHO long and still the best linear algebra book is

   Halmos, Finite Dimensional Vector Spaces (FDVS).

   It was written in 1942 when Halmos was an "assistant" to John von Neumann at the Institute for Advanced Study. It is intended to be finite dimensional vector spaces but done with the techniques of Hilbert space. The central result in the book, according to Halmos, is the spectral decomposition. One result at a time, the quality of von Neumann comes through. Commonly physicists have been given that book as their introduction to Hilbert space for quantum mechanics.

   But FDVS is a little too much for a first book on linear algebra, or maybe even a second book, should be maybe a third one.

   Also high quality is Nering, Linear Algebra and Matrix Theory. Again, the quality comes through: Nering was a student of Artin at Princeton. There Nering does most of linear algebra on just finite fields, not just the real and complex fields; finite fields in linear algebra are important in error correcting codes. So, that finite field work is a good introduction to abstract algebra.

   For a first book on linear algebra, I'd recommend something easy. The one I used was

   Murdoch, Linear Algebra for Undergraduates.

   It's still okay if can find it.

   For a first book, likely the one by Strang at MIT is good. Just use it as a first book and don't take it too seriously since are going to cover all of it and more again later.

   I can recommend the beginning sections on vector spaces, convexity, and the inverse and implicit function theorems in

   Fleming, Functions of Several Variables

   Fleming was long at the Brown University Division of Applied Math. The later chapters are on measure theory, the Lebesgue integral, and the exterior algebra of differential forms, and there are better treatments.

   Also there is now

   Stephen Boyd and Lieven Vandenberghe, Introduction to Applied Linear Algebra – Vectors, Matrices, and Least Squares

   at

   http://vmls-book.stanford.edu/vmls.pdf

   Since the book is new, I've only looked through it -- it looks like a good selection and arrangement of topics. And Boyd is good, wrote a terrific book, maybe, IMHO likely, the best in the world, on convexity, which is in a sense is half of linearity.

   Some course slides are available at

   http://vmls-book.stanford.edu/

   For reference for more, have a copy of

   Richard Bellman, Introduction to Matrix Analysis: Second Edition.

   Bellman was famous for dynamic programming.

   For computations in linear algebra, consider

   George E. Forsythe and Cleve B. Moler, Computer Solution of Linear Algebraic Systems

   although now the Linpack materials might be a better starting point for numerical linear algebra. Numerical linear algebra is now a well developed specialized field, and the Linpack materials might be a good start on the best of the field. Such linear algebra is apparently the main yardstick in evaluating the highly parallel supercomputers.

   After linear algebra go through

   Rudin, Principles of Mathematical Analysis, Third Edition.

   He does the Riemann integral very carefully, Fourier series, vector analysis via exterior algebra, and has the inverse and implicit function theorems (key to differential geometry, e.g., for relativity theory) as exercises.

   All of this material is to get to the main goals of measure theory, the Lebesgue integral, Fourier theory, Hilbert space and Banach space as in, say, the first, real (not complex) half of

   Rudin, Real and Complex Analysis

   But for that I would start with

   Royden, Real Analysis

   sweetheart writing on that math.

   Depending on the math department, those books might be enough to pass the Ph.D. qualifying exam in Analysis. It was for me: From those books I did the best in the class on that exam.

   Moreover, from independent study of Halmos, Nering, Fleming, Forsythe, linearity in statistics, and some more, I totally blew away all the students in a challenging second (maybe intentionally flunk out), advanced course in linear algebra and, then, did the best in the class on the corresponding qualifying exam, that is, where that second course was my first formal course in linear algebra.

   Lesson: Just self study of those books can give a really good background in linear algebra and its role in the rest of pure and applied math.

   No joke, linear algebra, and the associated vector spaces, is one of the most important courses for more work in pure and applied math, engineering, and likely the future of computing.

   Show morethread
5. graycatOct 2019
   "What part"?

   You wrote

   > Of the books that I explored, Baby Rudin came the closest to this ideal. While many of the individual chapters contains long runs of mostly linear development, the interdependence of the chapters is complex, and apparently very carefully considered.

   and I removed what was "complex".

   Many students hear bad things about Baby Rudin and don't try it. Of students who do try it, too many don't finish well. Of the students who do finish, only a small fraction take a second pass where they will get some of the clarity I typed in for you.

   For my Ph.D., there were five qualifying exams. I did the best on four of them, and one of those was Analysis and fairly close to Baby Rudin. The department didn't offer a course to help students prepare for the Analysis exam. So, I was likely the only student who had taken two passes through Baby Rudin; the first pass was in a course; I did okay; the second pass was on my own and slowly; it was fun! The second pass is the main reason I did the best on that exam. The next year the department tried teaching a course from Baby Rudin: They did that for only one year; I can believe that few or none of the students did well with the course. A friend from another department wanted to learn Baby Rudin, took the course, had a hard time, and dropped it. He went away unhappy, and that was sad and not really necessary.

   So, my experience with Baby Rudin suggests (i) take a formal course, (ii) use advice such as I gave here to see during the course some clarity in what is going on, how, and why; (iii) also study or glance at one or two other competitive or similar books, e.g., Spivak, Calculus on Manifolds, Fleming, Functions of Several Variables, Kolmogorov and Fomin, Apostol, and more, and (iv) take a second pass through Baby Rudin.

   Net, Baby Rudin has discouraged a lot of students, too many. To many students, Baby Rudin looks forbiddingly severe and abstract; students can't figure out what the heck is going on, how, or why. E.g., your view was "complex" -- it can be "simple". With some help, e.g., as I typed in here for you, Baby Rudin can be okay.

   Show morethread
6. graycatMar 2019
   I'll try to clear things up:

   First, Horn and Johnson is a bit much. I was in Horn's class. I had done a LOT in, call it, linear algebra and matrix theory in my career before the class, told the profs I didn't need the course, and they said it was a "second, advanced course" and smiled.

   The course was quite competitive and without trying at all and without intending to be competitive, I effortlessly blew away all the other students on graded homework, the tests, the midterm, the final exam, and the corresponding qualifying exam. At the end of the course Horn wrote about me IIRC "Best performance in the class by a wide margin. Knows this material cold."

   So, yes, it was an advanced course, actually had a lot of nice stuff in it, Horn's lectures were nicely precise and at times with some unusual, nice approaches, but to do well in the course it was sufficient just to have had a good background before.

   What background? For the main books, E. Nering (a student of E. Artin at Princeton), Halmos (an assistant to von Neumann at the Institute of Advanced Study at Princeton), Finite Dimensional Vector Spaces, basically also a finite introduction to Hilbert space and the spectral theorem there, Forsythe and Moler, Computer Solutions of Linear Algebraic Systems, and some good texts in multivariate statistics with regression analysis, discriminate analysis, factor analysis, analysis of variance. More in applications, e.g., the fast Fourier transform, more on curve fitting, linear systems in electronic engineering, antenna theory and beam forming, optimization, linear programming, unconstrained optimization, the Markowitz and Sharpe applications to finance, Lagrange multipliers, the Kuhn-Tucker conditions, etc. can also help.

   But Horn is not a good choice for a first text. For a first or second text I'd suggest, say, Hoffman and Kunze, Linear Algebra, Second Edition available for free on the Internet.

   For more, see my post on math in

   https://news.ycombinator.com/item?id=15116379

   and there sections

   (2) Linear Algebra

   (2.1) Linear Equations

   (2.2) Gauss Elimination

   (2.3) Vectors and Matrices

   (2.4) Vector Spaces

   (2.5) Eigen Values, Vectors

   (2.6) Texts

   To be brief, about the earliest and easiest start on linear algebra and matrix theory is just a high school style system of linear equations. The main solution technique is Gauss elimination. Matrix notation is a better notation for that subject.

   Here is essentially the role of matrix theory: Each of the old results in linear algebra can be written as a result, with nicer notation, in matrix theory. Can get the same results without matrix notation, but matrix notation makes it all much easier.

   Next, a broad statement is that the two pillars of the field of analysis in math are (1) linearity and (2) continuity. Well, linear algebra and matrix theory stands strongly on linearity and, as we move on in both the theory and applications, also continuity.

   Let's be clear on linearity via linear algebra and matrix theory: So, for positive integers m and n and an m x n matrix A we say that matrix A is a linear transformation (function) if for all n x 1 vectors x and y, and numbers a and b, we have that

   A(ax + by) = aAx + bAy

   Sure, to read this need the definitions of matrix sum and product; sum is trivial; product is not much harder and is really just what need to make Ax = b be the same as the high school system of linear equations.

   For the numbers, usually use either the set of real numbers R or the set of complex numbers C. But, sure, for numerical computation are essentially limited to the set of rational numbers Q. But in general need only what a course in abstract algebra calls a field: Each of R, C, and Q is such a field but also the set of integers modulo a prime number is a field, of interest in algebraic coding theory and cryptology.

   This definition of linearity generalizes in Hilbert space, Banach space, and functional analysis, and the more general definitions and results are important in quantum mechanics, differential equations in science and engineering, signal processing in electronic engineering, etc. Again, linearity is a pillar of analysis in math.

   Why pillars? In both theory and applications, linearity and continuity commonly hold and are astoundingly powerful properties. For such applications we have multivariate statistics, optimization, electronic engineering, antenna theory, beam forming, signals (each time invariant linear system has sines and cosines as eigenvectors; when a violinist on a concert stage plays some pure tones, the concert hall transmits those tones to you in the audience as a linear system so that what you hear are just the pure tones with the right frequencies but with some phase and amplitude changes; the Navy likes to know that for sonar signals; the USAF likes to know that for radar signals; cell phone people like to know that for their signals), Fourier theory, linear partial differential equations, superposition in quantum mechanics, etc. And when linearity does not hold, commonly it is a good, first approximation and the main means of iterative techniques. And if a problem is not linear, maybe after some simple transformation it will be.

   In some of the posts here, there is mention of matrix theory and basis, that is, a coordinate system. Well, can do that although is it not nearly as general as what physics likes to do with coordinate systems. But also can just decide not to do that, to take the vector space as just the n-tuples and not force thinking of the n-tuples as just coordinates of vectors in some basis. Or can do either approach depending on what is easier in the context.

   Here is a point should get: Suppose we start with just systems of linear equations. Then we say we are working with n-tuples of numbers. Then we call those n-tuples a vector space. Then using essentially just the main, relevant properties of those n-tuples, we write down the definition, axioms, of a vector space where we've said nothing about the vectors but have left them as just points.

   Well then we have two advantages: First, the definition of a vector space lets us talk about subspaces and in particular subspaces of the vector spaces of just the n-tuples, and we want to do that already, strongly with just Gauss elimination for linear equations. E.g., with the linear system Ax = b where m, n are positive integers, A is m x n, x is n x 1, and b is m x 1, the set of all x so that Ax = 0 (m x 1 of all zeros) is a vector subspace of all the n x 1 vectors (for the set of real numbers R, commonly called the set R^n). Call the set K the set of all x so that Ax = 0. If for some n x 1 u we have that Au = b, then from linearity we can argue that any v so that Av = b can be written as a sum of u and some vector in K. In this way we see all possible solutions of Ax = b. Actually at the end of Gauss elimination we can see K and u easily enough.

   Second, we get to consider vectors other than just n-tuples. E.g., we can consider the data of 1 second of music as a vector, a random variable as a vector, a color as a vector, the wave function of a photon or electron as a vector, etc.

   Then as the book continues, we get into eigenvalues and eigenvectors. Eigen is German essentially for special. They are special, and valuable. Maybe the nicest part is the polar decomposition: Each square matrix is a product UH where U in unitary and H is Hermitian. In class, when Horn got to that, I shouted out "That's my favorite theorem! The unitary part is an isometry, that is, doesn't change lengths or angles and is essentially a rigid motion, maybe just a rotation or reflection. The Hermitian part H is a shocking dream, amazing beyond belief: All H can do is take a circle and make it into an ellipse: The two axes of the ellipse are perpendicular (orthogonal) and the eigenvectors. Their lengths are the eigenvalues. And this generalizes to rounded footballs in three dimensions and all finite dimensions. And with the spectral theorem it generalizes to infinitely many dimensions and is the main reason in quantum mechanics the observables are eigenvalues. The polar decomposition is also the source of the powerful singular value decomposition, principle components analysis, factor analysis, analysis of saddle points in optimization (see W. Fleming, Functions of Several Variables), the matrix condition number in the numerical analysis of Gauss elimination, and much more in theory and applications.

   Hope this helps.

   Show morethread