Linear algebra and matrix theory
"This revision of a well-known text includes more sophisticated mathematical material. A new section on applications provides an introduction to the modern treatment of calculus of several variables, and the concept of duality receives expanded coverage. Notations have been changed to correspond to more current usage."--Publisher description.
Stats for nerds
1recommend
9mentions
2people
2009–2025years
33% recommend67% neutral
Mentions per year
What Hacker News says
- For abstract algebra, there is the polished
I.\ N.\ Herstein, {\it Topics in Algebra,\/}
(markup for TeX word processing).
For Galois theory, took an oral exam on what was in Herstein.
For linear algebra where the field is any of the rationals, reals, complex, and finite fields there is
Evar D.\ Nering, {\it Linear Algebra and Matrix Theory,\/} John Wiley and Sons, New York, 1964.\ \
As I recall, Nering was an Artin student at Princeton.
Some of the proofs for the rational, real, or complex fields don't work for finite fields so for those need special proofs.
Had a course in error correcting codes -- it was applied linear algebra where the fields were finite.
Linear algebra is usually about finite dimensional vector spaces with an inner product (some engineers say dot product), but the main ideas generalize to infinite dimensions and Hilbert and Banach spaces.
- Linear algebra, and more than one such book.
IMHO long and still the best linear algebra book is
Halmos, Finite Dimensional Vector Spaces (FDVS).
It was written in 1942 when Halmos was an "assistant" to John von Neumann at the Institute for Advanced Study. It is intended to be finite dimensional vector spaces but done with the techniques of Hilbert space. The central result in the book, according to Halmos, is the spectral decomposition. One result at a time, the quality of von Neumann comes through. Commonly physicists have been given that book as their introduction to Hilbert space for quantum mechanics.
But FDVS is a little too much for a first book on linear algebra, or maybe even a second book, should be maybe a third one.
Also high quality is Nering, Linear Algebra and Matrix Theory. Again, the quality comes through: Nering was a student of Artin at Princeton. There Nering does most of linear algebra on just finite fields, not just the real and complex fields; finite fields in linear algebra are important in error correcting codes. So, that finite field work is a good introduction to abstract algebra.
For a first book on linear algebra, I'd recommend something easy. The one I used was
Murdoch, Linear Algebra for Undergraduates.
It's still okay if can find it.
For a first book, likely the one by Strang at MIT is good. Just use it as a first book and don't take it too seriously since are going to cover all of it and more again later.
I can recommend the beginning sections on vector spaces, convexity, and the inverse and implicit function theorems in
Fleming, Functions of Several Variables
Fleming was long at the Brown University Division of Applied Math. The later chapters are on measure theory, the Lebesgue integral, and the exterior algebra of differential forms, and there are better treatments.
Also there is now
Stephen Boyd and Lieven Vandenberghe, Introduction to Applied Linear Algebra – Vectors, Matrices, and Least Squares
at
http://vmls-book.stanford.edu/vmls.pdf
Since the book is new, I've only looked through it -- it looks like a good selection and arrangement of topics. And Boyd is good, wrote a terrific book, maybe, IMHO likely, the best in the world, on convexity, which is in a sense is half of linearity.
Some course slides are available at
http://vmls-book.stanford.edu/
For reference for more, have a copy of
Richard Bellman, Introduction to Matrix Analysis: Second Edition.
Bellman was famous for dynamic programming.
For computations in linear algebra, consider
George E. Forsythe and Cleve B. Moler, Computer Solution of Linear Algebraic Systems
although now the Linpack materials might be a better starting point for numerical linear algebra. Numerical linear algebra is now a well developed specialized field, and the Linpack materials might be a good start on the best of the field. Such linear algebra is apparently the main yardstick in evaluating the highly parallel supercomputers.
After linear algebra go through
Rudin, Principles of Mathematical Analysis, Third Edition.
He does the Riemann integral very carefully, Fourier series, vector analysis via exterior algebra, and has the inverse and implicit function theorems (key to differential geometry, e.g., for relativity theory) as exercises.
All of this material is to get to the main goals of measure theory, the Lebesgue integral, Fourier theory, Hilbert space and Banach space as in, say, the first, real (not complex) half of
Rudin, Real and Complex Analysis
But for that I would start with
Royden, Real Analysis
sweetheart writing on that math.
Depending on the math department, those books might be enough to pass the Ph.D. qualifying exam in Analysis. It was for me: From those books I did the best in the class on that exam.
Moreover, from independent study of Halmos, Nering, Fleming, Forsythe, linearity in statistics, and some more, I totally blew away all the students in a challenging second (maybe intentionally flunk out), advanced course in linear algebra and, then, did the best in the class on the corresponding qualifying exam, that is, where that second course was my first formal course in linear algebra.
Lesson: Just self study of those books can give a really good background in linear algebra and its role in the rest of pure and applied math.
No joke, linear algebra, and the associated vector spaces, is one of the most important courses for more work in pure and applied math, engineering, and likely the future of computing.
- Evar D. Nering, Linear Algebra and Matrix Linear Algebra and Matrix Theory*
Kenneth Hoffmann And Ray Kunze. Linear Algebra, 2nd Edition, Prentice-Hall, Englewood Cliffs, New Jersey, 1971.
https://www.zuj.edu.jo/download/linear-algebra-2nd-edition-k...
Halmos, Finite Dimensional Vector Spaces
George E. Forsythe and Cleve B. Moler, Computer Solution of Linear Algebraic Systems
Paul R. Halmos, Naive Set Theory, Van Nostrand, Princeton, NJ, 1960.
More has been done since this book, but this book is a gorgeous introduction to axiomatic set theory. So, even people who want to dig into the latest work would do well to have this as the first book. And for people wanting to read any of the more advanced material here, knowledge of this book will be from good to have to important.
https://piazza-resources.s3.amazonaws.com/is25bi6c6o1oh/isv1...
Walter Rudin, Principles of Mathematical Analysis
The third edition is a lot better than the first two.
H. L. Royden, Real Analysis: Second Edition
Beautifully written, elegant, but maybe don't work way too hard on the exercises about upper/lower semi-continuity, and there is a better summary than Littlewood's three principles.
Bernard R. Gelbaum and John M. H. Olmsted, Counterexamples in Analysis
John C. Oxtoby, Measure and Category: A Survey of the Analogies between Topological and Measure Spaces
Walter Rudin, Real and Complex Analysis
Walter Rudin, Functional Analysis
Leo Breiman, Probability
Kai Lai Chung, A Course in Probability Theory, Second Edition
Jacques Neveu, Mathematical Foundations of the Calculus of Probability
Erhan Cinlar, Introduction to Stochastic Processes
J. L. Doob, Stochastic Processes
I. I. Gihman and A. V. Skorohod, The Theory of Stochastic Processes I, II
Donald E. Knuth, The TeX book
Donald E. Knuth, The Art of Computer Programming, Second Edition
Leo Breiman, "Statistical Modeling: The Two Cultures," Statistical Science, Vol. 16, No. 3, 199–231, 2001.
Paul R. Halmos, "The Theory of Unbiased Estimation", Annals of Mathematical Statistics, Volume 17, Number 1, pages 34-43, 1946.
Paul R. Halmos and L. J. Savage, "Application of the Radon-Nikodym Theorem to the Theory of Sufficient Statistics", The Annals of Mathematical Statistics, Volume 20, Number 2 (1949), 225-241.
https://projecteuclid.org/download/pdf_1/euclid.aoms/1177730...
- > Why Won’t Anyone Teach Her Math?
Some people, a lot of people, will gladly "teach her math" and do a good job of it.
E.g., I tried to, here: In the thread
"Why won’t anyone teach me math?"
at
https://news.ycombinator.com/item?id=30302079
I posted
"Part I"
https://news.ycombinator.com/item?id=30307316
and
"Part II"
https://news.ycombinator.com/item?id=30307349
The two parts are needed because of length limits of HN posts.
The OP (original post)
"Why won’t anyone teach me math?"
is about a coed at Princeton in a badly taught course in linear algebra.
It turns out, I did really well in linear algebra, and "Part I" explains how well I did and how I did that.
In "Part II" I give an outline of an introduction to linear algebra and matrix theory.
For my background I've taught applied math and computer science at three well known research universities and have a Ph.D. in pure/applied math from a fourth.
From how I learned math I have to say that there is a lot of having contempt for the students and dumping on them.
This dumping on the students, treating them with contempt, is unnecessary, unjustified, inexcusable, and bad for the students, researchers, teachers, schools, math, and the US.
The dumping isn't just in famous research universities: E.g., I went in grades 1-12 to the unique college prep school in the city. I got dumped on from the 4th grade on and maybe also in the first three grades but simply don't remember much about grades 1-3.
I wanted to learn but got dumped on so much that I just gave up trying to please the teachers and did what I could with the books for the courses. Turns out, that was a good thing for me to do, especially for learning math.
There were a lot of standardized tests, both aptitude and achievement, and apparently I did well on those, much better than how the teachers viewed me. Point: The opinions of teachers who dump on students can be highly inaccurate; some more objective measures are needed.
Due to the objective measures, in the 9th grade I got sent to a math tournament and in the 11th grade, to an NSF math & physics summer program. Point: The objective measures helped. Turns out, that was true throughout my education.
But for college I went to a private school that took lots of pride in doing well at teaching. Some of the teaching was quite good. Still, however, there was some dumping: E.g., the college got a new prof fresh from a famous research university, and he brought his dumping/contempt propensities with him. At one point I took a reading course from him from a then famous, advanced, notoriously difficult book in topology. So, for each chapter one week I gave a lecture to the prof on the math and the next week, on the exercises. I did well. Once another prof showed up. But even with that good self-teaching success, the prof at best only slightly relaxed his dumping/contempt mode. Point: In that reading course I was a spectacularly good student but the contempt still continued.
My Summary View:
I see no good reason or excuse for the dumping and contempt. I refused to do any such things in my teaching.
Generally I told the students: The material is in the book. Class is to help you learn. So, I'm here to explain and answer questions. There is homework which is to work as many of the exercises as you need to to learn the material, in particular to get good at working the exercises. The tests are problems similar to the exercises -- if you are good on the exercises, you should be good on the tests. From the numerical scores on the tests, I rank the students, and the purpose is to simplify the assignment of letter grades: So, if student A does better in the ranking than student B, then student A will get the same or better letter grade than student B.
But WHAT letter grades? My standards are, A means that know the material well; B, knows the material; C, knows some of the material; D, doesn't know much; F, doesn't know much and none of us want the student to get credit for the course. So, I look at the work on the tests, judge A-F and decide what final scores deserve what letter grades. Students are not in competition: E.g., I'm perfectly willing to assign As to all students who know the material well; would be pleased to do so; the situation would represent a big success. The department has no rules against my assigning As to everyone.
I'm available in class, after class, and in office hours. Generally the office hours are not heavily used.
The US colleges and universities have some really big heartburn about the comparative roles of teaching, applications, and research.
Research: Can argue that academic research is by a wide margin the most powerful contributor to civilization, health, wealth, understanding, etc. So, with lots of research universities where the profs are required to publish in peer-reviewed journals of original research and hopefully get prizes for work they have done and grants for the work they are doing will, year by year we get some astounding results. Quite broadly the most respected research is theory based on math, theory that mathematizes the field. Applications and teaching are regarded as much less desirable.
Applications. Schools of medicine take applications fully seriously -- that is, a lot of the teaching and research are done in research-teaching hospitals with patients at risk of dying. The field of medicine has a major and respected role for clinical, professional practice. The MD degree, to be a "doctor", is about practice, not research.
In schools of engineering, there can be some significant attention to applications. Still, generally engineering schools do not try nearly as hard as medical schools to purse practice, be clinical or professional.
Teaching. There usually is a lot of good teaching in colleges that emphasize teaching over research. But generally the research universities and their profs look down on teaching -- bad for the field, students, profs, schools, and the US.
Generally there is a belief, attitude, assumption, expectation, tradition, etc. that school is to be a competition, pick winners and losers, be challenging, be a filter, have the students put up with being dumped on, get lessons in going without sleep, etc. As a prof, I refused to play that game -- it was in the interest of all my students to work together to learn, come to my office hours, confess what they didn't know, and get additional explanations, problem solutions, etc.
My view now, from this thread and more, is that the US research universities are on thin ice, about ready for their "comeuppance". Via taxes, tuition, etc., the public is paying the bills and wants results -- contributions to health and wealth, national security, and especially good careers for the students.
Bad teaching, dumping, contempt, generally hurting the students, are creating some really bad PR for the research.
Next, for the teaching and a lot of what the customers want now is arriving via the Internet, e.g., YouTube. I did some teaching here in the "Part II" I wrote.
A lot of that teaching is NOT good, but the best of it is getting good. When there is some really good teaching on the Internet for free or nearly so, bad teaching, hurting the students, etc. will come to hurt the universities, even the Ivy League.
It's possible for a really good student, PBK, etc., brilliant, dedicated, to be so dumped on, treated with so much contempt, that their anxiety level gets really bad, they lose sleep, have physical symptoms, get depressed, clinically depressed, don't recover, and commit suicide. Some education. We're expected to pay for such abuse, via taxes, tuition, our health, our lives?
Bad teaching, hurting the students -- NOT acceptable, grade F.
- My view, from math, computing, and computer science is (1) that the author is being too critical and often throwing out a small but good baby with some not very dirty bathwater and (2) that he is not mentioning some larger problems, worse "sins".
Here would be my list of the worst sins or how to avoid them:
(1) A term is a word used with other than a standard dictionary definition. Technical fields are awash in terms. Make sure to try hard to define all the terms you use; with some judgment applied, can omit definitions of some terms certainly well understood by any reader with enough background to get anything from the paper. Poorly or undefined terms can be one of the best ways to lose readers. Here, hoping to get more readers, maybe inviting them into the field, bend over backwards in defining terms, that is, maybe give some term definitions for readers who, really, have little hope of getting much from the paper -- at least, when such a reader gives up they won't blame the author!
In addition, for any term new or relatively new to the field, with some judgment, might also include for the term motivation of its importance and examples of its usage -- i.e., make clear the importance, relevance, value, usage, etc. of the term.
(2) For nearly all acronyms, standard in the field or not, for the first use of the acronym include what it abbreviates. Here to be sure are doing well on this issue, strain and bend over backwards and include, say, even TCP (transmission control protocol), SMTP (simple mail transfer protocol), CPU (central processing unit) -- sure, I'm suggesting bending over backwards. Several slaps on the wrist, an hour in the corner with wearing a dunce cap, and a coating of tar and feathers for each acronym used but not defined with what it abbreviates.
(3) For how to write math, in two parts, (A) and (B):
(A) Take some theorem proving courses from some of the most precisely written texts, say, P. Halmos, Finite Dimensional Vector Spaces, W. Rudin, Principles of Mathematical Analysis, Neveu, Mathematical Foundations of the Calculus of Probability, E. Nering, Linear Algebra and Matrix Theory, Royden, Real Analysis where do a lot of the exercises as homework and where the professor DOES read and remark on your writing. Neveu was a Loeve student at Berkeley. Nering was an Artin student at Princeton. As I recall, Rudin's background was in Austria, although I don't know who his professors were. Of course, Royden was long at Stanford. For more, if have time and insist on some really good examples, read some of Bourbaki.
By the way, on use of we, that is standard. So we might have,
"Given topological spaces X and Y and a function f: X --> Y, we say that function f is continuous provided for each set B open in Y f^(-1)(B) is open in X.
So, that's a vanilla example of using we in mathematical writing.
(B) Do some things none of those texts do: Include some intuitive views, some helpful pictures, motivation via applications in math and also outside math, and outlines of research directions.
There is a lot of question about how appropriate is suggestion (B); in math my guess is that there is no question about the relevance, wisdom, importance, value, etc. of suggestion (A).
For the goal of writing math, suggestion (A) is important: Tough to expect good success without the texts I listed or other texts written with similar care. That level of care is extreme, tough to find and learn, and much tougher to do. Writing math with the care of those texts seems to have been understood and practiced significantly often only after about 1950 or 1940. And for at least one course from at least one such text, DO have the good homework grading of a good math professor.
Personal experience and lesson: While I'm not much interested in being a professor, I have published some papers, and from getting those papers reviewed my guess would be that about half of the reason the papers passed review, and they always did with no significant revision or difficulty, is that I wrote the math with nearly the care and precision of, say, W. Rudin. Point: To critical readers, any lapse in that level of care and precision can be like a worm in a baked apple, perhaps otherwise terrific from brown sugar, butter, etc. Or, might guess that, in nearly any field, the good work is less than 10% of the total, maybe less than 1%, with the rest flawed, maybe as bad as that apple with a worm. In writing math, it is way too easy to be in the bottom 99% or 90% just from the care and precision of the writing, and suggestion (A) is IMHO (in my humble opinion) a good way to have at least the writing quality keep you in the top 90+%.
For math used in computer science, for how to write that math carefully, there are of course examples from D. Knuth.
- Part II
Again, for variable t, think
That approach to the meaning of a variable works well enough for essentially all of math."There is a number; call it t".Calculus was invented mostly by I. Newton, mostly for physics, especially for explaining the motions of the planets. Since then calculus has become a pillar of civilization, especially Western Civilization.
Mostly in calculus, integration is finding the area (or more carefully defining the area) under a curve, maybe a parabola.
But can also have a line integral, say, the work do when carrying 100 pounds of hay to the top of the hay loft of the barn. Here we integrate from beginning to end of the work. If we let the hay fall, then neglecting friction, etc. get the work back as energy. So, have to pay attention to the direction of the integration, from the ground up to the hay loft or from the hay loft back to the ground. The two values have opposite signs.
You can teach yourself calculus: Just get 1-4, at least one good calculus texts and dig in -- read the chapters, follow the material, work nearly all the more difficult exercises, and maybe check 1-4 times a year with a good mathematician. I did that for freshman college calculus (but didn't check with anyone). I never took the course and, instead, started on sophomore calculus and made As. No problem.
(3) Linear Algebra. Likely the next course would be linear algebra. It would be good to do this subject 2-4 times from more elementary treatments to some of the more advanced material. Some of the more advanced material can cover linear programming optimization, classic two person game theory, the proof of the Nash result in game theory, some treatments of the fast Fourier transform in digital signal processing, group representations in the quantum mechanics of molecular spectroscopy, error correcting codes in algebraic coding theory, numerical methods, and more.
The start of linear algebra is just one or several linear equations in one or several variables. E.g., for two equations in three variables, unknowns:
For positive integers (whole numbers) m and n, given m such linear equations in n variables (unknowns), the number of solutions, depending also on the numerical constants in the equations, is none, one, or infinitely many.2x + 3y - 6z = 0 5x - 2y + 7z = 3The main way to find all the solutions is just Gauss elimination.
That covers a good chunk of a first course in linear algebra.
The notation
gets to be clumsy so we write instead2x + 3y - 6z = 0 5x - 2y + 7z = 3
write that with big, square brackets, and call it a matrix with two rows and three columns. The individual numbers are components. Of course, a matrix is conceptually close to an array with two subscripts common in programming languages. Maybe we call the matrix2 + 3 - 6 5 - 2 + 7
A. Since matrix A has 2 rows and 3 columns, we say that it is 2 x 3 ("2 by 3").2 + 3 - 6 5 - 2 + 7Then we write the x, y, z in a column
apply the big, square brackets, and call it a matrix with 3 rows and 1 column and maybe call it v. We write the right sidex y z
also with brackets, and maybe call it b.0 3Then we define a product, a matrix product Av so that
means just the same asAv = b
From then on we work with matrices and try to avoid notation like2x + 3y - 6z = 0 5x - 2y + 7z = 3
In this way, a course in linear algebra is commonly called a course, Linear Algebra and Matrix Theory.2x + 3y - 6z = 0 5x - 2y + 7z = 3A matrix with just one column is a column vector; with just one row, a row vector. In short, either is called a vector.
Linearity: Given m x n matrices A and B, m x 1 matrices u, v, and real numbers c, d, we have that
Here we regard Au as case of function A with argument u. That is, we could have said that matrix A defines function f so thatA(cu + dv) = cAu + dAv
Withf(u) = Au
we form products cA and dA and sumscA + dA
So, we have to define both of those: The definitions are close to just obvious.cA + dAThe equation
means that matrix A acts like a linear function. Big deal! The real world is awash in linear behavior, and linearity is an enormously powerful property mathematically. The rest of linear algebra and matrix theory is nearly all about the consequences of such linearity.A(cu + dv) = cAu + dAvThe fundamental theorem of algebra is that each (high school style) polynomial in variable x can be uniquely factored into a product of terms of the form (ax + b) where the a and b are possibly complex numbers. Essentially from this fact, the more advanced parts of linear algebra use the complex numbers and not just the real numbers.
At the end of a second course in linear algebra, will pay attention to the essentially geometrical notions of length and angle and, in particular, to orthogonality, that is, perpendicular. Big topics are the Gram-Schmidt process where find some orthogonal vectors and the polar decomposition.
The polar decomposition result says that a matrix acting on a circle will yield an ellipse, that is, for matrix A and vector u, if we let u take on the values of all the points on a circle, then the Au will generate all the values of all the points on an ellipse. The ellipse has two axes, and they are mutually perpendicular. So, if have the vectors of the two axes, then have enough to construct the whole ellipse.
In linear algebra, this situation generalizes to any finite dimension and yields the singular value decomposition, principle components, factor analysis, and more.
With more advanced work, this situation becomes the spectral theory of self-adjoint linear operatiors central to quantum mechanics.
(4) The Four Main Parts of Math
There are four main parts of math:
(A) Foundations, that is, deep in the basement with set theory.
(B) Algebra, as in high school, the integers, prime numbers, the fundamental theorem of arithmetic, greatest common divisor and least common multiple, the fundamental theorem of algebra, linear algebra and matrix theory, the generalizations of number systems such as groups, rings, fields (the rational, real, and complex numbers are all fields but so is the set of integers modulo a prime number), etc. Fermat's last theorem, settled by A. Wiles, is part of algebra; so are the deep, difficult questions about prime numbers, etc. There are applications in error correcting codes and cryptography.
(C) Geometry, e.g., high school plane geometry, analytic geometry, and differential geometry, a deep subject important for relativity.
(D) Analysis, as in calculus, differential equations, functional analysis, e.g., Banach and Hilbert spaces, partial differential equations, e.g., as in Maxwell's equations, fluid flow, and relativity, probability, statistics, stochastic processes, optimization, and more.
In differential equations we are given an equation with the derivatives of some function and want to find the function.
E.g., for real valued function y(t) and constants k and b, we might have
where y'(t) = d/dt y(t), the first derivative of function y(t). At one time, that little differential equation saved FedEx from going out of business. The solution is a lazy S curve and a first cut at viral growth.y'(t) = k y(t) (b - y(t))There is topology which is partly in geometry and analysis. The main idea of topology is continuity, that is, changing without sudden jumps or some cases of wildly fast oscillations.
There is algebraic geometry partly in algebra and geometry.
Much of number theory has deep connections with analysis.
Much of linear algebra is an introduction to functional analysis in analysis.
The older applied math is mostly analysis.
(5) Analysis. Usually after linear algebra will study advanced calculus and analysis. Broadly there are two approaches, (A) theory and (B) applications.
The theory is mostly to give fully careful proofs of the results and first generalizations of what you saw in calculus. For the theory maybe the most respected text is
R. Rudin, Principles of Mathematical Analysis, Third Edition.
For this, get the third edition and not either of the two earlier editions.
So, will discover that the integral learned in calculus is called the Riemann integral because B. Riemann made the theorems solid. Rudin also does the easy generalization to the Riemann-Stieltjes integral.
Related
Introduction to Applied Linear Algebra
7 recommend · 14
Computer solution of linear algebraic systems
2 recommend · 8
Topics in algebra
10 recommend · 15
Linear programming
5 recommend · 7
Finite-dimensional vector spaces
24 recommend · 89
Linear algebra
10 recommend · 20
