Stats for nerds
3recommend
10mentions
3people
2013–2023years
80% recommend20% neutral
Mentions per year
What Hacker News says
- Measure and Category by John Oxtoby. This book studies duality results between different notions of "small" sets in measure theory and topology. It's the first (and to some extent the only) math book where things just clicked and I didn't feel like I was drowning in a sea of notation and ideas. Here are some more thoughts on it: https://bcmullins.github.io/Top-Books-2019.
- Yes, the set of reals, as in the usual construction, is the only complete Archimedean ordered field.
And some amazing properties are (with my TeX markup) in
John C.\ Oxtoby, {\it Measure and Category:\ \ A Survey of the Analogies between Topological and Measure Spaces,\/} ISBN 3-540-05349-2, Springer-Verlag, Berlin, 1971.\ \
- Evar D. Nering, Linear Algebra and Matrix Linear Algebra and Matrix Theory*
Kenneth Hoffmann And Ray Kunze. Linear Algebra, 2nd Edition, Prentice-Hall, Englewood Cliffs, New Jersey, 1971.
https://www.zuj.edu.jo/download/linear-algebra-2nd-edition-k...
Halmos, Finite Dimensional Vector Spaces
George E. Forsythe and Cleve B. Moler, Computer Solution of Linear Algebraic Systems
Paul R. Halmos, Naive Set Theory, Van Nostrand, Princeton, NJ, 1960.
More has been done since this book, but this book is a gorgeous introduction to axiomatic set theory. So, even people who want to dig into the latest work would do well to have this as the first book. And for people wanting to read any of the more advanced material here, knowledge of this book will be from good to have to important.
https://piazza-resources.s3.amazonaws.com/is25bi6c6o1oh/isv1...
Walter Rudin, Principles of Mathematical Analysis
The third edition is a lot better than the first two.
H. L. Royden, Real Analysis: Second Edition
Beautifully written, elegant, but maybe don't work way too hard on the exercises about upper/lower semi-continuity, and there is a better summary than Littlewood's three principles.
Bernard R. Gelbaum and John M. H. Olmsted, Counterexamples in Analysis
John C. Oxtoby, Measure and Category: A Survey of the Analogies between Topological and Measure Spaces
Walter Rudin, Real and Complex Analysis
Walter Rudin, Functional Analysis
Leo Breiman, Probability
Kai Lai Chung, A Course in Probability Theory, Second Edition
Jacques Neveu, Mathematical Foundations of the Calculus of Probability
Erhan Cinlar, Introduction to Stochastic Processes
J. L. Doob, Stochastic Processes
I. I. Gihman and A. V. Skorohod, The Theory of Stochastic Processes I, II
Donald E. Knuth, The TeX book
Donald E. Knuth, The Art of Computer Programming, Second Edition
Leo Breiman, "Statistical Modeling: The Two Cultures," Statistical Science, Vol. 16, No. 3, 199–231, 2001.
Paul R. Halmos, "The Theory of Unbiased Estimation", Annals of Mathematical Statistics, Volume 17, Number 1, pages 34-43, 1946.
Paul R. Halmos and L. J. Savage, "Application of the Radon-Nikodym Theorem to the Theory of Sufficient Statistics", The Annals of Mathematical Statistics, Volume 20, Number 2 (1949), 225-241.
https://projecteuclid.org/download/pdf_1/euclid.aoms/1177730...
- How I learned: Early in my career, I was around DC doing mostly work in applied math and computing for US national security. No joke -- constantly the work was heavily probability, statistics, and stochastic processes. I had a good ugrad math major but no courses in any of those three subjects. So I was thrown into the deep end of the pool and was constantly struggling to understand. I did pick up a good overview and a lot of intuition. But the sources varied widely, in both the topics and the quality, over stacks of books and papers, documentation of software, etc.
Lesson: At least at first, one way to learn is just to jump in at the deep end and struggle using lots of texts, references, etc.
Sad Lesson: While nearly all the famous books were good, some of the books that, e.g., from the publisher, might have seemed good were not. The guy who wrote the stuff, I hope he got tenure -- can't be any other reason.
Later I got an applied math Ph.D. and had a terrific course in analysis and probability. So, the analysis part was, right, basically Royden, Real Analysis and the first half of Rudin, Real and Complex Analysis. There was also some material from Oxtoby, Measure and Category (ice cream and cake dessert -- super fun stuff).
The probability was right from the beginning sigma algebras, etc. So, a central topic was the Radon-Nikodym theorem and conditional expectation -- gorgeous once see it. So, there was beautiful coverage of the classic limit theorems, especially martingales.
Best course of any kind I ever took in school. The prof was a star student of E. Cinlar, long at Princeton.
For the course, the main texts in probability were from J. Neveu, L. Breiman, K. Chung, M. Loeve.
For statistics, for the applied stuff, I just remember the stacks of books I worked with early on, especially multivariate statistics. For the math, I just regard that as applied probability and sometimes just do my own derivations, sometimes at least a little new. I never found a statistics book I like or can recommend as the single, main book, e.g., like Rudin in analysis or Neveu in probability. All I can suggest is just to dig into the stacks of the most famous books and also glance at some of the software documentation.
I suspect that there is a really good statistics book to be written, and maybe someone has written it, or is writing it, but I haven't seen it.
Here is a simple derivation I typed in yesterday with an intuitive result in statistics that maybe people should keep in mind. In a sense this little derivation shows the strongest possible result in statistical estimation is, and may I have the envelope please [drum roll], and the discrete data version of the winner is just cross tabulation, assuming that have enough data.
The context is a person applying for credit. Might proceed similarly for, say, ad targeting, etc.
We assume that Y is a real valued random variable where E[Y^2], that is, the expectation, of Y^2 is finite -- meager assumption, especially for practice.
The Y is something about credit worthiness, e.g., loss on a loan, we are interested in.
We assume that X is a random variable taking possibly very general values, e.g., a credit history at uncountably infinitely many points in time in the past. We assume that we have the value of X -- that's our credit data on the person.
Let's do a little preliminary derivation: What value of real number a minimizes
E[(Y - a)^2]
Well, we have
E[(Y - a)^2]
= E[Y^2 - 2 Ya + a^2]
= E[Y^2] - 2aE[Y] + a^2
= E[Y^2] + E[Y]^2 - 2aE[Y] + a^2 - E[Y]^2
= E[Y^2] + (E[Y] - a)^2 - E[Y]^2
which we minimize with a = E[Y].
Or, for one interpretation, the minimum rotational moment of inertia is for rotation about the center of mass.
So, for our main concern, suppose we want to use the data we have X to approximate Y. So, we want real valued function f with domain the possible values of X so that f(X) approximates Y.
For the most accurate approximation, we want to minimize
E[(Y - f(X))]^2
Claim: For f(X) we want
f(X) = E[Y|X]
So, f(X), using X, is the best non-linear least squares approximation to Y.
Proof:
We start by using one of the properties of conditional expectation and then continue with just simple algebra:
E[(Y - f(X))^2]
= E[ E[Y^2 - 2Yf(X) + f(X)^2|X] ]
= E[ E[Y^2|X] - 2f(X)E[Y|X]
+ f(X)^2 ]
= E[ E[Y^2|X] E[Y|X]^2 - 2f(X)E[Y|X]
+ f(X)^2 - E[Y|X]^2 ]
= E[ E[Y^2|X]
+ (E[Y|X] - f(X))^2
- E[Y|X]^2 ]
which is minimized with
f(X) = E[Y|X]
Done.
- Part II
Starting there, as you go on in real analysis, you will place much less emphasis on continuity. The main work will replacing the Riemann integral by something that does much better on the edge or pathological cases.
Well, the Riemann integral was, right, a lot about area. Well, it's actually not a really good theory of area -- doesn't have all the nice properties we would wish for. So, that view of area gets polished up and replaced by measure theory. Next, that theory of area is used to define the Lebesgue integral, which is much nicer than the Riemann integral, e.g., gets rid of the assumptions of a compact set and continuity.
Lebesgue was a student of E. Borel and did his work near 1900. You will see the Heine-Borel theorem in Baby Rudin.
In simplest terms, the Lebesgue idea is to partition on the Y axis instead of the X axis like the Riemann integral does. Just why that works so well is cute. It also frees up the domain of the function to be much more general, and in 1933 A. Kolmogorov used that fact, finally, to give a good foundation for probability theory, the one accepted now in essentially all advanced work in probability, stochastic processes, and mathematical statistics.
Likely the nicest book to read on real analysis is H. Royden, Real Analysis. But also, later, read the first, real half of W. Rudin, Real and Complex Analysis.
Note: For real analysis, complex valued functions of a real variable are important; functions of a complex variable are quite different and not important; Rudin has some deep reasons to disagree with me. Rudin is no doubt correct; for students trying to learn, I'm correct!
Once you have the main ideas in mind, and maybe what I've given here will be enough, Rudin is more succinct and nicer to read than Royden. But read them both, Royden first. Go through Royden quickly since will do it all again in Rudin.
In Royden, notice his Littlewood's Three Principles as a cute view of what is going on in measure theory and the Lebesgue integral. Royden also has some exercises on upper and lower semi-continuity -- glance at those and get the main idea for the remaining connection with continuity. Notice the nice treatment of differentiation -- the connection between integration and differentiation is weaker for the Lebesgue integral because the integral is so much more general. Then notice the Carathéodory extension result -- that is the key to defining measures on the real line, in particular, Lebesgue measure.
Spend at least a weekend in
John C.\ Oxtoby, {\it Measure and Category:\ \ A Survey of the Analogies between Topological and Measure Spaces,\/} ISBN 3-540-05349-2, Springer-Verlag, Berlin, 1971.\ \
An amazing weekend.
Keep at hand
Bernard R.\ Gelbaum and John M.\ H.\ Olmsted, {\it Counterexamples in Analysis,\/} Holden-Day, San Francisco, 1964.\ \
Notice that for the Lebesgue integral, there is one, central proof technique in just four steps: Prove the theorem for a function that is just a positive constant defined on a set, say, just a box. By linearity, generalize to finite sums of such functions. Then by monotone continuity (see early on the monotone convergence theorem), prove the result for all non-negative functions. Last by linearity again, get the result for all integrable functions.
Okay, for integrable, there is a cute approach, trick: Define the integral for non-negative functions, even ones defined on the whole real line or all of R^n (for the set of real number R and a positive integer n) or an abstract measure space. Okay, easily enough, that integral can have value positive infinity (the Riemann integral gets sick here). Then do the same for a function that is all <= 0. Then given a function, write it as the sum of its positive part and its negative part. Integrate those two parts separately. Now, if at least one of those two integrals is finite, then add them for the integral of the given function. So, we are okay in all cases except where the positive part integrates to positive infinity and the negative part integrates to negative infinity -- we don't want to subtract infinities because permitting that ruins the laws we want for arithmetic.
For this four step proof technique, can knock off a really nice version of Fubini's theorem, that is, interchange of order of integration.
The Lebesgue integral with the dominated convergence theorem, which will see early on, give nice versions of differentiation under the integral sign.
Note: Commonly in physics, engineering, probability theory, etc. we write integrals over the whole real line. Alas, really, for Riemann integration, the appropriate theorems are rarely presented; they are not in Baby Rudin. Really, the Lebesgue integral is needed. So, you've been needing the Lebesgue integral for a long time.
Then get to apply the Lebesgue integral to the powerful Radon-Nikodym theorem (Rudin gives von Neumann's cute proof based, amazingly, in part on just polynomials), get the main, important duality theorems (the Lebesgue integral is the main linear operator!), Banach spaces, Hilbert spaces, and the Fourier integral.
Cover that and can claim you have a good start on real analysis. You will also have apparently so far the only good background for probability, stochastic processes, and mathematical statistics. E.g., the Radon-Nikodym theorem is crucial for conditional expectation, Markov processes, martingales (amazing things with one of the strongest inequalities in math and a super short proof of the strong law of large numbers), ergodic theory, sufficiency in statistics, a quite general proof of the Neyman-Pearson result in statistical hypothesis testing, and more, e.g., the role of Brownian motion in potential theory, stochastic optimal control, ....
Go for it!
- What's your take on this? Got a counter proof to Cantor?
As much as I am a fan of all things about cardinals, measure and category (Oxtoby's booklet on this stuff has been on my nightstand for several months, to a great pleasure), I still fail to recognize its technological and physical implications.
My take on this is modelled after the famous article "The dawning of the age of stochasticity" by David Mumford, where he argues that the current definition of real numbers leads to infuriating contradictions, especially in the light of probability theory. This article left me with the impression that in a near future, we will see a clever definition of "real-valued random variable" that does not depend on the definition of real numbers and avoids all kind of disgusting paradoxes.
Related
Measure theory
2 recommend · 10
Introduction to geometry
2 recommend · 2
Mathematical Analysis
6 recommend · 14
Topics in algebra
10 recommend · 15
Category Theory for Programmers
51 recommend · 90
Category theory
7 recommend · 9
Category theory for computing science
recommend · 7