Statistical methods

What Hacker News says

1. graycatAug 2019
   Most of the best math is not trivial and, thus, usually takes some effort to understand.

   There are some good authors of math for, say, calculus, linear algebra, differential equations, advanced calculus, advanced calculus mostly for applications, real analysis, optimization, probability, some topics in stochastic processes, introductory statistics, various more advanced topics in statistics.

   Mostly the books are short on motivation and applications, and as a result it is too easy to spend time on material likely not worth the time unless you can be sure both to live forever and remember forever.

   For calculus I liked Johnson and Kiokemeister. I taught from Protter and Morrey, and it was easier than J&K. Lots of people liked Thomas.

   For linear algebra, I liked E. Nering and, then, P. Halmos, Finite Dimensional Vector Spaces which really is baby Hilbert space theory. Take Nering seriously -- he was a student of E. Artin at Princeton. His treatment of linear algebra is balanced and polished. For one of his editions, he has some group representation theory in the back, good, and some linear programming, really bad.

   A lot of people like the MIT Strang book.

   For advanced calculus to help when studying physics, especially electricity and magnetism and engineering, I very much liked

   Tom M. Apostol, 'Mathematical Analysis: A Modern Approach to Advanced Calculus', Addison-Wesley, Reading, Massachusetts, 1957.

   He has more recent versions, but for physics and engineering I like the 1957 version and don't like the later versions at all.

   For ordinary differential equations, I liked

   Earl A. Coddington, 'An Introduction to Ordinary Differential Equations', Prentice-Hall, Englewood Cliffs, NJ, 1961.

   He makes variation of parameters look really nice -- then can understand the remark in the old movie The Day the Earth Stood Still. Ordinary differential equations is a huge, old field, and there is some question about how much of that deserves study now. Do notice that for systems of ordinary differential equations, get to apply some linear algebra in cute ways.

   For advanced calculus for applications, there is the old MIT Hildebrand -- he knows what he is talking about, is easy enough to read, and a good place to go if need one of his topics.

   In recent decades, the pure math departments wanted to teach advanced calculus as the theorems and proofs for freshman calculus. So there is Rudin, Principles of Mathematical Analysis, third edition (not the first two, maybe a later edition if there is one). So here's what is going on: He wants to develop the Riemann integral which is the one in freshman calculus. For that he wants to integrate over a closed interval on the real line, that is, some [a,b] which for real numbers a <= b is the set of a real numbers x so that a <= x <= b. Rudin will argue that [a,b] is a compact subset of the reals and show that the Riemann integral exists on all compact sets. So, the first chapters are big on compact sets. Then he talks about functions that are continuous and then ones that are uniformly continuous. With uniform continuity, the Riemann integral follows right away. Later he does some infinite sequences and series and then uses these for careful treatments of some important results, the exponential function, the number e, the sine and cosine, etc. Later he does integration of functions of several variables on manifolds for Stokes theorem and the related divergence theorem, a fully careful treatment of these theorems used in E&M. He does the Cartan exterior algebra: What is going on is that he wants to integrate a function g: M --> R where M is a manifold, that is, the range of some function f from some box, triangle, etc. to the space with the M. So, for this need the formula for change of variable for integrating with several variables, and that is a determinant of a square matrix. This integration is a multidimensional version of the line integral where direction of integration is important -- the exterior algebra is the multi-dimensional version of that. Can see that again in some treatments of general relativity in physics.

   I like Rudin's third edition: Once know what the heck he is driving at and how he is getting there, say, as above, then his high precision is welcome.

   For statistics, I suggest using some popular elementary book as a start. Then learn probability really well and from then on study particular topics in statistics as needed. The current directions in machine learning promise to make lots of particular topics important.

   For a first book on statistics, consider

   George W. Snedecor and William G. Cochran, 'Statistical Methods, Sixth Edition', ISBN 0-8138-1560-6, The Iowa State University Press, Ames, Iowa, 1971.

   My wife did really well with that. So, get a good start on statistics and, then, get to learn some analysis of variance (experimental design), an underrated topic.

   For a second book on statistics, consider

   Alexander M. Mood, Franklin A. Graybill, and Duane C. Boas, 'Introduction to the Theory of Statistics, Third Edition', McGraw-Hill, New York, 1974.

   Here, go quickly and get only the high points and don't expect the math to be very good -- in places it's pretty bad.

   For regression analysis and linear multivariate statistics more generally, there are several books, Maurice M.\ Tatsuoka, Donald F.\ Morrison, William W.\ Cooley and Paul R.\ Lohnes, N.\ R.\ Draper and H.\ Smith. So, in particular, get enough to understand that regression is a perpendicular projection and, thus, get the Pythagorean theorem again.

   For more on such statistics aimed at machine learning, get the Breiman CART -- Classification and Regression Trees, maybe much of the start of ML.

   With that much in statistics, will have seen a lot of applied probability and may be ready for the real stuff. For that, need measure theory, e.g., the first half of Rudin, Real and Complex Analysis or Royden, Real Analysis. Then read Breiman, Probability. After that might read some of Chung, Loeve, Neveu, and maybe some more. Then return to applications including statistics with a really solid foundation in probability, random variables, the classic limit results, and much more. Then can read and/or write lots of advanced topics in statistics.

   For optimization, a similar review is possible, but it's getting late.

   Show morethread
2. pingsweptOct 2010
   For stats:

   Statistical Methods by Snedecor and Cochran

   Statistical Data Analysis by Glen Cowan

   As a gentle introduction to statistics and probability, Statistics by Freedman, Pisani, and Purves.

   thread
3. graycatJun 2017
   Part I

   Beating linear programming as in C-PLEX on networks? In some cases, that's easy: On a lot of networks, the simplex algorithm specializes in an amazing way: A basic solution corresponds to a spanning tree. To consider adding a variable to the basis, just add an arc to the tree, get a circuit, run one unit of flow around the circuit and see if can save money, and if so run the flow around until hit a flow capacity constraint on one of the arcs in the circuit, remove that arc from the basis, and, presto, have a simplex pivot. If use the W. Cunningham (one of my profs) modification of strongly feasible bases, then are guaranteed won't cycle. It's easy to program and just blindingly fast -- e.g., the arithmetic is all just addition and subtraction. It's still the simplex algorithm so that C-PLEX could have been used, but for a big network problem will likely run ballpark thousands of times faster than C-PLEX.

   For more particular cases, maybe more can be done.

   Bertsekas has a polynomial algorithm for that problem. It may get applied to that wacko in Ping Pong Yang.

   Network flows is a an old, big, deep field e.g.,

   Ravindra K.\ Ahuja, Thomas L.\ Magnanti, James B.\ Orlin {\it Network Flows: Theory, Algorithms, and Applications,\/} ISBN 0-13-617549-X, Prentice Hall, New Jersey, 1993.\ \

   Mokhtar S.\ Bazaraa and John J.\ Jarvis, {\it Linear Programming and Network Flows,\/} ISBN 0-471-06015-1, John Wiley and Sons, New York, 1977.\ \

   For analysis of variance, there is, e.g.,

   George W.\ Snedecor and William G.\ Cochran, {\it Statistical Methods, Sixth Edition,\/} ISBN 0-8138-1560-6, The Iowa State University Press, Ames, Iowa, 1971.\ \

   No joke it's from Iowa since it's long been important in agriculture research and testing and in experimental design and analysis much more generally. E.g., people who propose new ways to do K-12 education commonly use these techniques to good advantage.

   Sure, first cut, it's regression analysis again, but with its special discreteness really it's quite different.

   It's powerful stuff.

   For the Gaussian assumptions in regression, don't get too concerned: The normal equations remain and don't need a Gaussian assumption -- there's still a projection and still get the Pythagorean theorem --

   total sum of squares = regression sum of squares + error sum of squares.

   The Gaussian assumption yields F ratio and t-tests, and ballpark in practice those remain robust mostly ignoring the Gaussian assumption. In some wild, edge cases, might want to check the robust claim, but otherwise do get some good information about the fit.

   Show morethread