Stats for nerds
1recommend
3mentions
3people
2009–2017years
33% recommend67% neutral
Mentions per year
What Hacker News says
- Early in my career, I asked that question. Eventually I got some answers.
Broadly usually the 'big bucks' are in things that are new at least in some important sense. To find such things, one recommended approach is to stand "on the shoulders of giants". Then you may still do some original work, but you have one heck of a base. My view is that Moore's law, the Internet, etc. are not nearly fully exploited, that enormous value remains, and the best way to get some of this value is to find a big unsolved problem where a good solution will be very valuable, stand on the shoulders of some giants, do some original work to get some powerful 'secret sauce' for solving the problem (which appears not to be easy to solve otherwise), start a business, please customers, get revenue, grow the business and sell it.
Here, however, just what problem, giants, original work, secret sauce are NOT easy to see. Broadly we are trying to see, no, MAKE, the future, and usually that is not easy. While we can see what applications have been made of statistics in the past, what statistics will be applied where in the future is not easy to see.
Yes, broadly statistics is highly promising. Or, in 'information technology' we want valuable, new information, are awash in cycles, bytes, bandwidth, infrastructure software, and data, and want to use these to create the information. Here, on appropriate problems, statistics can commonly, easily, totally 'blow away' anything else. The marriage between (A) 'information technology' in business and (B) statistics has yet to be consummated, even to reach to 'going steady' stage, assuming the traditional order of events.
For 100 years or so, people have come to statistics from various areas of work. Usually they had some data and some questions. Some of the areas have been educational and psychological testing, experiments and testing in agriculture and medicine, industrial quality control, model building in economics, experimental work in physics and chemistry, investing, attempts to create mathematically based sciences in the social sciences, especially economics, psychology, sociology, and political science. In sociology, old examples would be James Coleman, Pete Rossi (professors of my wife when she got her Ph.D.), Leo Goodman.
Likely the best fast, practical path into statistics is via books, courses, etc. intended for students in the social sciences. These students commonly do not have good backgrounds in mathematics. For the mathematical prerequisites, generally can get by, as a start, with just high school first year algebra. With this path can learn about probability distributions, the central limit theorem, the law of large numbers, statistical estimation and confidence intervals, hypothesis testing, cross tabulation, analysis of variance, regression analysis, principle components analysis, and more.
For more, statistics is a rock solid part of mathematics, as solid as any part of pure mathematics, e.g., topology, geometry, analysis, algebra, etc. That is, statistics is based solidly on theorems and proofs, sometimes relatively deep ones.
Statistics as theorems and proofs is called 'mathematical statistics'. Long standard has been (with TeX markup):
Alexander M.\ Mood, Franklin A.\ Graybill, and Duane C.\ Boas, {\it Introduction to the Theory of Statistics, Third Edition,\/} McGraw-Hill, New York.\ \
The main prerequisite for this book is just a not very good course in calculus, and the book actually makes not much use of calculus. Mostly all a student will need from calculus is that it can find the area under a curve. Since the book has long been standard, can't really ignore it, but it's ugly. And often, with just calculus, the book doesn't really give solid proofs of the results. E.g., their treatment of sufficient statistics has some nice intuition, but their proof is junk. The subject cries out for a good book, but I'm not trying to write one or waiting for someone else to.
Can get some of the flavor of mathematical statistics done with high quality, as mathematics, in, say (with TeX markup):
Jean-Ren\'e Barra, {\it Mathematical Basis of Statistics,\/} ISBN 0-12-079240-0, Academic Press, New York.\ \
Robert J.\ Serfling, {\it Approximation Theorems of Mathematical Statistics,\/} ISBN 0-471-02403-1, John Wiley and Sons, New York.\ \
P.\ Billingsley, {\it Convergence of Probability Measures, 2\raise0.5ex\hbox{ed},\/} ISBN: 0-471-19745-9, John Wiley, New York.\ \
R.\ S.\ Lipster and A.\ N.\ Shiryayev, {\it Statistics of Random Processes I, II,} ISBN 0-387-90226-0, Springer-Verlag, New York.\ \
However, pursued mathematically, statistics has some relatively advanced prerequisites some of which curiously are not popular in US university mathematics departments.
For the prerequisites,
High School. Should have had high school first and second year algebra (reasonable facility with algebraic manipulations, the binomial theorem, complex numbers, both of which will see again in important ways), plane geometry (where nearly all the work was proofs -- first place to learn about proofs), trigonometry (usually assumed in calculus and important in, say, analysis of organ tone harmonics and, thus, the most important example of an infinite dimensional Hilbert space), analytic geometry (especially the conic sections, especially ellipses which definitely will see again), and, if can, solid geometry (for more intuition in three dimensions).
College. Need a standard calculus course, not necessarily a very comprehensive or difficult one because will do the subject all over again, maybe two or three times, and more later, WITH the proofs!
Then need linear algebra, that is, how to work with data of several dimensions, which is just crucial. The big result is the polar decomposition, and there get to think about ellipses and get to use complex numbers. Also the course is an introduction to functional analysis and Hilbert space. Use any popular book to get started but in the end cover the classic, Halmos, "Finite Dimensional Vector Spaces'. Halmos wrote this when he was an assistant to von Neumann and intended it to be a finite dimensional introduction to Hilbert space (which once von Neumann had to explain to Hilbert) which it is. It also has some multi-linear algebra, of interest to exterior algebra now popular in relativity, but likely for nearly any business applications of statistics for the next several decades can skip that chapter.
Then need some advanced calculus. That is a poorly organized, huge, catch-all subject beyond any one course. The usual start is 'baby' Rudin, 'Principles of Mathematical Analysis'. So, that's calculus with the proofs. Warning: The book is severe, succinct, with zero pictures. Have to draw your own pictures in your head. The book is packed solidly with powerful material just awash in important applications from statistics, economics, and engineering to physics, but there is hardly a hint of the applications in the book. I enjoyed the book, but few people will enjoy it or even get through it. Hint: Get a really good teacher! Then for more, popular is Spivak, 'Calculus on Manifolds', mostly because it is short. Actually, it's too short. I prefer Fleming, 'Functions of Several Variables' until get to the exterior algebra chapter at which time, if care, can now get the thin
Henri Cartan, {\it Differential Forms,\/} ISBN 0-486-45010-4, Dover, Mineola, NY, 2006.\ \
in English.
Between Halmos, baby Rudin, and Spivak, you will have covered Harvard's Math 55 with a colorful description at
http://www.american.com/archive/2008/march-april-magazine-co...
Harvard tries to cover these three for freshman, but in most math departments the material will take you through all or nearly all of a focused undergraduate pure math major.
If somewhere take a course in abstract algebra, e.g., with a little group theory, then that might help!
Graduate School. Might learn a little more about topology, say, from Simmons, 'Introduction to Topology and Modern Analysis'. So, get good with metric spaces and get started on duality.
The next big step is a course in measure theory and functional analysis. The Simmons work will help. Baby Rudin will be crucial; Halmos is recommended. So, with measure theory, do calculus over again and in a very different and much more powerful way and a way just crucial, even central, for mathematical approaches to statistics. The functional analysis will concentrate on representation theorems, the Radon-Nikodym theorem, and Hilbert and Banach spaces. Long popular, from Stanford and sometimes aimed at statistics students, is Royden, 'Real Analysis'. It's gorgeous. Should also read the real half of Rudin, 'Real and Complex Analysis'; it's a few steps up in difficulty from baby Rudin. Again, hint: Get a good course from a good teacher who can get you over the material without getting stuck. Then go back and study the material 2-3 more times, apply it, do some original research in it, and finally begin to understand it.
We're talking high, top, center crown jewels of civilization here; the stuff is of just awesome power; my view is that it is, for the rest of this century, one of the main pillars of increases in economic productivity via the exploitation of Moore's law; on 'what to program', computer science is stuck and this material is the most promising way forward; as a famous restaurant owner once said about some Morey St. Denis, "you won't find better".
Now are ready for probability. I recommend:
Leo Breiman, {\it Probability,\/} ISBN 0-89871-296-3, SIAM, Philadelphia.\ \
M.\ Lo\`eve, {\it Probability Theory, I and II, 4th Edition,\/} Springer-Verlag, New York.\ \
Kai Lai Chung, {\it A Course in Probability Theory, Second Edition,\/} ISBN 0-12-174650-X, Academic Press, New York.\ \
Jacques Neveu, {\it Mathematical Foundations of the Calculus of Probability,\/} Holden-Day, San Francisco.\ \
Neveu is succinct, gorgeous, but not easy. This material is NOT popular in US departments of mathematics. At Princeton, see Cinlar.
Then should make some progress with stochastic processes: The big book is Gihman and and Skorohod, right, in three volumes, but mostly people settle for shorter treatments. Whatever, should learn about Poisson processes, Markov processes (discrete time, finite state space is enough to get started), Brownian motion, and martingales. Might also learn about second order stationary processes. A good course in stochastic processes is NOT easy to find, especially in mathematics departments.
Now are ready to attack statistics mathematically! I don't know of a good, single 'mathematical statistics' book at this level. Instead, there are many books -- I gave some above -- and then the journals. Thankfully, the field is relatively close to applications; so can take a practical problem and concentrate on what is relevant to it. One of my papers was some new work, at this level, in mathematical statistics for a problem in practical computing and computer science. The computer science community struggled terribly with the mathematics. So, it was some progress in computer science that community will have to struggle to understand.
One approach to work in computing is just to try things, that is, just to throw things against the wall and see if they appear to stick. Or, maybe the truth of the situation really is a simple statistical model. Likely that model will fit the data well. So, try many simple statistical models. We use these models mostly ignoring the mathematical assumptions; mostly we are proceeding 'heuristically', that is, with guessing. If any of the models fit well, then they can be considered candidates for the truth. So, are throwing things against the wall to see if they fit. This approach also called 'data mining'.
Problems:
(1) Will be quite limited in what statistical models can use. That is, will be drawing from a cookbook instead of being a real chef who can create good, new dishes appropriate for the available ingredients and customers!
(2) Don't have much, e.g., have not proceeded mathematically where from the deductive logic of assumptions and proofs actually know in advance some good things about the results. Something like breaking into a pharmacy, mixing up a lot of pills, taking them, and seeing if feel better! Uh, I'll pass and let you do that without me!
(3) May have gone through a lot of computer time in an 'exponential, combinatorial explosion' of efforts throwing against the wall.
(4) Have ignored a LOT in statistics that can add to what know about the results.
(5) Will be tempted to conclude have have found 'causality' but will likely not have.
(6) Will be tempted to conclude that have a model that predicts, but that is on shaky ground and risky and needs more work.
Applied to important problems, this approach can be dangerous.
There are not many healthy statistics departments. Much of the career interest is in biostatistics, especially related to FDA rules.
It appears that among the top statistics departments are Berkeley and UNC. Since Breiman and Brillinger are at Berkeley and since Stanford, long good in statistics, is not far away, if I were looking for a Ph.D. in statistics then I'd pick Berkeley.
There is a general problem getting a 'job' in a technical field and likely also with statistics. The assumption in US business is still as in factories 150 years ago: The supervisor knows more than the subordinate; the subordinate is supposed just to add common labor to what the supervisor says. In particular job descriptions are written by the supervisors, not the subordinates!
Well, there are nearly no supervisors in US business who have even as much as a weak little hollow hint of a tiny clue about the material described here. So, won't need that material to qualify for the job descriptions. Moreover, if actually know such material and let that fact leak out, then will likely not make it past the first level HR English major phone screen person who will tremble and conclude that you are not like the employees they have! If you do get hired and someone in your management chain discovers that you have used some mathematics they don't understand, you might be on the way out the door, especially is your work was valuable for the company!
Of course, the solution is to find a valuable application and start your own business. While maybe biomedical venture capital can understand crucial, core technical content, in information technology venture capital, likely you will be trying to explain this stuff to, say, history majors who worked in fund raising, marketing, general management, or financial analysis or have a background in just relatively elementary parts of computing. Just will NOT find more than six people, maybe not more than zero people, in US venture capital who can work the exercises in Royden or explain the strong law of large numbers. Sorry 'bout that! So, if you explain that the value of your venture is the powerful material in your 'secret sauce', then you will be regarded as a kook, far outside the mainstream of venture funded entrepreneurs, discarded, maybe even laughed at. As it is, some of the venture people are making money now, and the rest just want to be more like the ones who are making money. Looking for anything really new, powerful, and valuable is just NOT in the picture.
So, once you have some results in users, customers, revenue, etc., then maybe you can get some venture funding; just why at that point, owning 100% of the business, you would take venture funding, a Board that can fire you, etc. is less clear! Or venture funding is not for everyone! Or venture firms prefer to give money to people who don't need it!
For the real power of the 'secret sauce', you have just to keep that a secret!
Once mathematicians have yachts, at the venture firms math will be to info tech like biochem is to biotech. In the meanwhile note that a valuable application of statistics can put you on the Forbes 400 where there are not many people! Generally if you are making a valuable application of advanced or new statistics, then you will not know many people who understand what you are doing. Or, if lots of people understood it, then it wouldn't be valuable!
- I agree with a lot of that.
But hypothesis testing remains important because in some contexts with too little data that's about the best can do. For parametric tests, there are some cases when that's okay.
The asymptotic stuff is mostly okay: Can't figure out precisely what the darned thing will do for case n so let n go to infinity and maybe can see what happens. Then say, "for large n, this is about what happens" -- crude but better than nothing.
The central limit theorem and more says that the Gaussian is really important. Okay. Done. We know that. But, right, for too long too much of statistics made nearly a religion out of the Gaussian. Bummer.
Today happened to see the discussion panel at Stanford at
https://www.youtube.com/watch?v=hxXIJnjC_HI
It's fun stuff, with some big names.
They are laughing at ML: ML discovered statistics like Columbus discovered America. Columbus didn't discover America since when he landed millions of people were already living there. And when CS AL/ML discovered statistics, there was already a huge field there.
Moreover, the situation is reversed: Columbus brought better ship, etc. technology to the Americas than the Americas had, but CS AL/ML are bringing nearly all highly inferior technology to statistics, pure/applied math, etc.
CS AI/ML is nearly all new labels for adulterated old wine.
So, take some data on two variables, plot on an X-Y graph, fit a straight line, get the coefficients of the line as in first year algebra, and, presto, bingo, "The AI revolution is here!!!!", CS had had a machine "learn" the coefficients and they started with "training" data, that is trained the machine!!!!!! Where can I get one of those airline seat barf bags? Upchuck time!
Uh, I have a pretty good professional library; much of my house is lined with -bookshelves. The main topics are pure/applied math. Some of the books relevant to statistics include (with TeX markup)
Alexander M.\ Mood, Franklin A.\ Graybill, and Duane C.\ Boas, {\it Introduction to the Theory of Statistics, Third Edition,\/} McGraw-Hill, New York, 1974.\ \
N.\ R.\ Draper and H.\ Smith, {\it Applied Regression Analysis,\/} John Wiley and Sons, New York, 1968.\ \
C.\ Radhakrishna Rao, {\it Linear Statistical Inference and Its Applications:\ \ Second Edition,\/} ISBN 0-471-70823-2, John Wiley and Sons, New York, 1967.\ \
Henry Scheff\'e, {\it Analysis of Variance,\/} John Wiley and Sons, New York, 1967.\ \
Yvonne M.\ M.\ Bishop, Stephen E.\ Fienberg, Paul W.\ Holland, {\it Discrete Multivariate Analysis:\ \ Theory and Practice,\/} ISBN 0-262-52040-0, MIT Press, Cambridge, Massachusetts, 1979.\ \
Stephen E.\ Fienberg, {\it The Analysis of Cross-Classified Data,\/} ISBN 0-262-06063-9, MIT Press, Cambridge, Massachusetts, 1979.\ \
Leo Breiman, Jerome H.\ Friedman, Richard A.\ Olshen, Charles J.\ Stone, {\it Classification and Regression Trees,\/} ISBN 0-534-98054-6, Wadsworth \& Brooks/Cole, Pacific Grove, California, 1984.\ \
R.\ B.\ Blackman and J.\ W.\ Tukey, {\it The Measurement of Power Spectra:\ \ From the Point of View of Communications Engineering,\/} Dover, New York, 1959.\ \
William W.\ Cooley and Paul R.\ Lohnes, {\it Multivariate Data Analysis,\/} John Wiley and Sons, New York, 1971.\ \
Maurice M.\ Tatsuoka, {\it Multivariate Analysis: Techniques for Educational and Psychological Research,\/} John Wiley and Sons, 1971.\ \
E.\ L.\ Lehmann, {\it Testing Statistical Hypotheses,\/} John Wiley, New York, 1959.\ \
E.\ L.\ Lehmann, {\it Nonparametrics: Statistical Methods Based on Ranks,\/} ISBN 0-8162-4994-6, Holden-Day, San Francisco, 1975.\ \
Jaroslav H\'ajek and Zbyn\v ek \v Sid\'ak, {\it Theory of Rank Tests,\/} Academia, Prague, 1967.\ \
Sidney Siegel, {\it Nonparametric Statistics for the Behavioral Sciences,\/} McGraw-Hill, New York, 1956.\ \
Donald F.\ Morrison, {\it Multivariate Statistical Methods: Second Edition,\/} ISBN 0-07-043186-8, McGraw-Hill, New York, 1976.\ \
Harry H.\ Harman, {\it Modern Factor Analysis: Second Edition, Revised,\/} The University of Chicago Press, Chicago, 1967.\ \
George E.\ P.\ Box and Gwilym M.\ Jenkins, {\it Time Series Analysis --- Forecasting and Control: Revised Edition,\/} ISBN 0-8162-1104-3, Holden-Day, San Francisco, 1976.\ \
Jean-Ren\'e Barra, {\it Mathematical Basis of Statistics}, ISBN 0-12-079240-0, Academic Press, New York, 1981.\ \
So, what is really significant recent ML is adding to this quite old material? How much of the good stuff in that material has CS AI/ML even read and understood so far? Uh, what am I missing? I can think of a little but not much.
CS AI/ML going for applications of statistics is a lot like Stanley Tools, long in the hammer, screwdriver, saw, drill, and nail gun business, going into the residential construction business building, selling new houses and claiming that they have something new! That is, because they make good nail guns they conclude that they are the best at building houses. They know nothing of excavation, masonry, framing carpentry, windows, doors, roofing, plumbing, electrical, HVAC, dry wall, painting, circular stairs, walnut paneling, landscaping, or even the building codes, but they've got some good nail guns!
The math I derived and published in multi-dimensional, distribution-free anomaly detection has next to nothing to do with any of those books above; instead, I did original work and drew from ergodic theory, abstract algebra, and a classic result of Ulam. For the math for my startup, it's also original and even farther from those books above. In both cases, the core results are presented as theorems with proofs. Such math is mostly not what I'm seeing in CS AI/ML.
I DID see some cute, recent work in analysis, maybe from ML, of the over fitting problem; I don't need that work for my startup, but I do intend to go back and read that work.
My main points are:
(1) If AI/ML have some applied math tools that are solid, good, and new, terrific. Then those cases will add to the many thousands of such tools already on the shelves of the research libraries.
(2) The most important paradigm for a solid future for such tools is new, solid, correct, and powerful theorems and proofs building on solid material in math. That is, we want better stuff than is already in the libraries.
- I looked through Downey's book.
(A) Definition of Probability.
As far as I could tell, the book defined probability in only one of two ways:
(1) Relative frequency assuming finitely many trials (page 13) and
(2) Bayesian prior intuitive belief (page 52).
No, here the book is doing readers a disservice.
(B) Exponential Distribution
The 'derivation' of the exponential distribution on page 37:
"I’ll start with the exponential distribution because it is easy to work with. In the real world, exponential distributions come up when we look at a series of events and measure the times between events, which are called inter-arrival times. If the events are equally likely to occur at any time, the distribution of inter-arrival times tends to look like an exponential distribution."
is a bit too vague.
(C) Random.
The book also uses 'random' too frequently, loosely, and unnecessarily. Instead, essentially we can just drop the word 'random' and avoid questions about what 'random' means and if some data is 'truly random'.
For (A), essentially universally in advanced work, there is only one approach:
We have a non-empty set of 'trials'. Then an 'event' is a subset of the set of all trials. The set of all trials is an also an event. The set of all events is closed under countable unions and relative complements. In particular, the empty set is an event.
A 'probability' is a function P from the set of all events to the interval [0,1]. For an event A, P(A) is its probability. The probability of the set of all trials is 1.
P is 'countably additive': For countably infinitely many pair-wise disjoint events A_i, for i = 1, 2, ..., the probability of the the union of the A_i is the sum of the P(A_i).
A 'random variable' X is a function from the set of all trials to the real numbers such that for any real number x the set of all trials w such that X(w) <= x is an event. Usually we suppress the notation of the trials and just write the event X <= x. This is our only use of 'random', and we give no more definition of it.
In practice, essentially any number at all that we observe can be regarded as a random variable. In particular, the number might have been something observed about a person. The statement on page (52) that
"Anything involving people is pretty much off the table."
would be a silly foundation for probability.
For Bayesian and prior beliefs, can regard such a belief as an estimate of a conditional probability where we condition on what we know. E.g., we know a lot about people so that given a person, we can guess that with probability over 99% the person is less than 10 feet tall. This point does not mean that we have a 'Bayesian' foundation for probability. It's better just to drop mention of 'Bayesian'.
The 'distribution' of a random variable X is the function F_X(x) = P(X <= x).
Full details are in any of, say,
Jacques Neveu, 'Mathematical Foundations of the Calculus of Probability', Holden-Day.
Leo Breiman, 'Probability', ISBN 0-89871-296-3, SIAM.
M. Loeve, 'Probability Theory, I and II, 4th Edition', Springer-Verlag.
Kai Lai Chung, 'A Course in Probability Theory, Second Edition', ISBN 0-12-174650-X, Academic Press.
Yuan Shih Chow and Henry Teicher, 'Probability Theory: Independence, Interchangeability, Martingales', ISBN 0-387-90331-3, Springer-Verlag.
Essentially what is going on is that we are defining P as a non-negative real 'measure' with 'total mass 1' as in any of:
Paul R. Halmos, 'Measure Theory', D. Van Nostrand Company, Inc.
H. L. Royden, 'Real Analysis: Second Edition', Macmillan.
Walter Rudin, 'Real and Complex Analysis', ISBN 07-054232-5, McGraw-Hill.
This approach to probability is the one used in essentially all advanced work, e.g.,
Ioannis Karatzas and Steven E. Shreve, 'Brownian Motion and Stochastic Calculus, Second Edition', ISBN 0-387-97655-8.
Jean-Rene Barra, 'Mathematical Basis of Statistics', ISBN 0-12-079240-0, Academic Press.
Early in Neveu can see some ways we get essentially pushed into this foundation for probability.
With this approach, for a random variable X, we define its expectation E[X] as just its integral with respect to measure P in the sense of measure theory. Then essentially by a routine 'change of variable' we can get E[X] as an integral in terms of the distribution F_X and the measure it defines on the reals.
With this approach, if, say, for some positive integer n we measure the height of n people, then we say that we have the values of the n random variables
X_1, X_2, ..., X_n
and do not say that we have the values of n trials of one random variable X.
Indeed, with this approach, all our experience, all of the universe, is only one trial.
Given a set of events, we can define what it means for the set to be 'independent'. And we can proceed similarly for random variables.
Then we can give a more careful statement of the central limit theorem and also state the law of large numbers. We can also say why we use an average to estimate an expectation, e.g., as in
Paul R. Halmos, "The Theory of Unbiased Estimation", 'Annals of Mathematical Statistics', Volume 17, Number 1, pages 34-43, 1946.
On page 12 there is:
"The mean of this sample is 100 pounds, but if I told you 'The average pumpkin in my garden is 100 pounds,' that would be wrong, or at least misleading. In this example, there is no meaningful average because there is no typical pumpkin."
No: An 'average' does not have to be 'typical' to be 'meaningful'. The expectation of the weight X of a pumpkin in the garden remains meaningful. And the law of large numbers will still apply.
For the exponential distribution, there is a good 'qualitative' derivation: If we have arrivals where the inter-arrival times are stationary and independent, then the inter-arrival times are independent and have the same exponential distribution. Good details are early in the chapter on Poisson processes in
Erhan Cinlar, 'Introduction to Stochastic Processes', ISBN 0-13-498089-1, Prentice-Hall.
Downey's book places a lot of emphasis on distributions, e.g., Weibull, and descriptions, e.g., skewness. So an implication is, given some data, we should try to find its distribution and various summary properties of it, but usually this is unpromising. It is good to have the concept of a distribution but, given some data, usually not good to try to find its distribution. Instead, we get our results by manipulating the data in ways where we need to know little or nothing about the distribution.
The strong emphasis on hypothesis testing is curious. There is more in:
E. L. Lehmann, 'Testing Statistical Hypotheses', John Wiley and Sons.
E. L. Lehmann, 'Nonparametrics: Statistical Methods Based on Ranks', ISBN 0-8162-4994-6.
Downey's book mentioned the chi-square distribution: The main point is that if X_1, X_2, ..., X_k are random variables with Gaussian distribution with mean 0 and variance 1, then
Y = X_1^2 + X_2^2 + ... + X_k^2
has chi-squared distribution with k degrees of freedom.
Downey's book mentioned the variance of a finite population; while variance is of high interest, the variance of a finite population is not. What is usually wanted is an estimate of variance, and for that the formula in the book should be dividing by n - 1 instead of n; this change yields an 'unbiased' estimate.
It would be good for students to get at least an intuitive understanding of the most powerful hypothesis testing via the Neyman-Pearson result: Using an analogy from real estate investing, regard the false alarm rate as money to be spent and spend the money to get the best return on investment buy buying the property with the highest ROI first, then the one with the second highest, etc. Yes, in principal this process leads to a knapsack problem which is NP-complete. Likely the nicest proof of the Neyman-Pearson result is from the Hahn decomposition; this follows from the Radon-Nikodym result with a famous proof by von Neumann in Rudin.
The mention of 'resampling' is curious: There are cases where we won't have independent random variables but will have 'exchangeable' random variables, and exchangeability can be enough.
One example is extending nonparametric tests to multidimensional data and using it for anomaly detection in server farms and networks where multidimensional data is ubiquitous, e.g.,
N. B. Waite, "A Real-Time System-Adapted Anomaly Detector", 'Information Sciences', volume 115, April, 1999, pages 221-259.
Related
Mathematical foundations of the calculus of probability
3 recommend · 30
Probability
5 recommend · 11
Approximation theorems of mathematical statistics
2 recommend · 2
A Course in Probability Theory
3 recommend · 11
Probability Theory
77 recommend · 193
The Elements of Statistical Learning
144 recommend · 266
