Stats for nerds
2recommend
6mentions
3people
2011–2019years
67% recommend33% neutral
Mentions per year
What Hacker News says
- The world is awash in texts and papers on applied statistics. My notes were an introduction to provide a start on what is needed to read more in applied statistics.
For much of engineering, e.g., quality control, the statistical hypothesis testing I outlined is common.
There are now classic texts on hypothesis testing:
E. L. Lehmann, Testing Statistical Hypotheses, John Wiley, New York, 1959.
E. L. Lehmann, Nonparametrics: Statistical Methods Based on Ranks, ISBN 0-8162-4994-6, Holden-Day, San Francisco, 1975.
Sidney Siegel, Nonparametric Statistics for the Behavioral Sciences, McGraw-Hill, New York, 1956.
The last is also just hypothesis testing; apparently the book is still referenced.
From some statistical consultants inside GE, there is
Gerald J. Hahn and Samuel S. Shapiro, Statistical Models in Engineering, John Wiley & Sons, New York, 1967.
For machine learning, so far the approach appears to be cases of empirical curve fitting.
The linear case of versions of regression analysis remain of interest. There the matrix theory I outlined is central. And regression analysis goes way back, 100 or so years, and is awash in uses of statistical hypothesis tests for the regression (F-ratio), individual coefficients, (t-tests), and confidence intervals on predicted values.
For using probabilistic and statistical methods in the non-linear fitting, that is more advanced and likely an active field of research.
In my own work, I use and/or create new statistical methods when and as I need them. My current work has some original work in, really, applied probability that also can be considered statistics. And when I was doing AI at IBM, I created some new mathematical statistics hypothesis tests to improve on what we were doing in AI. Later I published the work. Parts of the machine learning community might call that paper a case of supervised learning, but the paper is theorems and proofs in applied probability that give results on false alarm rates and detection rates, that is, statistical hypothesis tests (both multi-dimensional and distribution-free). So, that paper can be regarded as relevant to machine learning, but the prerequisites are a ugrad pure math major, about two years of graduate pure math with measure theory and functional analysis, a course in what Leo Breiman called graduate probability, and some work in stochastic processes. At one point I use a classic result of S. Ulam that the French probabilist Le Cam called tightness -- there is a nice presentation in
Patrick Billingsley, Convergence of Probability Measures, John Wiley and Sons, New York, 1968.
The work was for anomaly detection in complex systems so could be regarded as for both engineering and machine learning. But I can't give all the background here.
- Okay, PG has an hypothesis test.
There's a large literature for that, e.g.,
E. L. Lehmann, Testing Statistical Hypotheses.
E. L. Lehmann, Nonparametrics: Statistical Methods Based on Ranks.
Sidney Siegel, Nonparametric Statistics for the Behavioral Sciences.
In this case, PG will be more interested in the non-parametric case, i.e., distribution-free where we make no assumptions about probability distributions.
We start an hypothesis test with an hypothesis, commonly called the null hypothesis which is an assumption that there is no effect or, in PG's case, no bias. Then with that assumption, we are able to do some probability calculations.
Then we look at the real data and calculate the probability of, say, the evidence of bias being as large as we observed. If that probability is small, say, less than 1%, then we reject the null hypothesis, that is, reject the assumption of no bias, and conclude that the null hypothesis is false and that there is bias. The role of the assumption about the sample is so that we know that the problem is bias and not something about the sample.
In hypothesis testing, about all that matters are just two numbers -- the probability of Type I error and that of Type II error. We want both probabilities to be as low as possible.
Type I Error: We reject the null hypothesis when it is true, e.g., we conclude bias when there is none.
Type II Error: We fail to reject (i.e., we accept) the null hypothesis when it is false.
When looking for bias, Type I error can be called a false alarm of bias, and Type II error can be called a missed detection of bias.
In PGs case, suppose we have 100 startups and five of those have women founders. Suppose for each of the startups we have the data from "their subsequent performance is measured".
Our null hypothesis is that the expected performance of the women is the same as that of the men.
So, let's find those two averages and take the difference, say, the average of the women less the average of the men.
PG says if this difference is positive, then there was bias, but PG has not given us any estimate of the probability of Type I error, that is, of the probability (or rate) of a false alarm.
I mean we don't want to get First Round Capital in trouble with Betty Friedan, Gloria Steinem, Marissa Mayer, Sheryl Sandberg, Hillary Clinton, Ivanka Trump, or Lady Gaga unjustly! :-).
Let's call this difference our test statistic.
So, let's find the probability of a false alarm:
So, let's put all 100 measurements in a pot, stir the pot vigorously (we can use a computer for this), pull out five numbers and average, pull out the other 95 numbers and average, take the difference in the two averages, that of the five less that of the 95, and do this, say, 1000 times. Ah, computers are cheap; let's be generous and do this 10,000 times.
For a random number, how about starting with a 32 bit integer, with appropriately long precision arithmetic multiply by 5^15, add 1, take modulo 2^47, and scale as we want?
So, we get an empirical distribution of these differences, from the five less the 95. Looking at the distribution, we see what the probability is of getting a difference as high or high or higher than our test statistic. If that probability is low, say, 1% or less, then we reject the null hypothesis of no bias and conclude bias with our estimate of probability of Type I error 1% or less.
If with the 1% we reject, then it looks like First Round has done a transgression, will get retribution from Betty, et al., and needs to seek redemption and Betty, et al., are happy to have their suspicions confirmed. Else First Round looks like the good guys, are "certified statistically fair to women", may get more deal flow from women, and Betty, et al., can be happy that First Round is so nice!
Notice that either way Betty, et al., are "happy". That's called "happy women, happy life"! Or, heads, the women win, tails they lose, and in no event is there a huge crowd of angry women in front of First Round's offices with a bonfire of lingerie screaming "bias"!
When we reject the null hypothesis, we want to know that the reason was men versus women and not something else, e.g., a biased sample. So here is where we use our assumption of independence with the same mean.
Now we have a handle on Type I error.
Here we have done a non-parametric statistical hypothesis test, i.e., have made no assumptions, except the means, about the distributions of the male/female CEO performance measurements.
And we can select our desired false alarm rate in advance and get that rate almost exactly.
For Type II error, that is more difficult.
Bottom line, what we really want is, for whatever rate of false alarms we are willing to tolerate, the lowest rate of missed detections we can get.
Can we do that? With enough more data, yup. There is a classic result due to J. Neyman (long at Berkeley) and K. Pearson (early in statistics) that shows how.
How? Regard false alarm rate as money and think of investing in SF real estate. We put our money done on the opportunities with highest expected ROI until we have spent all our money. Done. For details, an unusually general proof can follow from the Hahn decomposition from the Radon-Nikodym theorem in measure theory, e.g., Rudin, Real and Complex Analysis. Right, in the discrete case, we have a knapsack problem, known to be in NP-complete.
What we have done with our pot stirring is called resampling, and for more such look for B. Efron, long at Yale, and P. Diaconis, once at Harvard, now long at Stanford.
Tom, with a reputation as a hacker, likes to work late, say, till 2 AM. So, we look at the intrusion alerts each minute between 2 AM and 3 AM (something like the performance of the women) and compare with those of the other minutes of 24 hours (like the performance of the men) much as above and ask if Tom is trying to hack the servers.
Or, we have a server farm and/or a network, and we want to detect problems never seen before, e.g., zero day problems. So, we have no data at all on the problems we are trying to detect because we have never seen any of those before.
So, to do a good job, let's pick some system we want to monitor and for that system, get data on, say, each of 10 variables at, say, 20 times a second. Now what?
Our work with bias in women venture applications used just one number for our measurement and test statistic. So we were uni-dimensional. Here we have 10 numbers and need to be multi-dimensional.
Well, in principle we should be able to do much better (pair of Type I and Type II error rates) with 10 numbers than just one. The usual ways will require us to have, with our null hypothesis, the probability distribution of the 10 numbers, but can only get something like that from smoking funny stuff -- not even big data is that big.
So, we want to need no assumptions about distribution, that is, be distribution-free.
So, we want some statistical a hypothesis test that is both multi-dimensional and distribution free.
Can we do that? Yup.
"You mean you can select false alarm rate in advance and get that rate essentially exactly, as in PG's bias example?" Yup.
"Could that be used in a real server farm or network to detect zero day problems -- security, performance, hard/software failures, system management errors?" Yup -- just what it was invented for.
"Attempted credit card fraud?" Ah, once a guy in an audience thought so!
How? Ah, sadly there is no more room in this post!
What else might we do with hypothesis tests? Well, look around at, right, big data or just small data.
Do we have a case of big data analytics or artificial intelligence (AI)?
Ah, I've given a sweetheart outline of statistical hypothesis testing, and now you are suggesting some things really low grade? Where did I go wrong to deserve such an insult?
- Sure, but we should take that off-line.
What is just crucial, really, what it's all about, that is, where the value is, necessarily, inescapably is just in the two rates, the false alarm rate and the detection rate. If can't say at least good things about false alarm rate, then have a definite maybe.
The OP struggled with the false alarm rate -- my work solved that problem.
Really, essentially inescapably and necessarily, the core of the work has to be a statistical hypothesis test.
So, what is that? We make an assumption, typically, that the system we are monitoring is healthy. Commonly that assumption is called the null hypothesis, that is, null in the sense of nothing wrong.
That assumption leads to some mathematical assumptions that can be used as hypotheses in some theorems with proofs.
Then, that assumption with some real data let's us calculate the probability of getting data like we just observed.
If that probability is relatively small, then we have (A) the system is healthy and we just observed something quite rare or (B) the system is sick. If the probability is really small and/or the health of the system is really important, then we guess that case (A) is too rare to be believed, we reject the assumption, the null hypothesis, and conclude (B) that the system is sick.
That's a short description of statistical hypothesis testing for 100+ years a main pillar of pure and applied statistics and a lot in science, both social and physical.
But for monitoring computer server farms, this context very much need the test to be both multidimensional and distribution-free. The world wide collection of those was from tiny down to zero -- likely and apparently zero. So, I invented a nice, large collection of them.
The core of the work was really some novel derivations, right, with theorems and proofs, in, call it, applied probability.
Then I worked out some algorithms to make the computing fast.
The work is likely and apparently the best approach to do zero day monitoring for health and wellness in high end server farms and networks. And there may be some other applications.
Zero day is the case where we are looking for problems never seen before. So, it's sometimes called behavioral monitoring.
Sure, should monitor for all problems have seen before, and there we should be able to do relatively well on the two rates. But, after those old problems, we still need to look for new ones -- sorry 'bout that.
I did this work a long time ago and presented it, or put it in front of, a lot of people, but no one was much interested.
From time to time, people have, as in the OP, rediscovered the problem and done this and that with it. E.g., there was a project of profs Fox and Patterson at Stanford and Berkeley funded by Google, Microsoft, and Sun and in part considered monitoring -- looking at their work, I thought that mine was better.
Sure, there was
http://www.sans.org/resources/idfaq/behavior_based.php
where it was claimed that behavioral monitoring had high false alarm rates. Nope -- my work doesn't! Instead, with my work, get to select false alarm rates over a wide range, including quite small.
I have another project, and for it I recently completed writing the version 1.0 software -- 18,000 programming language statements in 80,000 lines of text (with comments, blank lines, etc). I want to go live ASAP. This project is much easier for people to like! E.g., don't have to get people interested in rates of false alarms and detections. If the project grows, then I will implement and deploy my anomaly detection ideas for my server farm.
Otherwise, for now, I'm no longer interested in that anomaly work mostly because no one else is. Others are welcome, from time to time, to rediscover the problem, struggle with false alarm rate, see no good, general results on detection rate, etc., think that the key is artificial intelligence or machine learning or some such (I didn't use either), etc. Uh, it's statistical hypothesis testing, guys. Sorry 'bout that.
For more, see, say some of the classics:
E. L. Lehmann, Testing Statistical Hypotheses.
E. L. Lehmann, Nonparametrics: Statistical Methods Based on Ranks.
Sidney Siegel, Nonparametric Statistics for the Behavioral Sciences.
Once I presented my work to Cisco. Their reaction was that my work was "just statistical". Well, what the heck do you expect? Want something deterministic, 0 or 1, for healthy or sick? Do you realize what the heck you are asking for?
Another time I presented the work and the reaction was, "What we really want is diagnosis." Gads. You are Sony -- or some other big organization, take your pick from various headlines from current to going way back -- and your server farm is under attack with gigabytes or terabytes on the way out the door to hackers, your whole company is at risk of going belly up, and you don't want to know that there's a serious problem until you can also be told just what the heck to do about it? Sure that's your view?
Gee, you don't want to know that your car's about out of gas unless you are also told that you can get more gas right away? Ever drive in rural Montana? Like long walks? What about engine oil pressure or coolant temperature -- don't want to know about those, either? Or your blood pressure? Not until can be told just what to do about it? Sure about that?
It's an important problem. I did the work. It's good work. But if people are not much interested, then so be it -- I'm not into pushing big rocks up long hills for no good reason.
Suspicion: A guy has a sore on his foot. He knows it's there. It hurts. But he lives with it. He's tried this and that, but he still has the sore. Someone has a good cure, but he is so used to putting up with the sore that he ignores the good solution. Besides, he didn't see a way to cure the sore so doesn't believe that anyone else could either.
Even worse, my solution is not really computer science but is, really, some applied math, actually applied probability, and such work, especially new such work, is rare in practice.
More generally, good, new stuff is rare, so rare that people are very reluctant to consider that they should pay any attention to it.
Fine: Let them live with a sore foot.
- I looked through Downey's book.
(A) Definition of Probability.
As far as I could tell, the book defined probability in only one of two ways:
(1) Relative frequency assuming finitely many trials (page 13) and
(2) Bayesian prior intuitive belief (page 52).
No, here the book is doing readers a disservice.
(B) Exponential Distribution
The 'derivation' of the exponential distribution on page 37:
"I’ll start with the exponential distribution because it is easy to work with. In the real world, exponential distributions come up when we look at a series of events and measure the times between events, which are called inter-arrival times. If the events are equally likely to occur at any time, the distribution of inter-arrival times tends to look like an exponential distribution."
is a bit too vague.
(C) Random.
The book also uses 'random' too frequently, loosely, and unnecessarily. Instead, essentially we can just drop the word 'random' and avoid questions about what 'random' means and if some data is 'truly random'.
For (A), essentially universally in advanced work, there is only one approach:
We have a non-empty set of 'trials'. Then an 'event' is a subset of the set of all trials. The set of all trials is an also an event. The set of all events is closed under countable unions and relative complements. In particular, the empty set is an event.
A 'probability' is a function P from the set of all events to the interval [0,1]. For an event A, P(A) is its probability. The probability of the set of all trials is 1.
P is 'countably additive': For countably infinitely many pair-wise disjoint events A_i, for i = 1, 2, ..., the probability of the the union of the A_i is the sum of the P(A_i).
A 'random variable' X is a function from the set of all trials to the real numbers such that for any real number x the set of all trials w such that X(w) <= x is an event. Usually we suppress the notation of the trials and just write the event X <= x. This is our only use of 'random', and we give no more definition of it.
In practice, essentially any number at all that we observe can be regarded as a random variable. In particular, the number might have been something observed about a person. The statement on page (52) that
"Anything involving people is pretty much off the table."
would be a silly foundation for probability.
For Bayesian and prior beliefs, can regard such a belief as an estimate of a conditional probability where we condition on what we know. E.g., we know a lot about people so that given a person, we can guess that with probability over 99% the person is less than 10 feet tall. This point does not mean that we have a 'Bayesian' foundation for probability. It's better just to drop mention of 'Bayesian'.
The 'distribution' of a random variable X is the function F_X(x) = P(X <= x).
Full details are in any of, say,
Jacques Neveu, 'Mathematical Foundations of the Calculus of Probability', Holden-Day.
Leo Breiman, 'Probability', ISBN 0-89871-296-3, SIAM.
M. Loeve, 'Probability Theory, I and II, 4th Edition', Springer-Verlag.
Kai Lai Chung, 'A Course in Probability Theory, Second Edition', ISBN 0-12-174650-X, Academic Press.
Yuan Shih Chow and Henry Teicher, 'Probability Theory: Independence, Interchangeability, Martingales', ISBN 0-387-90331-3, Springer-Verlag.
Essentially what is going on is that we are defining P as a non-negative real 'measure' with 'total mass 1' as in any of:
Paul R. Halmos, 'Measure Theory', D. Van Nostrand Company, Inc.
H. L. Royden, 'Real Analysis: Second Edition', Macmillan.
Walter Rudin, 'Real and Complex Analysis', ISBN 07-054232-5, McGraw-Hill.
This approach to probability is the one used in essentially all advanced work, e.g.,
Ioannis Karatzas and Steven E. Shreve, 'Brownian Motion and Stochastic Calculus, Second Edition', ISBN 0-387-97655-8.
Jean-Rene Barra, 'Mathematical Basis of Statistics', ISBN 0-12-079240-0, Academic Press.
Early in Neveu can see some ways we get essentially pushed into this foundation for probability.
With this approach, for a random variable X, we define its expectation E[X] as just its integral with respect to measure P in the sense of measure theory. Then essentially by a routine 'change of variable' we can get E[X] as an integral in terms of the distribution F_X and the measure it defines on the reals.
With this approach, if, say, for some positive integer n we measure the height of n people, then we say that we have the values of the n random variables
X_1, X_2, ..., X_n
and do not say that we have the values of n trials of one random variable X.
Indeed, with this approach, all our experience, all of the universe, is only one trial.
Given a set of events, we can define what it means for the set to be 'independent'. And we can proceed similarly for random variables.
Then we can give a more careful statement of the central limit theorem and also state the law of large numbers. We can also say why we use an average to estimate an expectation, e.g., as in
Paul R. Halmos, "The Theory of Unbiased Estimation", 'Annals of Mathematical Statistics', Volume 17, Number 1, pages 34-43, 1946.
On page 12 there is:
"The mean of this sample is 100 pounds, but if I told you 'The average pumpkin in my garden is 100 pounds,' that would be wrong, or at least misleading. In this example, there is no meaningful average because there is no typical pumpkin."
No: An 'average' does not have to be 'typical' to be 'meaningful'. The expectation of the weight X of a pumpkin in the garden remains meaningful. And the law of large numbers will still apply.
For the exponential distribution, there is a good 'qualitative' derivation: If we have arrivals where the inter-arrival times are stationary and independent, then the inter-arrival times are independent and have the same exponential distribution. Good details are early in the chapter on Poisson processes in
Erhan Cinlar, 'Introduction to Stochastic Processes', ISBN 0-13-498089-1, Prentice-Hall.
Downey's book places a lot of emphasis on distributions, e.g., Weibull, and descriptions, e.g., skewness. So an implication is, given some data, we should try to find its distribution and various summary properties of it, but usually this is unpromising. It is good to have the concept of a distribution but, given some data, usually not good to try to find its distribution. Instead, we get our results by manipulating the data in ways where we need to know little or nothing about the distribution.
The strong emphasis on hypothesis testing is curious. There is more in:
E. L. Lehmann, 'Testing Statistical Hypotheses', John Wiley and Sons.
E. L. Lehmann, 'Nonparametrics: Statistical Methods Based on Ranks', ISBN 0-8162-4994-6.
Downey's book mentioned the chi-square distribution: The main point is that if X_1, X_2, ..., X_k are random variables with Gaussian distribution with mean 0 and variance 1, then
Y = X_1^2 + X_2^2 + ... + X_k^2
has chi-squared distribution with k degrees of freedom.
Downey's book mentioned the variance of a finite population; while variance is of high interest, the variance of a finite population is not. What is usually wanted is an estimate of variance, and for that the formula in the book should be dividing by n - 1 instead of n; this change yields an 'unbiased' estimate.
It would be good for students to get at least an intuitive understanding of the most powerful hypothesis testing via the Neyman-Pearson result: Using an analogy from real estate investing, regard the false alarm rate as money to be spent and spend the money to get the best return on investment buy buying the property with the highest ROI first, then the one with the second highest, etc. Yes, in principal this process leads to a knapsack problem which is NP-complete. Likely the nicest proof of the Neyman-Pearson result is from the Hahn decomposition; this follows from the Radon-Nikodym result with a famous proof by von Neumann in Rudin.
The mention of 'resampling' is curious: There are cases where we won't have independent random variables but will have 'exchangeable' random variables, and exchangeability can be enough.
One example is extending nonparametric tests to multidimensional data and using it for anomaly detection in server farms and networks where multidimensional data is ubiquitous, e.g.,
N. B. Waite, "A Real-Time System-Adapted Anomaly Detector", 'Information Sciences', volume 115, April, 1999, pages 221-259.
- I can't find the textbook this specific professor assigned but the assigned textbooks for this semester's section are:
Testing Statistical Hypotheses (Cl) by Lehmann
Theory Of Point Estimation by Lehmann
Theoretical Statistics by Keener
- I agree with a lot of that.
But hypothesis testing remains important because in some contexts with too little data that's about the best can do. For parametric tests, there are some cases when that's okay.
The asymptotic stuff is mostly okay: Can't figure out precisely what the darned thing will do for case n so let n go to infinity and maybe can see what happens. Then say, "for large n, this is about what happens" -- crude but better than nothing.
The central limit theorem and more says that the Gaussian is really important. Okay. Done. We know that. But, right, for too long too much of statistics made nearly a religion out of the Gaussian. Bummer.
Today happened to see the discussion panel at Stanford at
https://www.youtube.com/watch?v=hxXIJnjC_HI
It's fun stuff, with some big names.
They are laughing at ML: ML discovered statistics like Columbus discovered America. Columbus didn't discover America since when he landed millions of people were already living there. And when CS AL/ML discovered statistics, there was already a huge field there.
Moreover, the situation is reversed: Columbus brought better ship, etc. technology to the Americas than the Americas had, but CS AL/ML are bringing nearly all highly inferior technology to statistics, pure/applied math, etc.
CS AI/ML is nearly all new labels for adulterated old wine.
So, take some data on two variables, plot on an X-Y graph, fit a straight line, get the coefficients of the line as in first year algebra, and, presto, bingo, "The AI revolution is here!!!!", CS had had a machine "learn" the coefficients and they started with "training" data, that is trained the machine!!!!!! Where can I get one of those airline seat barf bags? Upchuck time!
Uh, I have a pretty good professional library; much of my house is lined with -bookshelves. The main topics are pure/applied math. Some of the books relevant to statistics include (with TeX markup)
Alexander M.\ Mood, Franklin A.\ Graybill, and Duane C.\ Boas, {\it Introduction to the Theory of Statistics, Third Edition,\/} McGraw-Hill, New York, 1974.\ \
N.\ R.\ Draper and H.\ Smith, {\it Applied Regression Analysis,\/} John Wiley and Sons, New York, 1968.\ \
C.\ Radhakrishna Rao, {\it Linear Statistical Inference and Its Applications:\ \ Second Edition,\/} ISBN 0-471-70823-2, John Wiley and Sons, New York, 1967.\ \
Henry Scheff\'e, {\it Analysis of Variance,\/} John Wiley and Sons, New York, 1967.\ \
Yvonne M.\ M.\ Bishop, Stephen E.\ Fienberg, Paul W.\ Holland, {\it Discrete Multivariate Analysis:\ \ Theory and Practice,\/} ISBN 0-262-52040-0, MIT Press, Cambridge, Massachusetts, 1979.\ \
Stephen E.\ Fienberg, {\it The Analysis of Cross-Classified Data,\/} ISBN 0-262-06063-9, MIT Press, Cambridge, Massachusetts, 1979.\ \
Leo Breiman, Jerome H.\ Friedman, Richard A.\ Olshen, Charles J.\ Stone, {\it Classification and Regression Trees,\/} ISBN 0-534-98054-6, Wadsworth \& Brooks/Cole, Pacific Grove, California, 1984.\ \
R.\ B.\ Blackman and J.\ W.\ Tukey, {\it The Measurement of Power Spectra:\ \ From the Point of View of Communications Engineering,\/} Dover, New York, 1959.\ \
William W.\ Cooley and Paul R.\ Lohnes, {\it Multivariate Data Analysis,\/} John Wiley and Sons, New York, 1971.\ \
Maurice M.\ Tatsuoka, {\it Multivariate Analysis: Techniques for Educational and Psychological Research,\/} John Wiley and Sons, 1971.\ \
E.\ L.\ Lehmann, {\it Testing Statistical Hypotheses,\/} John Wiley, New York, 1959.\ \
E.\ L.\ Lehmann, {\it Nonparametrics: Statistical Methods Based on Ranks,\/} ISBN 0-8162-4994-6, Holden-Day, San Francisco, 1975.\ \
Jaroslav H\'ajek and Zbyn\v ek \v Sid\'ak, {\it Theory of Rank Tests,\/} Academia, Prague, 1967.\ \
Sidney Siegel, {\it Nonparametric Statistics for the Behavioral Sciences,\/} McGraw-Hill, New York, 1956.\ \
Donald F.\ Morrison, {\it Multivariate Statistical Methods: Second Edition,\/} ISBN 0-07-043186-8, McGraw-Hill, New York, 1976.\ \
Harry H.\ Harman, {\it Modern Factor Analysis: Second Edition, Revised,\/} The University of Chicago Press, Chicago, 1967.\ \
George E.\ P.\ Box and Gwilym M.\ Jenkins, {\it Time Series Analysis --- Forecasting and Control: Revised Edition,\/} ISBN 0-8162-1104-3, Holden-Day, San Francisco, 1976.\ \
Jean-Ren\'e Barra, {\it Mathematical Basis of Statistics}, ISBN 0-12-079240-0, Academic Press, New York, 1981.\ \
So, what is really significant recent ML is adding to this quite old material? How much of the good stuff in that material has CS AI/ML even read and understood so far? Uh, what am I missing? I can think of a little but not much.
CS AI/ML going for applications of statistics is a lot like Stanley Tools, long in the hammer, screwdriver, saw, drill, and nail gun business, going into the residential construction business building, selling new houses and claiming that they have something new! That is, because they make good nail guns they conclude that they are the best at building houses. They know nothing of excavation, masonry, framing carpentry, windows, doors, roofing, plumbing, electrical, HVAC, dry wall, painting, circular stairs, walnut paneling, landscaping, or even the building codes, but they've got some good nail guns!
The math I derived and published in multi-dimensional, distribution-free anomaly detection has next to nothing to do with any of those books above; instead, I did original work and drew from ergodic theory, abstract algebra, and a classic result of Ulam. For the math for my startup, it's also original and even farther from those books above. In both cases, the core results are presented as theorems with proofs. Such math is mostly not what I'm seeing in CS AI/ML.
I DID see some cute, recent work in analysis, maybe from ML, of the over fitting problem; I don't need that work for my startup, but I do intend to go back and read that work.
My main points are:
(1) If AI/ML have some applied math tools that are solid, good, and new, terrific. Then those cases will add to the many thousands of such tools already on the shelves of the research libraries.
(2) The most important paradigm for a solid future for such tools is new, solid, correct, and powerful theorems and proofs building on solid material in math. That is, we want better stuff than is already in the libraries.
Related
Nonparametrics
1 recommend · 9
Statistical methods
2 recommend · 3
Probability
5 recommend · 11
Statistics of random processes
1 recommend · 2
Nonparametric statistics for the behavioral sciences
2 recommend · 12
Linear statistical inference and its applications
1 recommend · 10
