Convergence of Probability Measures

What Hacker News says

1. graycatApr 2015
   I know some people in (1) the top of business in the US, (2) high end US research academics, and (3) information technology (IT).

   (A) Bluntly likely none of those people knows anyone at First Round Capital. (B) Bluntly, with the exception of Howard Morgan, the qualifications of the people at First Round Capital in business, research academics, and technology are not high enough to be respected by the people I know. Net, the people at First Round Capital and myself know nearly none of the same people.

   Moreover, for my technology startup, there is no one at First Round Capital with business, academic, or technology qualifications good enough for me to hire for a significant position. I'd have a tough time respecting any of them to take their advice on my startup.

   Once, just once, I did call a person I know, the founder, COB, CEO of a major, world famous company and asked for an introduction to his CIO. The CIO and I talked and had a nice review of history, and he gave me an introduction to a partner at a venture firm with a partner on the BoD of the company. Net, the introduction meant nothing -- the venture partner paid no attention to the introduction or my project at all; we did communicate but just as in a cold call.

   I never again wanted to bother any of the high end people I know, bother them to introduce me to a venture firm just to get past some absurd hoop the venture firm erected but ignored.

   Net, venture firms who want introductions get put at the bottom of my list; the venture partners and I nearly never know or respect the same people; I'm not going to pester the good, important people I know and respect to have them waste their time communicating with people as poorly qualified in business, research, and IT as all but a small number of venture partners. No way.

   Next, in contacting venture firms just via cold calls, I've had little trouble getting through and getting a response including several hour long conference calls. In particular well known venture partners at well known venture firms are aware of my work on my project, all without any introductions.

   Net, it appears to me that introductions are not really necessary and not very helpful.

   What was helpful was having a team: For a while I did that, but too soon I encountered the common problem -- disputes. So, now I'm a solo founder. There are some serious advantages being a solo founder, but it does appear that getting a phone conversation with a venture partner as a solo founder is more difficult. For whatever reasons, venture firms don't like solo founder startups.

   Moreover, it appears to me that, really, venture firms have their feet locked in concrete that, with only some rare exceptions, they just will not pay much attention to an IT startup before the software is developed and there is significant traction growing rapidly. E.g., venture partner Fred Wilson at his Union Square Ventures recently made it clear on his blog AVC.com that he just will not fund software development. Okay by me.

   But at

   http://a16z.com/2014/07/30/the-happy-demise-of-the-10x-engin...

   with

   "The Happy Demise of the 10X Engineer"

   By Sam Gerstenzang

   is in part:

   "This is the new normal: fewer engineers and dollars to ship code to more users than ever before. The potential impact of the lone software engineer is soaring. How long before we have a billion-dollar acquisition offer for a one-engineer startup? How long before the role of an engineer, artisanally crafting custom solutions, vanishes altogether?"

   So, Andreessen-Horowitz is admitting the possibility of a solo founder creating a billion dollar startup.

   I don't see just why not: Venture firms want the code written and traction significant and growing rapidly. If the startup is ad supported, then it doesn't take much traction to let a solo (single) founder startup have cash enough for organic growth, that is, without equity funding. Then, if enough of the 3+ billion Internet users like the work a lot, presto, bingo, a billion dollar startup.

   Indeed, for my startup, I believe that, as the founder, I need to "know my business" (a traditional criterion), and that includes the code, the servers, how to please the users, how to please the paying customers, how to handle the billing, bookkeeping, accounting, legal, etc. So, I'm not seeing where a larger founding team is necessary. First hire? Likely an Office Manager, and not a co-founder.

   There is some irony: Necessarily venture firms are looking for highly exceptional projects, but their means of looking are mostly to compare with simplistic patterns from the past -- not promising. And as in

   http://www.kauffman.org/newsroom/2012/07/institutional-limit...

   and

   http://www.avc.com/a_vc/2013/02/venture-capital-returns.html...

   on average the venture firms are not making much money doing this.

   Thankfully the US NSF, NIH, DARPA, various other parts of the US DoD, commonly ignore simplistic patterns from the past and, instead, actually get expert evaluations of projects submitted on paper. Moreover, the history is that projects that do well on such evaluations have much better batting average or ROI than US IT venture capital.

   E.g., for the Manhattan Project, the US DoD (then the War Department) didn't reject the project because it looked nothing like the long history of bombs.

   The CIA didn't reject the SR-71 because it looked very different from anything in the past of aviation.

   The US Navy didn't reject the first satellite navigation system because it looked nothing like anything in the history of navigation.

   NSF prize winning research rarely looks much like the past.

   The Human Genome project funded by the NIH looked very different from anything in the past.

   Intel is shooting for 10nm -- again, new ground significantly different from anything in the past.

   But only a tiny fraction of US IT venture partners have the ability, say, of a project reviewer for a leading peer-reviewed journal of original research or the NSF or NIH, to review leading-edge technical material. That's been their business model. Okay.

   Yes, venture firms on their Web sites commonly claim to have "deep domain knowledge". Curious: Only a tiny fraction of US IT venture partners have the qualifications even to be admitted to the graduate program where I got my Ph.D.

   Indeed, Google search

   "deep domain knowledge" venture

   gives "About 40,600 results". I've read the backgrounds of hundreds of US IT venture partners, and I doubt that I've seen over 10 that have the qualifications for "deep domain knowledge" in anything very technical. I've seen a lot of lawyers, history majors, MBAs, international studies majors, English majors, etc. and darned few math, physics, engineering, or computer science majors.

   It looks like in their educations, only a tiny fraction of the US IT VCs liked the STEM fields.

   I published a paper in computer science. The paper has a typo. Here I make a public bet, of one dollar, that no US IT venture partner on their own can find the typo. Should such a person wish to try, then reply here, and I will send a PDF of the paper.

   Hint: It would be good to be able to find, say,

   Patrick Billingsley, 'Convergence of Probability Measures'

   fun and easy reading, and for that should have greatly enjoyed, say, at least the first (real) half of

   Walter Rudin, 'Real and Complex Analysis'

   and

   Walter Rudin, 'Principles of Mathematical Analysis'.

   Come on US IT VCs: You've got "deep domain knowledge", high determination, are all-go, never stop, never give up, commonly leap tall buildings at a single bound, have over the top self esteem, etc. Should be a piece of cake for you, right?

   I'll save you some time:

   (A) Look up the definitions of countability and of a sigma algebra. Then show that there are no countably infinite sigma algebras.

   (B) For positive integer n, the real numbers R, R^n with the usual topology, and subset C of R^n closed in that topology, show that there exists function f: R^n --> R zero on C, positive otherwise, and infinitely differentiable. Notice that examples of C include Cantor sets of positive measure, the Mandelbrot set, and sample paths of Brownian motion. Curious result.

   If you find (A) and (B) easy, then by all means, also with your "deep domain knowledge", read Rudin and go for the typo also using Billingsley.

   What does the paper with the typo have to do with IT? It's likely the best thing so far for detecting zero day problems in large server farms and networks.

   Point: Only a tiny fraction of US IT venture partners have the "deep domain knowledge" necessary to evaluate new work in information technology.

   So, they have to evaluate based mostly just on traction.

   As far as I can tell, nearly everything else US IT VCs say they want to see is just smoke to cover the one thing they really want -- traction significant and growing rapidly.

   Point: Due to the possibility of a solo founder with tiny burn rate, waiting for traction will be too late.

   Can a solo founder of an IT startup hope to be successful without equity funding? Should be: All across the US, cross roads to the largest cities, solo founders do well mowing grass, selling pizza or hamburgers, pumping gas, paving driveways, ..., big-truck, little-truck distribution businesses, etc. without equity funding. IT should be an advantage.

   Show morethread
2. graycatMay 2014
   > No need to guess whether the underlying assumptions of the model are true, just see if it works.

   No, there are assumptions also for this approach, and these assumptions also need to be checked.

   Then in applying the model that has been checked, there are still more assumptions, that need to be checked.

   And checking models is often essentially hypothesis testing where need more assumptions and, commonly, some experience to know what hypothesis tests to use. E.g., tests in regression can be F ratios and/or t-tests, and need to check some assumptions or at least robustness here.

   E.g., in power spectral estimation, see, Blackman and Tukey, 'The Measurement of Power Spectra', there is a severe trade off between resolution and stability, another case of a trade off between bias and variance. Tough to 'automate' that consideration.

   So, take a box of data. Divide it in half. Use the first half for the 'fitting' or 'training' data. Test on the second half. Not deep; I advised some people in finance to do that in about 1982; not nearly new; and I very much doubt I was the first; gotta believe that J. Tukey, L. Breiman, and others thought of this long before I did.

   And if the test fails, then try to fit again. Run all night and in the morning, presto, have a model that does well on the test data. Now what have we? Not a trivial question to answer.

   Why least squares instead of something else? A good answer is not easy.

   I'm not criticizing biology, and it's done some excellent science, e.g., see the E. Lander lectures in his MIT course in biology.

   Statistics is a good field with a lot of good work by a lot of bright people with excellent backgrounds in pure and applied math.

   The first time I heard about 'machine learning' I guessed they meant filling in the optimal value function for stochastic dynamic programming. Okay, that is a kind of 'learning'. But, that's not what they had in mind.

   Next an example was the children's interactive computer game Animals where a child thinks of an animal, answers some questions about the animal to chase down a binary tree, stored in the program, to a leaf, and have the computer guess the animal. If the computer is wrong it asks the child "What is true for your animal and false for my guess.", adds to the binary tree, and thus, 'learns'. Call it a parlor trick.

   A little like 'self driving cars': Really 'self-driving'? Nope, not close. Only on streets where everything has been mapped down to 1 cm or better including all the painted lines, all the curbs, all the traffic signs, and all the traffic lights, etc. In no way does the 'self driving car' actually look at a new street scene, make sense out of it, and use that 'learning' to drive. Instead, so far a self driving car is about as amazing as a train on a track.

   We're talking hype, old wine, filtered, in new bottles, and not much that is new and good. Or, the old academic joke goes, "the new is not good and the good, not new.". The last time I looked at a leading 'machine learning' prof, I had conclude that he needed to return to ugrad school, be a math major, and learn how to read/write math.

   E.g., why maximum likelihood estimation? There are reasons, but I saw no hint of them in the Ng lectures at Stanford.

   We've known a lot about pure and applied statistics and how to use them for a very long time. The good work in statistics is high quality pure/applied math, e.g., Billingsly, 'Convergence of Probability Measures', e.g. with Ulam's result Le Cam called 'tightness', Brillinger on time series, Serfling on limit theorems, Rao on linear methods, Breiman on CART, and much more. I'm not seeing comparable quality in 'machine learning'; indeed, from all I've seen only a tiny fraction of the computer science machine learning people have the math prerequisites for good research in statistics.

   Yes, I've published peer-reviewed original research in statistics, indeed, applied to a problem in computer science.

   Show morethread
3. HilbertSpaceNov 2009
   Early in my career, I asked that question. Eventually I got some answers.

   Broadly usually the 'big bucks' are in things that are new at least in some important sense. To find such things, one recommended approach is to stand "on the shoulders of giants". Then you may still do some original work, but you have one heck of a base. My view is that Moore's law, the Internet, etc. are not nearly fully exploited, that enormous value remains, and the best way to get some of this value is to find a big unsolved problem where a good solution will be very valuable, stand on the shoulders of some giants, do some original work to get some powerful 'secret sauce' for solving the problem (which appears not to be easy to solve otherwise), start a business, please customers, get revenue, grow the business and sell it.

   Here, however, just what problem, giants, original work, secret sauce are NOT easy to see. Broadly we are trying to see, no, MAKE, the future, and usually that is not easy. While we can see what applications have been made of statistics in the past, what statistics will be applied where in the future is not easy to see.

   Yes, broadly statistics is highly promising. Or, in 'information technology' we want valuable, new information, are awash in cycles, bytes, bandwidth, infrastructure software, and data, and want to use these to create the information. Here, on appropriate problems, statistics can commonly, easily, totally 'blow away' anything else. The marriage between (A) 'information technology' in business and (B) statistics has yet to be consummated, even to reach to 'going steady' stage, assuming the traditional order of events.

   For 100 years or so, people have come to statistics from various areas of work. Usually they had some data and some questions. Some of the areas have been educational and psychological testing, experiments and testing in agriculture and medicine, industrial quality control, model building in economics, experimental work in physics and chemistry, investing, attempts to create mathematically based sciences in the social sciences, especially economics, psychology, sociology, and political science. In sociology, old examples would be James Coleman, Pete Rossi (professors of my wife when she got her Ph.D.), Leo Goodman.

   Likely the best fast, practical path into statistics is via books, courses, etc. intended for students in the social sciences. These students commonly do not have good backgrounds in mathematics. For the mathematical prerequisites, generally can get by, as a start, with just high school first year algebra. With this path can learn about probability distributions, the central limit theorem, the law of large numbers, statistical estimation and confidence intervals, hypothesis testing, cross tabulation, analysis of variance, regression analysis, principle components analysis, and more.

   For more, statistics is a rock solid part of mathematics, as solid as any part of pure mathematics, e.g., topology, geometry, analysis, algebra, etc. That is, statistics is based solidly on theorems and proofs, sometimes relatively deep ones.

   Statistics as theorems and proofs is called 'mathematical statistics'. Long standard has been (with TeX markup):

   Alexander M.\ Mood, Franklin A.\ Graybill, and Duane C.\ Boas, {\it Introduction to the Theory of Statistics, Third Edition,\/} McGraw-Hill, New York.\ \

   The main prerequisite for this book is just a not very good course in calculus, and the book actually makes not much use of calculus. Mostly all a student will need from calculus is that it can find the area under a curve. Since the book has long been standard, can't really ignore it, but it's ugly. And often, with just calculus, the book doesn't really give solid proofs of the results. E.g., their treatment of sufficient statistics has some nice intuition, but their proof is junk. The subject cries out for a good book, but I'm not trying to write one or waiting for someone else to.

   Can get some of the flavor of mathematical statistics done with high quality, as mathematics, in, say (with TeX markup):

   Jean-Ren\'e Barra, {\it Mathematical Basis of Statistics,\/} ISBN 0-12-079240-0, Academic Press, New York.\ \

   Robert J.\ Serfling, {\it Approximation Theorems of Mathematical Statistics,\/} ISBN 0-471-02403-1, John Wiley and Sons, New York.\ \

   P.\ Billingsley, {\it Convergence of Probability Measures, 2\raise0.5ex\hbox{ed},\/} ISBN: 0-471-19745-9, John Wiley, New York.\ \

   R.\ S.\ Lipster and A.\ N.\ Shiryayev, {\it Statistics of Random Processes I, II,} ISBN 0-387-90226-0, Springer-Verlag, New York.\ \

   However, pursued mathematically, statistics has some relatively advanced prerequisites some of which curiously are not popular in US university mathematics departments.

   For the prerequisites,

   High School. Should have had high school first and second year algebra (reasonable facility with algebraic manipulations, the binomial theorem, complex numbers, both of which will see again in important ways), plane geometry (where nearly all the work was proofs -- first place to learn about proofs), trigonometry (usually assumed in calculus and important in, say, analysis of organ tone harmonics and, thus, the most important example of an infinite dimensional Hilbert space), analytic geometry (especially the conic sections, especially ellipses which definitely will see again), and, if can, solid geometry (for more intuition in three dimensions).

   College. Need a standard calculus course, not necessarily a very comprehensive or difficult one because will do the subject all over again, maybe two or three times, and more later, WITH the proofs!

   Then need linear algebra, that is, how to work with data of several dimensions, which is just crucial. The big result is the polar decomposition, and there get to think about ellipses and get to use complex numbers. Also the course is an introduction to functional analysis and Hilbert space. Use any popular book to get started but in the end cover the classic, Halmos, "Finite Dimensional Vector Spaces'. Halmos wrote this when he was an assistant to von Neumann and intended it to be a finite dimensional introduction to Hilbert space (which once von Neumann had to explain to Hilbert) which it is. It also has some multi-linear algebra, of interest to exterior algebra now popular in relativity, but likely for nearly any business applications of statistics for the next several decades can skip that chapter.

   Then need some advanced calculus. That is a poorly organized, huge, catch-all subject beyond any one course. The usual start is 'baby' Rudin, 'Principles of Mathematical Analysis'. So, that's calculus with the proofs. Warning: The book is severe, succinct, with zero pictures. Have to draw your own pictures in your head. The book is packed solidly with powerful material just awash in important applications from statistics, economics, and engineering to physics, but there is hardly a hint of the applications in the book. I enjoyed the book, but few people will enjoy it or even get through it. Hint: Get a really good teacher! Then for more, popular is Spivak, 'Calculus on Manifolds', mostly because it is short. Actually, it's too short. I prefer Fleming, 'Functions of Several Variables' until get to the exterior algebra chapter at which time, if care, can now get the thin

   Henri Cartan, {\it Differential Forms,\/} ISBN 0-486-45010-4, Dover, Mineola, NY, 2006.\ \

   in English.

   Between Halmos, baby Rudin, and Spivak, you will have covered Harvard's Math 55 with a colorful description at

   http://www.american.com/archive/2008/march-april-magazine-co...

   Harvard tries to cover these three for freshman, but in most math departments the material will take you through all or nearly all of a focused undergraduate pure math major.

   If somewhere take a course in abstract algebra, e.g., with a little group theory, then that might help!

   Graduate School. Might learn a little more about topology, say, from Simmons, 'Introduction to Topology and Modern Analysis'. So, get good with metric spaces and get started on duality.

   The next big step is a course in measure theory and functional analysis. The Simmons work will help. Baby Rudin will be crucial; Halmos is recommended. So, with measure theory, do calculus over again and in a very different and much more powerful way and a way just crucial, even central, for mathematical approaches to statistics. The functional analysis will concentrate on representation theorems, the Radon-Nikodym theorem, and Hilbert and Banach spaces. Long popular, from Stanford and sometimes aimed at statistics students, is Royden, 'Real Analysis'. It's gorgeous. Should also read the real half of Rudin, 'Real and Complex Analysis'; it's a few steps up in difficulty from baby Rudin. Again, hint: Get a good course from a good teacher who can get you over the material without getting stuck. Then go back and study the material 2-3 more times, apply it, do some original research in it, and finally begin to understand it.

   We're talking high, top, center crown jewels of civilization here; the stuff is of just awesome power; my view is that it is, for the rest of this century, one of the main pillars of increases in economic productivity via the exploitation of Moore's law; on 'what to program', computer science is stuck and this material is the most promising way forward; as a famous restaurant owner once said about some Morey St. Denis, "you won't find better".

   Now are ready for probability. I recommend:

   Leo Breiman, {\it Probability,\/} ISBN 0-89871-296-3, SIAM, Philadelphia.\ \

   M.\ Lo\`eve, {\it Probability Theory, I and II, 4th Edition,\/} Springer-Verlag, New York.\ \

   Kai Lai Chung, {\it A Course in Probability Theory, Second Edition,\/} ISBN 0-12-174650-X, Academic Press, New York.\ \

   Jacques Neveu, {\it Mathematical Foundations of the Calculus of Probability,\/} Holden-Day, San Francisco.\ \

   Neveu is succinct, gorgeous, but not easy. This material is NOT popular in US departments of mathematics. At Princeton, see Cinlar.

   Then should make some progress with stochastic processes: The big book is Gihman and and Skorohod, right, in three volumes, but mostly people settle for shorter treatments. Whatever, should learn about Poisson processes, Markov processes (discrete time, finite state space is enough to get started), Brownian motion, and martingales. Might also learn about second order stationary processes. A good course in stochastic processes is NOT easy to find, especially in mathematics departments.

   Now are ready to attack statistics mathematically! I don't know of a good, single 'mathematical statistics' book at this level. Instead, there are many books -- I gave some above -- and then the journals. Thankfully, the field is relatively close to applications; so can take a practical problem and concentrate on what is relevant to it. One of my papers was some new work, at this level, in mathematical statistics for a problem in practical computing and computer science. The computer science community struggled terribly with the mathematics. So, it was some progress in computer science that community will have to struggle to understand.

   One approach to work in computing is just to try things, that is, just to throw things against the wall and see if they appear to stick. Or, maybe the truth of the situation really is a simple statistical model. Likely that model will fit the data well. So, try many simple statistical models. We use these models mostly ignoring the mathematical assumptions; mostly we are proceeding 'heuristically', that is, with guessing. If any of the models fit well, then they can be considered candidates for the truth. So, are throwing things against the wall to see if they fit. This approach also called 'data mining'.

   Problems:

   (1) Will be quite limited in what statistical models can use. That is, will be drawing from a cookbook instead of being a real chef who can create good, new dishes appropriate for the available ingredients and customers!

   (2) Don't have much, e.g., have not proceeded mathematically where from the deductive logic of assumptions and proofs actually know in advance some good things about the results. Something like breaking into a pharmacy, mixing up a lot of pills, taking them, and seeing if feel better! Uh, I'll pass and let you do that without me!

   (3) May have gone through a lot of computer time in an 'exponential, combinatorial explosion' of efforts throwing against the wall.

   (4) Have ignored a LOT in statistics that can add to what know about the results.

   (5) Will be tempted to conclude have have found 'causality' but will likely not have.

   (6) Will be tempted to conclude that have a model that predicts, but that is on shaky ground and risky and needs more work.

   Applied to important problems, this approach can be dangerous.

   There are not many healthy statistics departments. Much of the career interest is in biostatistics, especially related to FDA rules.

   It appears that among the top statistics departments are Berkeley and UNC. Since Breiman and Brillinger are at Berkeley and since Stanford, long good in statistics, is not far away, if I were looking for a Ph.D. in statistics then I'd pick Berkeley.

   There is a general problem getting a 'job' in a technical field and likely also with statistics. The assumption in US business is still as in factories 150 years ago: The supervisor knows more than the subordinate; the subordinate is supposed just to add common labor to what the supervisor says. In particular job descriptions are written by the supervisors, not the subordinates!

   Well, there are nearly no supervisors in US business who have even as much as a weak little hollow hint of a tiny clue about the material described here. So, won't need that material to qualify for the job descriptions. Moreover, if actually know such material and let that fact leak out, then will likely not make it past the first level HR English major phone screen person who will tremble and conclude that you are not like the employees they have! If you do get hired and someone in your management chain discovers that you have used some mathematics they don't understand, you might be on the way out the door, especially is your work was valuable for the company!

   Of course, the solution is to find a valuable application and start your own business. While maybe biomedical venture capital can understand crucial, core technical content, in information technology venture capital, likely you will be trying to explain this stuff to, say, history majors who worked in fund raising, marketing, general management, or financial analysis or have a background in just relatively elementary parts of computing. Just will NOT find more than six people, maybe not more than zero people, in US venture capital who can work the exercises in Royden or explain the strong law of large numbers. Sorry 'bout that! So, if you explain that the value of your venture is the powerful material in your 'secret sauce', then you will be regarded as a kook, far outside the mainstream of venture funded entrepreneurs, discarded, maybe even laughed at. As it is, some of the venture people are making money now, and the rest just want to be more like the ones who are making money. Looking for anything really new, powerful, and valuable is just NOT in the picture.

   So, once you have some results in users, customers, revenue, etc., then maybe you can get some venture funding; just why at that point, owning 100% of the business, you would take venture funding, a Board that can fire you, etc. is less clear! Or venture funding is not for everyone! Or venture firms prefer to give money to people who don't need it!

   For the real power of the 'secret sauce', you have just to keep that a secret!

   Once mathematicians have yachts, at the venture firms math will be to info tech like biochem is to biotech. In the meanwhile note that a valuable application of statistics can put you on the Forbes 400 where there are not many people! Generally if you are making a valuable application of advanced or new statistics, then you will not know many people who understand what you are doing. Or, if lots of people understood it, then it wouldn't be valuable!

   Show morethread
4. graycatOct 2021
   Ulam has a result in measure theory the French mathematician LeCam called tightness. The result is in

   Patrick Billingsley, Convergence of Probability Measures, John Wiley and Sons, New York, 1968.

   The result was used in a course I took in measure theory based probability theory. Later I used the result in one paper to show that a goofy statistical hypothesis test was not trivial.

   The URL I've had of the picture of Ulam, von Neumann, and Feynman at Los Alamos during WWII moved, but another copy is at

   https://3.bp.blogspot.com/-PRA6JxXrePw/Trm_LwKgxyI/AAAAAAAAC...

   Show morethread
5. graycatMar 2020
   No, that misses a big point:

   (1) Now, in the present context, quite broadly, the testing is important.

   (2) The news media reports lots of testing, e.g., from China, South Korea, Italy, the US, etc., e.g., lots of the testing from early on.

   (3) IIRC in some important respects, the good tests are super tough to do: E.g., Dr. Brix mentioned in one of the White House task force presentations that the test the US is using involves amplification which likely means the PCR (polymerase chain reaction), which before 1983 by Kary Mullis, was essentially impossible and remains astounding, amazing, and non-trivial, and then analyzing the results of the amplification, likely also non-trivial.

   (4) So, we have to suspect that a lot of the tests done early on were not so good. By not so good, we have to mean rates of false positives and/or false negatives too high.

   (4) IRCC, Dr. Brix did mention at one of those presentations that they, the US efforts, are willing to use only "approved" tests and that some of the tests submitted had false positives of, IRCC, 60%. Here we are on the way to coin flipping range.

   E.g., there is a news story about a guy offering to do tests in his college dorm room. I can't say that that is impossible, but it sounds not so good.

   Also for some days, a big news media issue was essentially "Where are the tests? Why can't we all get tested? When will we all get tested? What is the big hangup on the tests? Why do the tests take so long? Where has the Administration messed up? ...."

   (5) And of course, it is easy to get a rate of false positives of 0% -- just always report no virus found. Presto, bingo, 0% false positives.

   (6) The media has been reporting the results of lots of tests, getting headlines, news stories, eyeballs, attention, creating anxiety, getting ad revenue, etc. -- that is, dirty stuff.

   So, we all have to wonder, for a lot of those early tests from out in Wherever Land, what the heck was the quality? Were they really doing PCR? In particular, what the heck were the rates of false positives and false negatives? We just CANNOT just take for granted that the tests done had any meaningful quality. Then we have a tough time using the reported data from such questionable tests to evaluate the transmission rate, etc. of the virus and, thus, plan how and estimate when we might defeat the virus. In this context, important, quite broadly important.

   (7) So, very briefly, I mentioned, as Dr. Brix did in one of the presentations, that the rate of false positives is important and even at 0% can be from a silly (technical term, trivial) test.

   This mention never claimed that the medical profession is not aware of the rates of false positives and false negatives. And I'm fairly sure that the medical statistics people, some of the best statisticians there are, are quite aware of the now classic Neyman-Pearson result on how, for whatever rate of false positives specified, can get the lowest possible rate of false negatives. I have a relatively general proof from the Hahn decomposition from the Radon-Nikodym theorem (Rudin in Real and Complex Analysis gives the novel von Neumann proof).

   And for such things there is a lot more, e.g., the A. Wald work on optimal

   sequential testing* (a stochastic optimal control problem).

   Moreover I have some qualifications in the field since I published peer-reviewed original research, in Information Sciences, on anomaly detection, both multi-variate and distribution-free. And there I used the S. Ulam result tightness that even in the medical statistics community is likely not well known (it's in P. Billingsley, Convergence of Probability Measures). So, I have some technical qualifications to talk about test quality.

   But here I'm not trying to address, comment on, inform, educate, or critique the medical profession. Instead:

   My real point is, as Dr. Brix also seems to have had in mind, for the news media reporting we just must keep in mind test quality in particular rates of false positives and false negatives.

   Simple point. Maybe some people here read in some other intentions and objected to those. Again, simple point: In this context, we have to be careful about test quality, not just within the medical profession but, now, especially for what comes from the news media. Simple point. And in the context, an important point.

   It appears that some people have missed this simple point and presumed I was making some other point.

   Know of anything wrong with that simple, important point?

   Show morethread
6. graycatNov 2019
   The world is awash in texts and papers on applied statistics. My notes were an introduction to provide a start on what is needed to read more in applied statistics.

   For much of engineering, e.g., quality control, the statistical hypothesis testing I outlined is common.

   There are now classic texts on hypothesis testing:

   E. L. Lehmann, Testing Statistical Hypotheses, John Wiley, New York, 1959.

   E. L. Lehmann, Nonparametrics: Statistical Methods Based on Ranks, ISBN 0-8162-4994-6, Holden-Day, San Francisco, 1975.

   Sidney Siegel, Nonparametric Statistics for the Behavioral Sciences, McGraw-Hill, New York, 1956.

   The last is also just hypothesis testing; apparently the book is still referenced.

   From some statistical consultants inside GE, there is

   Gerald J. Hahn and Samuel S. Shapiro, Statistical Models in Engineering, John Wiley & Sons, New York, 1967.

   For machine learning, so far the approach appears to be cases of empirical curve fitting.

   The linear case of versions of regression analysis remain of interest. There the matrix theory I outlined is central. And regression analysis goes way back, 100 or so years, and is awash in uses of statistical hypothesis tests for the regression (F-ratio), individual coefficients, (t-tests), and confidence intervals on predicted values.

   For using probabilistic and statistical methods in the non-linear fitting, that is more advanced and likely an active field of research.

   In my own work, I use and/or create new statistical methods when and as I need them. My current work has some original work in, really, applied probability that also can be considered statistics. And when I was doing AI at IBM, I created some new mathematical statistics hypothesis tests to improve on what we were doing in AI. Later I published the work. Parts of the machine learning community might call that paper a case of supervised learning, but the paper is theorems and proofs in applied probability that give results on false alarm rates and detection rates, that is, statistical hypothesis tests (both multi-dimensional and distribution-free). So, that paper can be regarded as relevant to machine learning, but the prerequisites are a ugrad pure math major, about two years of graduate pure math with measure theory and functional analysis, a course in what Leo Breiman called graduate probability, and some work in stochastic processes. At one point I use a classic result of S. Ulam that the French probabilist Le Cam called tightness -- there is a nice presentation in

   Patrick Billingsley, Convergence of Probability Measures, John Wiley and Sons, New York, 1968.

   The work was for anomaly detection in complex systems so could be regarded as for both engineering and machine learning. But I can't give all the background here.

   Show morethread