Radically elementary probability theory
Using only the very elementary framework of finite probability spaces, this book treats a number of topics in the modern theory of stochastic processes. This is made possible by using a small amount of Abraham Robinson's nonstandard analysis and not attempting to convert the results into conventional form.
Stats for nerds
6recommend
9mentions
7people
2011–2025years
78% recommend22% neutral
Mentions per year
What Hacker News says
- Alright, I'll bite:
To defend Wildberger a bit (because I am an ultrafinitist) I'd like to state first that Wildberger has poor personal PR ability.
Now, as programmers here, you are all natural ultrafinitists as you work with finite quantities (computer systems) and use numerical methods to accurately approximate real numbers.
An ultrafinitist says that that's really all there is to it. The extra axiomatic fluff about infinities existing are logically unnecessary to do all the heavy lifting of the math that we are familiar with. Wildberger's point (and the point of all ultrafinitist claims) is that it's an intellectual and pedagogical disservice to teach and speak of, e.g. Real Numbers, as if they're actually involving infinite quantities that you can never fully specify. We are always going to have to confront the numerical methods part, so it's better to make teaching about numbers methodologically aligned with how we actually measure and use them.
I have personally been working on building various finite equivalents to familiar math. I recommend anyone to read Radically Elementary Probability Theory by Nelson to get a better sense of how to do finite math, at least at the theoretical level. Once again, on a practical level to do with directly computing quantities, we've only ever done finite math.
- Side comment: anyone interested in calculus via infinitesimals may also be interested in taking a look at Radically Elementary Probability Theory by Ed Nelson: https://web.math.princeton.edu/~nelson/books/rept.pdf
- I really appreciated this piece. Thank you to OP for writing and submitting it.
The thing that piqued my interest was the side remark that the Dirac delta is a “distribution“, and that this is an unfortunate name clash with the same concept in probability (measure theory).
My training (in EE) used both Dirac delta “functions” (in signal processing) and distributions in the sense of measure theory (in estimation theory). Really two separate forks of coursework.
I had always thought that the use of delta functions in convolution integrals (signal processing) was ultimately justified by measure theory — the same machinery as I learned (with some effort) when I took measure theoretic probability.
But, as flagged by the OP, that is not the case! Mind blown.
Some of this is the result of the way these concepts are taught. There is some hand waving both in signal processing, and in estimation theory, when these difficult functions and integrals come up.
I’m not aware of signal processing courses (probably graduate level) in which convolution against delta “functions” uses the distribution concept. There are indeed words to the effect of either,
- Dirac delta is not a function, but think of it as a limit of increasingly-concentrated Gaussians;
- use of Dirac delta is ok, because we don’t need to represent it directly, only the result of an inner product against a smooth function (i.e., a convolution)
But these excuses are not rigorously justified, even at the graduate level, in my experience.
*
Separately from that, I wonder if OP has ever seen the book Radically Elementary Probability Theory, by Edward Nelson (https://web.math.princeton.edu/~nelson/books/rept.pdf). It uses nonstandard analysis to get around a lot of the (elegant) fussiness of measure theory.
The preface alone is fun to read.
- Completely agree, speaking as a mathematician who put in the time to understand the details of sigma-algebras. It's an implementation detail. We don't need to talk about inductors and MOSFETs to describe how bubble-sort works, and we don't need to talk about sigma-algebras to talk about conditional probabilities or expectations.
A wacky alternative implementation of probability theory can be found in Nelson's book "Radically Elementary Probability Theory" - I highly recommend it as a way to see how little the underlying framework matters.
- In retrospect, my other comment was stupidly obtuse. Both too technical (in the sense of specificity) and too unstructured (in the sense of presentation order). A more appropriate path from CS might be analogous (well, inverse if anything) to the path Robert Goldblatt has taken. It dips into nonstandard analysis, but not totally without reason. Some subset of the following, with nLab and Wikipedia supplementing as necessary:
0. Milewski's "Category Theory for Programmers"[0]
1. Goldblatt's "Topoi"[1]
2. McLarty's "The Uses and Abuses of the History of Topos Theory"[2] (this does not require [1], it just undoes some historical assumptions made in [1] and, like everything else by McLarty, is extraordinarily well-written)
3. Goldblatt's "Lectures on the Hyperreals"[3]
4. Nelson's "Radically Elementary Probability Theory"[4]
5. Tao's "Ultraproducts as a Bridge Between Discrete and Continuous Analysis"[5]
6. Some canonical machine learning text, like Murphy[6] or Bishop[7]
7. Koller/Friedman's "Probabilistic Graphical Models"[8]
8. Lawvere's "Taking Categories Seriously"[9]
From there you should see a variety of paths for mapping (things:Uncertainty) <-> (things:Structure). The Giry monad is just one of them, and would probably be understandable after reading Barr/Wells' "Toposes, Triples and Theories"[10].
The above list also assumes some comfort with integration. Particularly good books in line with this pedagogical path might be:
9. Any and all canonical intros to real analysis
10. Malliavin's "Integration and Probability"[11]
11. Segal/Kunze's "Integrals and Operators"[12]
Similarly, some normative focus on probability would be useful:
12. Jaynes' "Probability Theory"[13]
13. Pearl's "Causality"[14]
---
[0] https://bartoszmilewski.com/2014/10/28/category-theory-for-p...
[1] https://www.amazon.com/Topoi-Categorial-Analysis-Logic-Mathe...
[2] http://www.cwru.edu/artsci/phil/UsesandAbuses%20HistoryTopos...
[3] https://www.amazon.com/Lectures-Hyperreals-Introduction-Nons...
[4] https://web.math.princeton.edu/%7Enelson/books/rept.pdf
[5] https://www.youtube.com/watch?v=IS9fsr3yGLE
[6] https://www.amazon.com/Machine-Learning-Probabilistic-Perspe...
[7] https://www.amazon.com/Pattern-Recognition-Learning-Informat...
[8] https://www.amazon.com/Probabilistic-Graphical-Models-Princi...
[9] http://www.emis.de/journals/TAC/reprints/articles/8/tr8.pdf
[10] http://www.tac.mta.ca/tac/reprints/articles/12/tr12.pdf
[11] https://www.springer.com/us/book/9780387944098
[12] https://www.amazon.com/Integrals-Operators-Grundlehren-mathe...
[13] http://www.med.mcgill.ca/epidemiology/hanley/bios601/Gaussia...
[14] https://www.amazon.com/Causality-Reasoning-Inference-Judea-P...
- > If you want to go beyond calculus, almost everything (in this and related areas) is about sequences, limits, limiting processes, functions, and transformations. There, non-standard analysis tends not to help ...
Why do you say this? I ask because I've found internal set theory, Edward Nelson's axiomatic version of nonstandard analysis, to be a lovely tool for doing typical sorts of things in analysis.
You have to learn to wield the "standard" predicate [0], which is too dark an art for some mathematicians, I suppose. But, in my opinion, nonstandard characterizations of notions like convergence and continuity are delightfully simple and direct.
It also turns out that when you have nonstandard numbers at hand, infinity is an over-powerful abstraction for some purposes. Nelson came up with a new formalism for probability theory [1], for example, that makes finite spaces powerful enough to capture what's interesting for most purposes. Similarly, finite but unlimited sequences often are "long enough" to incorporate all the interesting behavior of infinite sequences.
0. Alain Robert's Nonstandard Analysis is a good starting point.
1. See his short book Radically Elementary Probability Theory. I love this book, and didn't much like probability theory before reading it.
Related
Probability Theory
77 recommend · 193
Elementary probability
1 recommend · 2
An introduction to probability theory and its applications
8 recommend · 11
A Course in Probability Theory
3 recommend · 11
A first course in probability
9 recommend · 16
Elementary probability theory with stochastic processes
2 recommend · 2
