David Mumford, a Fields Medal mathematician at Brown, writes about math, probability, vision, AI, and consciousness.
HN links his essay 'The Dawning of the Age of Stochasticity' in debates over whether probabilistic models are scientific, and his Grothendieck obituary for its summary of that mathematician's work.
Even Russel and Norvig is still applicable for the fundamentals, and with the rise of agenic efforts would be extremely helpful.
The updates to even the Bias/Variance Dilemma (Geman 1992) are minor if you look at the original paper:
https://www.dam.brown.edu/people/documents/bias-variance.pdf
They were dealing with small datasets or infinite datasets, and double decent only really works when the patterns in your test set are similar enough to those in your training set.
While you do need to be mindful about some of the the older opinions, the fundamentals are the same.
For fine tuning or RL, the same problems with small datasets or infinite datasets, where concept classes for training data may be novel, that 1992 paper still applies and will bite you if you assume it is universally invalid.
Most of the foundational concepts are from the mid 20th century.
The availability of mass amounts of data and new discoveries have modified the assumptions and tooling way more than invalidating previous research. Skim that paper and you will see they simply dismissed the mass data and compute we have today as impractical at the time.
Find the book that works best for you, learn the concepts and build tacit experience.
Lots of efforts are trying to incorporate symbolic and other methods too.
IMHO Building breadth and depth is what will save time and help you find opportunities, knowledge of the fundamentals is critical for that.
The author, David Mumford, is known for his distinguished work in algebraic geometry, and was awarded the Fields Medal in 1974. [2]
Pattern theory was formulated by Ulf Grenander to describe knowledge of the world as patterns. [3]
Prof. Mumford explains that "[s]everal essential ideas brought me to realize how Grenander's Pattern Theory was the right way to understand almost all cognitive skills and especially vision. One was the emphasis on pattern synthesis as well as pattern analysis." [0]. Second, "was that natural signals given by functions f vary not only by random additive perturbations but often by composition with random rearrangements of their domain. The resulting probability distribution in the vector space of signals is nothing like Gaussian. Its support is usually a twisted snakey submanifold. This puts the lie to all simplistic Gaussian pattern recognition algorithms." [0]. And third, "graphical structures were everywhere in the representations of ideas in cognitive domains" [0].
His most recent work seems to be "Pattern Theory, the Stochastic Analysis of Real World Signals" [1]
[0] https://www.dam.brown.edu/people/mumford/vision/pattern.html
[1] https://www.amazon.com/dp/1568815794/ref=sr_1_1?ie=UTF8&qid=1351792472&sr=8-1&keywords=Pattern+Theory+Mumford+Desolneux
[2] https://en.wikipedia.org/wiki/David_Mumford
[3] https://en.wikipedia.org/wiki/Pattern_theory
This biographical sketch is well-written, concise, and complete (or at least it appears to be). It includes many quotes and photos that I hadn’t seen before.
But I can imagine that some readers will be left wondering what was so remarkable about this guy’s work. Unfortunately, it’s hard to explain without some exposure to pure mathematics.
David Mumford, who pops up in this article, has a lengthy blog post[1] on this exact problem, stemming from his experience writing Grothendieck’s obituary for Nature. It’s an interesting read if you have some math background.
In any case, I think the New York Times obituary by Edward Frenkel[2] does a nice job of giving a taste of his work to a lay audience (by tackling the problem of defining Grothendieck’s schemes, just like Mumford did).
[1] http://www.dam.brown.edu/people/mumford/blog/2014/Grothendieck.html
[2] https://www.nytimes.com/2014/11/25/science/the-lives-of-alexander-grothendieck-a-mathematical-visionary.html
I am reminded of the recent obituary of Grothendieck: http://www.dam.brown.edu/people/mumford/blog/2014/Grothendieck.html
Two extremely well-known mathematicians were asked by Nature to write an obituary for the late Grothendieck. They wrote it to be mostly understood by a nonmathematical audience, but Nature ultimately rejected the obituary on it being too technical.
It is almost as if each discipline has become an island. When I submit a math paper, I do not at all imagine a nonmathematician will read it. But then again, I also do not write it so that nonmathematicians could read it. I suspect this is largely the same across different fields.
I'm also not certain if this is necessarily bad; it would be tremendously hard and time-consuming to make each article understandable to another field. Or maybe this is precisely the problem?
The debate is mostly about:
Are opaque probabilistic models scientific?
David Mumford's stance:
"This paper is a meant to be a polemic which argues for a very fundamental
point: that stochastic models and statistical reasoning are more relevant i) to
the world, ii) to science and many parts of mathematics and iii) particularly
to understanding the computations in our own minds, than exact models and
logical reasoning"
https://www.dam.brown.edu/people/mumford/beyond/papers/2000b--DawningAgeStoch-NC.pdf
i recommend this paper to understand brain-state-in-a-box[0]. In my studies of linear algebra / calculus, we had optimum calculus reaching error minimum.
help me out, i learnt it a long time ago, would "Optimum in der Infinitesimalrechnung" be optimum calculus?
[0] https://www.dam.brown.edu/people/elie/am41%202012/gBSB.pdf
(edit: wording)