Michael Nielsen

I think you're being a little too quick to jump to the answer. There are plenty of functions where this isn't true. Two easy examples are jump discontinuities (one that looks like this ____----- but more wiggly) (known as piecewise continuous) and functions with hole. These cannot be approximated to arbitrary accuracy, even with an infinite number of them.

You need to pay close attention to the wording in the approximation theorems.

Hornik[0] which does the proof you're discussing says

> This paper rigorously establishes that standard multilayer feedforward networks with as few as one _hidden layer_ using arbitrary squashing functions are capable of approximating any __Borel measurable function__ from one __finite__ dimensional space to another to any desired degree of accuracy

The confusion is probably in the Borel sigma-algebra. Borel means that it includes all finite open intervals in the real numbers. So (0,1) but not [0,1]. Open means boundary is not included! The interval also needs to be continuous, so our discontinuities violate the assumptions. Funahashi's[1] and Cybenko's[2] also require continuous functions.

This is actually a really important thing that gets ignored because it "seems obvious." Or maybe we just see it too often. But it is __critical__ to pay attention to assumptions and limitations. There is a lot of that going off the rails lately with ML and it's going to give us some roadblocks (we're already seeing some[side note]). EVERYTHING (and I mean…

Thanks for going to some trouble explaining your point of view.

I'll try to clarify mine: yes, a plus sign vs. a capital sigma can be quickly explained. Let's imagine you're reading a short paper about, say, signal processing, and it says in a footnote or prefatory matter "we use tau for 2 pi". Also it uses the sum notation I brought up. Oh, and the exponential is different too: maybe one of these https://math.stackexchange.com/questions/30046/alternative-notation-for-exponents-logs-and-roots/1158802 or maybe just the electrical-engineering angle symbol for exp(i theta).

Now you start reading the actual new material but even the fourier series looks all different and you're like "why is the author imposing this cognitive tax on me?" You can work through it, but why? The author must be a weirdo.

That'd be a reasonable reaction. But if notational trivia matter to those of us "with a freakish knack for manipulating abstract symbols" (http://worrydream.com/KillMath/) then they must also matter as a barrier to people who are more average in that regard. I agree if you're saying that something like sigma vs. plus sign is far down on the list of ways to improve mathematical communication -- even any notational reforms would not come first. (Though see http://cognitivemedium.com/ for some more thoughts on how computers doing what paper can't opens up new possibilities.) I also agree that learning must be active -- learning isn't just a matter of more efficiently pouring knowledge…

I really enjoyed physicist Michael Nielsen's "Using spaced repetition systems to see through a piece of mathematics," as the post lets the reader learn about the thought process behind a researcher who is highly accomplished in a mathematics-heavy field, in both academia and industry (2019): https://cognitivemedium.com/srs-mathematics

I plan to re-read the post as I gain a better understanding of higher-level mathematics over time, but from my first reads, I could relate and better put to words the idea that using spaced repetition to study technical concepts (though I personally always make sure it's not the main method of study, versus practice exercises) can help you better grasp connections between concepts over time.

A statement from the author's article that I still think about from time to time is: "I can’t emphasize enough the value of finding multiple different ways of thinking about the “same” mathematical ideas."

---

From a completely different angle, Nicky Case's "How to remember anything forever-ish" is a fun introductory post for giving an overview of what spaced repetition is, in case a friend or colleague is curious about the study method (2018): https://ncase.me/remember/

---

For a much older article that contains the fundamentals of making good elements/items/flashcards, Piotr Wozniak's "20 rules of knowledge formulation" is an article that I've referred to time and time again. I treat the rules more as guidelines, as I personally don't use images or…

>You narrated the ideas back

No, I rewound my visual memory, to see what I was doing, and then I did it again, to provide the same stimulus, to retrieve the lost memory.

I've met a few people who have nonverbal cognition, which is also not visual.

It's a bit like this, a "direct manipulation of ideas".

> The words or the language, as they are written or spoken, do not seem to play any role in my mechanism of thought. The psychical entities which seem to serve as elements in thought are certain signs and more or less clear images which can be “voluntarily” reproduced and combined… The above-mentioned elements are, in my case, of visual and some of muscular type. Conventional words or other signs have to be sought for laboriously only in a secondary stage, when the mentioned associative play is sufficiently established and can be reproduced at will.

https://cognitivemedium.com/srs-mathematics

Though as I mentioned, I had this more as a child, and I've become overreliant on language, and this "direct" facility has starkly deteriorated. But as the author explains in the rest of this article, this fluency can be regained by sheer force of will, by simply working on a very narrow problem space obsessively for weeks at a time.

(We don't know if that works for everyone, or if it's something weird like perfect pitch. I suspect everyone could gain a great deal of fluency -- that the chunking would reach such a high order level as to make the mnaipulation feel transparent. But…

A daily anki habit. The habit to check my cards, but also to add new ones. It's perhaps not a skill exactly, but it feels like a superpower nonetheless.

I started creating flashcards when I began studying spanish for my wife. --- a very common introduction to flashcards is via language learning.

But these days I put anything into a card that I feel is useful. It could be an actual event (like recently) with the russian attempted putsch. I might create a card like:

----

On
{
{c1::June 2023}},
{
{c2::Yevgeny Prigozhin}} the leader of
{
{c3::Wagner}} group staged a rebellion by marching his troups towards Moscow after taking
{
{c4::Rostov-on-don}}.

Within hours a treaty was brokered by
{
{c5::Belarus}} president
{
{c6::Alexander Lukashenko}}

----

these facts are all salient after reading about such an important event, in the moment. But now when I talk about this event years from now, it will spring to mind. I will also be forced to think about it from time to time, and remember the crazy (big) things that happened.

I'll also use flashcards these days to study a new API, or learn a domain model in a new company.

Important is to prune and actively delete cards that no longer make sense. So I suppose the skill is realizing how to use anki. It is a bit of a living entity.

It's a habit/skill I've honed for over 10 years now.

Some recommended reading for those wanting to add flashcards into their lives.

1. https://cognitivemedium.com/srs-mathematics

2.…

Indeed the typical wisdom in math is that the best way to understand a group (and the reason we usually care about one) is its actions.

There's some quote to the effect of: "Just as with men, a group is known by its actions." This was the only reference I could find off the top of my head https://gowers.wordpress.com/2011/11/06/group-actions-i/, but I've definitely heard the sentiment repeated in a couple different texts.

The sentiment expressed in this article might useful to hear if you're solely interested in learning how to use monad transformers or whatever in Haskell, eg, I know a fair amount of math, but very little category theory, and manage use monads just fine without any deep intuition as to why they're an important formalism. But if you want to develop your problem-solving skills, for example, I think "quiet contemplation" of axioms is pretty much always worse than working through several examples and absorbing some good exposition.

Here's a recent article that I found very though-provoking: http://cognitivemedium.com/invisible_visible/invisible_visible.html. They quote some mathematicians discussing their visual intuition for one property of groups.