
In Meta Math!, Gregory Chaitin, one of the world's foremost mathematicians, leads us on a spellbinding journey of scientific discovery and illuminates the process by which he arrived at his groundbreaking theories.All of science is based on mathematics, but mathematicians have become painfully aware that math itself has serious limitations. This notion was first revealed in the work of two giants of twentieth-century mathematics: Kurt Godel and Alan Turing. Now their successor, Gregory Chaitin, digs even deeper into the foundations of mathematics, demonstrating that mathematics is riddled with randomness, enigmas, and paradoxes.Chaitin's revolutionary discovery, the Omega number, is an exquisitely complex representation of unknowability in mathematics. His investigations shed light on what, ultimately, we can know about the universe and the very nature of life. But if unknowability is at the core of Chaitin's theories, the great gift of his book is its completely engaging knowability. In an infectious and enthusiastic narrative, Chaitin introduces us to his passion for mathematics at its deepest and most philosophical level, and delineates the specific intellectual and intuitive steps he took toward the discovery of Omega. In the final analysis, he shows us that mathematics is as much art as logic, as much experimental science as pure reasoning.
Of course this is silly, but interesting nonetheless. And we routinely speak about such high-dimensional spaces in research and engineering. Or we can imagine optimization as traversing a pre-existing search space. It may be structured as a graph or perhaps a Euclidean space. And in that space we can imagine a loss surface, that sits there in peace all along, with its global minimum somewhere. And instead of "constructing" a solution, we are simply hiking in this space and trying to spot that valley. But this is a bit fictional. We never physically "instantiate" this surface. It's an imagined abstraction. In reality we just have a vector and some rules as to how we change that vector. But we can imagine those changes to be movements in an imagined space.
It's like the idea that the sculptor doesn't create the sculpture, the sculpture was there all along, he just had to remove the superfluous matter to reveal what was already there (i.e. the atoms belonging to the final sculpture).
The most interesting thing is kind of on the border, between these absurdly large spaces and the more manageable ones that are feasible to enumerate.
Another similar mindblow thing was when I forgot the password to a file that I encrypted. It's a fascinating thing that the bit pattern on the disk is functionally random now, and cracking it would take longer than the age of the universe. But if only I knew the password, it would only take just a second. There is a definite sequence of keystrokes I can execute to bring the universe in a state where the content will appear on my screen, it's so close, yet it's so-so far if you don't remember the password. Just a little difference in your brain state and it flips from trivial to hopeless.
PS, if you like thinking about such things, I recommend Meta-Math by Gregory Chaitin, it's very fun (providing an address VS constructing the thing is basically the gist of algorithmic information theory).
I especially like the Turing / Godel equivalency stuff he discusses.