Stats for nerds
1recommend
6mentions
3people
2018–2025years
17% recommend83% neutral
Mentions per year
What Hacker News says
- A lot of what I'm reading now relates directly back to my response to the recent "What are you working on?" thread[0]. So you'll see reflections of my recent interest in metaphysics, epistemology, and various cognitive science topics.
Would I recommend any or all of this stuff? Well, if you're particularly interested in cognitive science, metaphysics, and/or epistemology with an eye towards working on AGI, then yes. Otherwise, "it depends".
Metaphysics: A Contemporary Introduction[1]
Epistemology: A Contemporary Introduction[2]
The Organization of Behavior: A Neuropsychological Principle[3]
Sensation and Perception[4]
Cognitive Psychology[5]
Memory[6]
[0]: https://news.ycombinator.com/item?id=33425626
[1]: https://www.goodreads.com/book/show/31627001-metaphysics
[2]: https://www.goodreads.com/book/show/23741073-epistemology
[3]: https://www.goodreads.com/book/show/15928201-the-organizatio...
[4]: https://www.goodreads.com/book/show/2654548-sensation-and-pe...
[5]: https://www.goodreads.com/book/show/4546859-cognitive-psycho...
[6]: https://www.goodreads.com/book/show/22164410-memory-paperbac...
- On that particular day I was going through a bunch of different books, trying to work out what my plan was going to be for the next little while. Of those ones, I wound up finishing A Theory of Cognitive Dissonance and the Sensation and Perception one. I also knocked out a pretty good chunk of the one titled just Memory. The rest got bumped down the stack a bit.
I kinda getting back to the place now where I want to revisit The Organization of Behavior. That's the seminal work by Hebb that introduced Hebbian Learning and I'm on this big quest now to revisit a lot of old school approaches to learning & neural networks (in something at least approximating chronological order, although I won't be super strict about it) and code up implementations of each. So basically, some sort of Hebbian Learning system, a "McCulloch & Pitts Neuron", a Perceptron with the Perceptron Convergence Algorithm, Selfridge's Pandemonium Architecture, and so on, gradually working my way up to the current SOTA.
I'm about to finish up the Minsky & Papert Perceptrons book, and once I finish that I will probably read Volume 2 of the Parallel Distributed Processing series, then go back to Hebb.
FWIW, that Memory book was pretty fascinating. The general subject of human memory is, both simply taken for its own sake, and taken as inspiration for approaches to AI. I'm slightly more interested in AI than human memory qua human memory, but in either case it's fascinating material.
- Continuing on from last time[1].
Still working on Bosworth, this "box of sensors" project for working with embodied AI. The goal is still to collect fundamental data that represents how an entity "experiences" the physical world around it, with an eye towards trying to make some progress in the area of how it can learn a naive / intuitive metaphysics of the world, naive epistemology, naive physics, and so on. To be even more specific, what I'd love to discover is the answer to a question along the lines of "What's the bare minimum of "stuff" that a learning entity has to come hard-wired with (analogous to whatever human children are hard-wired with through evolution) in order to support the learning of all higher level concepts?"
The other angle I'm hoping to dig into a little bit concerns knowledge representation, and specifically how to represent "things" (whether those things are abstract concepts, concrete entities, or "other") are both the neural / "pattern matching" level AND the symbolic / logical reasoning level. That is to say, I'm interested in the idea of neuro-symbolic systems and how to overcome the "impedance mismatch" that is inherent there.
Am I arrogant enough to think I'll figure all that out? Not necessarily, but these are the areas that I'm exploring because I think these are fascinating topics.
So with all that said, in addition to working on the physical apparatus and associated software for the "box of sensors" thing, I've been going back and doing a lot of reading on metaphysics and epistemology. I also just read Leon Festinger's[2] book A Theory of Cognitive Dissonance, and am increasingly convinced that cognitive dissonance is an important part of how learning works. I can't shake the idea that there's a crucial connection between cognitive dissonance and "belief revision"[3]. I'm also going back to first sources and reading Donald O. Hebb's The Organization of Behavior which contains his ideas on learning that have come to be called Hebbian Learning[4].
Anyway, that's more or less what I'm working on at the moment. The only other thing of note is working on learning/re-learning a lot of maths stuff. That's been an ongoing thing for me for some time, and most recently I went back to a really basic book on Logic (Logic for Dummies actually) to refresh my memory of all of those basic rules of propositional logic and predicate logic. The goal is to move into some set theory stuff soon, and one of those various "How to write proofs" / "a transition to proof based maths" books and work on getting myself up to where I can understand proofs. Not trying to become a mathematician per-se, but just want to understand a lot of this stuff at a higher level than I currently do. And right alongside all of that is an ongoing review of really basic stuff like high-school algebra / trig all the way up through calculus, linear algebra, differential equations, etc. My formal maths education ended with Calc I and that was like 25 years ago, so I have some ground to cover.
[1]: https://news.ycombinator.com/item?id=32967331
[2]: https://en.wikipedia.org/wiki/Leon_Festinger
- When Hebb talks about "reverberation" in neural circuits, he still thinks in advance of our current knowledge of oscillatory neurodynamics. Here he speculates about the short term memory trace that holds position dynamically, prior to physical changes in the synapse:
"It might be supposed that the mnemonic trace is a lasting pattern of reverberatory activity without fixed locus like a cloud formation or eddies in a millpond"
From Hebb's 1948 "Organization of Behavior"
- The author (rightly in my view) doesn't group Hayek under Austrian.
What you're talking about sounds like transaction costs and information asymmetry, which are interesting. Hayek's work is interesting to me, even as someone who would self describe as a socialist.
His decentralized market solution to the economic calculation problem, the information from prices (price signals) tell producers and consumers to increase/decrease supply and demand.
But markets aren't perfectly competitive, there are monopolies. Markets are embedded in societies, they don't exist outside of them. Markets don't guarantee an optimal equilibrium. 50% of R&D spending is by the government.
He also did early work on neural networks, published The Sensory Order in 1952, 3 years after Hebb's The Organization of Behavior
- I think the most interesting and relevant strand of philosophy of mind is the connectionism of Rumelhart, McClelland and the PDP Research Group (including "godfather of deep learning" Geoff Hinton), exemplified in the tome "Parallel Distributed Processing" released in 1986
It even has an accompanying manual on how to implement artificial neural networks.
http://www.cnbc.cmu.edu/~plaut/IntroPDP/index.html
More along this line is found in David Marr's Vision: A Computational Investigation into the Human Representation and Processing of Visual Information
https://www.youtube.com/watch?v=9WcIiSCDqhE
There is a spiritual successor to the PDP books called "Rethinking Innateness: A connectionist perspective on development" lead by Jeffery Elman who did some of the first research on Recurrent Neural Networks (RNNs). It makes heavy reference to Chomsky's work on language and innateness
https://crl.ucsd.edu/~elman/Papers/book/index.shtml
This line of work even has heavyweight ancestors in Donald Hebb's work on the neural basis of learning in "The Organization of Behavior" (1949) and the noted austrian economist(?)/classical liberal and anti-communist Frederick Hayek covered similar lines in his book "The Sensory Order" (1952)
I think this tradition is carried on in the work of Yann Lecun, who recently released a talk titled "Deep Learning, Structure and Innate Priors"
http://www.abigailsee.com/2018/02/21/deep-learning-structure...
Related
Behave
37 recommend · 84
The Organized Mind
6 recommend · 18
The Organization Man
7 recommend · 26
Atomic Habits
191 recommend · 386
Human behavior
2 recommend · 4
The Psychology of Everyday Things
14 recommend · 29
Images of organization
5 recommend · 7
