Parallel distributed processing

What Hacker News says

1. seertaakAug 2025
   Rumelhart et al wrote "Parallel Distributed Processing"; there's a chapter where he proves that the backprop algorithm maximizes "harmony", which is simply a different formulation of error minimization.

   I remember reading this book enthusiastically back in the mid 90s. I don't recall struggling with the proof, it was fairly straightforward. (I was in senior high school year at the time.)

   thread
2. robgJun 2023
   Highly recommend the exercises in Rumelhart and McClelland - Parallel Distributed Processing: Explorations in the Microstructure of Cognition from 1986-1987 (two volumes)

   https://direct.mit.edu/books/book/4424/Parallel-Distributed-...

   thread
3. robgJan 2023
   Coming from cognitive neuroscience surprised that Explorations in Parallel Distributed Processing by McClelland and Rumelhart doesn’t get more attention as a classic in bridging old school AI approaches with the modern paradigm.

   https://psycnet.apa.org/record/1988-97441-000

   thread
4. PeterisPFeb 2022
   I was just reading 1986 book Parallel Distributed Processing (https://mitpress.mit.edu/books/parallel-distributed-processi...) and was quite surprised by the relevance of their arguments how learning systems should be built and structured, which map quite accurately to how we actually developed them in the following decades after we got more computing power, more data and engineering know-how.
   thread
5. f00_Mar 2018
   I think the most interesting and relevant strand of philosophy of mind is the connectionism of Rumelhart, McClelland and the PDP Research Group (including "godfather of deep learning" Geoff Hinton), exemplified in the tome "Parallel Distributed Processing" released in 1986

   It even has an accompanying manual on how to implement artificial neural networks.

   http://www.cnbc.cmu.edu/~plaut/IntroPDP/index.html

   More along this line is found in David Marr's Vision: A Computational Investigation into the Human Representation and Processing of Visual Information

   https://www.youtube.com/watch?v=9WcIiSCDqhE

   There is a spiritual successor to the PDP books called "Rethinking Innateness: A connectionist perspective on development" lead by Jeffery Elman who did some of the first research on Recurrent Neural Networks (RNNs). It makes heavy reference to Chomsky's work on language and innateness

   https://crl.ucsd.edu/~elman/Papers/book/index.shtml

   This line of work even has heavyweight ancestors in Donald Hebb's work on the neural basis of learning in "The Organization of Behavior" (1949) and the noted austrian economist(?)/classical liberal and anti-communist Frederick Hayek covered similar lines in his book "The Sensory Order" (1952)

   I think this tradition is carried on in the work of Yann Lecun, who recently released a talk titled "Deep Learning, Structure and Innate Priors"

   http://www.abigailsee.com/2018/02/21/deep-learning-structure...

   Show morethread
6. signa11Sep 2014
   may i also humbly suggest the two vol. series called "parallel distributed processing" (rumelhart et al) which provides an excellent overview of early NN research.
   thread