Words and rules
How does language work, and how do we learn to speak? Why do languages change over time, and why do they have so many quirks and irregularities? In this book, the profound mysteries of language are explored.
Stats for nerds
1recommend
7mentions
4people
2016–2021years
29% recommend71% neutral
Mentions per year
What Hacker News says
- If you find this sort of thing interesting, I can highly recommend Steven Pinker's Words and Rules.
- "Compression" applies very much to grammatical variants of words. It's why the only bizarre irregular verbs in a language tend to be the ones that get used all the time - be, do, go etc - because for anything more obscure the brain just forgets the special case and applies the general rule.
Steven Pinker's book Words and Rules is a great layman-oriented read if this sort of thing interests you.
- I'm on vacation from my dayjob this week, so I have a lot of time to catch up on side interests for the next couple of days.
Right now I'm mostly still just deep-diving into various bits of "stuff" related to AI. I just finished reading a The SOAR Cognitive Architecture by John Laird, and Engineering General Intelligence Part 2 by Ben Goertzel, as well as Words and Rules by Stephen Pinker. Also spent some time building and installing all of OpenCog, and installing SOAR. The goal is to start messing around with both, and start actually implementing some simple agents.
I'm also now reading Foundations of Computational Linguistics: Human-Computer Communication in Natural Language by Roland Hausser, and have a book on Cognitive Modeling queued up.
And I need to find some time to get back to working with Apache River (the OSS version of what Sun used to call JINI).
If I'm feeling inspired one day soon, I might finally finish wrapping up the software for my "convection oven to reflow oven" conversion project. The hardware parts have all been done for months, and even most of the code is written, but I sort of mentally stalled out on that and just haven't bothered to push everything across the finish line. Really need to wrap that all up.
- I didn't exactly build anything in terms of creating anything new. I'm in a phase right now where I'm doing a lot of reading and background research. So I just finished reading Laird's SOAR book[1], and the second volume of Goertzel's Engineering General Intelligence[2]. Now I'm working on finishing Pinker's Words and Rules[3].
On the more hands-on front, all I really did was download the source for the various components of OpenCog[4], build them, and get all of that stuff installed. The plan for the next little while is to start playing around with building simple systems with OpenCog and SOAR[5] and gain a deeper understanding of both.
So yeah, wish I had something more exciting to talk about, but that's just where things are for me right now.
[1]: https://www.amazon.com/gp/product/0262122960/
[2]: https://www.amazon.com/Engineering-General-Intelligence-Part...
[3]: https://www.amazon.com/Words-Rules-Ingredients-Language-Scie...
[4]: https://opencog.org/
- OK, here ya go. This list is, for now, just a raw dump of titles and authors. When I have some more time, I'll turn this into a proper blog post with more commentary, sorting / grouping of the titles, links to complementary resources etc.
One bit of warning... the first half of this list includes everything "AI" related I've read (or am currently reading) for about the past two years. I say "AI" and not "AGI" because I haven't always bothered to make the distinction. So some of this stuff may not be relevant to you.
FYI, this list (especially the first half) is a heterogeneous lot that varies considerably along the "pop sci" to "academic" axis, as well as the "applied" to "theoretical" axis.
I'm also leaving off (for now anyway) most the "related, but not specifically about AI" stuff, like books on Probability, Bayesian Statistics, Graph Theory, etc., and books that are more about contemporary "Machine Learning" and "Deep Learning" topics. I do believe that there is a connection though, and that DL may play a role in building a truly intelligent system, so when I write the expanded blog post, I'll include more of that stuff.
OK, With no further ado, the "AI stuff I've read recently" list.
0. Blackboard Systems - Robert Engelmore & Tony Morgan
1. The Age of Spiritual Machines - Ray Kurzweil
2. The Emotion Machine - Marvin Minsky
3. The Society of Mind - Marvin Minsky
4. Possible Minds: 25 Ways of Looking at AI - John Brockman (ed)
5. AI Super-Powers: China, Silicon Valley and the New World Order - Kai-Fu Lee (more meta than actually about AI per-se)
6. Artificial Intelligence In Practice - Bernard Marr
7. Architects of Intelligence - Martin Ford
8. Gödel, Escher, Bach - Douglas Hofstadter
9. Superintelligence - Nick Bostrom
10. The Master Algorithm - Pedro Domingos
11. On Intelligence - Jeff Hawkins
12. Unified Theories of Cognition - Alan Newell
13. The AGI Revolution - Ben Goertzel
14. How To Create A Mind - Ray Kurzweil
15. Case-Based Reasoning: A Textbook - Michael M. Richter & Rosina O. Weber
16. The Age of Intelligent Machines - Ray Kurzweil
17. The Singularity is Near - Ray Kurzweil
18. An Introduction to MultiAgent Systems - Michael J. Wooldridge
19. Engineering General Intelligence - Volume 1 - Ben Goertzel
20. Artificial Intelligence: A Modern Approach - Peter Norvig & Stuart Russell
----
Below here is the stuff that is in the "to read" queue. This part tends to run more to textbooks and more academic material and less of the "pop sci" stuff. The choices on this list were inspired by other reading and research I've done, which led me to believe this group of titles includes material that will be useful. YMMV.
21. Cause Effect Pairs in Machine Learning - Isabelle Guyon, Alexander Statnikov, & Berna Bakir Batu (eds)
22. Propositional, Probabilistic, and Evidential Reasoning: Integrating Numerical and Symbolic Approaches - Weiru Liu
23. Human Compatible - Stuart Russell
23. Rebooting AI - Gary Marcus & Ernest Davis
24. Artificial Intelligence - Melanie Mitchell
25. Protocol Analysis - K. Anders Ericsson & Herbert A. Simon
26. The Robot's Dilemma: The Frame Problem in Artificial Intelligence - Zenon W. Pylyshyn
27. Image and Mind - Stephen Michael Kosslyn
28. Elements of Episodic Memory - Endel Tulving
29. Things and Places: How The Mind Connects With The World - Zenon W. Pylyshyn
30. Information Algebras: Generic Structures for Inference - Jurg Kohlas
31. Probabilistic Logics and Probabilistic Networks - Rolf Haenni, Jan-Willem Romeijn, Gregory Wheeler & Jon Williamson
32. Knowledge-Based Systems in Artificial Intelligence - Randall Davis & Douglas B. Lenat
33. Building Large Knowledge-Based Systems - Douglas B. Lenat & R.V. Guha
34. Elements of Information Theory - Thomas M. Cover & Joy A. Thomas
35. The Bayesian Choice - Christian P. Robert
35. Machine Learning for Commonsense Reasoning Processes - Xenia Naidenova
36. Reinforcement Learning and Optimal Control - Dimitri P. Bertsekas
37. An Introduction to Kolmogorov Complexity and Its Applications - Ming Li & Paul Vitanyi
38. The Mechanical Mind: A Philosophical Introduction to Minds, Machines, and Mental Representation - Tim Crane
39. Constraining Cognitive Theories - Zenon W. Pylyshyn
40. Neuro-Dynamic Programming - Dimitri P. Bertsekas & John Tsitsiklis
41. The Language Instinct - Steven Pinker
42. How The Mind Works - Steven Pinker
43. Images and Words - Steven Pinker
44. Words and Rules - Steven Pinker
45. The Blank Slate - Steven Pinker
46. Consciousness Explained - Daniel Dennett
47. How to Build a Brain: A Neural Architecture for Biological Cognition - Chris Eliasmith
48. Introduction to Artificial Intelligence - Eugene Charniak & Drew McDermott
49. Metamagical Themas - Douglas Hofstadter
50. I Am A Strange Loop - Douglas Hofstadter
51. Fluid Concepts And Creative Analogies: Computer Models Of The Fundamental Mechanisms Of Thought - Douglas Hofstadter
52. Principles of Synthetic Intelligence - Joshua Bach
53. Anatomy of the Mind - Ron Sun
54. Generic Inference: A Unifying Theory for Automated Reasoning - Marc Pouly & Jurg Kohlas
55. Knowledge Representation, Reasoning, and the Design of Intelligent Agents: The Answer-Set Programming Approach - Michael Gelfond & Yulia Kahl
56. Semantic Networks: An Evidential Formalization and its Connectionist Realization - Lockendra Shastri
57. Automated Planning and Acting - Malik Ghallab, Dana Nau, & Paolo Traverso
58. How Can The Human Mind Occur in the Physical Universe? - John Anderson
59. Spiking Neuron Models - Wulfram Gerstner & Werner Kistler
60. Neural Engineering - Chris Eliasmith & Charles H. Anderson
61. Principles of Neural Design - Peter Sterling & Simon Laughlin
62. Semantic Information Processing - Marvin Minsky
63. The SOAR Cognitive Architecture - John Laird
64. The Architecture of Cognition - John Anderson
65. Human Associative Memory - John Anderson & Gordon E. Bower
66. Knowledge Representation & Reasoning - Ronald J. Brachman & Hector J. Levesque
67. Cognitive Architectures - Maria Isabel Aldinhas Ferreira, Joao Silva Sequeira, & Rodrigo Ventura (eds)
68. Scripts, Goals, and Planning - Roger C. Schank & Robert P. Abelson
69. Reasoning About Plans - James F. Allen, Henry A. Kautz, Richard N. Pelavin, & Josh D. Tenenberg
70. Generating Abstraction Hierarchies: An Automated Approach to Reducing Search in Planning - Craig A. Knoblock
71. Representations of Commonsense Knowledge - Ernest Davis
72. Approaches to Knowledge Representation - G.A. Ringland & D.A. Duce
73. Knowledge Representation - John F. Sowa
74. Conceptual Structures: Information Processing in Mind and Machine - John F. Sowa
75. Inside Computer Understanding - Roger C. Schank & Christopher K. Riesbeck
76. Integrating Rules and Connectionism for Robust Commonsense Reasoning - Ron Sun
77. Probabilistic Logic Networks - Ben Goertzel, et al.
78. The Subtlety of Sameness: A Theory and Computer Model of Analogy Making - Robert M. French
79. Analogy Making as Perception - Melanie Mitchell
80. Real World Reasoning - Ben Goertzel, et al.
81. Artificial General Intelligence - Ben Goertzel & Cassio Pennachin
82. The Hidden Pattern: A Patternist Philosophy of Mind - Ben Goertzel
83. Engineering General Intelligence - Volume 2 - Ben Goertzel, Cassio Pennachin, & Nil Geisweiller
84. Computation and Cognition - Zenon W. Pylyshyn
85. Universal Artificial Intelligence - Marcus Hutter
86. The Cambridge Handbook of Computational Psychology - Ron Sun
87. Handbook of Neuroevolution Through Erlang - Gene I. Sher
88. Readings in Cognitive Science: A Perspective from Psychology and Artificial Intelligence - Allan Collins & Edward E. Smith -- this one contains a LOT of seminal papers in the field, including one of the earliest papers on Semantic Networks by Ross Quillian. I actually put this on the wrong part of the list, as I'm about halfway through reading this one, but just remembered it and didn't want to renumber everything. Heh.
Eh, that's a pretty good chunk of the stuff I've got lying around here, waiting to be read. There's a handful of others, but my fingers are getting numb.
- I don't have any strong opinion about this, but it's suggestive that AlphaGo combines a classical AI search with neural nets. Another example: Steven Pinker's theory that human language uses a combo of neural-net-style and logic-style processing (Words and Rules). This isn't to say that Skynet will run part of itself on Prolog -- more like, to get the best performance over the broadest range of domains will need multiple techniques.
One advantage of logic + deduction is potential clarity ahead of time about what the system might do, and ability to explain its actions. If AI safety is your top concern, those seem at least potentially valuable even when other techniques can build powerful systems sooner. (I wish more conventional software had that kind of transparency.)
Related
The Language Instinct
44 recommend · 108
The Sense of Style
45 recommend · 74
The Stuff of Thought
10 recommend · 18
The Blank Slate
42 recommend · 100
Politics and the English language
112 recommend · 263
On Writing Well
137 recommend · 235
Style
38 recommend · 57
