Stats for nerds
2recommend
5mentions
3people
2012–2020years
60% recommend40% neutral
Mentions per year
What Hacker News says
- Fuzzy "logic" is best used when you take out all the references to logic and just think about fuzzy sets -- then it's just fuzzy set theory, which is useful when you need to model partial set membership.
I enjoyed reading Probabilistic Logic Networks (https://www.amazon.com/Probabilistic-Logic-Networks-Comprehe...) as it introduces a possible framework to sanely deal with different kinds of truth values about different things which aren't necessarily a single number.
- I thought I'd pop in and mention this set of lectures (now a book, huh) from Scott Aaronson on quantum mechanics and quantum computing: http://www.scottaaronson.com/democritus/ Particularly Lecture 9 goes into the basic mathematics of quantum mechanics, and explains why it's more appropriate to consider it an extension to probability theory using complex numbers and call them amplitudes, rather than classic probabilities. Not being a physicist myself I don't know if this is really helpful in clearing up professional scientists' intuitions, but it's helpful for me at least in grasping the basics.
You're on the right track by noting that quantum theorists can use probability to communicate their findings. A single classical probability like 0.7 is a summary, not the whole picture. For the whole picture, you need a log of observations with time stamps, which is a tremendous amount of data because taken to the extreme it's a balancing act between maximizing the data you know about you and your surroundings and minimizing the number of planck unit time steps to collect the data. Even with more realistic amounts of data it's still a lot to pack around, so you can summarize it by constructing a probability distribution which is more convenient mathematically, and you can summarize that probability distribution into a handful of numbers if you need to because those are even easier to pack around. (Like the single probability 0.7, or two numbers u and s that characterize a Normal distribution, etc.)
So if you think of probability as just summarizing your past observations and constraining your expectations for future observations (because what else can?), I think it's easy to see how two people can have different probabilities that represent a belief in a proposition or an abstract system or something else. If both are being honest (I'm packing a lot of assumptions in that word since this is basically Aumann's agreement theorem) and the probabilities are different, there are a couple reasons why: either at least one of them has information (observations) the other one does not, or they started out with different prior probabilities. In the former case, they can reach agreement by sharing data. In the latter case, they can eventually reach agreement by obtaining more data or choosing a common prior. With enough successive Bayesian updates, you can arrive at the same conclusion regardless of where you started. (In practice, of course, things are more hairy and not this simplistic, which is what the submitted pdf is addressing in part.)
I find it hard to grok what it means for something to have a physical frequency property. I can understand it as a statistic of an abstract system, i.e. a probability, but in light of Many Worlds, I don't think a frequency can fundamentally be a property of something in the same way energy can be. But since the energy of a photon is related to the photon's "frequency" via its "wavelength", is energy really fundamental or is the relation just a convenient means of making the wave mechanics useful? I've read Feynman tell how the whole idea of forces is just a somewhat circular simplification that's not really fundamental, even though sometimes it seems like it is. While I have a high degree of certainty about the physics accurately describing reality as she is, my meta-belief / confidence about that high certainty isn't nearly as high simply because I feel a lot of confusion about the ultimate nature of reality. But at least at a higher level than fundamental physics, it's clear to me that saying a "fair coin" has p=0.5 per unique side is really just summarizing we don't have enough knowledge of the flipper and the environment it is flipped in to predict the result with certainty in the way we can predict the result of the output of a logic OR gate with low- and high- voltage inputs with certainty. This is different than the uncertainty of which branch you are within a Many Worlds universe, where it's more of a physical resolution problem rather than one of needing more knowledge, similar to not being able to find out what happens between planck unit time steps (if anything happens and if the question means anything).
Ben Goertzel's book Probabilistic Logic Networks is another resource I'd recommend. Sorry if my branching reply is a bit big, but I'm constantly trying to clarify things for myself as well. ;)
- So I figured it out. Basically, they take the idea of AGI seriously, and actually consider and talk about the repercussions, and therefore you dismiss them and their ideas as fringe and not worth investigating. I know that, because if you had investigated at all, you would see that all of those projects had really interesting results and these people are not being vague and hand-waving.
Not all of those projects I listed identify themselves as AGI. However, they should go in the same group.
And anyway, all of those projects have demonstrated progress. If you looked into them at all then you would see that. Ben Goertzel is using some aspects of his AGI research in mainstream (narrow) AI projects. OpenCog has released a number of solid demonstrations of current features. And Goertzel isn't hand-waving or bullshitting in his numerous books and scientific papers, for example Probabilistic Logic Networks: A Comprehensive Framework for Uncertain Inference (336 pages).
Hawkins has demonstrated very interesting progress with his software and has a commercial application https://www.numenta.com/grok_info.html
Voss is using his system at Adaptive AI as a commercial enterprise.
Qualcomm is funding Brain Corporation (Izhikevich et al) so obviously they are taking it seriously. A bakery in Tokyo has tested Brain Corporation's machine vision technology to power a semi-automated cashier system
- OK, here ya go. This list is, for now, just a raw dump of titles and authors. When I have some more time, I'll turn this into a proper blog post with more commentary, sorting / grouping of the titles, links to complementary resources etc.
One bit of warning... the first half of this list includes everything "AI" related I've read (or am currently reading) for about the past two years. I say "AI" and not "AGI" because I haven't always bothered to make the distinction. So some of this stuff may not be relevant to you.
FYI, this list (especially the first half) is a heterogeneous lot that varies considerably along the "pop sci" to "academic" axis, as well as the "applied" to "theoretical" axis.
I'm also leaving off (for now anyway) most the "related, but not specifically about AI" stuff, like books on Probability, Bayesian Statistics, Graph Theory, etc., and books that are more about contemporary "Machine Learning" and "Deep Learning" topics. I do believe that there is a connection though, and that DL may play a role in building a truly intelligent system, so when I write the expanded blog post, I'll include more of that stuff.
OK, With no further ado, the "AI stuff I've read recently" list.
0. Blackboard Systems - Robert Engelmore & Tony Morgan
1. The Age of Spiritual Machines - Ray Kurzweil
2. The Emotion Machine - Marvin Minsky
3. The Society of Mind - Marvin Minsky
4. Possible Minds: 25 Ways of Looking at AI - John Brockman (ed)
5. AI Super-Powers: China, Silicon Valley and the New World Order - Kai-Fu Lee (more meta than actually about AI per-se)
6. Artificial Intelligence In Practice - Bernard Marr
7. Architects of Intelligence - Martin Ford
8. Gödel, Escher, Bach - Douglas Hofstadter
9. Superintelligence - Nick Bostrom
10. The Master Algorithm - Pedro Domingos
11. On Intelligence - Jeff Hawkins
12. Unified Theories of Cognition - Alan Newell
13. The AGI Revolution - Ben Goertzel
14. How To Create A Mind - Ray Kurzweil
15. Case-Based Reasoning: A Textbook - Michael M. Richter & Rosina O. Weber
16. The Age of Intelligent Machines - Ray Kurzweil
17. The Singularity is Near - Ray Kurzweil
18. An Introduction to MultiAgent Systems - Michael J. Wooldridge
19. Engineering General Intelligence - Volume 1 - Ben Goertzel
20. Artificial Intelligence: A Modern Approach - Peter Norvig & Stuart Russell
----
Below here is the stuff that is in the "to read" queue. This part tends to run more to textbooks and more academic material and less of the "pop sci" stuff. The choices on this list were inspired by other reading and research I've done, which led me to believe this group of titles includes material that will be useful. YMMV.
21. Cause Effect Pairs in Machine Learning - Isabelle Guyon, Alexander Statnikov, & Berna Bakir Batu (eds)
22. Propositional, Probabilistic, and Evidential Reasoning: Integrating Numerical and Symbolic Approaches - Weiru Liu
23. Human Compatible - Stuart Russell
23. Rebooting AI - Gary Marcus & Ernest Davis
24. Artificial Intelligence - Melanie Mitchell
25. Protocol Analysis - K. Anders Ericsson & Herbert A. Simon
26. The Robot's Dilemma: The Frame Problem in Artificial Intelligence - Zenon W. Pylyshyn
27. Image and Mind - Stephen Michael Kosslyn
28. Elements of Episodic Memory - Endel Tulving
29. Things and Places: How The Mind Connects With The World - Zenon W. Pylyshyn
30. Information Algebras: Generic Structures for Inference - Jurg Kohlas
31. Probabilistic Logics and Probabilistic Networks - Rolf Haenni, Jan-Willem Romeijn, Gregory Wheeler & Jon Williamson
32. Knowledge-Based Systems in Artificial Intelligence - Randall Davis & Douglas B. Lenat
33. Building Large Knowledge-Based Systems - Douglas B. Lenat & R.V. Guha
34. Elements of Information Theory - Thomas M. Cover & Joy A. Thomas
35. The Bayesian Choice - Christian P. Robert
35. Machine Learning for Commonsense Reasoning Processes - Xenia Naidenova
36. Reinforcement Learning and Optimal Control - Dimitri P. Bertsekas
37. An Introduction to Kolmogorov Complexity and Its Applications - Ming Li & Paul Vitanyi
38. The Mechanical Mind: A Philosophical Introduction to Minds, Machines, and Mental Representation - Tim Crane
39. Constraining Cognitive Theories - Zenon W. Pylyshyn
40. Neuro-Dynamic Programming - Dimitri P. Bertsekas & John Tsitsiklis
41. The Language Instinct - Steven Pinker
42. How The Mind Works - Steven Pinker
43. Images and Words - Steven Pinker
44. Words and Rules - Steven Pinker
45. The Blank Slate - Steven Pinker
46. Consciousness Explained - Daniel Dennett
47. How to Build a Brain: A Neural Architecture for Biological Cognition - Chris Eliasmith
48. Introduction to Artificial Intelligence - Eugene Charniak & Drew McDermott
49. Metamagical Themas - Douglas Hofstadter
50. I Am A Strange Loop - Douglas Hofstadter
51. Fluid Concepts And Creative Analogies: Computer Models Of The Fundamental Mechanisms Of Thought - Douglas Hofstadter
52. Principles of Synthetic Intelligence - Joshua Bach
53. Anatomy of the Mind - Ron Sun
54. Generic Inference: A Unifying Theory for Automated Reasoning - Marc Pouly & Jurg Kohlas
55. Knowledge Representation, Reasoning, and the Design of Intelligent Agents: The Answer-Set Programming Approach - Michael Gelfond & Yulia Kahl
56. Semantic Networks: An Evidential Formalization and its Connectionist Realization - Lockendra Shastri
57. Automated Planning and Acting - Malik Ghallab, Dana Nau, & Paolo Traverso
58. How Can The Human Mind Occur in the Physical Universe? - John Anderson
59. Spiking Neuron Models - Wulfram Gerstner & Werner Kistler
60. Neural Engineering - Chris Eliasmith & Charles H. Anderson
61. Principles of Neural Design - Peter Sterling & Simon Laughlin
62. Semantic Information Processing - Marvin Minsky
63. The SOAR Cognitive Architecture - John Laird
64. The Architecture of Cognition - John Anderson
65. Human Associative Memory - John Anderson & Gordon E. Bower
66. Knowledge Representation & Reasoning - Ronald J. Brachman & Hector J. Levesque
67. Cognitive Architectures - Maria Isabel Aldinhas Ferreira, Joao Silva Sequeira, & Rodrigo Ventura (eds)
68. Scripts, Goals, and Planning - Roger C. Schank & Robert P. Abelson
69. Reasoning About Plans - James F. Allen, Henry A. Kautz, Richard N. Pelavin, & Josh D. Tenenberg
70. Generating Abstraction Hierarchies: An Automated Approach to Reducing Search in Planning - Craig A. Knoblock
71. Representations of Commonsense Knowledge - Ernest Davis
72. Approaches to Knowledge Representation - G.A. Ringland & D.A. Duce
73. Knowledge Representation - John F. Sowa
74. Conceptual Structures: Information Processing in Mind and Machine - John F. Sowa
75. Inside Computer Understanding - Roger C. Schank & Christopher K. Riesbeck
76. Integrating Rules and Connectionism for Robust Commonsense Reasoning - Ron Sun
77. Probabilistic Logic Networks - Ben Goertzel, et al.
78. The Subtlety of Sameness: A Theory and Computer Model of Analogy Making - Robert M. French
79. Analogy Making as Perception - Melanie Mitchell
80. Real World Reasoning - Ben Goertzel, et al.
81. Artificial General Intelligence - Ben Goertzel & Cassio Pennachin
82. The Hidden Pattern: A Patternist Philosophy of Mind - Ben Goertzel
83. Engineering General Intelligence - Volume 2 - Ben Goertzel, Cassio Pennachin, & Nil Geisweiller
84. Computation and Cognition - Zenon W. Pylyshyn
85. Universal Artificial Intelligence - Marcus Hutter
86. The Cambridge Handbook of Computational Psychology - Ron Sun
87. Handbook of Neuroevolution Through Erlang - Gene I. Sher
88. Readings in Cognitive Science: A Perspective from Psychology and Artificial Intelligence - Allan Collins & Edward E. Smith -- this one contains a LOT of seminal papers in the field, including one of the earliest papers on Semantic Networks by Ross Quillian. I actually put this on the wrong part of the list, as I'm about halfway through reading this one, but just remembered it and didn't want to renumber everything. Heh.
Eh, that's a pretty good chunk of the stuff I've got lying around here, waiting to be read. There's a handful of others, but my fingers are getting numb.
- The field is now called AGI. It isn't mentioned in this article. Everyone seems to be ignoring the whole field of AGI (artificial general intelligence). Or maybe they truly are ignorant of it.
Anyway, suffice to say, AI and AGI didn't stop progressing, and Chomsky is no longer any sort of expert in those fields.
Even Norvig isn't up to speed on the most advanced approaches to AGI, but at least he enters the same room with people who are aware of the field. For example, he gave a talk at the recent Singularity Summit.
The Fifth Conference on Artificial General Intelligence is going to be in Oxford in December. http://agi-conference.org/2012/
Here is some information for people who are interested in pertinent ideas related to AGI.
http://www.amazon.com/How-Create-Mind-Thought-Revealed/dp/06...
>OpenCog is a diverse assemblage of cognitive algorithms, each embodying their own innovations — but what makes the overall architecture powerful is its careful adherence to the principle of cognitive synergy.
>The human brain consists of a host of subsystems carrying out particular tasks — some more specialized, some more general in nature — and connected together in a manner enabling them to (usually) synergetically assist rather than work against each other.
http://wiki.opencog.org/w/Probabilistic_Logic_Networks
> PLN is a novel conceptual, mathematical and computational approach to uncertain inference. In order to carry out effective reasoning in real-world circumstances, AI software must robustly handle uncertainty. However, previous approaches to uncertain inference do not have the breadth of scope required to provide an integrated treatment of the disparate forms of cognitively critical uncertainty as they manifest themselves within the various forms of pragmatic inference. Going beyond prior probabilistic approaches to uncertain inference, PLN is able to encompass within uncertain logic such ideas as induction, abduction, analogy, fuzziness and speculation, and reasoning about time and causality.
http://wiki.opencog.org/w/AtomSpace
Conceptually, knowledge in OpenCog is stored within large [weighted, labeled] hypergraphs with nodes and links linked together to represent knowledge. This is done on two levels: Information primitives are symbolized in individual or small sets of nodes/links, and patterns of relationships or activity found in [potentially] overlapping and nesting networks of nodes and links. (OCP tutorial log #2).
http://www.izhikevich.org/publications/large-scale_model_of_...
Large-Scale Model of Mammalian Thalamocortical Systems
> The understanding of the structural and dynamic complexity of mammalian brains is greatly facilitated by computer simulations. We present here a detailed large-scale thalamocortical model based on experimental measures in several mammalian species. The model spans three anatomical scales. (i) It is based on global (white-matter) thalamocortical anatomy obtained by means of diffusion tensor imaging (DTI) of a human brain. (ii) It includes multiple thalamic nuclei and six-layered cortical microcircuitry based on in vitro labeling and three-dimensional reconstruction of single neurons of cat visual cortex. (iii) It has 22 basic types of neurons with appropriate laminar distribution of their branching dendritic trees. The model simulates one million multicompartmental spiking neurons calibrated to reproduce known types of responses recorded in vitro in rats. It has almost half a billion synapses with appropriate receptor kinetics, short-term plasticity, and long-term dendritic spike-timing-dependent synaptic plasticity (dendritic STDP). The model exhibits behavioral regimes of normal brain activity that were not explicitly built-in but emerged spontaneously as the result of interactions among anatomical and dynamic processes. We describe spontaneous activity, sensitivity to changes in individual neurons, emergence of waves and rhythms, and functional connectivity on different scales.
http://www.sciencebytes.org/2011/05/03/blueprint-for-the-bra...
Essentials of General Intelligence: The direct path to AGI
http://www.adaptiveai.com/RealAI_chap_ver2c.htm
>General intelligence, as described above, demands a number of irreducible features and capabilities. In order to proactively accumulate knowledge from various (and/ or changing) environments, it requires:
>1. Senses to obtain features from ‘the world’ (virtual or actual),
>2. A coherent means for storing knowledge obtained this way, and
>3. Adaptive output/ actuation mechanisms (both static and dynamic).
>Such knowledge also needs to be automatically adjusted and updated on an ongoing basis; new knowledge must be appropriately related to existing data. Furthermore, perceived entities/ patterns must be stored in a way that facilitates concept formation and generalization. An effective way to represent complex feature relationships is through vector encoding (Churchland 1995).
>Any practical applications of AGI (and certainly any real-time uses) must inherently be able to process temporal data as patterns in time – not just as static patterns with a time dimension. Furthermore, AGIs must cope with data from different sense probes (e.g., visual, auditory, and data), and deal with such attributes as: noisy, scalar, unreliable, incomplete, multi-dimensional (both space/ time dimensional, and having a large number of simultaneous features), etc. Fuzzy pattern matching helps deal with pattern variability and noise.
>Another essential requirement of general intelligence is to cope with an overabundance of data. Reality presents massively more features and detail than is (contextually) relevant, or that can be usefully processed. This is why the system needs to have some control over what input data is selected for analysis and learning – both in terms of which data, and also the degree of detail. Senses (‘probes’) are needed not only for selection and focus, but also in order to ground concepts – to give them (reality-based) meaning.
http://en.wikipedia.org/wiki/Hierarchical_temporal_memory
> A typical HTM network is a tree-shaped hierarchy of levels that are composed of smaller elements called nodes or columns. A single level in the hierarchy is also called a region. Higher hierarchy levels often have fewer nodes and therefore less spacial resolvability. Higher hierarchy levels can reuse patterns learned at the lower levels by combining them to memorize more complex patterns.
> Each HTM node has the same basic functionality. In learning and inference modes; sensory data comes into the bottom level nodes. In generation mode; the bottom level nodes output the generated pattern of a given category. The top level usually has a single node that stores the most general categories (concepts) which determine, or are determined by, smaller concepts in the lower levels which are more restricted in time and space. When in inference mode; a node in each level interprets information coming in from its child nodes in the lower level as probabilities of the categories it has in memory.
>Each HTM region learns by identifying and memorizing spatial patterns - combinations of input bits that often occur at the same time. It then identifies temporal sequences of spatial patterns that are likely to occur one after another.
Related
Probability Theory
77 recommend · 193
Probabilistic reasoning in intelligent systems
7 recommend · 11
A mathematical introduction to logic
6 recommend · 11
2020–2026
Probabilistic Machine Learning
27 recommend · 40
Probabilistic Graphical Models
16 recommend · 42
Logic in Computer Science
5 recommend · 9
