Inductive Logic Programming

What Hacker News says

1. YeGoblynQueenneFeb 2022
   My pleasure!

   - First, some more recent work, mostly overviews.

   1. The following is the most recent overview of the field I'm aware of:

   Inductive logic programming at 30 (Cropper et al, 2020)

   https://www.doc.ic.ac.uk/~shm/Papers/ilp30.pdf

   2. And a slightly shorter version of the same paper that summarises new trends:

   Turning 30: New Ideas in Inductive Logic Programming (Cropper et al, 2020)

   https://www.ijcai.org/Proceedings/2020/0673.pdf

   3. Here's a short introduction to the relatively new ILP direction of learning Answer Set Programming:

   Inductive Logic Programming in Answer Set Programming (Corapi et al, 2011)

   https://link.springer.com/chapter/10.1007/978-3-642-31951-8_...

   4. This is an overview of Meta-Interpretive Learning (MIL), a new approach to ILP that overcomes many difficulties of earlier approaches (Full disclosure: my own work is on MIL, though not the article linked):

   Meta-Interpretive Learning: achievements and challenges (Stephen Muggleton, 2017)

   https://www.doc.ic.ac.uk/~shm/Papers/rulemlabs.pdf

   5. And this is a (short vesion) of a paper on δILP, a neural-net based ILP system:

   Learning Explanatory Rules from Noisy Data (Evans and Grefenstette, 2018)

   https://www.ijcai.org/Proceedings/2018/0792.pdf

   - Next, some earlier work that is still relevant:

   6. This is the inaugural paper of the field, that first named it (a little heavy reading though):

   Inductive Logic Programming (Stephen Muggleton, 1990)

   https://www.doc.ic.ac.uk/~shm/Papers/ilp.pdf

   7. Here's an early paper on predicate invention, an important technique in ILP (only recently fully realised via MIL):

   Predicate Invention in ILP - an Overview (Irene Stahl, 1993)

   https://link.springer.com/chapter/10.1007%2F3-540-56602-3_14...

   8. And an early overview of learning recursion (and performing predicate invention) that also lists several early ILP systems:

   Inductive synthesis of recursive logic programs:achievements and prospects (Flener and Yilmaz, 1999)

   https://core.ac.uk/download/pdf/82810434.pdf

   That should be enough to get you started. I recommend reading in the order I linked to the various articles. I tried to give links to documents that I know can be read for free.

   Unfortunately most of the material on ILP is either in scholarly articles, or, where there are textbooks, they tend to be older. That sounds bad, but there has been much new work recently with several new approaches.

   Let me know if you're looking for more specific information. See my signature for contact details- I'm happy to answer emails about ILP :)

   Show morethread
2. YeGoblynQueenneJan 2019
   Lise Getoor has been doing the Lord's* work for a while now, presenting statistical relational learning work at the major machine learning conferences. She had the keynote speech at NIPS last year, supporting the use of structural bias in statistical learning:

   https://www.youtube.com/watch?v=t4k5LKCpboc

   Although she had to riff through a couple of very unfortunate slides quickly (identifying political affiliation of social network users is a classic task in relational learning, but the keynote came hot in the heels of the Cambridge Analytica scandal and her slides went down like a led baloon) it was a great keynote and one I was happy to see.

   There's more work on combining relational, logic-based machine learning and statistical machine learning by Luc de Raedt, Kristian Kerstig and Stephen Muggleton, among many others in the Inductive Logic Programming community. There's some good pointers on this website for Lise Getoor's introductory book to the field:

   https://www.cs.umd.edu/srl-book/

   ___________

   * Lord Sauron's.

   Show morethread
3. theszFeb 2026
   You said AI: https://github.com/stassa/louise

     Louise (Patsantzis & Muggleton 2021) is a machine learning system that learns Prolog programs.
   
     Louise is a Meta-Interpretive Learning (MIL) system. MIL (Muggleton et al. 2014), (Muggleton et al. 2015), is a new setting for Inductive Logic Programming (ILP) (Muggleton, 1991). ILP is a form of weakly-supervised machine learning of logic programs from examples of program behaviour (meaning examples of the inputs and outputs of the programs to be learned). Unlike conventional, statistical machine learning algorithms, ILP approaches do not need to see examples of programs to learn new programs and instead rely on background knowledge, a library of pre-existing logic programs that they reuse to compose new programs.
   

   This is what was done by Douglas Lenat from late 1970-s on [1]. He did his work using Lisp, this thing does something close using Prolog.

   [1] https://en.wikipedia.org/wiki/Eurisko

   Show morethread
4. YeGoblynQueenneDec 2024
   "ILP" (as in Inductive Logic Programming not Integer Linear Programming) was first named in 1991 in a paper by Stephen Muggleton ("Inductive Logic Programming and Progol). The paper properly launched the field and generated a great deal of excitement at the time.

   There were precursors. At least Ehud Shapiro's doctoral thesis ("Automated Debugging") in the 1980's and Gordon Plotkin's doctoral thesis in the 1970's ("Automated Methods of Inductive Inference"). Sorry for not giving the exact years off the top of my head but I think it was 1983 and 1976, respectively.

   The point you are making is very right however because modern machine learning as a field started in the 1980's with the fall of expert systems, in fact it basically started as an effort to overcome one of the major limitations of expert systems, the so-called "knowledge acquisition bottleneck", which is to say, the difficulty of creating and maintaining huge databases of expert knowledge (in the form of production rules).

   In any case the seminal textbook in the field for the first 20 years, Tom Mitchell's Machine Learning came out in 1997 (https://www.cse.iitb.ac.in/~cs725/notes/slides/tom_mitchell/...) and includes probabilistic, neural-net based and symbolic, logic-based approaches. So not only machines could "learn" way before 2003 but they could also learn in many different ways than what Ilya Sutskever means.

   We can go further back, to Donald Michie's 1961 MENACE (the first Reinforcement Learning system, implemented on a computer made of matchboxes with coloured beads used to encode state) and Arthur Samuel's 1959 checkers player (a paper on which gave the name to the field of machine learning).

   Lots of learning all over the place long, long before 2003.

   Show morethread
5. YeGoblynQueenneMar 2016
   The data is in that format because Quinlan pre-processed it for use in FOIL, which expects first-order logic terms as input, but I don't see what that's a problem? Turning natural language to word embeddings is a very similar process: you might as well argue that the fact that a statistical system does well with word vectors is not that interesting (it is, it's a brilliant idea).

   >> it is not clear how we could extend a logic-based system to handle the messiness of natural language.

   There's a lot of recent stuff in Muggleton's Latest Advances in Inductive Logic Programming and a ton of work from others, before and after that.

   There's no problem in handling "messiness" with logic. Like I note elsewhere, your computer is a logic machine and it handles messiness just fine (no offense meant).

   Show morethread