The Asimov Institute is a research group that published the Neural Network Zoo, a chart of neural network architectures.
HN commenters link the Neural Network Zoo as a nice visualization and helpful chart of network types with brief explanations of each.
>> It seems to me that if the programmer understands the algorithm, then the
algorithm is limited by the programmer!
Well, a learning algorithm is always going to be limited by something. In AI
literature any such something is grouped under "inductive bias" and there is
no machine learning algorithm that doesn't incorporate some kind of inductive
bias. Bayesian learners have their priors, distance learners have their
disance functions, Support Vector Machines in particular have their kernels,
and of course neural networks have their intricate architectures painstakingly
hand-engineered and fine-tuned to a particular domain, or even a specific
dataset [1]. Indeed it is probably impossible to have learning without
inductive bias [2] [3].
The opinion in the short piece above is representative of a current trend in
machine learning, of taking "the human out of the loop", which loosely
translates in trying to learn everything end-to-end, only from examples, while
pretending that no attempt is made at any point to guide the learner to find
a consistent hypothesis that explains the examples.
Unfotunately, in practicce, this ideal remains a fantasy. All the progress
achieved with deep learning in the last few years would not have been possible
without the discovery (by chance or concerted effort) of good biases that are
conducive to learning in specific domains, e.g. convolutional layers for image
recognition, or Long-Short Term Memory cells for sequence learning, etc.
And where…
AGI hasn't been delayed. It's been overpromised. And the proponents will continue to overpromise right up till when it arrives. We need a breakthrough in AI research of at least two magnitudes before even tackling the AGI problem.
One is we need to understand the mechanisms of neural networks at least an order of magnitude better than we currently do. Best article on the subject that concisely describes NNs is The NN zoo[0]. I have no idea what needs to happen for us to collectively understand the mechanisms better. We need an Einstein moment I think where some researcher has a happy thought and a decade where the rest of us catch up (at minimum).
The other is we need to understand the problem we are solving. At it's core it seems very easy to define intelligence but you very soon realize that it's an endless Matryoska with no discernible root. It's very easy to define intelligence in terms of solving elementary cogitation tasks and this is what we are doing writ-large. This leads to a culture of solving Weak AI tasks, hoping that the aggregation of the solutions will lead to AGI. Well, it's only going to do so if we are engineering solutions that will solve the larger problem and if we don't understand it well enough it's going to be a really random occurrence finding it.
[0] https://www.asimovinstitute.org/neural-network-zoo/
Superficially examining the Clos Network article, if Factorio is amenable to creating such networks, then it seems like it would be possible to create some of the simpler neural network designs, such as shown here: [1] Maybe having something like a weighting of the resource density arriving at a certain location to change the output? From the mechanics I can see here [2] it looks like Mergers / Un-Mergers and three belt speeds could probably do density weighted decision making.
[1] https://www.asimovinstitute.org/wp-content/uploads/2019/04/NeuralNetworkZoo20042019.png
[2] https://wiki.factorio.com/Belt_transport_system#Splitters
> Why should experts bias NN architectural parameters if there are "Parameter-free" neural network graphical models?
The Asimov Institute > The Neural Network Zoo: https://www.asimovinstitute.org/neural-network-zoo
"A mostly complete chart of neural networks" (2019) includes Hopfield nets! https://www.asimovinstitute.org/wp-content/uploads/2019/04/NeuralNetworkZo19High.png
Category:Neural network architectures: https://en.wikipedia.org/wiki/Category:Neural_network_architectures
Types of artificial neural networks: https://en.wikipedia.org/wiki/Types_of_artificial_neural_networks
This is amazing, such a nice presentation. It reminds me of the Neural Network Zoo [1], which was also a nice visualization of different architectures.
[1] https://www.asimovinstitute.org/neural-network-zoo/
You might find "The neural network zoo" helpful; it's a chart showing the different types of neural networks, along with a brief discussion of each type: https://www.asimovinstitute.org/neural-network-zoo/