Name: Neural network design
ISBN: 9780971732117

Neural network design

This book provides a clear and detailed coverage of fundamental neural network architectures and learning rules. In it, the authors emphasize a coherent presentation of the principal neural networks, methods for training them and their applications to practical problems.

What Hacker News says

NevermarkJan 2026

[EDIT] Updated original comment to include matrix dimensions.

If you want a serious text that goes through the relevant linear algebra and optimization mathematics in depth up front, Neural Network Design, 2nd edition is a good one. [Disclaimer, co-author]. We took great pains to walk through every conceptual and mathematical topic before we apply those concepts to machine learning. We use MATLAB a lot, which may or may not be helpful.

Another potential option is "Linear Algebra and Optimization for Machine Learning", which looks good and also starts out with linear algebra before machine learning. I haven't read it, but the first 2020 edition gets good reviews, and a second 2026 edition just came out, apparently with a fair amount of positive revision. Given the speed of change, that's nice to see.

thread

mindcrimeNov 2021

Maybe Clever Algorithms: Nature-Inspired Programming Recipes[1]. These techniques might not be useful to everybody, all the time, but they can be very handy in places.

Edit:

A couple of other suggestions.

Managing Gigabytes: Compressing and Indexing Documents and Images[2]

Mining of Massive Datasets[3]

Algorithm Design Manual[4]

Network Algorithmics[5]

Neural Network Design[6]

I think all of these fall into the category of "Won't be applicable to everyone, but can be good for those who need this kind of stuff."

[1]: https://github.com/clever-algorithms/CleverAlgorithms

[2]: https://people.eng.unimelb.edu.au/ammoffat/mg/

[3]: http://www.mmds.org/

[4]: https://www.amazon.com/Algorithm-Design-Manual-Steven-Skiena...

[5]: https://www.amazon.com/Network-Algorithmics-Interdisciplinar...

[6]: https://hagan.okstate.edu/NNDesign.pdf

thread

mindcrimeFeb 2016

Assuming that you consider Machine Learning to be either a subset of AI (as I do) or a sibling field, and want to learn aspects of ML, then consider Andrew Ng's Machine Learning course on Coursera. It's a great introduction and doesn't require a ton in terms of prerequisites. You'll see some multi-variable calculus and linear algebra, but he does the calculus derivations for you, and there's a pretty adequate review of the relevant parts of Linear Algebra.

In addition, if you don't already have a background in Calculus and Linear Algebra, then supplement the Ng course with the Khan Academy stuff on Calculus and Linear Algebra, or other courses you can pick up on Coursera or Edx or whatever.

If you get really interested in neural networks (which are all the rage these days) after the Ng class, there's a freely available book on Neural Network design that you could look at. It doesn't cover all the very latest techniques, but it would help you build the foundation of understanding.

http://hagan.okstate.edu/nnd.html

There's also a MOOC around the Learning From Data book that you could check out.

http://amlbook.com/

https://work.caltech.edu/telecourse.html

OTOH, if you're making a sharp distinction between "Classic AI" and "Machine Learning" and you really care mainly about the classical stuff, then you might want to start with the Berkeley CS188 class. You can take it through EdX (https://www.edx.org/course/artificial-intelligence-uc-berkel...) or just watch the videos and download the notes and stuff from http://ai.berkeley.edu/home.html

And if you just want to dive into reading some classic papers and stuff, check out:

http://publications.csail.mit.edu/ai/pubs_browse.shtml

and/or

http://ijcai.org/past_proceedings

Another good resource is

http://aitopics.org/

thread

mzielDec 2015

Not-technical

"The World's Toughest Endurance Challenges" - Very nicely illustrated

"Thing Explainer: Complicated Stuff in Simple Words" - Extremely enjoyable. People here certainly are aware of xkcd and it's type of humor.

Technical

"Functional programming in Scala" - good read for Scala beginners, especially after "Scala for the Impatient"

"Learning Spark" - the best book on Spark so far

"Assessing and Improving Prediction and Classification" - couple of interesting ideas for ML and Data Science

"Neural Network Design" - Quite basic, but I like the flow and introducing mathematical concepts just before they're needed.

thread

bgalbraithApr 2011

An excellent resource to get a more comprehensive review of ANNs is Neural Network Design (http://hagan.okstate.edu/nnd.html). Unfortunately, the book itself is out of print, but that link has some sample chapters and used copies are floating around. The book was written by the authors of the Neural Network Toolbox for MATLAB, so naturally all the examples and demos referenced are in MATLAB.

thread

plinkplonkJan 2010

I wrote a supplementary blog post to Brad's ( http://pindancing.blogspot.com/2010/01/learning-about-machin...) with the list of books I found useful (no amazon referral links if anyone is worried) with brief descriptions of each. I work in somewhat different domains and so have a different list of books.

I hope someone finds this useful.

For those who want a summary,

Proof Technique

(a)Velleman's "How to Prove It" (b)Gries and Schneider's "A Logical Approach to Discrete Math"

Math

(c) Calculus (best "lite" book - Calculus by Strang (free download), best "heavy" books - (d) Calculus by Spivak, (e) Principles of Mathematical Analysis a.k.a "Baby Rudin")

(f) Discrete Math (ALADM above + (g) a good book on Algorithms, Cormen will do - though working through it comprehensively is ... hard!

(h) Linear Algebra (First work through Strang's book, then (i) Axler's)

(j) Probability (see Bradford's very comprehensive recommendations) and

(k) Statistics (I would reccomend Devore and Peck for the total beginner but it is a damn expensive book. So hit a library or get a bootlegged copy to see if it suits you before buying a copy, see brad's list for advanced stuff.)

(l) Information Theory (MacKay's book is freely available online)

Basic AI

(m)AIMA 3d Edition (I prefer this to Mitchell)

Machine Learning

(n) "Pattern Recognition and Machine Learning" by Christopher Bishop,

(o)"Elements of Statistical Learning" (free download).

(p) Neural Network Design by Hagan Demuth and Kneale,

(q) Neural Networks, A Comprehensive Foundation (2nd edition) - By Haykin (there is a newer edition out but I don't know anything about that, this is the one I used)

(r) Neural Networks for Pattern Recognition ( Bishop).

At this point you are in good shape to read any papers in NN. My reccomendations - anything by Yann LeCun and Geoffrey Hinton. Both do amazing research.

Reinforcement Learning

(s) Reinforcement Learning - An Introduction by Barto and Sutton (follow up with "Recent Advances In reinforcement Learning" (PDF) which is an old paper but a GREAT introduction to Hierarchical Reinforcement learning)

(t) Neuro Dynamic Programming by Bertsekas

Computer Vision

(u) Introductory Techniques for 3-D Computer Vision, by Emanuele Trucco and Alessandro Verri.

(v) An Invitation to 3-D Vision by Y. Ma, S. Soatto, J. Kosecka, S.S. Sastry. (warning TOUGH!!)

Robotics.

(w) Probabilistic Graphical Models: Principles and Techniques (Adaptive Computation and Machine Learning) - not about robotics per se but useful to understand the next book

(x) Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) by Thrun, Burgard and Fox

PS: I own all these books (except AIMA 3 for which I only have pre pub pdfs) and if any HN folks in Bangalore want to browse before they buy anything (friggin expensive when you add amazon's postage) send me email.

PPS: on languages, I think Bradford is on the money with regards to reccomending functional languages. I would just say, also know C well. Saved my ass a few times.

thread