
It’s an older book, but it’s a deep dive into the math and intuition of neural networks. We used it for a grad-level applied math course in neural networks in 2013 (just as deep learning was emerging). It has tons of great visualizations and interesting exercises, is very readable, and is the best price (free).
I recommend reading a chapter or two to see how you like it, it has a bit of a different flavor compared to more modern deep learning books.
He uses a nice graphical approach that is easily understandable yet formal. It's been many years since I've read it to learn for an exam at university but I remember it was an enjoyable read and I wished I had more time to spend on the book.
[1] http://www.amazon.com/Neural-Networks-A-Systematic-Introduct...