Reinforcement Learning: An Introduction, by Richard S. Sutton and Andrew G. Barto
https://webdocs.cs.ualberta.ca/~sutton/book/ebook/the-book.h...
Algorithms for Reinforcement Learning, by Csaba Szepesvari
http://www.ualberta.ca/~szepesva/papers/RLAlgsInMDPs.pdf
edit: added other book