
1) Foundations of Multithreaded, Parallel, and Distributed Programming by Gregory R. Andrews - This is one of my favourites even though old. You get to learn/compare the different paradigms in one single book.
2) Parallel Programming: Concepts and Practice by Bertil Schmidt et al. - This is a more recent book with good explanations/coverage including CUDA.
3) The Art of Multiprocessor Programming by Maurice Herlihy et al. - Well known classic though with a main focus on shared-memory architectures.
Along with the above you also need more detailed language/library specific implementation books;
a) C++ Concurrency in Action by Anthony Williams.
b) Programming with POSIX Threads by David Butenhof.
c) UNIX Systems Programming: Communication, Concurrency and Threads by Kay Robbins and Steve Robbins.
Finally; for studying concurrency at the OS level where software meets hardware, i have found nothing better than;
i) Unix Systems for Modern Architectures: Symmetric Multiprocessing and Caching for Kernel Programmers by Curt Schimmel.
Stroustrup's _The C++ Programming Language_, twice in a row, because I didn't want to be left out for not having taken Computer Science.
Butelhof's _Programming with Posix Threads_, because another book referenced its explanation of why Win32 threading had a bug because it could not, at the time, signal and wait on a mutex in an atomic way. There was something deep to find in those rowboats and bucket stories.
GoF's _Design Patterns: Elements of Reusable Object-Oriented Software_, because when I complained to a mentor that I was struggling to organize a dozen scientist programmers, he handed it to me, laughing.
John Robbins, _Debugging Applications_, first edition, because it was the first time for me that code didn't have the disclaimer, "of course you should insert error-handling code here." Basic x86 assembly in one chapter!