Name: Programming Massively Parallel Processors
ISBN: 9780323912310

Programming Massively Parallel Processors

Programming Massively Parallel Processors: A Hands-on Approach shows both student and professional alike the basic concepts of parallel programming and GPU architecture. Various techniques for constructing parallel programs are explored in detail. Case studies demonstrate the development process, which begins with computational thinking and ends with effective and efficient parallel programs. Topics of performance, floating-point format, parallel patterns, and dynamic parallelism are covered in depth. For this new edition, the authors are updating their coverage of CUDA, including the concept of unified memory, and expanding content in areas such as threads, while still retaining its concise, intuitive, practical approach based on years of road-testing in the authors' own parallel computing courses. Teaches computational thinking and problem-solving techniques that facilitate high-performance parallel computing Updated to utilize CUDA version 10.0, NVIDIA's software development tool created specifically for massively parallel environments Features new content on unified memory, as well as expanded content on threads, streams, warp divergence, and OpenMP Includes updated and new case studies

What Hacker News says

imjonseJun 2025

These should keep you busy for months:

https://www.gpumode.com/ resources and discord community Book: Programming massively parallel processors nvidia cuda docs are very comprehensive too https://github.com/srush/GPU-Puzzles

thread

signa11Feb 2025

this book:

    Programming Massively Parallel Processors by Wen-mei W. Hwu , David B. Kirk , Izzat El Hajj

seems to be tailor mode for folks transitioning from cpu -> gpu arch.

thread

zackangeloFeb 2025

My go to is Programming Massively Parallel Processors by Wen-Mei Hwu, excellent really approachable introduction. [0]

[0] https://a.co/d/9fmbZqg

thread

mandliyaJun 2024

CUDA programming (writing CUDA kernels) might be a good direction too.

GPU race is getting really hot and there is a lot of work being done to squeeze every ounce of performance especially for LLM training and inference.

One resource I would recommend is “Programming massively parallel processors” [1]

I am also learning it as my hobby project and uploading my notes here [2]

[1] https://shop.elsevier.com/books/programming-massively-parall...

[2] https://github.com/mandliya/PMPP_notes

thread

cataPhilMar 2024

Thanks! I highly recommend the book Programming Massively Parallel Processors

thread

boberoniJan 2024

If you like textbooks, I would recommend "Programming Massively Parallel Processors" by Hwu, Kirk, and Hajj. [1] The most recent edition was published in 2022.

If you like lecture videos, I would recommend Hajj's YouTube playlist of 2021 lectures [2]. He works through a subset of the textbook.

This will give you a good foundation of GPU hardware architecture and CUDA programming. The knowledge is somewhat transferable to other areas of high-performance computing.

[1] https://www.amazon.com/Programming-Massively-Parallel-Proces...

[2] https://www.youtube.com/playlist?list=PLRRuQYjFhpmubuwx-w8X9...

thread