
Dave Eberly's 3D Game Engine Design was the first professional guide to the essential concepts and algorithms of real-time 3D engines and quickly became a classic of game development. Dave's new book 3D Game Engine Architecture continues the tradition with a comprehensive look at the software engineering and programming of 3D engines... This book is a complete guide to the engineering process, starting with a walk-through of the graphics pipeline showing how to construct the core elements of 3D systems, including data structures, the math system, and the object system. Dave explains how to manage data with scene graphs, how to build rendering and camera systems, and how to handle level of detail, terrain, and animation... Advanced rendering effects such as vertex and pixel shaders are also covered as well as collision detection and physics systems. The book concludes with a discussion of application design, development tools, and coding standards for the source code of the new version of the Wild Magic engine included on the CD-ROM. Wild Magic is a commercial-quality game engine used by many companies and is a unique resource for the game development community.Note: CD-ROM/DVD and other supplementary materials are not included.
https://www.amazon.com/Game-Engine-Architecture-Engineering-...
Not only is this a fantastic introduction to the math and the data structures involved in computer graphics, the book also walks you through the construction of an in-software renderer!
An older book like Eberly's 3D Game Engine Design was more a loosely knit compendium of theory and algorithms. Eberly's more recent 3D Game Engine Architecture is closer to Gregory's in intent. Unfortunately, it's more a reflection of the idiosyncrasies of the author and his personal code base than an investigation of game engines as developed and used in the industry.
I've also heard good things about Mike McShaffry's Game Coding Complete. McShaffry is a game industry veteran, so he knows what he's talking about. However, it seems to be targeted at rank beginners, so a lot of the material might not be very useful to you.
Below that tier, of course, you have an endless array of how-to books written by incompetents. The worst is the "here's my shitty homebrewed codebase" variety.
For rendering in games, you'll need a separate book. Real-Time Rendering is the canonical reference. The first and second editions were my favorites. The third edition added a lot of information on recent techniques but at the expense of reading too much like an annotated bibliography, skating from topic to topic while rarely going into enough detail to be useful and often listing several alternative solutions to a given problem without comparing their relative merits and trade-offs. The first editions flowed more like textbooks and were thus better suited as introductions. Some of that might be hindsight bias on my part; I'm curious what others think who first learned the subject from the third edition.
I also want to give a shout-out to Christer Ericson's Real-Time Collision Detection book. It's full of clear explanations and practical, hard-earned knowledge that you can't get anywhere else. Plus, the accompanying code is robust and well written, a far cry from the usual crap that either straight-up doesn't work or is inextricably entangled with a massive code base (I'm looking at you, Eberly!).