An introduction to information theory
Behind the familiar surfaces of the telephone, radio, and television lies a sophisticated and intriguing body of knowledge known as information theory. This is the theory that has permeated the rapid development of all sorts of communication, from color television to the clear transmission of photographs from the vicinity of Jupiter. Even more revolutionary progress is expected in the future. To give a solid introduction to this burgeoning field, J. R. Pierce has revised his well-received 1961 study of information theory for an up-to-date second edition. Beginning with the origins of the field, Dr. Pierce follows the brilliant formulations of Claude Shannon and describes such aspects of the subject as encoding and binary digits, entropy. language and meaning, efficient encoding, and the noisy channel. He then goes beyond the strict confines of the topic to explore the ways in which information theory relates to physics, cybernetics, psychology, and art. Mathematical formulas are introduced at the appropriate points for the benefit of serious students. A glossary of terms and an appendix on mathematical notation are provided to help the less mathematically sophisticated. J. R. Pierce worked for many years at the Bell Telephone Laboratories, where he became Director of Research in Communications Principles. He is currently affiliated with the engineering department of the California Institute of Technology. While his background is impeccable, Dr.
Stats for nerds
15recommend
19mentions
17people
2014–2025years
79% recommend21% neutral
Mentions per year
What Hacker News says
- 1. Differential Privacy (2025) by Simson Garfinkel
A readable but thorough introduction to differential privacy.
2. Tales of the Weird: An Uncanny Introduction (2023)
A collection of short stories from the British Library to introduce the many flavors of weird fiction.
3. An Introduction to Information Theory: Symbols, Signals and Noise (1980) by John R. Pierce
A fun survey of information theory with great anecdotes about Bell Labs in mid-century.
See my full list here: https://bcmullins.github.io/interesting-books-2025/
- Beginners might want to start with the following works;
1) An Introduction to Information Theory, Symbols, Signals and Noise by John R. Pierce - A classic general text to understand the concepts and build intuition. Other books by the same author are also excellent.
2) Information Theory: A Tutorial Introduction by James V. Stone - A good general introduction. The author has similar tutorial books on other subjects which are also good.
3) A Student's Guide to Coding and Information Theory by Stefan Moser and Po-ning chen - A concise guide. The other books in the "student's guide" series from cambridge are also good.
- What's the story angle/line/tanent/"e"/"pi"/etc.? (linear algegra, Lewis Carol math, Claud, Shannon, etc. ) -- Physical reminder/context for why 'booked ticket to get punched'?
Target audience? -- is journey a story in which 'instructions/learning' are actions used to get the reader to partake in story? aka humanities novel as wrapper to 70's wall of documentation volumes.
----
> Should I just focus on creating the content and making sure it’s useful/valuable and treat the book as just another method of presentation?
Is this for a 'Go Forth' TeD book of the month talk?
Any further clarification on type theory / category background? (cs, art, architecture, fonts, LaTex, ascii, utf, ...)
"The Power of Making Thinking Visible: Practices to Engage and Empower All Learners" by Ron Ritchhart (Author), Mark Church (Author)
"The Elements of Style" by William Strunk Jr. (Author), E. B. White (Author)
"Thinking with Type: Critical Guide for Designers, Writers, Editors, & Students Paperback" by Ellen Lupton
"Unleash the Power of Storytelling: Win Hearts, Change Minds, Get Results" by Rob Biesenbach
----
> ... book just a nice to have thing? Wide variety of catecorical/topi types.
Physicall book can be less expensive / less hassel than mangaging drmo on elecronic tablet/format only to have reference auto-erased. Dont have to worry about battery leaking/exploding. Thankfully, electronic formats allow for circumventing the imaginary wierd mechanics of trying to carry around wall's worth of pearls within multiple physical book encylopedia volumes without spending one's life working out at gym to be able to effortlessly carry/handle the encylopedia volume(s).
Categorical/Topi 'sea' type stuff:
"Alice in Wonderland" by Lewis Carol;
"Alice in Puzzle-Land: A Carrollian Tale for Children Under Eighty" by Raymond M. Smullyan
"To Mock a Mockingbird" by Raymond Smullyan;
"The New Turing Omnibus: Sixty-Six Excursions in Computer Science" by A.K. Dewdney
"Once Upon an Algorithm: How Stories Explain Computing" by Martin Erwig
"The Annotated Gödel: A Reader's Guide to his Classic Paper on Logic and Incompleteness Paperback" by Hal Prince (Author)
"Gödel, Escher, Bach: An Eternal Golden Braid" by Douglas R Hofstadter
"Information Theory: A Tutorial Introduction (2nd Edition)" by James V Stone
"An Introduction to Information Theory: Symbols, Signals and Noise" Subsequent Edition by John R. Pierce
"Coding the Matrix: Linear Algebra through Applications to Computer Science" by Philip N. Klein
"Land of Lisp: Learn to Program in Lisp, One Game at a Time!" by Conrad Barski M.D.
"Build Your Own Lisp" by Mr Daniel Holden
"LISP IS MY FAVORITE LANGUAGE IN LIFE FOUR MONTHS TO LEARN: Funny beginner's noteook to Learn LISP Programming Step-by-Step(PROGRAMMING LANGUAGE)"
----
- My introduction to Shannon and information theory through this book [1], which talks briefly about language entropy, is what set me on the path to my career.
[1]. An Introduction to Information Theory, Symbols, Signals & Noise By John Robinson Pierce · 1980 https://www.google.com/books/edition/An_Introduction_to_Info...
- There's an awesome introductory book on information theory which in one chapter resolves Maxwell's demon. The argument is that the energy needed to reliably send a single bit of information (which is needed for making the detector talk to the trap door) depends on the temperature of the fluid (because you have to overcome background noise), and it turns out the minimum energy needed for this communication is exactly equal to the amount that can be recovered from separating out a molecule.
If my understanding is correct (admittedly a big if), it seems the same problem applies to gambling. The energy it takes to communicate whether a gamble has paid off is at least as big as what can be recovered from the gamble.
The book is "An Introduction to Information Theory" by John R Pierce, and it's the most I've ever learned from a $10 paperback.
- _Introduction to Information Theory: Symbols, Signals and Noise (Dover Books on Mathematics)_ by Pierce is another such book.
It is of course only an introduction to the field, but it had an immense impact on how I saw information (hence, the universe) after I read. Primarily because it showed to me the many faces of Information Theory- music, psychology, geometry, language, cybernetics, etc.
Related
Symbols, signals, and noise
12 recommend · 16
Information Theory
2 recommend · 3
Introduction to the theory of computation
66 recommend · 96
Data structure techniques
2 recommend · 3
Advanced data structures
3 recommend · 4
The mathematical theory of communication
47 recommend · 118
