Information Theory

What Hacker News says

1. sargstuffJun 2024
   A Mathematical Theory of Communication (1948) by C. E. Shannon [0][1]

   Information theory: a Tutorial Introduction (2nd Edition) by James V Stone bit more modern introduction/hands-on approach.

   -----

   [0] : paper : https://people.math.harvard.edu/~ctm/home/text/others/shanno...

   current book version of paper is 16th addition.

   [1] : Why [0] matters : https://www.quantamagazine.org/how-claude-shannons-informati...

   Show morethread
2. activatedgeekJun 2023
   Another excellent book that I recently read was "Information Theory: A Tutorial Introduction" by James V. Stone [1]. This book is a fairly light read and builds up everything from scratch which was very nice.

   Shannon's original paper [2] is often touted to be very parseable, but it is always enlightening to read the same basics many times from multiple sources, especially something as foundational as source coding (and broader information theory) in a more modern language.

   I also finally discovered the meaning of "differential" in differential entropy [2], the continuous counterpart of discrete probability masses, something that I always swept under the rug.

   [1]: https://www.librarything.com/work/16165301/book/242295611 [2]: https://ieeexplore.ieee.org/document/6773024 [3]: https://sanyamkapoor.com/kb/differential-entropy

   Show morethread
3. teleforceJun 2021
   The author of the paper has a book with the same title:

   https://www.amazon.com/Information-Theory-Introduction-James...

   thread