Weapons of Math Destruction
A former Wall Street quant sounds an alarm on the mathematical models that pervade modern life — and threaten to rip apart our social fabric We live in the age of the algorithm. Increasingly, the decisions that affect our lives—where we go to school, whether we get a car loan, how much we pay for health insurance—are being made not by humans, but by mathematical models. In theory, this should lead to greater fairness: Everyone is judged according to the same rules, and bias is eliminated. But as Cathy O’Neil reveals in this urgent and necessary book, the opposite is true. The models being used today are opaque, unregulated, and uncontestable, even when they’re wrong. Most troubling, they reinforce discrimination: If a poor student can’t get a loan because a lending model deems him too risky (by virtue of his zip code), he’s then cut off from the kind of education that could pull him out of poverty, and a vicious spiral ensues. Models are propping up the lucky and punishing the downtrodden, creating a “toxic cocktail for democracy.” Welcome to the dark side of Big Data. Tracing the arc of a person’s life, O’Neil exposes the black box models that shape our future, both as individuals and as a society. These “weapons of math destruction” score teachers and students, sort résumés, grant (or deny) loans, evaluate workers, target voters, set parole, and monitor our health.
Stats for nerds
68recommend
132mentions
112people
2016–2026years
57% recommend41% neutral2% critical
Mentions per year
What Hacker News says
- This has been the past and present for a long at this point. "Sorry there's nothing we can do, the system won't let me."
Also see Weapons of Math Destruction [0].
[0]: https://www.penguinrandomhouse.com/books/241363/weapons-of-m...
- ”AI” started taking judges jobs at least over ten years ago. See tools like COMPAS.
Read Weapons of Math Destruction.
- Related discussion (517 pts):
https://news.ycombinator.com/item?id=41891694
https://aworkinglibrary.com/reading/unaccountability-machine
(A very short overview of Dan Davies' book, quoted in TFA, that came up with the term)
EDIT: complementing book mentioned in that thread
Cathy O'Neil's "Weapons of Math Destruction" (2016, Penguin Random House) is a good companion to this concept, covering the "accountability sink" from the other side of those constructing or overseeing systems.
Cathy argues that the use of algorithm in some contexts permits a new scale of harmful and unaccountable systems that ought to be reigned in.
- Oh yeah definitely. There is a very good book about this actually, Weapons of Math Destruction, where the author Cathy O'Neil talks about exactly this - how biases are laundered through algorithms.
Simply put: - Police Enforcment unfairly policed black neighborhoods, overrepresenting them in historical records - Machine learning models were run on said historical records - Police Enforcement used said models to justify their policing, saying it was algorithmically determined.
I may be blanking on a few details, so if anybody can add anything please do, it has been a while since I've read it. These laundering algorithms will cause untold damage and should've been banned immediately.
- Cathy O'Neil's "Weapons of Math Destruction" (2016, Penguin Random House) is a good companion to this concept, covering the "accountability sink" from the other side of those constructing or overseeing systems.
Cathy argues that the use of algorithm in some contexts permits a new scale of harmful and unaccountable systems that ought to be reigned in.
https://www.penguinrandomhouse.com/books/241363/weapons-of-m...
- "Weapons of Math Destruction" covered some of these problems, good book.
Essentially automating existing biases, and in a way its even more insidious because companies can point to the black box and say "how could it be biased, its a computer!".
Then you have companies trying to overcorrect the other way like Google with their AI images fiasco.
Related
Mindstorms
89 recommend · 201
The art of computer programming
336 recommend · 1,799
Kill It with Fire
10 recommend · 16
Metamagical Themas
45 recommend · 98
Code Complete
393 recommend · 894
Artificial Intelligence
60 recommend · 153
Code
243 recommend · 385
