Uncovering the Dark Side of Big Data: How Mathematical Models are Worsening Inequality and Undermining Democracy

Ian Doyle: A compelling read, I couldn't put it down and read it in two sittings. O'Neill uses lots of clear real world examples to highlight her concerns. Highly recommend.

United Kingdom on Sep 25, 2023

JAA Tats: If you have only a rudimentary knowledge of statistics it will be tougher going but it clearly shows how big data WILL take us backwards when fed into tomorrow's AI products.. it circumvents human intervention and confuses causes and effects and correlations.

United States on Jul 02, 2023

Peer Sylvester: Die Autorin ist Mathematikerin und hat bis zur Finanzkrise für einen Hedge Fond gearbeitet. Sie weiß also worüber sie spricht. Sie verwendet die Bezeichnung "Weapons of Math Destructio" oder WMD für Algorythmen, die mit Hilfe von Daten Leute in grobe Kategorien einteilen und so immensen Schaden anrichten können - nicht zuletzt weil die genauen Algorythmen im Dunkeln bleiben. Weder weiß man genau, was sie messen, noch wissen die Ausgesiebten oft, dass sie aussortiert wurden, geschweige denn warum. Einige Leute profitieren natürlich, aber in der Regel sind die Opfer zahlreicher und dramatischer. Das ist nicht immer ganz neu, aber gut dargestellt und argumentiert.
Mein Hauptproblem ist tatsächlich, dass sich das alles praktisch ausschließlich auf die USA bezieht (im letzten Kapitel über Wahlwerbung lassen sich auch Rückschlüsse auf Europa schließen). Und so ist das Gefühl ein bisschen wie beim Ansehen einer Folge Last Week Tonight mit John Oliver: Interessant, erschreckend, aber man weiß, dass es einen nicht wirklich betrifft, da wir hier deutlich klarere Gesetze bezüglich Arbeitsschutz etc. haben.

Germany on Jul 25, 2022

Somjit Amrit: Weapons of Math Destruction is a book which is smartly titled in a way that it would pique anyone’s attention.
Cathy O’Neil ( PhD., Harvard) is an accomplished academic-mathematician , and it is a bit strange that a mathematician would so eloquently highlight the frightening reality of algorithms which could run amok and trample people of their choices, distort truths when they are expected to remove biases and reinforce the vicious feedback loop in an endless spool.
The examples problems are wide ranging from ranking of colleges and their impact of universities , faculties and most importantly students, to credit scores and the way they are defined and regulate, the flawed banking system governed by statistical fallacies.
The fact that Big Data processes codify the past and they do not invent future, but could masquerade inventing future. Defining the future would need moral imagination and that is something humans can provide. The fairness and morality of the Data model is often sacrificed in the alter of the God of Mamon , profits , efficiency rule the roost.
For a mathematician to courageously describe the ills of misusing mathematical principles is something...

India on Mar 01, 2022

Concobus: The underlying problem that the author addresses is an important one. Briefly, her concern is that opaque algorithms are blindly trusted and are broadly used to rank or classify us, which may have negative social consequences. But it seems like this topic is explored more adequately and even-handedly in the book "Hello World" by Hannah Fry.

My fundamental complaint with the book is that the author seems to be deeply confused about the meaning of the word "unfair", and "unfairness" underlies her major complaint with these algorithms... So this is not unimportant confusion. (She also makes a number of unjustified and unverified claims.)

For example, she complains that it is "unfair" for a recidivism model (used by some courts to help determine sentencing based on the probability that you will commit another crime) to be based on your answers to questions like "when did you have your first-ever encounter with the police?" because that question is based on your upbringing (which she claims is irrelevant and is banned in court), and because it potentially discriminates against poor people and racial minorities who are more likely to have had a run-in with police at an...

Canada on Oct 27, 2020

John P. Jones III: …and a book that delivers by delving into the numerous ways that mathematical algorithms impact our lives, sometimes very negatively and often without appeal. All too often the ones who formulate the algorithms view the injustices as just so much collateral damage, sacrificed on the altar of economic efficiency. At the end of the review, I’ll provide an example of my own.

Cathy O’Neil has a PhD in math from Harvard, taught at Barnard, decided to make three times the money by working as a “quant” on Wall Street, specifically for the hedge fund D. E. Shaw. Of the numerous wry observations she makes in the book, she compares working at D.E. Shaw to the structure of Al Qaeda. Information was tightly controlled in individual “cells.” No one (probably even the big boys) understood the entire structure which prevented someone “walking” to a rival. The financial meltdown of 2008, when suddenly the quants, and others, realized that a strawberry picker named Alberto Ramirez, making $14,000 a year, really couldn’t afford the $720,000 he financed in Rancho Grande, CA,, and therefore the “Triple A” rating on the bonds issued based on the mortgage was phony,...

United States on May 15, 2020

Eliza Krzanowska: This book is about the dark side of the Internet and computer technology, and it is not for the faint-hearted. Its author, Cathy O’Neil, is a veteran of data business. With a PhD in number theory from Harvard, she began her career as a university math professor at Baruch College, CUNY. After a few years she moved on to the D.E. Shaw hedge fund, where she witnessed firsthand the destruction that computers and Big Data had unleashed upon the world’s economy. She then held a string of positions as a data scientist for several data processing companies, before finally becoming a journalist. The story told in Weapons of Math Destruction is an account of her experience with Big Data, big money and computers.
Most of us know what Big Data is: a massive collection of information about anything we do from telephone conversations, private emails, chats, credit card records, Twitter activity, purchasing habits, ‘likes’, behavioral patterns, commuting and shopping habits, or professional contacts. The list is endless. We may see this vast collection of data about us as somewhat threatening to our privacy and somehow impinging on our ‘rights’. But we are uncertain how and on...

United Kingdom on Nov 21, 2017

David Zetland: I was excited to read this book as soon as I heard Cathy O'Neill, the author, interviewed on EconTalk.

O'Neill's hypothesis is that algorithms and machine learning can be useful, but they can also be destructive if they are (1) opaque, (2) scalable and (3) damaging. Put differently, an algorithm that determines whether you should be hired or fired, given a loan or able to retire on your savings is a WMD if it is opaque to users, "beneficiaries" and the public, has an impact on a large group of people at once, and "makes decisions" that have large social, financial or legal impacts. WMDs can leave thousands in jail or bankrupt pensions, often without warning or remorse.

As examples of non-WMDs, consider bitcoin/blockchain (the code and transactions are published), algorithms developed by a teacher (small scale), and Amazon's "recommended" lists, which are not damaging (because customers can decide to buy or not).

As examples of WMDs (many of which are explained in the book), consider Facebook's "newsfeed" algorithm, which is opaque (based on their internal advertising model), scaled (1.9 billion disenfranchised zombies) and damaging (echo-chamber,...

United States on Feb 08, 2017