This is a mandatory book during a course on democracy, that I actually read approximately three years ago and thus never reviewed (this website didn’t exist then), so I thought it was time for a proper review.
Cathy O’Neil is a computer scientist and mathematician, who left the academic life for the financial industry in the early 2000’s, working with computers, for companies making lots of money. There she discovered what is now called Big Data and later became troubled by the purposes and intents of algorithms. After realising the even more troublesome side effects on society, she thus wrote this book, with the secondary title How Big Data increases inequality and threatens democracy.
Through ten chapters, O’Neil takes the reader through what a data model is and how it can affect people in real life, such as the effects of university ranking models and the possibility of getting an adequate education, evaluations of teachers, online advertising, criminal injustice and justice and getting insurance, among other things. How come a data model deems a teacher unsuccesful or a job applicant unfit? Is the model correctly constructed or does it inherit its lack of perspective, and mathematical incoherence, from the creator? Data models with destructive effects on people’s lives are what she calls weapons of math destruction, WMD.
In large, I agree with her and appreciate her arguments and conclusions. Negative feedback loops can infer that black men are more prone to commit crimes because the police has indicated black neighbourhoods as more exposed to petty crimes, sending police patrols to these neighbourhoods rather than white communities with more hidden crimes not marked on a map. This kind of feedback loop creates or maintains inequalities, which have destructive consequences for society.
Sometimes, though, she contradicts herself. The extremes in statistical data are more likely to be pointed out and punished, she writes, although she also writes (rightly) that black men become an average in criminal statistics, simply being the median and mean, rather than the extreme. In a black community with more black men than white men, black men are the average. In a sense, being an average person, financially for instance, in a big data model can be very punishing, while being an extreme in form of extremely rich is better.
On average (huh!) though, this book is still highly relevant, even though we’ve moved into the “age of AI”. AI-programmes rely on the same errors and statistical inferences as the programmes O’Neil discusses. Personally, I think the book is good for social scientists. She presents statistical models used by scientists and businesses, and how easily they can turn into stupid models discriminating people. It’s nice to get a mathematicians perspective and logical thinking.
Conclusion: It still stands. Brief as that.