The Glass Cage: Where Automation Is Taking Us review – on course for disaster

If you want to understand the human consequences of automation, says Nicholas Carr, look up. Airlines and plane manufacturers have been at the forefront of the technology that has gradually replaced human judgment with a set of computer algorithms.

Where planes have led, he claims, all else is now following. Computers can now parse legal documents and develop trial strategies. They can design buildings, trade shares and diagnose diseases. The process that began with Japanese robots taking over the work of skilled labourers is now extending into all areas of working life. Work once reserved for highly qualified professionals, that pays well, and that is rewarding to do, the work that the sort of people who are likely to read this book do, is simply the next class of jobs to fall to the inexorable logic of the machine.

In 2010 Carr wrote The Shallows, a timely and influential book about what the internet is doing to our brains. Here he extends his focus to consider what automation is doing to humans. As computers have taken over the cockpit, he notes, pilots’ salaries and job satisfaction have fallen, and while commercial flying has become safer, when something goes wrong, as it can and will, the humans sitting at the controls are less capable of taking over.

He cites the example of the May 2009 Airbus A330 accident in which an Air France plane flying from Rio to Paris plunged into the ocean with the loss of all 228 on board. The autopilot, it turned out, had failed, and faced with the task of taking over, the pilots suffered what the investigators described as “a total loss of cognitive control of the situation”. They had, in a nutshell, lost the ability to fly the plane.

The “de-skilling” that occurs when pilots no longer fly planes is happening across the board. The first few chapters of The Glass Cage have a familiar ring to them. If you haven’t come across Mihaly Csikszentmihalyi’s studies on the paradox of work – that whatever we believe, we tend to be happier and more fulfilled when we’re at work, and more anxious when we’re not – then you haven’t read enough popular psychology. But gradually, Carr builds up a subtler, more provocative and more interesting case than first appears. We all know that GPS has made us lazy at reading maps. And even Carr’s story of the Inuit ignoring the handed-down wisdom of millennia to defer to their screens doesn’t feel surprising. But Carr ponders what effect our loss of understanding about where we are in the world has upon us. How finding our way in the world is actually a fundamental part of our humanity. Navigation, it appears, may be implicated in memory. One of the first functions Alzheimer’s sufferers lose is the ability to know where they are. Carr cites a research psychiatrist and memory expert, Veronique Bohbot, who fears that as we use our navigational skills less and less, our hippocampuses will begin to atrophy, leading to ever earlier dementia in future generations.

It’s just one of the risks, Carr suggests, of becoming “a ward of our phones”: we are disembodying ourselves, removing ourselves from the world, living via screens and interfaces, and losing touch with many of the things that make us human. Even Google’s search engine used to require thought. You had to compose a question; not now. You simply type a keystroke or two. And even that may soon no longer be necessary, according to Ray Kurzweil, the futurist who now works for Google. He believes “the majority of search queries will be answered without actually asking”.

Carr isn’t anti-technology. He’s just wary of a world in which machines take centre stage. Where even human relations are governed by algorithms, as in Facebook’s “frictionless sharing”, where “friendship becomes more like the attachment between consumers and products”. Easy come, easy go. An ideological belief that technology is fundamentally “benevolent, self-healing, autonomous” is dangerously seductive; it allows us to feel optimistic about the future, he suggests, while relieving us of the responsibility for it. And it suits “our new plutocrats”, giving them “a heroic narrative in which they play starring roles”.

Carr points out: “It strains credulity to imagine today’s tech moguls with their libertarian leanings and impatience with government agreeing to the kind of vast wealth redistribution scheme that would be necessary to fund the self-actualising leisure time pursuits of the jobless multitudes.”

And computers are always programmed by people. By corporations. Automation brings with it a whole new set of philosophical and ethical considerations we’ve barely begun to consider. Who will write the algorithm that controls Google’s self-driving car? Will it be programmed to save the child who runs into the road, even if that endangers the driver? Or the other way round?

What will it mean when warfare, as seems ever more likely, is automated? When casualties are no longer a reason not to go to war? Carr doesn’t go there but it’s hard not to read the chapter on lethal autonomous robots – technology that already exists – without thinking of the perpetual warfare of Orwell’s Nineteen Eighty-Four.

Who is it serving, this new technology, asks Carr. Us? Or the companies that make billions from it? Billions that have shown no evidence of trickling down. The question shouldn’t be “who cares?” he says at one point. It should be: how far from the world do we want to retreat?

The Glass Cage by Nicholas Carr is published by Bodley Head (£20). Click here to order it for £16