Increasingly that data has been used not just to analyze behavior but to predict it, and, most worrisome, to change it. Zuboff shared several examples, such as advertisers creating stops in the Pokemon Go game to generate foot traffic to their stores, a Facebook proposal to alert advertisers when analytics indicate a consumer’s brand loyalty is in jeopardy, allowing them to head off the switch to another product, and the data-collecting plans that — before public outcry — were part of Google’s planned 12-acre “smart city” development in Toronto.
In recent years, governments have caught on to the potential of monitoring citizens and shaping their behavior, with troubling implications for religious and ethnic minorities who might be targeted, as reportedly has been happening in China.
“One scientist described it, ‘We can engineer the context around a particular behavior and force change that way. We are learning how to write the music — and then we let the music make them dance,’” Zuboff said. “This new power does not employ jackbooted soldiers to threaten us with terror and murder. It has no intention of sending people to knock on your doors in the middle of the night, to drag you to the gulag or the camp. It works through the medium of the digital to impose its will remotely, more likely to be toting a cappuccino than a gun.”
But at long last, Zuboff said, the crowds she’s seen on her travels to speak about her 2019 book, “The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power,” have begun voicing a deepening concern about who owns their data.
Crowds have sometimes topped 1,000, she said, and their mood has been expressed in a list of words she solicits about their feelings on the subject. The list includes “anxiety, fear, manipulation, democracy, resistance, and rebellion.”
“These are words I hear over and over again,” Zuboff said.
Zuboff spoke Monday at the Harvard Kennedy School’s Carr Center for Human Rights Policy during an event co-sponsored by the Ash Center for Democratic Governance and Innovation. The event is part of the Carr Center’s yearlong examination of technology’s impact on society, called “Towards Life 3.0: Ethics and Technology in the 21st Century.”
Carr Center Director Mathias Risse, the Lucius N. Littauer Professor of Philosophy and Public Administration, said the talk highlighted the evolution of public opinion on the subject. A decade ago, he said, tech optimism abounded, fed by the belief that the free flow of information would provide citizens the knowledge to improve their lives, including making government more responsive to their needs.
Today, he said, tech pessimism has taken over as governments and corporations have illustrated that they too understand the potential power of controlling the internet’s knowledge flow and showed their ability to wield that power.
“People are deeply worried about these things,” Risse said.
Zuboff said she’s encouraged by her lecture crowds because the solution to the problem lies in our hands, in the much-maligned practice of democracy, which she called the “only antidote.” Laws that clarify who owns an individual’s data — two have been filed at the federal level in the last year — can shift the landscape and create a tech environment in which people determine where and with whom to share their private information — DNA data for medical research but not for commercial consumption, for example.
“Right now most discussions already begin with ‘data’ — data ownership, data portability, data accessibility, and so forth — and my view is that once the sentence begins with ‘data’ then we’ve already lost,” Zuboff said. “What should become data in the first place, that is where the line has to be drawn.”