Written by Mihalis Kritikos,

Surveillance capitalism and Europe's moral duty to shape a human-centric digital future

‘The European Union represents humanity’s best hope to prevent lawless, unprecedented computational concentrations of knowledge and power from becoming as irreversible and poisonous to our societies as the toxic concentrations of carbon dioxide in our atmosphere have become to our earth.’ That was one the main statements made by Shoshana Zuboff, Charles Edward Wilson Professor Emerita of Business Administration at Harvard Business School and award-winning author of The Age of Surveillance Capitalism’, at the STOA Annual Lecture 2020 that took place (virtually) on 9 December 2020. This year’s Annual Lecture, ‘Digital human rights and the future of democracy: Lessons from the pandemic’, was dedicated to the memory of the first Head of the STOA Secretariat, Richard (‘Dick’) Holdsworth. It focused on the disruptive effects of the digital revolution upon democracy and examined the challenges associated with the growing datafication and platformisation of our societies, and the need to reclaim data sovereignty in the era of artificial intelligence.

Given that the ongoing pandemic has accelerated our dependence on the digital world and our transformation from data subjects to data suppliers, the event proved to be a timely occasion to discuss how Europe could react to the uncontrolled extraction and use of data by technological giants. It took place only a few days before the publication of the European Commission’s proposals for a Digital Services Act and a Digital Markets Act. Both legislative proposals put forward a comprehensive set of new rules for all digital services – including online platforms – that operate in the European Union. Soshanna Zuboff referred to these legislative proposals as the EU’s chance to open the black box of surveillance capitalism and assert the right of democratic societies to control their own destinies – as regulators have done in countless other industries.

In her opening remarks, STOA Chair Eva Kaili (S&D, Greece) noted that digital processes and technologies are no longer simply monitoring our behaviour; through the use of predictive analytics, they are also pervasively influencing it. Expressing her concern about the accelerating commodification of personal data, Eva Kaili stated that the extensive use of highly targeted behaviour modification techniques have led to a threat of algocracy, i.e. ‘rule by algorithm’, which challenges the rule of law and erodes human agency, privacy and even democracy, in many ways. She noted that Shoshanna Zuboff’s participation in the 2020 STOA Annual Lecture as a keynote speaker marked an important milestone in STOA’s engagement with bold thinkers, whose critical views and ‘wake-up calls’ remind us of the fragility of our common values, as well as of our rights as citizens, our responsibilities as policy-makers, and our obligations to future generations.

Following the STOA Chair’s intervention, the European Commission Vice-President for the European Way of Life, Margaritis Schinas, delivered the opening speech of the event. He noted the EU’s special responsibility to fit digitisation to our European values, to our democratic principles and to our European way of life. In his insightful speech, Vice-President Schinas highlighted the need to invest in people rather than in technologies or infrastructures. He placed a special emphasis on the need to chart a course for a digital transformation that is genuinely European – not as a matter of competitiveness but as an existential question. Vice-President Schinas then referred to the Commission’s various initiatives and legislative plans that will put in place a model that is human-centric and empowers Europeans as digital citizens.

In her remarkable lecture, Professor Zuboff characterised the uncontrolled extraction and use of data by technology giants as an ‘epistemic’ coup, where digitally activated data extraction leads to an unprecedented dominance of knowledge by the private sector, and allows a few data-enabled individuals to disregard humanity, democratic values and the rule of law, in the name of increasing their profits. She attributed the creation of ‘hidden extraction mechanisms’ that exert a new kind of power over people to the economic imperatives of surveillance capitalism. Her exceptional analysis revealed that a group of corporate actors (along with their complex, far-reaching ecosystems) are ‘intent upon subordinating private human experience to the goals of datafication, computational production and sales’. The nub of Professor Zuboff’s argument is that surveillance capitalism’s target is human nature itself – with Shoshanna Zuboff calling out the ‘data business’ playbook of ‘hidden extraction mechanisms’, which she said is robbing us of the ability to fight back. Professor Zuboff argued that surveillance exceptionalism, as established in the United States of America after 9/11, has become a practice free of regulatory constraints, that steals private human experiences and transforms them into data. Prediction products extracted by surveillance capitalists from human behaviour data are then sold on what she calls ‘behavioural futures markets’. Shoshanna Zuboff further noted that ‘we are paralysed in the iron cage of an epistemic dominance that is based on unequal knowledge and produces unequal power, and that the pandemic has led to a growth in surveillance revenues and to epistemic chaos’. She suggested that Europe should take the lead in adopting strong laws that focus on upstream data extraction practices and on the formal recognition of epistemic rights.

Professor Zuboff’s keynote speech was followed by a panel discussion on various aspects of key new technologies, with the participation of two additional eminent experts: Fredrik Heintz, Associate Professor of Computer Science at Linköping University, Sweden, and Karen Yeung, Professor of Law, Ethics and Informatics at Birmingham Law School, United Kingdom. Fredrik Heintz noted that explicability and investing in education are key principles when designing a governance framework for artificial intelligence. Karen Yeung stated that legal institutions can help us maintain the social foundation of our democracy. In her closing remarks, Eva Kaili noted that Shoshanna Zuboff’s insightful and passionate voice is needed to help European law-makers set rules on algorithmic transparency and shape the digital age in an inclusive and responsible manner. In her view, Professor Zuboff’s keynote speech can also act as a reminder of the fact that we have a lot of work to do in regulating technology, and more particularly in protecting our citizens and our societies by taming the unaccountable power of tech companies.

If you missed the lecture this time, you can access the presentations and watch a recording of the Annual Lecture on the STOA page.