Mike Ananny and I have an oped in the Globe and Mail on the ethics and governance of AI.  We wrote it in response to the Federal government’s recent funding announcement for AI research and commercialization.

Ethics and governance are getting lost in the AI frenzy

Taylor Owen and Mike Ananny, The Globe and Mail, March 30, 2017

On Thursday, Prime Minister Justin Trudeau announced the government’s pan-Canadian artificial intelligence strategy.

This initiative, which includes a partnership with a consortium of technology companies to create a non-profit hub for artificial intelligence called the Vector Institute, aims to put Canada at the centre of an emerging gold rush of innovation.

There is little doubt that AI is transforming the economic and social fabric of society. It influences stock markets, social media, elections, policing, health care, insurance, credit scores, transit, and even drone warfare. AI may make goods and services cheaper, markets more efficient, and discover new patterns that optimize much of life. From deciding what movies get made, to which voters are valuable, there is virtually no area of life untouched by the promise of efficiency and optimization.

Yet while significant research and policy investments have created these technologies, the short history of their development and deployment also reveals serious ethical problems in their use. Any investment in the engineering of AI must therefore be coupled with substantial research into how it will be governed. This means asking two key questions.

First, what kind of assumptions do AI systems make?

Technologies are not neutral. They contain the biases, preferences and incentives of their makers. When technologists gather to analyze data, they leave a trail of assumptions about which data they think is relevant, what patterns are significant, which harms should be avoided and which benefits should be prioritized. Some systems are so complex that not even their designers fully understand how they work when deployed “in the wild.”

For example, Google cannot explain why certain search results appeared over others, Facebook cannot give a detailed account of why your newsfeed may look different from one day to the next, and Netflix is unable to explain exactly why you got one movie recommendation over another.

While the opacity of movie choices may seem innocuous, these same AI systems can have serious ethical consequence. When a self-driving car decides to choose the life of a driver over a pedestrian; when skin colour or religious affiliation influences criminal-sentencing algorithms; when insurance companies set rates using an algorithm’s guess about your genetic make-up; or, when people and behaviours are flagged as ‘abnormal’ by algorithms, AI is making an ethical judgment.

This leads to a second question: how should we hold AI accountable?

The data and algorithms driving AI are largely hidden from public view. They are proprietary and protected by corporate law, classified by governments as essential for national security, and often not fully understood even by the technologists who make them. This is important because the existing ethics that are embedded in our governance institutions place human agency at their foundation. As such, it makes little sense to talk about holding computer code accountable. Instead, we should see AI as a people-machine hybrid, a combination of human choices and automated decisions.

Who or what can be held accountable in this cyborg mix? Is it individual engineers who design code, the companies that employ them and deploy the technology, the police force that arrests someone based on an algorithmic suggestion, the government that uses it to make a policy? An unwanted movie recommendation is nothing like an unjust criminal sentence. It makes little sense to talk about holding systems accountable in the same way when such different types of error, injustice, consequences and freedom are at stake.

This reveals a troubling disconnect between the rapid development of AI technologies and the static nature of our governance institutions. It is difficult to imagine how governments will regulate the social implications of an AI that adapts in real time, based on flows of data that technologists don’t foresee or understand. It is equally challenging for governments to design safeguards that anticipate human-machine action, and that can trace consequences across multiple systems, data-sets, and institutions.

We have a long history of holding human actors accountable to Canadian values, but we are largely ignorant about how to manage the emerging ungoverned space of machines and people acting in ways we don’t understand and cannot predict.

We welcome the government’s investment in the development of AI technology, and expect it will put Canadian companies, people and technologies at the forefront of AI. But we also urgently need substantial investment in the ethics and governance of how artificial intelligence will be used.

Emily Bell and I have written a Tow Center report exploring how Silicon Valley has reengineered journalism. We look at how publishers have been absorbed into the platform ecosystem, how ad tech has shaped both media economics and political campaigns, and do a deep dive into Facebook and the 2016 election. In short, it’s a structural problem.

Executive summary is below, the full report is here, and as a pdf here.

The Platform Press:  How Silicon Valley Reshaped Journalism 

Executive Summary

The influence of social media platforms and technology companies is having a greater effect on American journalism than even the shift from print to digital. There is a rapid takeover of traditional publishers’ roles by companies including Facebook, Snapchat, Google, and Twitter that shows no sign of slowing, and which raises serious questions over how the costs of journalism will be supported. These companies have evolved beyond their role as distribution channels, and now control what audiences see and who gets paid for their attention, and even what format and type of journalism flourishes.

Publishers are continuing to push more of their journalism to third-party platforms despite no guarantee of consistent return on investment. Publishing is no longer the core activity of certain journalism organizations. This trend will continue as news companies give up more of the traditional functions of publishers.

This report, part of an ongoing study by the Tow Center for Digital Journalism at Columbia Journalism School, charts the convergence between journalism and platform companies. In the span of 20 years, journalism has experienced three significant changes in business and distribution models: the switch from analog to digital, the rise of the social web, and now the dominance of mobile. This last phase has seen large technology companies dominate the markets for attention and advertising and has forced news organizations to rethink their processes and structures.

Findings

• Technology platforms have become publishers in a short space of time, leaving news organizations confused about their own future. If the speed of convergence continues, more news organizations are likely to cease publishing—distributing, hosting, and monetizing—as a core activity.
• Competition among platforms to release products for publishers is helping newsrooms reach larger audiences than ever before. But the advantages of each platform are difficult to assess, and the return on investment is inadequate. The loss of branding, the lack of audience data, and the migration of advertising revenue remain key concerns for publishers.
• The influence of social platforms shapes the journalism itself. By offering incentives to news organizations for particular types of content, such as live video, or by dictating publisher activity through design standards, the platforms are explicitly editorial.
• The “fake news” revelations of the 2016 election have forced social platforms to take greater responsibility for publishing decisions. However, this is a distraction from the larger issue that the structure and the economics of social platforms incentivize the spread of low-quality content over high-quality material. Journalism with high civic value—journalism that investigates power, or reaches underserved and local communities—is discriminated against by a system that favors scale and shareability.
• Platforms rely on algorithms to sort and target content. They have not wanted to invest in human editing, to avoid both cost and the perception that humans would be biased. However, the nuances of journalism require editorial judgment, so platforms will need to reconsider their approach.
• Greater transparency and accountability are required from platform companies. While news might reach more people than ever before, for the first time, the audience has no way of knowing how or why it reaches them, how data collected about them is used, or how their online behavior is being manipulated. And publishers are producing more content than ever, without knowing who it is reaching or how—they are at the mercy of the algorithms.

In the wake of the election, we have an immediate opportunity to turn the attention focused on tech power and journalism into action. Until recently, the default position of platforms (and notably Facebook) has been to avoid the expensive responsibilities and liabilities of being publishers. The platform companies, led by Facebook and Google, have been proactive in starting initiatives focused on improving the news environment and issues of news literacy. However, more structural questions remain unaddressed.

If news organizations are to remain autonomous entities in the future, there will have to be a reversal in information consumption trends and advertising expenditure or a significant transfer of wealth from technology companies and advertisers. Some publishers are seeing a “Trump Bump” with subscriptions and donations rising post-election, and there is evidence of renewed efforts of both large and niche publishers to build audiences and revenue streams away from the intermediary platform businesses. However, it is too soon to tell if this represents a systemic change rather than a cyclical ripple.

News organizations face a critical dilemma. Should they continue the costly business of maintaining their own publishing infrastructure, with smaller audiences but complete control over revenue, brand, and audience data? Or, should they cede control over user data and advertising in exchange for the significant audience growth offered by Facebook or other platforms? We describe how publishers are managing these trade-offs through content analysis and interviews.

While the spread of misinformation online became a global story this year, we see it as a proxy for much wider issues about the commercialization and private control of the public sphere.

My edited book for Columbia University Press with Emily Bell, Journalism After Snowden: The Future of the Free Press in the Surveillance State, on journalism and state surveillance has recently been released. It includes chapters by a pretty wonderful group of journalists and scholars, including: Steve Coll, Cass Sunstein, Clay Shirky, Alan Rusbridger, Jill Abramson, Glenn Greenwald, Ethan Zuckerman, Julia Angwin, David Sanger, Edward Snowden, Jonathan Zittrain, Lee Bollinger among many others. I feel very lucky to have worked with them on this.

It goes without saying that the stakes are even higher now, but it’s worth remembering that the surveillance architecture and vulnerability of journalism that it causes is a long-running, international, and bi-partisan affair.

Details of the book are available here.

The launch event for the book will be in New York on March 7th and will include a panel with Julia Angwin, Barton Gellman, and Ben Wizner. Details and tickets are here.

The oped below was written after the publishing of The Shattered Mirror: News, Democracy and Trust in the Digital Age, on which i served as a research principal.  I have been surprised that what I believe is the most radical and potentially transformative idea in the report has received almost no attention.  So Elizabeth Dubois and I wrote an oped about the recommendation for the CBC to move to publishing under a creative commons licence. The piece below was first published in The Toronto Star.

For the past year I have had the fortune to work with Ed Greenspon, the Public Policy Forum, and a wonderful group of scholars and journalists on a report on the state of journalism in Canada. The resulting report was recently released, The Shattered Mirror: News, Democracy and Trust in the Digital Age. My role was mainly to support the analysis of digital journalism both in Canada and within the broader platform (Facebook, Google, etc) ecosystem. I hosted a workshop in Vancouver to dive specifically into the challenges digital start-ups face in Canada and I attended many of the roundtables that the PPF hosted across the country.  I learned a ton, and while I don’t agree with everything in the report, i think that it represents the most detailed current assessment of the state of the industry in Canada, and some of the recommendations, if adopted, would lead to significant transformation of the digital journalism space in this country.  The site for the report is here and a PDF can be downloaded here.

I have an article in the Columbia Journalism Review that explores virtual reality, Facebook, the challenges of doing journalism in and on virtual realities, and the importance of holding platform companies accountable for the worlds they are building.  Full piece is here, below is from the introduction:

As Facebook and others begin researching and developing technologies that could augment our lives in significant ways, a new space is opening up for journalism. And unlike early virtual journalism experiments in Second Life, which ultimately mimicked traditional “real world” reporting, journalism inside these new virtual worlds will require an entirely different set of skills and approaches, and will challenge three core journalistic concepts: representation, witnessing, and accountability.

First, virtual reality challenges the ways in which journalists think about representation. At the core of VR’s unique power is a deception–that the user believes she is experiencing something she is not. The goal of journalism in VR, therefore, is to inform the user by blurring the act of journalistic representation. But journalists cannot appropriate the physiological power of virtual reality without also thinking seriously about how leveraging it for journalistic purposes changes the way the world is represented.

Second, virtual reality challenges journalists’ ability to serve as witnesses with agency. It is entirely unclear what tools will be needed to observe events and institutions in a virtual space that is created by a confluence of human intervention and algorithmic control. If the boundaries between observation, participation, audience, and social structure fundamentally break down in virtual worlds, it is uncertain whether virtual reality journalism can be done by a human at all.

Third, as Facebook begins to build a virtual world and signals its ambitions to augment human capabilities, there has never been a greater need for accountability journalism, both within virtual spaces and for the companies building them. These virtual experiences will be designed and increasingly automated to be as addictive as possible. They will be marketed aggressively and widely, and could radically change our lives. The technologies driving them will undoubtedly be used by governments and militaries to seek ever greater control. But as the cluster of Silicon Valley companies building these futures rises to significant and largely unchecked social, political, and economic power, technology journalism has proven insufficient to hold them responsible for their actions.

Whether or not these technology futures emerge, they are being discussed and researched at one of the largest companies in the world, with a user base of over 1.5 billion people and rich data about much of the world’s consumption, movements, knowledge, and networks. How these virtual worlds are designed and created, and how humans will evolve to engage with these new technologies, pose fundamental problems for journalism.

I have a short article in Policy Magazine summarizing a broader argument that I have been making about the limitations of a siloed digital foreign policy. The argument draws extensively on Disruptive Power.  In short, I argue that a digital foreign policy:

will first and foremost require a rethinking of the approach to online governance. It means addressing the misalignment between our international institutions and the actors and technologies that currently have power. The status quo governance discourse delegitimizes many of the emerging actors with real power, and because of this it is blind to some of the core policy challenges of the 21st century.

It also means assessing what new technologies or socio-technological processes currently sit  utside of our international governance structures. Algorithms, autonomous weapons, quantum computing and cryptocurrencies all exist in ungoverned spaces that fundamentally challenge the legitimacy and authority of the state. What does governance in this rapidly evolving space look like? Finally, taking digital foreign policy seriously means moving beyond siloed digital foreign policies. The idea that surveillance policy, digital diplomacy, autonomous weapons development and digital humanitarianism can be discussed in isolated departmental silos is absurd. They all intimately effect each other, are based on the same data flows and algorithmic tools, and contradictions between them seriously harm our credibility and impact in the world.

Put another way: What does a Whole of Government Digital Strategy look like—one that addresses surveillance, IP, C-51, dual-use technologies, cyber war, autonomous weapons and online finance? Taking this question seriously, with all of the complexities it entails, is a pre-requisite for for any country seeking to engage with responsibility, legitimacy and continued relevance in the emerging global digital system.

The full article is available here: “Towards a Whole of Government Digital Strategy“

I have a review of Don and Alex Tapscott’s new book, Blockchain Revolution: How the Technology Behind Bitcoin Is Changing Money, Business and the World in this month’s Literary Review of Canada.  The full piece is here, but in short:

The book is at its best when explaining the complex concepts underpinning blockchain, but ultimately falls victim to an irreconcilable conflict at the heart of the blockchain movement—the growing divide in the cryptocurrency community between those who want to normalize its use and those who remain steadfast in its revolutionary potential. Put more starkly, while some view blockchain as a more efficient tool for financial services, market exchanges, online purchases and information sharing that can be adopted by established actors, others see it as a fundamental challenge to the hierarchical nature of power itself. These two visions of blockchain cannot coexist.

I was on The Agenda with Steve Paikin a couple of weeks ago to talk digital technology, power and some Canadian foreign policy. We got into both my Disruptive Power and The World Won’t Wait books.

At the end though, I was asked about Trump. My answer relates to Brexit as well though.  The growing desire to burn down the house stems in part from a failure of governments to deliver on their promise & rhetoric to large segments of society. This, in turn, is because states have diminishing power to control citizens and events. Their monopoly over collective action is over. While much of the resulting nihilism is manifest in the right, this is a particular challenge for the left, who’s central proposition is deeply rooted in the role of the state.