Writing the Book on Artificial Intelligence: LBF Quantum’s Nick Bostrom

In Feature Articles by Mark Piesing

“The process of writing decides what is to be written next.” Hence, says Nick Bostrom, Artificial Intelligence isn’t as big an “existential risk” for publishing as for other fields. Maybe.
Nick Bostrom at Oxford's Future of Humanity Institute. Image: Mark Piesing

Nick Bostrom at Oxford’s Future of Humanity Institute. Image: Mark Piesing

For those of us accustomed to attending London Book Fair’s “Publishing for Digital Minds” conference each year, there’s a mild retraining task in the event’s new name, “Quantum: Publishing & So Much More.” The day is again under the able direction of Orna O’Brien, however, and after LBF chief Jacks Thomas welcomes the crowd, she and co-chairs Bill Thompson and Alison Jones will hand off to Nick Bostrom for the opening keynote. We asked Mark Piesing to speak with the professor in Oxford. And when it comes to fears of AI’s potential, Piesing comes back to us here with a quantum of solace for publishing. —Porter Anderson

By Mark Piesing | @MarkPiesing

‘A Risk That We Will Smash Into the Wall’

“Artificial Intelligence could in the end replace most, if not all, of the high-level functions of an editor, but if you get to that point then the whole game changes not just for an industry but for humanity as a whole, because it will have become superintelligent,” says Oxford academic and prophet of an AI apocalypse Nick Bostrom.

Projected on to the wall behind him is a live stream from the Google DeepMind AlphaGo challenge being played in Seoul. DeepMind has become the first computer program to beat a professional human player at the game of Go, this month scoring a 4-to-1 victory over one of the world’s top champions.

Bostrom SuperintelligenceBostrom defines superintelligence as an intelligence much smarter than the best human brains in practically every field, including scientific creativity, general wisdom, and social skills. It’s also the title of his book Superintelligence: Paths, Dangers, Strategies (Oxford University Press), which was published in 2014 and made The New York Times bestseller list.

If you haven’t been following the rise of the new technology, then the threat of an AI takeover in the next 100 years may come as a surprise or seem a joke, even a piece of science-fiction. It shouldn’t. Bostrom’s book—a call to arms to the leaders of industry, government, and the military about the possible threat posed by AI to the future of humanity—led Raffi Khatchadourian in November to dub Bostrom “a philosopher of remarkable influence” in his feature on the professor, The Doomsday Invention.

Bostrom has been listed on Foreign Policy’s Top 100 Global Thinkers list twice, and he was the youngest person included on Prospect magazine’s World Thinkers list.

And on April 11th, Bostrom’s keynote address at London Book Fair’s Quantum Conference will help make good on the event’s tag line, “Publishing & So Much More.” He certainly is that. Success has made him a hot ticket.

The slightly reserved, precise, and very sharp Bostrom is the founding director of the Future of Humanity Institute (FHI) in Oxford, which explores often neglected “existential risks” to the future of humanity.

Bostrom has defined existential risks as ones that could annihilate life on Earth or permanently and drastically curtail its potential, such as supervolcanoes, pandemics caused by bio-weapons, nanotechnology, and of course AI. He explored this same ground in an earlier book, Global Catastrophic Risks. Comprising 26 essays by world experts and edited with Milan Cirkovic, it’s enough to keep you up all night.

‘Existential Risks’

The Institute itself is situated over a gym in a maze of Oxford streets that date back a thousand years and it shares space with the Centre for Effective Altruism. Not surprisingly given his schedule, Bostrom is already behind schedule when he walks in to begin our interview holding his trademark super large smoothie. His assistant hovers nervously behind me as he has a phone call booked straight after our chat. Bostrom says:

Nick Bostrom

Nick Bostrom

“The book is my attempt to think through all the issues that might arise as we move towards machine superintelligence. The first couple of chapters talk about how we get there – the rest focuses on what happens when we do, when machines do surpass humans. This is called the control problem. How can we create controls so that machine superintelligence will be beneficial to humanity as opposed to destroying us?

“After all, AI has an upside. It is a very general-purpose technology. Almost everything we expect technology to do can be done better if the technology is smarter and adapts to the user. In the long term, all the things we want to achieve, like space settlement, we will need machine intelligence for.

“However, [while] the development of machine superintelligence may well be a portal that civilization will have to pass through to meet its full potential, there is a risk that rather than go through the portal we will smash into the wall. We could have a terminal catastrophe which will lead to the destruction of everything. It may even be the scenario that happens unless we put in a heroic effort.

“AI is advancing quickly. It is very fashionable as all the big tech companies have some kind of AI effort. What happens when AI succeeds is now taken a lot more seriously than a sci-fi type thing to talk about over drinks. AI is fashionable. So the key players are starting to realize that maybe there should be a few things that we should start doing now.”

He’s more sanguine, he says, about the implications of DeepMind’s victory with AlphaGo than many of the rather lurid headlines.

“DeepMind’s victory in AlphaGo is significant as a milestone rather than for the technology itself,” Bostrom says. “It shows the progress that has occurred in machine learning as it is—a general-purpose software, not handcrafted material put together to play Go. With some minor modifications, the same applications could play other games and be used for other applications. Traditional AI would have to be reworked entirely when you wanted to use it for a new application.”

‘Publishing & So Much More’

When the stakes are so high in his field, does it seem at all urgent to talk about publishing? Given the threats he routinely explores, Bostrom has only recently started thinking about how AI might affect publishing, he says. But he does have some good news:

Bostrom book 2“Certainly there are lower-level editorial tasks that AI should be able to do soon, and that includes an interesting application of a more advanced version of spell-checking, where the AI suggests changes in sentence structure as well the spelling.

“While I could see AI augmenting or replacing humans in visual arts, in writing it would be harder, as the process of writing decides what is to be written next. The current automated writing software is a very limited form of writing—basically putting data into sentences.

“However, it would be a mistake to reduce this to an examination of the impact on one sector when its impact on humanity could be so great. At the moment, the planet is in our hands and our technology gives use the power to organize, plan and shape it. If an AI can shape the world in the way it wants, then we might get paved over, just like we pave over ant colonies—not because we hate them, it is just a dispassionate act.”

Whether we would have any warning of the apocalypse is a different matter.

“There is some uncertainly whether there will be a sudden advance in AI or whether there would be warning signs building up to it. If it goes well, it will be a controlled detonation which will produce a desirable outcome for humanity.

“We don’t want to be in a position in 50 years’ time of thinking ‘it’s too late now – if only we’d started to think about it in 2016’ .”

Early Bird rates end this week, on Friday March 25th, for London Book Fair’s (#LBF16) Quantum Conference (#Quantum16). Information on the event, set for April 11th at Olympia London, is here. 



About the Author

Mark Piesing

Mark Piesing is a freelance journalist (and teacher) based in Oxford, UK now writing mainly about technology, culture and the intersection between the two for some of the biggest brands in the UK media such as The Economist, Wired.co.uk, and The Guardian. He also contributes to Warwick Business School's Core magazine. WBS is one of the top business schools in the UK.