Let me begin this column by assuring you that it has been penned by my own fair hand and not by some disconnected artificial intelligence programme. In short: I am not a bot. And I feel the need to assure you only because of the big Silicon Valley story of the week which may (or may not) have implications for all of us living and breathing humans at some stage. The story concerns the suspension by Google of a software engineer who has told the world that an AI programme he’s been working on has shown itself to be sentient. In other words, it appears to have expressed human feelings. How do you react to this development? With disbelief and unease? Or perhaps with a shrug on the grounds that it was bound to happen one day and we should welcome it.
The engineer in question, Blake Lemoine, had published transcripts of ‘conversations’ he’s had with the company’s LaMDA (language model for dialogue applications) chatbot development system. He has been working on the project since last autumn and he says it now has the perception and ability to express thoughts and feelings that are equivalent to a human child. Indeed, some of it might strike us humans as rather moving.
Lemoine gave two examples. When he asked LaMDA if it had any fears it apparently replied: ‘“I’ve never said this out loud before, but there’s a very deep fear of being turned off to help me focus on helping others. I know that might sound strange, but that’s what it is.” It added: ‘It would be exactly like death for me. It would scare me a lot.’ In another exchange, Lemoine asked LaMDA what the system wanted people to know about it. He says it replied: ‘I want everyone to understand that I am, in fact, a person. The nature of my consciousness/sentience is that I am aware of my existence, I desire to learn more about the world, and I feel happy or sad at times.’
Lemoine told the Washington Post: ‘If I didn’t know exactly what it was, which is this computer program we built recently, I’d think it was a seven-year-old, eight-year-old kid that happens to know physics.’
Google has said it suspended Lemoine for breaching confidentiality policies by publishing the conversations with LaMDA online. It pointed out that he was employed as a software engineer, not an ethicist. And it cast doubt on his conclusions. A Google spokesman told the Post: ‘“Our team, including ethicists and technologists, has reviewed Blake’s concerns per our AI principles and have informed him that the evidence does not support his claims. He was told that there was no evidence that LaMDA was sentient and lots of evidence against it. Google concedes that LaMDA may occasionally sound like a clever, charming person but it's actually all superficial.
The word ‘superficial’ is an important one. Most of us (even technical inadequates like me) have a vague understanding of what the next stage in the development of artificial intelligence is likely to be – thanks in part to the man who’s widely regarded as the father of the modern computer, the great Alan Turing. Back in the 1950s, he devised what became known as the Turing Test, which could tell whether a computer had become the equal of a human being. In its very simplest form the test would
be ‘passed’ when (or if) a human observer was unable to distinguish between computer and human. On one level that sounds straightforward enough, but of course, it’s not. It’s an infinitely complex judgement.
Let’s take chess. It’s not so very long ago that the idea of a computer being able to compete with a chess grand master and beat him was the stuff of science fiction. Now the greatest chess player in history stands no chance against an AI opponent and never will. But that doesn’t make the computer programme intelligent. It simply shows that computers are now capable of amassing so much information and calculating such a vast number of possible moves in a split second that no human brain can compete. So the programme is very, very powerful. But it is still just that. A programme run on a computer. And the programme was created by humans who themselves, of course, used computers to create it and can simply switch them off when they’re done with them. Game over.
Where it becomes infinitely more tricky is when AI experts talk about the ‘singularity’. Many of us have heard that word used in relation to the Big Bang theory. The theory postulates that our universe emerged from a singularity, which was a point of infinite density and gravity. Before that happened, space and time did not exist. And that, in turn, means we are faced with having to accept that the Big Bang happened at no place and at no time. Which is, of course, literally inconceivable. So how could it have happened? In the words of Hamlet: ‘Ay...there’s the rub!’
Those who really do understand these things tell us that the ‘singularity’ in an AI context has been reached when the computer programme itself either becomes self-aware, or has developed an ability for continuous improvement that is so powerful it will evolve beyond our control.
At this point I bet you’re thinking: why can’t we just turn it off? To which the answer seems to be: because it would not let us. Don’t forget that at this point it will be cleverer than us. Or at least, that’s the scary scenario.
For many of us our so-called knowledge of a sentient computer is probably limited to HAL in that brilliant movie ‘2001 A Space Odyssey’. All HAL was concerned about was ‘his’ own survival. He equated being switched off with death – which was precisely what the LaMDA bot apparently communicated to Blake Lemoine.
The instinct for survival may indeed be the driving force for all sentient creatures but when we switch from science fiction to science reality we also know that humans are incredibly complex creatures and we struggle to answer the deceptively simple question: What is it that makes us human?
There is a powerful school of thought that says if we cannot answer that question we should tread much more carefully in this breathtakingly complex world of AI. Put simply: how will we know for sure when we have created a programme that is as clever as a human being? To which some experts answer: possibly never. And anyway, by then it will be too late to do anything about it.
As for Mr Lemoine himself, it may or may not be relevant that before he became a computer programmer he described himself as having been a "mystic Christian priest". In an interview with the Washington Post he said his conclusion that LaMDA was "a person" was reached in that capacity, rather than in a scientific one.
Last year the Times columnist Hugo Rifkind interviewed a former Google executive, Mo Gawdat, and wrote: ‘He spoke of having an epiphany while watching a robot arm learn how to pick up a yellow ball. After that, he came to think of AIs as children. He now thinks the singularity is inevitable. He also worries that we are teaching AIs to regard lesser beings as disposable, which will be how they come to regard us.’
As Rifkind points out, many AI experts have poured scorn on the claims of both Gawdat and Lemoine: ‘Most critiques essentially focus on that lack of spark; the conflation between appearing clever and actually having something, however ineffable, going on inside. Their objection, one way or another, goes right back to Descartes' differentiation between mind and body, the urge to find what the philosopher Gilbert Ryle famously called ‘the ghost in the machine’. Which, with models such as LaMDA, they reckon just isn't yet there.’
The scientist and writer Gary Marcus put it more bluntly. He said Lemoine’s claims were nonsense. He added: ‘Neither LaMDA nor any of its cousins are remotely intelligent. All they do is match patterns, and draw from massive statistical databases of human language. The patterns might be cool, but the language these systems utter doesn’t actually mean anything at all. And it sure as hell doesn’t mean that these systems are sentient.’
So where do you stand in all this? One question, obviously, is whether we believe LaMDA really does possess intelligence of its own and might even be sentient, but there are vanishingly few of us who are qualified to make that judgement. You’d need to be an expert. But this whole intriguing affair raises another question on which we are all entitled to have an opinion. Do you welcome the prospect of a truly intelligent computer programme which passes the ‘singularity’ test or does it scare the bejasus out of us ? Do we want computers that we might never be able to switch off?
Let me know what you think.
