davidbau.com

Analyzing the Robot Apocalypse

On a long drive this last weekend, I chatted with my ten-year-old about the only thing I really know anything about, which is how computers work. We talked about all the parts that go into a computer we had built for playing video games, and he joked that we should figure it all out "before the robot apocalypse happens."

I thought this joke was an interesting window into his ten-year old thoughts, so I pressed him on it: "How will we know when it's the robot apocalypse? Maybe it has already happened.''

His response was simple: "It will happen when computers are sentient.''

Sentient is a sophisticated word for a ten-year old! What does he think it means? "What does it mean to be sentient? That's what I'm asking.''

Cody has a quick answer: "A robot is sentient if it's self-aware.''

Ah, a common science-fiction trope. "And what does self-awareness mean? We just talked about how the BIOS on the motherboard starts up by figuring out which CPU, GPU, memory, disk, and I/O you have plugged in. Doesn't that mean that computers are already self-aware?'' I thought I would start a debate with him about self-awareness, but he surprised me by having a different answer.

He said, "That's not all. To be sentient a computer can't just follow simple rules. It has to do complicated things, like the way the computer sees it when you attack it with a big army in AOE.''

Cody's Four Levels of Sentience

At ten, Cody has played lots of games against computer AIs, and so he has a remarkably subtle understanding of what AI can do. There is no stumping him. So over our hour-long drive, we chatted more about it, and I was able to glean a four-level model of what sentience means to Cody.

Level 1: Self-awareness. Can you think about your own thinking? A computer scientist might say this is roughly a way of saying that a computer can see its own program; or do recursion; or that it is Turing-complete. Cody quickly noticed that sentience is not just about having inward thoughts, but about the sophistication of a computer's relationship with data.

Level 2: Generalization. Can you handle messy data? Instead of following a simple set of brittle rules, sentience seems to be about having robust rules that let you make decisions in new situations. Cody explains that an AI game opponent needs to be flexible, since no game situation is exactly the same. Any rules need to generalize. The program needs to be soft and flexible, not rigid.

Level 3: Adaptability. Can you learn new patterns? A sentient program could certainly be surprised if you sneak up on it. But Cody explains that a sentient AI it would not be repeatedly surprised by the same thing. Soon enough, "like within an hour," says Cody, it would recognize a new pattern and adapt to it. So a sentient AI should never stop training.

Level 4: Opinions. Can you imagine? Cody recently watched a remarkable video by Jennifer Doudna, one of the inventors of CRISPR, who is calling for a clinical moratorium. Cody explains that she is able to imagine a future world of inventions that does not exist yet, and this leads her to form a strong opinion about something that has never been seen. So imagination is another part of being sentient, he explained.

Cody's four levels of sentient AI are: (1) know yourself; (2) act based on patterns, not hard rules; (3) learn by adapting to new patterns; (4) extrapolate based on imagined situations that have not yet been observed.

I think it is a pretty good roadmap when preparing for the coming robot apocalypse!

Posted by David at January 16, 2018 12:12 PM