Can AI Evolve?

[ intro ] Artificial intelligence is really, well, intelligent. It’s beaten humans at chess and poker and AlphaGo famously beat one of the world’s
top players at the super-complex game Go IBM Watson even won Jeopardy, and now a different
version of it is designing personalized cancer treatments. These are tasks humans spend decades mastering,
and AI is beating them left and right. Except, for as good as we are at teaching
AI to do complicated tasks, we’re terrible at getting them to learn
even the most basic, toddler-level skills. Like, ask AlphaGo to recognize a cat — let
alone a specific person — and it will have no idea what to do.

This problem is called Moravec’s paradox,
after one computer scientist who studied it in the 1980s. And while it seems to have a pretty straightforward
solution, it’s definitely not an easy one. Because to make a program that thinks like
a toddler… we’ll likely have to teach AI to evolve. On the most basic level, the reason for Moravec’s
paradox is simple: We don’t know how to program general intelligence. We’re great at getting AI to do one thing,
but most toddler-level skills — like facial recognition — involve learning new things and then transferring
them to other contexts. Getting computers to do that is one goal of
what’s called general AI. And in 1988, Hans Moravec pointed out that
there’s a simple reason it’s so hard: evolution. His point was that things that seem really
easy to us are actually the result of thousands of years of development. So even though most kids can easily tell the
difference between yellow and blue, or a friend and a stranger, those aren’t actually simple
skills. They only seem basic to us because our species
has spent tens of thousands of years refining them.

Meanwhile, we’ve only been making computers
for about a century, tops. So there’s no way we could have figured
out general AI in that time. Then again, Moravec didn’t think time was
the only issue: He also thought researchers were approaching
the problem the wrong way. In the ‘80s, developers were mainly working
from the top down, just trying to copy the mental processes of
fully-formed human brains. But Moravec believed that the most successful
approach would be to work from the bottom up. In other words, instead of building a complex
brain from scratch, he thought we should mimic evolution. Just like in nature, we would start small,
then add complexity to our AI little by little, all the while challenging these programs to

We could even study how the human brain does
this and apply those lessons to machines. Which, obviously, is not difficult whatsoever. Still, it does seem to be a solution that
works. Because the more computer scientists base
their AI on our brains… the smarter they seem to get. A lot of this research is focused on neural
networks. These are systems that can teach themselves
to recognize patterns, and they’re modeled after how our brains learn information. When you learn something new, your brain strengthens
the connections between its neurons. v. Over time, those connections grow stronger
and stronger, and will likely stay in your brain for a while. On the flip side, if your brain realizes a
piece of information isn’t worth keeping — like where you parked that one time three
months ago — it can remove receptors.

That way, the connection can get overwritten
with something more helpful. These stable and dynamic connections allow
your brain to keep what it needs, and get rid of the stuff it doesn’t. And neural networks work in a similar way. They start off with some basic framework for
how to do a task, and then they practice that task to refine
the connections between their artificial neurons. Like, say you want to train a network to identify
dog breeds. To do it, you would first give the system
some basic guidelines. Then, you would feed it a bunch of pictures,
and the AI would try to identify each one. At first, it would be terrible at this. But with each image, the network would make
small tweaks to connections between its neurons, called weights. For example, it might make size more important
than paw color. After thousands or millions of pictures, those
tweaks would eventually be good enough for the network to identify dog breeds accurately. The program would have strengthened the connections
it needed, and scaled back the ones it didn’t, just
like your brain.

Neural networks are a big step toward general
AI, but they’re not perfect. Actually, in a lot of ways, they’re pretty
narrow, because most of them can still only do one thing. While your brain can make connections about
all kinds of information at once, many neural networks have connections that
are too weak and dynamic. I means that all of their weights get adjusted
with every new piece of data. So if you suddenly started feeding your network
pictures of cats instead of dogs, it would adjust all of the weights you worked
so hard to perfect.

Every connection would now be about cats. This problem even has a dramatic name: catastrophic
forgetting. But the cool thing is, we can use other knowledge
about our brains to solve it. One approach took inspiration from the fact
that the brain doesn’t just go for a grab-bag of whatever neurons are available. Instead, it activates different sets of neurons
for different tasks. In a 2018 study published in PNAS, researchers
showed that you can do this in a neural network, too: You can make one task activate one set of
neurons, and another task activate another set. By combining this approach with previous methods
from other teams, these researchers were able to program a network
that achieved 90% accuracy on 500 tasks. Which isn’t perfect, but is promising. And the more we learn about how our brains
refine connections, the better these methods are going to get. Of course, catastrophic forgetting isn’t
the only barrier to general AI. Another challenge is getting systems to learn
from more than just examples. Not every task has a huge dataset for a network
to sort through, and anyway, who has time for that? If a program is going to think like a human,
it has to start grasping the rules that govern whether an answer is correct.

And in 2016, UK researchers came up with a
way to achieve that. They relied on two concepts: a recurrent neural
network and reinforcement learning. A recurrent neural network uses feedback loops
to keep tabs on what just happened and how that should inform an AI’s next move. They’re used a lot in language processing. For example, if a program started a sentence
with a noun, it would remember that. Then, it would use the rules of English grammar
to tell itself that the next word should probably be a verb. Reinforcement learning is how a network can
figure out the best next move on its own. It guesses the right answer and then immediately
receives feedback by getting some kind of reward signal or lack thereof. Then, it uses that feedback to learn what
to do next time. Going back to the language example, if an
AI guessed that “Olivia dog” was a good sentence, it wouldn’t get a reward signal. But if it said, “Olivia ran,” it would. This is the approach the AI system AlphaGo
used to beat the world’s top Go player. In this 2017 study, the UK team trained a
recurrent network using reinforcement learning, but they also got it to use a different, secondary
reinforcement learning algorithm at the same time.

In that way, one part of the AI learned how
to respond to different examples, while the other part learned how those examples
fell into a larger rule structure. They called this deep meta-reinforcement learning, and their approach helped the network quickly
learn and adapt to seven very different tasks. It could do things like navigate a labyrinth
with a changing goal and pull a series of rigged slot machines to get the maximum reward. While it might not seem as obvious, these
systems are based on our brains, too. When a system gets a reward in reinforcement
learning, it’s like how your brain uses chemicals like dopamine to give you a reward. That reinforcement encourages you to practice
certain behaviors. So by building a similar system into neural
networks, researchers are hoping to encourage them to keep learning and adapting.

Now, all of the techniques we’ve talked
about are great. A program with general intelligence should
totally be able to process multiple kinds of information, and should be able to learn
new rules. But to build a truly evolving system, we’ll
also need to make AI curious. Because having an intrinsic desire to figure
out how stuff works and fits together is a big part of how we learn and explore. Think about kids. They’ll go turn over a rock and poke at
the bugs underneath it because exploring sounds fun, not because they were promised a reward for

Of course, we can’t just push AI out into
the world and tell it to be home before supper. So instead, scientists are using video games
to teach them curiosity. In 2017, Berkeley researchers managed to do
this using Super Mario Brothers. They trained their AI to predict what each
frame of the game would look like as it explored. But instead of generating a reward for being
right, this AI got a reward for being wrong. That is, the less reality matched its prediction,
the bigger the reward it got. Essentially, it was rewarded for being surprised. This led the system to explore new parts of
the game, which means the team basically programmed
it to be curious about its environment. Unfortunately, it could never even beat the
first level. But hey, it’s a start. Now, other projects are trying different ways
of getting AI to be self-sufficient. In January 2019, a team from Columbia University
successfully got a robot arm to create an internal model of itself.

It figured out what it looked like and how
it worked without any outside input. It did this by trying out a thousand different
movements, recording each one to figure out which ones worked and which ones were physically
impossible. Kind of like the robot version of a baby playing
with its hands. Once it was all done, the arm could successfully
pick up and place small balls into containers and write with a marker — even though researchers never told the
arm what it could do. The team even replaced one of the arm’s
parts with a deformed piece, and it quickly adapted to the change. The robot learned what it was on its own,
and that made it easier for it to adjust to new situations. By building programs that process information
like our brains, and teaching these programs to be curious instead of just correct, scientists are heading down a road where AI
might one day be able to evolve.

Someday, we might make a machine that learns
to treat diseases just by learning a little about biochemistry, or one that can design cars by studying engineering. Essentially, we’d be making the AI equivalent
of students — programs that learn to synthesize and apply
information. But there’s a lot to figure out, because
in reality, we don’t know everything about our brains work — let alone how to apply
those things to machines. We know a lot, sure, but to really make an
AI that thinks like us… well, we’re going to have to understand
ourselves a little better first.

And that’s a whole different field of research. Thanks for watching this episode of SciShow! If you want to keep exploring the universe
with us, you can go to and subscribe. [ outro ].

Leave a Reply

Your email address will not be published. Required fields are marked *