“How good are you — and how can we make you better — at anticipating when your machines will fail?”
An interview with Chaz Firestone, PhD
Asst. Professor in the Dept. of Psychological & Brain Sciences, Johns Hopkins University
By: Michelle Johnson (4/5/19)
Did you always want to be a scientist when you were younger?
My training is in both science and in philosophy, and I think as a child I imagined myself as a scientist, but I really liked the humanities too. I think when you’re younger you don’t always know what the options are that are available to you, career-wise or even interest-wise. Are you allowed to be the kind of person who’s interested in both science and in philosophy? One thing that I really appreciated when I went to college — and that I now appreciate as an educator — is experiencing a brand new field that doesn’t even exist for high school students. For me, that was cognitive science, which blends insights from psychology, computer science, neuroscience, philosophy, anthropology, and many other fields. You really can have it all!
What are the main questions you’re studying now?
What we’ve been working on recently is the question of what you as a human know about intelligent machines. Something that you and I do all the time is try to read each other’s minds. We try to say, “What’s this person thinking right now?” What we’ve become interested in is what happens when you take that attitude towards a machine.
So far we’ve been focusing on a new and fascinating problem in artificial intelligence research: When researchers make machines that seem to be very intelligent, other researchers come up with ways to “fool” those machines so that the machines will make mistakes. Vision is a good example: There are machines that have recently become very good at looking at a picture and telling you what’s in them; they can tell you that something is a picture of a dog, a cat, a table, a chair — or even a weapon, or a tumor. But then there are special images that “fool” those machines — pictures that basically look like gibberish, but that the machine also will call a dog, or a cat, or a table, or a chair.
What we want to know is whether an ordinary person can predict which images will fool machines, which will not, and in general what machines will say in these situations. What will happen when a machine looks at one of these pictures that it’s supposed to get wrong? Will you be able to tell that it will get it wrong?
Examples of images that “fool” AI trained to classify familiar objects in pictures & labels indicating the machine’s classification of each image. (Zhou & Firestone, 2019)
What might be an example of this in daily life?
Something that you have to do when a new technology enters your life is learn when it will work and when it will fail. You have to know that when you’re talking on your cell phone, you can’t get in an elevator because it might drop the call; you have to know that when you issue a voice command to Alexa, you can’t do it with your mouth full, and you can’t do it using slang. The same thing is going to be true for AI technologies that enter your life, whether it be through a self-driving car, or even a machine that helps a radiologist decide whether a tissue sample is healthy or diseased. We’re interested in this question: how good are you — and how can we make you better — at anticipating when your machines will fail?
What has most surprised you working in this field?
There’s this dream you have when you start a lab and join an academic department that when you show up on day one, it’s going to be this party of ideas and you are going to be engaged in this awesome intellectual conversation with all of your students and colleagues. But is that really what it’s like? To be honest, for me at least, that is what it’s like! In other words, I’m surprised at how unsurprised I am, by which I mean that I really value my colleagues, my department, my lab, and the people I work with.
If you could choose to do anything else besides your current job, what would it be?
My main athletic hobby is grappling. It’s kind of like wrestling. In wrestling, your goal is to pin your opponent and to put their shoulders on the mat. In my sport, your goal is to make your opponent give up; your goal is to put your opponent in such an uncomfortable position that they have to say, “Stop,” and then you win. There are some people who can do that as their job, they’re professional grapplers. That would be pretty cool.
To learn more about Dr. Firestone and his work, click here.
Click here to go back to the “Interviews with Scientists” page.