Chaz Firestone
Department of Psychological and Brain Sciences
Johns Hopkins University

Taking a machine’s perspective

How similar is the human mind to the sophisticated machine-learning systems that can behave like it? Biologically inspired neural network models — including ever-more-popular DeepNets — have taken our field by storm, reaching human-level benchmarks in recognizing new objects and scenes. These advances support technologies such as self-driving cars and machine medical diagnosis; but they are also interesting for psychology, as they may serve as candidate models for the human mind itself.

However, unlike humans, DeepNets can fail in surprising, fascinating, and downright bizarre ways. Perhaps most striking is their susceptibility to “adversarial examples” — carefully crafted images that look like nonsense patterns to humans but are recognized as familiar objects by machines, or that look like one object to humans (e.g., an orange) and a different object to machines (e.g., a missile). This extreme divergence between human and machine classification fundamentally challenges these new advances, not only as applied image-recognition systems but also as models of our own minds.

Surprisingly, however, little work has empirically investigated human classification of the stimuli that fool machines; does human and machine performance ultimately diverge? Or could humans engage in some “machine theory of mind” and predict the machine’s preferred labels? Here, I’ll show how human and machine classification are robustly related: I will present data showing that, across many prominent and diverse imagesets, human subjects can predict how machines will (mis)classify images — including truly bizarre images described in the literature as “totally unrecognizable to human eyes”. I suggest that human intuition is a surprisingly reliable guide to machine perception, and I explore the consequences of these results for psychology, neuroscience, computer vision, “explainable AI”, and our shared cyberpunk future.

The talk will begin at 12:00pm. A pizza lunch will be served at 11:45am.