Thomas Serre
Cognitive & Psychological Sciences and Computer Science Departments
Carney Center for Computational Brain Science
Brown University

Feedforward and feedback processes in visual reasoning

The progress made in deep learning has led to significant achievements in various engineering applications. For instance, convolutional neural networks and transformer networks, two types of feedforward neural networks, are now achieving – and sometimes even surpassing – human accuracy on different visual recognition tasks. However, in this talk, I will demonstrate that these neural networks have limited capacity to solve what appear to be simple visual reasoning problems. Our team has developed a computational neuroscience model of the feedback circuitry found in the visual cortex. The model was built with constraints from the anatomy and physiology of the visual cortex and was shown to account for various visual illusions. This provided computational evidence for a new canonical circuit shared across different visual modalities. I will demonstrate that this computational neuroscience model can be transformed into a modern end-to-end trainable deep recurrent network architecture that addresses some of the weaknesses of current state-of-the-art feedforward networks. This indicates that neuroscience can provide powerful new concepts and techniques for computer science and artificial intelligence.