Alon Hafri
Department of Linguistics and Cognitive Science
University of Delaware

Where language and vision meet

There may be no two domains more central to cognitive science than language and visual perception. They are typically studied in isolation, yet at some level, they must connect. How? In my talk, I’ll discuss work suggesting that language and vision share core representational principles and conceptual content. First, I’ll present evidence that visual perception automatically extracts relational properties such as Agent, Patient, supporting, and containing, in ways that instantiate principles that are typically associated with linguistic representations (such as abstractness, categoricality, and structure). Second, I’ll explore the consequences of such parallels for how language and vision connect in the mind. Using the concept of symmetry as a case study, I will show that language and vision link to the same modality-neutral conceptual representations, as revealed by a striking correspondence between linguistic and perceptual judgments of the same visual displays. Finally, I’ll discuss the implications of these parallels and connections for areas as varied as language acquisition and scene perception.