Coordinating action and language

Abstract

The overarching aim of this chapter is to highlight the strong interfacing between linguistic and motor systems, focusing especially on the degree to which manipulations of a visual context interact with linguistic manipulations in order to modulate the properties of manual movements elicited in response to a linguistic signal. First, we provide a review of recent work in the domain of embodied cognition that highlights the engagement of motor systems during language comprehension, including gestures, eye movements, and postural sway. In the context of this body of work, we then focus on the notion that continuous and non-ballistic computer-mouse movement trajectories can serve as a viable motoric index of various aspects of language processing. The results of studies that have recorded mouse-movement trajectories in visual-world displays will be reviewed, with the goal of highlighting the methodological benefits of data contained in the recorded trajectories, along with their theoretical consequences. New data will be presented from a study in which both eye- and hand-movements were simultaneously recorded, speaking to the relationship between the two movement-related dependent measures. Finally, we will highlight the degree to which mouse-movements around a visual display can be informative for domains other than syntactic processing, focusing on new data from research designed to investigate the processing of verbal aspect.