PDF《Semantic information mediates visual attention during spoken...》

Semantic information mediates visual attention during spoken word recognition in Chinese: Evidence from the printed-word version of the visual-world paradigm Wei Shen1,2 &

Qingqing Qu1 &

Xingshan Li1 Published online:

18 March

2016 # The Psychonomic Society, Inc.

2016 Abstract In the present study, we investigated whether the activation of semantic information during spoken word recog- nition can mediate visual attention'

s deployment to printed Chinese words. We used a visual-world paradigm with printed words, in which participants listened to a spoken target word embedded in a neutral spoken sentence while looking at a visual display of printed words. We examined whether a se- mantic competitor effect could be observed in the printed- word version of the visual-world paradigm. In Experiment 1, the relationship between the spoken target words and the printed words was manipulated so that they were semantically related (a semantic competitor), phonologically related (a pho- nological competitor), or unrelated (distractors). We found that the probability of fixations on semantic competitors was significantly higher than that of fixations on the distractors. In Experiment 2, the orthographic similarity between the spoken target words and their semantic competitors was manipulated to further examine whether the semantic competitor effect was modulated by orthographic similarity. We found significant semantic competitor effects regardless of orthographic simi- larity. Our study not only reveals that semantic information can affect visual attention, it also provides important new in- sights into the methodology employed to investigate the se- mantic processing of spoken words during spoken word recognition using the printed-word version of the visual- world paradigm. Keywords Semantic processing . Printed-word paradigm . Spoken word recognition . Chinese . Eye movements Examining the types of information involved in spoken word recognition and their effects on visual attention can help us better understand how information from the visual and audi- tory modalities interacts to influence eye movements. In the present study, we investigated whether the activation of se- mantic information in spoken word recognition can mediate the deployment of visual attention to printed Chinese words. For our research we utilized the visual-world paradigm, which has been used to study the interplay of cognitive pro- cesses such as language processing and attention that have traditionally been investigated separately (Allopenna, Magnuson, &

Tanenhaus, 1998;

Cooper, 1974;

Tanenhaus, Spivey-Knowlton, Eberhard, &

Sedivy, 1995;

see Henderson &

Ferreira, 2004, for a review). In this paradigm, participants listen to an utterance while looking at a visual display, and their eye movements are tracked. It is well- established that the proportions of eye fixation on objects are constrained by semantic, orthographic, or phonological rela- tions between auditory information and object names. For instance, Allopenna, Magnuson, and Tanenhaus (1998) showed participants a screen with four objects: a target refer- ent ( beaker ), a phonological competitor sharing the initial syllables with the target ( beetle ), a phonological competitor sharing the tail syllables with the target ( speaker ), and an unrelated distractor ( carriage ). The participants were asked to follow the instruction of please pick up the beaker. Their study showed that both types of phonological competitors attracted more fixations than the unrelated distractors. * Qingqing Qu quqq@psych.ac.cn * Xingshan Li lixs@psych.ac.cn