The Timecourse of Sentence Processing in the Brain

This chapter discusses the current state of the art with regard to the timecourse of sentence processing in the brain. It outlines the challenges associated with studying timecourse information at the sentence level from a neurobiological perspective and describes competing theoretical and empirical perspectives in this domain. In addition to drawing on findings from neurophysiological methods (electroencephalography [EEG]; magnetoencephalography [MEG]), insights from eye movement measures during natural reading are also taken into account. The chapter concludes that while we are currently unable to make absolute claims about the timecourse of sentence processing from a neurobiological pers…

Exploring the nature of the ‘subject’-preference: Evidence from the online comprehension of simple sentences in Mandarin Chinese

In two visual ERP studies, we investigated whether Mandarin Chinese shows a subject-preference in spite of the controversial status of grammatical relations in this language. We compared ERP responses at the position of the verb and the second NP in object-verb-subject (OVS) and subject-verb-object (SVO) structures. While SVO is the basic word order in Chinese and OV with subject-drop is possible, OVS is strongly dispreferred. At the position of the verb, which disambiguated towards an object or a subject reading of NP1, Experiment 1 revealed an N400 for both subject-initial control conditions in comparison with the critical object-initial condition. Experiment 2 showed that this result was…

Agreement or no agreement. ERP correlates of verb agreement violation in German Sign Language

Previous studies on agreement violation in sign languages report neurophysiological responses similar to those observed for spoken languages. In contrast, the two current event-related potential studies (ERP) on agreement violations in German Sign Language sentences present results that allow for an alternative explanation. In experiment A, we investigated the processing of agreement verbs ending in an unspecified location different to the location associated with the referent. Incorrect agreement verbs engendered a posterior positivity effect (220–570 ms post nonmanual cues) and a left anterior effect (300–600 ms post the subsequent sign onset). In experiment B, we investigated a violation…

Animacy-based predictions in language comprehension are robust: contextual cues modulate but do not nullify them

Couldn't a humble coconut hurt a gardener? At least in the first instance, the brain seems to assume that it should not: we perceive inanimate entities such as coconuts as poor event instigators ("Actors"). Ideally, entities causing a change in another entity should be animate and this assumption not only influences event perception but also carries over to language comprehension. We present three auditory event-related brain potential (ERP) studies on the processing of inanimate and animate subjects and objects in simple transitive sentences in Tamil. ERP responses were measured at the second argument (event participant) in all three studies. Experiment 1 employed all possible animacy comb…

The N400 as a correlate of interpretively relevant linguistic rules: evidence from Hindi.

Classical views on the electrophysiology of language assume that different event-related potential (ERP) components index distinct linguistic subdomains. Hence, left-anterior negativities are often viewed as correlates of rule-based linguistic knowledge, whereas centro-parietal negativities (N400s) are taken to reflect (non-rule-based) semantic memory or aspects of lexical-semantic predictability. The present ERP study of case marking in Hindi challenges this clear-cut dichotomy. Though determined by a grammatical rule, the choice of subject case in Hindi is also interpretively relevant as it constrains the range of possible interpretations of the subject. For incorrect subject cases, we ob…

Prosodic Phrasing and Transitivity in Head-Final Sentence Comprehension – ERP Evidence from German Ambiguous DPs

In this article, we present an ERP study investigating the neurophysiological correlates of the interplay between prosodic phrasing and case information in head-final sentence processing. We examine German DP1-DP2-V constructions, in which the second DP can either be interpreted as a possessive modifier of the first DP (one-participant reading), or as a verbal argument (two-participant reading). An N400 on the determiner of the second DP is observed when either case information or prosodic phrasing biased toward the establishment of a two-participant reading, thus reflecting local processing costs associated with the introduction of a transitive event. By contrast, when the DPs are integrat…

Response to Skeide and Friederici: The myth of the uniquely human “direct” dorsal pathway

In their comment on our recent article [1], Skeide and Friederici [2] claim ‘that some important data not discussed by Bornkessel-Schlesewsky et al. strongly support the view that there are clear qualitative, and not merely quantitative, differences between [human and nonhuman primate] species with respect to both the intrinsic functional connectivity of frontal and temporal cortices, and their direct structural connection via a dorsal white matter fiber tract.’ This obviously refers to work by Friederici and colleagues [3] emphasizing the functional importance of a direct connection between the posterior superior temporal cortex (pSTC) and Brodmann area (BA) 44 in humans, and its absence i…

Subjective impressions do not mirror online reading effort: concurrent EEG-eyetracking evidence from the reading of books and digital media

In the rapidly changing circumstances of our increasingly digital world, reading is also becoming an increasingly digital experience: electronic books (e-books) are now outselling print books in the United States and the United Kingdom. Nevertheless, many readers still view e-books as less readable than print books. The present study thus used combined EEG and eyetracking measures in order to test whether reading from digital media requires higher cognitive effort than reading conventional books. Young and elderly adults read short texts on three different reading devices: a paper page, an e-reader and a tablet computer and answered comprehension questions about them while their eye movemen…

Meaningful physical changes mediate lexical-semantic integration: top-down and form-based bottom-up information sources interact in the N400

Models of how the human brain reconstructs an intended meaning from a linguistic input often draw upon the N400 event-related potential (ERP) component as evidence. Current accounts of the N400 emphasise either the role of contextually induced lexical preactivation of a critical word (Lau, Phillips,& Poeppel, 2008) or the ease of integration into the overall discourse context including a wide variety of influencing factors (Hagoort & van Berkum, 2007). The present ERP study challenges both types of accounts by demonstrating a contextually independent and purely form-based bottom-up influence on the N400: the N400 effect for implausible sentence-endings was attenuated when the critical sente…

Age-Related Changes in Predictive Capacity Versus Internal Model Adaptability: Electrophysiological Evidence that Individual Differences Outweigh Effects of Age

Hierarchical predictive coding has been identified as a possible unifying principle of brain function, and recent work in cognitive neuroscience has examined how it may be affected by age related changes. Using language comprehension as a test case, the present study aimed to dissociate age-related changes in prediction generation versus internal model adaptation following a prediction error. Event related brain potentials (ERPs) were measured in a group of older adults (60-81 years; n = 40) as they read sentences of the form "The opposite of black is white/yellow/nice." Replicating previous work in young adults, results showed a target related P300 for the expected antonym ("white"; an eff…

The role of animacy in the real time comprehension of Mandarin Chinese: Evidence from auditory event-related brain potentials.

Two auditory ERP studies examined the role of animacy in sentence comprehension in Mandarin Chinese by comparing active and passive sentences in simple verb-final (Experiment 1) and relative clause constructions (Experiment 2). In addition to the voice manipulation (which modulated the assignment of actor and undergoer roles to the arguments), both arguments were either animate or inanimate. This allowed us to examine the interplay of animacy with thematic interpretation. In Experiment 1, we observed no effect of animacy at NP1, but N400 effects for inanimate actor arguments in second position. This result mirrors previous findings in German, thus suggesting that an initial undergoer univer…

Electrophysiology Reveals the Neural Dynamics of Naturalistic Auditory Language Processing: Event-Related Potentials Reflect Continuous Model Updates.

The recent trend away from ANOVA-based analyses places experimental investigations into the neurobiology of cognition in more naturalistic and ecologically valid designs within reach. Using mixed-effects models for epoch-based regression, we demonstrate the feasibility of examining event-related potentials (ERPs), and in particular the N400, to study the neural dynamics of human auditory language processing in a naturalistic setting. Despite the large variability between trials during naturalistic stimulation, we replicated previous findings from the literature: the effects of frequency, animacy, word order and find previously unexplored interaction effects. This suggests a new perspective …

Sentence judgments and the grammar of poetry: Linking linguistic structure and poetic effect

The present article aims to show that the elicitation of intuitive literary-aesthetic sentence judgments taps into readers’ poetry-specific linguistic register, and how such judgment methods can be used to support and constrain future theory formation in experimental poetics. In two experiments, we examined effects of deviant and parallelistic linguistic features on readers’ grammatical and literary-aesthetic evaluation of single sentences.In Experiment 1, participants rated carefully selected and modified lines of German poetry for either acceptability or poeticity (n = 40 each) on a 7-point scale; original lines featured grammatical deviations that were absent in modified versions. All in…

The Role of Prominence Information in the Real-Time Comprehension of Transitive Constructions: A Cross-Linguistic Approach

Approaches to language processing have traditionally been formulated with reference to general cognitive concepts (e.g. working memory limitations) or have based their representational assumptions on concepts from generative linguistic theory (e.g. structure determines interpretation). Thus, many well-established generalisations about language that have emerged from cross-linguistic/typological research have not as yet had a major influence in shaping ideas about online processing. Here, we examine the viability of using typologically motivated concepts to account for phenomena in online language comprehension. In particular, we focus on the comprehension of simple transitive sentences (i.e…

Think globally: Cross-linguistic variation in electrophysiological activity during sentence comprehension

This paper demonstrates systematic cross-linguistic differences in the electrophysiological correlates of conflicts between form and meaning (“semantic reversal anomalies”). These engender P600 effects in English and Dutch (e.g. Kolk et al., 2003 ; Kuperberg et al., 2003), but a biphasic N400 – late positivity pattern in German (Schlesewsky and Bornkessel-Schlesewsky, 2009), and monophasic N400 effects in Turkish (Experiment 1) and Mandarin Chinese (Experiment 2). Experiment 3 revealed that, in Icelandic, semantic reversal anomalies show the English pattern with verbs requiring a position-based identification of argument roles, but the German pattern with verbs requiring a case-based identi…

Reconciling time, space and function: a new dorsal-ventral stream model of sentence comprehension.

We present a new dorsal–ventral stream framework for language comprehension which unifies basic neurobiological assumptions (Rauschecker & Scott, 2009) with a cross-linguistic neurocognitive sentence comprehension model (eADM; Bornkessel & Schlesewsky, 2006). The dissociation between (time-dependent)syntactic structure-building and (time-independent) sentence interpretation assumed within thee ADM provides a basis for the division of labour between the dorsal and ventral streams in comprehension.We posit that the ventral stream performs time-independent unifications of conceptual schemata,serving to create auditory objects of increasing complexity. The dorsal stream engages in the time-depe…

Event-related potentials reflecting the processing of phonological constraint violations

How are violations of phonological constraints processed in word comprehension? The present article reports the results of an event-related potentials (ERP) study on a phonological constraint of German that disallows identical segments within a syllable or word (CC(i)VC(i)). We examined three types of monosyllabic late positive CCVC words: (a) existing words [see text], (b) wellformed novel words [see text] and component (c) illformed novel words [see text] as instances of Obligatory Contour Principle non-word (OCP) violations. Wellformed and illformed novel words evoked an N400 effect processing in comparison to existing words. In addition, illformed words produced an enhanced late posteri…

The Argument Dependency Model

This chapter summarizes the architecture of the extended Argument Dependency Model (eADM), a model of language comprehension that aspires toward neurobiological plausibility. It combines design principles from neurobiology with insights on cross-linguistic diversity. Like other current models, the eADM posits that auditory language processing proceeds along two distinct streams in the brain emanating from auditory cortex: the antero-ventral and postero-dorsal streams. Both streams are organized hierarchically and information processing takes place in a cascaded fashion. Each stream has functionally unified computational properties congruent with its role in primate audition. While the dorsa…

An alternative perspective on “semantic P600” effects in language comprehension

Abstract The literature on the electrophysiology of language comprehension has recently seen a very prominent discussion of “semantic P600” effects, which have been observed, for example, in sentences involving an implausible thematic role assignment to an argument that would be a highly plausible filler for a different thematic role of the same verb. These findings have sparked a discussion about underlying properties of the language comprehension architecture, as they have generally been viewed as a challenge to established models of language processing and specifically to the notion that syntax precedes semantics in the comprehension process. In this paper, we review the literature on se…

Agreement or no agreement. ERP correlates of verb agreement violation in German Sign Language

Previous studies on agreement violation in sign languages report neurophysiological responses similar to those observed for spoken languages. In contrast, the two current event-related potential studies (ERP) on agreement violations in German Sign Language sentences present results that allow for an alternative explanation. In experiment A, we investigated the processing of agreement verbs ending in an unspecified location different to the location associated with the referent. Incorrect agreement verbs engendered a posterior positivity effect (220–570 ms post nonmanual cues) and a left anterior effect (300–600 ms post the subsequent sign onset). In experiment B, we investigated a violation…

Yes, you can? A speaker’s potency to act upon his words orchestrates early neural responses to message-level meaning

Evidence is accruing that, in comprehending language, the human brain rapidly integrates a wealth of information sources-including the reader or hearer's knowledge about the world and even his/her current mood. However, little is known to date about how language processing in the brain is affected by the hearer's knowledge about the speaker. Here, we investigated the impact of social attributions to the speaker by measuring event-related brain potentials while participants watched videos of three speakers uttering true or false statements pertaining to politics or general knowledge: a top political decision maker (the German Federal Minister of Finance at the time of the experiment), a well…

Lexical prediction via forward models: N400 evidence from German Sign Language

Models of language processing in the human brain often emphasize the prediction of upcoming input for example in order to explain the rapidity of language understanding. However,the precise mechanisms of prediction are still poorly understood. Forward models,which draw upon the language production system to setup expectations during comprehension, provide a promising approach in this regard. Here, we present an event- related potential (ERP) study on German Sign Language (DGS) which tested the hypotheses of a forward model perspective on prediction. Sign languages involve relatively long transition phases between one sign and the next, which should be anticipated as part of a forward model-…

The Role of Gamma Oscillations During Integration of Metaphoric Gestures and Abstract Speech

Metaphoric (MP) co-speech gestures are commonly used during daily communication. They communicate about abstract information by referring to gestures that are clearly concrete (e.g., raising a hand for "the level of the football game is high"). To understand MP co-speech gestures, a multisensory integration at semantic level is necessary between abstract speech and concrete gestures. While semantic gesture-speech integration has been extensively investigated using functional magnetic resonance imaging, evidence from electroencephalography (EEG) is rare. In the current study, we set out an EEG experiment, investigating the processing of MP vs. iconic (IC) co-speech gestures in different cont…

Minimality as vacuous distinctness: Evidence from cross-linguistic sentence comprehension

Abstract Psycholinguistic theorising has long been shaped by the assumption that the processing system endeavours to minimise structures/relations during online comprehension. Within the scope of a recent cross-linguistic, neurocognitive model of sentence comprehension (Bornkessel and Schlesewsky, 2006), we also proposed that the assumption of a very general ‘Minimality’ principle can account for a variety of psycholinguistic findings from a range of languages. In the present paper, we review empirical evidence for this notion of Minimality, before going on to discuss its limitations. On the basis of this discussion, we propose that, rather than constituting an independent processing princi…

Discovering prominence and its role in language processing: an individual (differences) approach

Abstract It has been suggested that, during real time language comprehension, the human language processing system attempts to identify the argument primarily responsible for the state of affairs (the “actor”) as quickly and unambiguously as possible. However, previous work on a prominence (e.g. animacy, definiteness, case marking) based heuristic for actor identification has suffered from underspecification of the relationship between different cue hierarchies. Qualitative work has yielded a partial ordering of many features (e.g. MacWhinney et al. 1984), but a precise quantification has remained elusive due to difficulties in exploring the full feature space in a particular language. Feat…

Prominence Facilitates Ambiguity Resolution: On the Interaction Between Referentiality, Thematic Roles and Word Order in Syntactic Reanalysis

In two eye-tracking experiments, we investigated the relationship between the subject preference in the resolution of subject-object ambiguities in German embedded clauses and semantic word order constraints (i.e., prominence hierarchies relating to the specificity/referentiality of noun phrases, case assignment and thematic role assignment). Our central research question concerned the timecourse with which prominence information is used and particularly whether it modulates the subject preference. In both experiments, we replicated previous findings of reanalysis effects for object-initial structures. Our findings further suggest that noun phrase prominence does not alter initial parsing s…

Conflicts in language processing: A new perspective on the N400-P600 distinction

Conflicts in language processing often correlate with late positive event-related brain potentials (ERPs), particularly when they are induced by inconsistencies between different information types (e.g. syntactic and thematic/plausibility information). However, under certain circumstances, similar sentence-level interpretation conflicts (inanimate subjects) engender negativity effects (N400s) instead. The present ERP study was designed to shed light on this inconsistency. In previous studies showing monophasic positivities (P600s), the conflict was irresolvable and induced via a verb. whereas N400s were elicited by resolvable, argument-induced conflicts. Here, we therefore examined irresolv…

Implementation is crucial but must be neurobiologically grounded

The P600-as-P3 hypothesis revisited: single-trial analyses reveal that the late EEG positivity following linguistically deviant material is reaction time aligned.

The P600, a late positive ERP component following linguistically deviant stimuli, is commonly seen as indexing structural, high-level processes, e.g. of linguistic (re)analysis. It has also been identified with the P3 (P600-as-P3 hypothesis), which is thought to reflect a systemic neuromodulator release facilitating behavioural shifts and is usually response time aligned. We investigated single-trial alignment of the P600 to response, a critical prediction of the P600-as-P3 hypothesis. Participants heard sentences containing morphosyntactic and semantic violations and responded via a button press. The elicited P600 was perfectly response aligned, while an N400 following semantic deviations …

Sentence-Level Effects of Literary Genre: Behavioral and Electrophysiological Evidence

The current study used event-related brain potentials (ERPs) and behavioral measures to examine effects of genre awareness on sentence processing and evaluation. We hypothesized that genre awareness modulates effects of genre-typical manipulations. We manipulated instructions between participants, either specifying a genre (poetry) or not (neutral). Sentences contained genre-typical variations of semantic congruency (congruent/incongruent) and morpho-phonological features (archaic/contemporary inflections). Offline ratings of meaningfulness (n = 64/group) showed higher average ratings for semantically incongruent sentences in the poetry vs. neutral condition. ERPs during sentence reading (n…

The exceptional nature of the first person in natural story processing and the transfer of egocentricity

Human language enables us to externalise self-internal information (e.g. emotions or beliefs that are not readily accessible to others). Thus, language bridges the gap between the self and the other (e.g. Frith and Frith, 2010) in a way that possibly no other communication system can provide. In many languages, the difference between the self and others is directly reflected in the distinction between first (“I”), second (“you”) and third person (“he, she”) marking. In the present study, we compared ERPs to first, second and third person pronouns during the comprehension of an audio-book version of The Little Prince. Our results revealed a strong P300 response following first person pronoun…

Linguistische Daten aus experimentellen Umgebungen: Eine multiexperimentelle und multimodale Perspektive

Word order and Broca’s region: Evidence for a supra-syntactic perspective

It has often been suggested that the role of Broca's region in sentence comprehension can be explained with reference to general cognitive mechanisms (e.g. working memory, cognitive control). However, the (language-related) basis for such proposals is often restricted to findings on English. Here, we argue that an extension of the database to other languages can shed new light on the types of mechanisms that an adequate account of Broca's region should be equipped to deal with. This becomes most readily apparent in the domain of word order variations, which we examined in German verb-final sentences using event-related fMRI. Our results showed that activation in the pars opercularis--a core…

Prominence vs. aboutness in sequencing: a functional distinction within the left inferior frontal gyrus

Prior research on the neural bases of syntactic comprehension suggests that activation in the left inferior frontal gyrus (lIFG) correlates with the processing of word order variations. However, there are inconsistencies with respect to the specific subregion within the IFG that is implicated by these findings: the pars opercularisor the pars triangularis. Here, we examined the hypothesis that the dissociation between parsopercularis and pars triangularis activation may reflect functional differences between clause-medial and clause-initial word order permutations, respectively. To this end, we directly compared clause-medial and clause-initial object-before-subject orders in German in a wi…

Cross-linguistic variation in the neurophysiological response to semantic processing: Evidence from anomalies at the borderline of awareness

The N400 event-related brain potential (ERP) has played a major role in the examination of how the human brain processes meaning. For current theories of the N400, classes of semantic inconsistencies which do not elicit N400 effects have proven particularly influential. Semantic anomalies that are difficult to detect are a case in point ("borderline anomalies", e.g. "After an air crash, where should the survivors be buried?"), engendering a late positive ERP response but no N400 effect in English (Sanford, Leuthold, Bohan, & Sanford, 2011). In three auditory ERP experiments, we demonstrate that this result is subject to cross-linguistic variation. In a German version of Sanford and colleagu…

The role of animacy in online argument interpretation in Mandarin Chinese

The present event-related brain potential (ERPs) study demonstrates that online argument interpretation in verb-final structures in Mandarin Chinese is modulated by two factors: a preference for Undergoer-before-Actor orders and a preference for animate Actor arguments. Participants listened to sentences with NP(animate)-NP(inanimate)-verb or NP(inanimate)-NP(animate)-verb orders embedded in minimal contexts. Sentences were disambiguated towards either an Actor-initial or an Undergoer-initial order by the clause-final verb. Between 450 and 700 ms post verb onset, we observed an anterior negativity for sentences violating both preferences (inanimate-Actor-initial structures) vs. sentences fu…

Text type attribution modulates pre-stimulus alpha power in sentence reading

Prior knowledge and context-specific expectations influence the perception of sensory events, e.g., speech, as well as complex higher-order cognitive operations like text reading. Here, we focused on pre-stimulus neural activity during sentence reading to examine text type-dependent attentional bias in anticipation of written stimuli, capitalizing on the functional relevance of brain oscillations in the alpha (8–12 Hz) frequency range. Two sex- and age-matched groups of participants (n = 24 each) read identical sentences on a screen at a fixed per-constituent presentation rate while their electroencephalogram was recorded; the groups were differentially instructed to read “sentences” (genre…

Neurobiological roots of language in primate audition : common computational properties

Here, we present a new perspective on an old question: how does the neurobiology of human language relate to brain systems in nonhuman primates? We argue that higher-order language combinatorics, including sentence and discourse processing, can be situated in a unified, cross-species dorsal-ventral streams architecture for higher auditory processing, and that the functions of the dorsal and ventral streams in higher-order language processing can be grounded in their respective computational properties in primate audition. This view challenges an assumption, common in the cognitive sciences, that a nonhuman primate model forms an inherently inadequate basis for modeling higher-level language…

Preface: The neurobiology of syntax.

The neural mechanisms of word order processing revisited: electrophysiological evidence from Japanese.

We present two ERP studies on the processing of word order variations in Japanese, a language that is suited to shedding further light on the implications of word order freedom for neurocognitive approaches to sentence comprehension. Experiment 1 used auditory presentation and revealed that initial accusative objects elicit increased processing costs in comparison to initial subjects (in the form of a transient negativity) only when followed by a prosodic boundary. A similar effect was observed using visual presentation in Experiment 2, however only for accusative but not for dative objects. These results support a relational account of word order processing, in which the costs of comprehen…

Parafoveal versus foveal N400s dissociate spreading activation from contextual fit.

Using concurrent electroencephalogram and eye movement measures to track natural reading, this study shows that N400 effects reflecting predictability are dissociable from those owing to spreading activation. In comparing predicted sentence endings with related and unrelated unpredicted endings in antonym constructions ('the opposite of black is white/yellow/nice'), fixation-related potentials at the critical word revealed a predictability-based N400 effect (unpredicted vs. predicted words). By contrast, event-related potentials time locked to the last fixation before the critical word showed an N400 only for the nonrelated unpredicted condition (nice). This effect is attributed to a parafo…

Domain-general neural correlates of dependency formation: Using complex tones to simulate language

There is an ongoing debate whether the P600 event-related potential component following syntactic anomalies reflects syntactic processes per se, or if it is an instance of the P300, a domain-general ERP component associated with attention and cognitive reorientation. A direct comparison of both components is challenging because of the huge discrepancy in experimental designs and stimulus choice between language and ‘classic’ P300 experiments. In the present study, we develop a new approach to mimic the interplay of sequential position as well as categorical and relational information in natural language syntax (word category and agreement) in a non-linguistic target detection paradigm using…

Agreement or no agreement. ERP correlates of verb agreement violation in German Sign Language

Previous studies on agreement violation in sign languages report neurophysiological responses similar to those observed for spoken languages. In contrast, the two current event-related potential studies (ERP) on agreement violations in German Sign Language sentences present results that allow for an alternative explanation. In experiment A, we investigated the processing of agreement verbs ending in an unspecified location different to the location associated with the referent. Incorrect agreement verbs engendered a posterior positivity effect (220–570 ms post nonmanual cues) and a left anterior effect (300–600 ms post the subsequent sign onset). In experiment B, we investigated a violation…