The facilitative effect of gestures on the neural processing of semantic complexity in a continuous narrative
© 2019 Elsevier Inc. Gestures are elemental components of social communication and aid comprehension of verbal messages; however, little is known about the potential role of gestures in facilitating processing of semantic complexity in an ecologically valid setting. The goal of this study was to investigate whether cognitive load, as indexed by semantic complexity, is modulated by the presentation of gestures accompanying speech. Twenty healthy participants watched 16 video clips of a short narrative while instructed to carefully listen to and watch the narrator while functional magnetic resonance imaging (fMRI) data were acquired. The videos contained passages with and without various co-s…
Modality-specific dysfunctional neural processing of social-abstract and non-social-concrete information in schizophrenia
Highlights • Social/non-social information processing in three modalities was investigated in SZ. • SZ showed reduced activation for social information only in gesture modality. • Reduced activation in SZ was observed for non-social information only in speech. • Neural Neural processing in bimodal condition is not different between patients and controls.
Evidence for gesture-speech mismatch detection impairments in schizophrenia.
Patients with schizophrenia suffer from impairments in the perception and production of gestures. The extent to which patients can access the semantic association between speech and co-verbal gestures in concrete or abstract/metaphorical meaning contexts is unknown. We investigated 1) how patients differ from controls in gesture matching performance, 2) how performance differs in the context of abstract versus concrete meaning, and 3) whether formal thought disorder (FTD) symptom severity predicts task impairment. Forty-five patients with schizophrenia spectrum disorders (two subgroups, mild and severe) took part in this study. Participants were presented with video clips, each showing an a…
Feeling addressed! The neural processing of social communicative cues in patients with major depression
Abstract The feeling of being addressed is the first step in a complex processing stream enabling successful social communication. Social impairments are a relevant characteristic of patients with major depressive disorder (MDD). Here, we investigated a mechanism which—if impaired—might contribute to withdrawal or isolation in MDD, namely, the neural processing of social cues such as body orientation and gesture. During funtional magnetic resonance imaging (fMRI) data acquisition, 33 patients with MDD and 43 healthy control subjects watched video clips of a speaking actor: one version with a gesture accompanying the speech and one without gesture. Videos were filmed simultaneously from two …
Neural Basis of Speech-Gesture Mismatch Detection in Schizophrenia Spectrum Disorders
AbstractPatients with schizophrenia spectrum disorders (SSD) exhibit an aberrant perception and comprehension of abstract speech-gesture combinations associated with dysfunctional activation of the left inferior frontal gyrus (IFG). Recently, a significant deficit of speech-gesture mismatch detection was identified in SSD, but the underlying neural mechanisms have not yet been examined. A novel mismatch-detection fMRI paradigm was implemented manipulating speech-gesture abstractness (abstract/concrete) and relatedness (related/unrelated). During fMRI data acquisition, 42 SSD patients (schizophrenia, schizoaffective disorder, or other non-organic psychotic disorder [ICD-10: F20, F25, F28; DS…
The EEG and fMRI signatures of neural integration: An investigation of meaningful gestures and corresponding speech
Abstract One of the key features of human interpersonal communication is our ability to integrate information communicated by speech and accompanying gestures. However, it is still not fully understood how this essential combinatory process is represented in the human brain. Functional magnetic resonance imaging (fMRI) studies have unanimously attested the relevance of activation in the posterior superior temporal sulcus/middle temporal gyrus (pSTS/MTG), while electroencephalography (EEG) studies have shown oscillatory activity in specific frequency bands to be associated with multisensory integration. In the current study, we used fMRI and EEG to separately investigate the anatomical and o…
Modality-specific dysfunctional neural processing of social and non-social information in schizophrenia
Abstract Background Schizophrenia (SZ) is characterized by marked social dysfunctions encompassing potential deficits in the processing of social and non-social information, especially in everyday settings where multiple modalities are present. To date, the neurobiological basis of these deficits remains elusive. Methods In a functional magnetic resonance imaging (fMRI) study, 17 patients with schizophrenia or schizoaffective disorder, and 18 matched controls watched videos of an actor speaking, gesturing (unimodal), and both speaking and gesturing (bimodal) about social or non-social events in a naturalistic way. Participants had to judge whether each video contains person-related (social)…