Semi-blind Independent Component Analysis of functional MRI elicited by continuous listening to music

This study presents a method to analyze blood-oxygen-level-dependent (BOLD) functional magnetic resonance imaging (tMRI) signals associated with listening to continuous music. Semi-blind independent component analysis (ICA) was applied to decompose the tMRI data to source level activation maps and their respective temporal courses. The unmixing matrix in the source separation process of ICA was constrained by a variety of acoustic features derived from the piece of music used as the stimulus in the experiment. This allowed more stable estimation and extraction of more activation maps of interest compared to conventional ICA methods.

Tag2Risk: Harnessing social music tags for characterizing depression risk

Musical preferences have been considered a mirror of the self. In this age of Big Data, online music streaming services allow us to capture ecologically valid music listening behavior and provide a rich source of information to identify several user-specific aspects. Studies have shown musical engagement to be an indirect representation of internal states including internalized symptomatology and depression. The current study aims at unearthing patterns and trends in the individuals at risk for depression as it manifests in naturally occurring music listening behavior. Mental well-being scores, musical engagement measures, and listening histories of Last.fm users (N=541) were acquired. Soci…

Capturing the musical brain with Lasso: Dynamic decoding of musical features from fMRI data.

We investigated neural correlates of musical feature processing with a decoding approach. To this end, we used a method that combines computational extraction of musical features with regularized multiple regression (LASSO). Optimal model parameters were determined by maximizing the decoding accuracy using a leave-one-out cross-validation scheme. The method was applied to functional magnetic resonance imaging (fMRI) data that were collected using a naturalistic paradigm, in which participants' brain responses were recorded while they were continuously listening to pieces of real music. The dependent variables comprised musical feature time series that were computationally extracted from the…

Musical expertise modulates functional connectivity of limbic regions during continuous music listening.

Music is known to be an important facet of all human cultures (Merriam, 1964). Listening to music in order to influence moods, evoke strong emotions, and derive pleasure is becoming increasingly common, especially in this day and age when access to music is easy and quick. In recent years, exploring the neural correlates of musical emotions has attracted the attention of neuroscientists (Brattico & Pearce, 2013; Koelsch, Fritz, v. Cramon, Muller, & Friederici, 2006). However, the majority of these studies have not accounted for the effect of musical expertise, despite increasing evidence of structural and functional differences between musicians and nonmusicians, particularly in the regions…

On application of kernel PCA for generating stimulus features for fMRI during continuous music listening

Abstract Background There has been growing interest towards naturalistic neuroimaging experiments, which deepen our understanding of how human brain processes and integrates incoming streams of multifaceted sensory information, as commonly occurs in real world. Music is a good example of such complex continuous phenomenon. In a few recent fMRI studies examining neural correlates of music in continuous listening settings, multiple perceptual attributes of music stimulus were represented by a set of high-level features, produced as the linear combination of the acoustic descriptors computationally extracted from the stimulus audio. New method fMRI data from naturalistic music listening experi…

A functional MRI study of happy and sad emotions in music with and without lyrics

Musical emotions, such as happiness and sadness, have been investigated using instrumental music devoid of linguistic content. However, pop and rock, the most common musical genres, utilize lyrics for conveying emotions. Using participants’ self-selected musical excerpts, we studied their behavior and brain responses to elucidate how lyrics interact with musical emotion processing, as reflected by emotion recognition and activation of limbic areas involved in affective experience. We extracted samples from subjects’ selections of sad and happy pieces and sorted them according to the presence of lyrics. Acoustic feature analysis showed that music with lyrics differed from music without lyric…

The chronnectome of musical beat

Keeping time is fundamental for our everyday existence. Various isochronous activities, such as locomotion, require us to use internal timekeeping. This phenomenon comes into play also in other human pursuits such as dance and music. When listening to music, we spontaneously perceive and predict its beat. The process of beat perception comprises both beat inference and beat maintenance, their relative importance depending on the salience of beat in the music. To study functional connectivity associated with these processes in a naturalistic situation, we used functional magnetic resonance imaging to measure brain responses of participants while they were listening to a piece of music contai…

The reliability of continuous brain responses during naturalistic listening to music

Low-level (timbral) and high-level (tonal and rhythmical) musical features during continuous listening to music, studied by functional magnetic resonance imaging (fMRI), have been shown to elicit large-scale responses in cognitive, motor, and limbic brain networks. Using a similar methodological approach and a similar group of participants, we aimed to study the replicability of previous findings. Participants' fMRI responses during continuous listening of a tango Nuevo piece were correlated voxelwise against the time series of a set of perceptually validated musical features computationally extracted from the music. The replicability of previous results and the present study was assessed b…

On application of kernel PCA for generating stimulus features for fMRI during continuous music listening

Background There has been growing interest towards naturalistic neuroimaging experiments, which deepen our understanding of how human brain processes and integrates incoming streams of multifaceted sensory information, as commonly occurs in real world. Music is a good example of such complex continuous phenomenon. In a few recent fMRI studies examining neural correlates of music in continuous listening settings, multiple perceptual attributes of music stimulus were represented by a set of high-level features, produced as the linear combination of the acoustic descriptors computationally extracted from the stimulus audio. New method fMRI data from naturalistic music listening experiment were…

Connectivity Patterns During Music Listening: Evidence for Action-Based Processing in Musicians

Musical expertise is visible both in the morphology and functionality of the brain. Recent research indicates that functional integration between multi-sensory, somato-motor, default-mode (DMN), and salience (SN) networks of the brain differentiates musicians from non-musicians during resting state. Here, we aimed at determining whether brain networks differentially exchange information in musicians as opposed to non-musicians during naturalistic music listening. Whole-brain graph-theory analyses were performed on participants' fMRI responses. Group-level differences revealed that musicians' primary hubs comprised cerebral and cerebellar sensorimotor regions whereas non-musicians' dominant …

Maladaptive music listening strategies are modulated by individual traits

Music listening is a great resource for mental well-being, pleasure, and self-regulation, but it may also be maladaptive. Depression, for instance, has been shown to relate to music use that is characterized by rumination, avoidance, and mood worsening. However, we know little of the role of individual differences in such maladaptive music use. Hence, this study focused on examining the role of personality, empathic traits, emotional contagion, and the types of musical reward as predictors of maladaptive music listening. Participants ( N = 318) answered an online survey comprising measures for the abovementioned traits in addition to the Healthy-Unhealthy Music Scale (HUMS) as a measure of…

Trait empathy modulates music-related functional connectivity

It has been well established through behavioural studies that empathy significantly influences music perception. Such individual differences typically manifest as variability in whole brain functional connectivity patterns. To date, nobody has examined the modulatory effect of empathy on functional connectivity patterns during continuous music listening. In the present study, we seek to investigate the global and local connectivity patterns of 36 participants whose fMRI scanning was done by employing the naturalistic paradigm wherein they listened to a continuous piece of music. We used graph-based measures of functional connectivity to identify how cognitive and affective components of emp…

sj-pdf-1-pom-10.1177_03057356211065061 ��� Supplemental material for Maladaptive music listening strategies are modulated by individual traits

Supplemental material, sj-pdf-1-pom-10.1177_03057356211065061 for Maladaptive music listening strategies are modulated by individual traits by Vinoo Alluri, Anant Mittal, Azhagammal SC, Jonna K. Vuoskoski and Suvi Saarikallio in Psychology of Music

Predicting Individual Differences from Brain Responses to Music using Functional Network Centrality

Individual differences are known to modulate brain responses to music. Recent neuroscience research suggests that each individual has unique and fundamentally stable functional brain connections irrespective of the task they perform. 77 participants’ functional Magnetic Resonance Imaging (fMRI) responses were measured while continuously listening to music. Using a graph-theory-based approach, we modeled whole-brain functional connectivity. We then calculate voxel-wise eigenvector centrality and subsequently use it to classify gender and musical expertise using binary Support Vector Machine (SVM). We achieved a cross-validated classification accuracy of 97% and 96% for gender and musical exp…

Decoding Individual differences and musical preference via music-induced movement.

AbstractMovement is a universal response to music, with dance often taking place in social settings. Although previous work has suggested that socially relevant information, such as personality and gender, are encoded in dance movement, the generalizability of previous work is limited. The current study aims to decode dancers’ gender, personality traits, and music preference from music-induced movements. We propose a method that predicts such individual difference from free dance movements, and demonstrate the robustness of the proposed method by using two data sets collected using different musical stimuli. In addition, we introduce a novel measure to explore the relative importance of dif…

It's Sad but I Like It The Neural Dissociation Between Musical Emotions and Liking in Experts and Laypersons

Emotion-related areas of the brain, such as the medial frontal cortices, amygdala, and striatum, are activated during listening to sad or happy music as well as during listening to pleasurable music. Indeed, in music, like in other arts, sad and happy emotions might co-exist and be distinct from emotions of pleasure or enjoyment. Here we aimed at discerning the neural correlates of sadness or happiness in music as opposed those related to musical enjoyment. We further investigated whether musical expertise modulates the neural activity during affective listening of music. To these aims, 13 musicians and 16 non-musicians brought to the lab their most liked and disliked musical pieces with a …

Dynamics of brain activity underlying working memory for music in a naturalistic condition

We aimed at determining the functional neuroanatomy of working memory (WM) recognition of musical motifs that occurs while listening to music by adopting a non-standard procedure. Western tonal music provides naturally occurring repetition and variation of motifs. These serve as WM triggers, thus allowing us to study the phenomenon of motif tracking within real music. Adopting a modern tango as stimulus, a behavioural test helped to identify the stimulus motifs and build a time-course regressor of WM neural responses. This regressor was then correlated with the participants' (musicians') functional magnetic resonance imaging (fMRI) signal obtained during a continuous listening condition. In…

Mining Mental States using Music Associations

Key issues in decomposing fMRI during naturalistic and continuous music experience with independent component analysis

Background: Independent component analysis (ICA) has been often used to decompose fMRI data mostly for the resting-state, block and event-related designs due to its outstanding advantage. For fMRI data during free-listening experiences, only a few exploratory studies applied ICA.New method: For processing the fMRI data elicited by 512-s modern tango, a FFT based band-pass filter was used to further pre-process the fMRI data to remove sources of no interest and noise. Then, a fast model order selection method was applied to estimate the number of sources. Next, both individual ICA and group ICA were performed. Subsequently, ICA components whose temporal courses were significantly correlated …

sj-pdf-1-pom-10.1177_03057356211065061 ��� Supplemental material for Maladaptive music listening strategies are modulated by individual traits

Supplemental material, sj-pdf-1-pom-10.1177_03057356211065061 for Maladaptive music listening strategies are modulated by individual traits by Vinoo Alluri, Anant Mittal, Azhagammal SC, Jonna K. Vuoskoski and Suvi Saarikallio in Psychology of Music

Differential Effects of Trait Empathy on Functional Network Centrality

Previous research has shown that empathy, a fundamental component of human social functioning, is engaged when listening to music. Neuroimaging studies of empathy processing in music have, however, been limited. fMRI analysis methods based on graph theory have recently gained popularity as they are capable of illustrating global patterns of functional connectivity, which could be very useful in studying complex traits such as empathy. The current study examines the role of trait empathy, including cognitive and affective facets, on whole-brain functional network centrality in 36 participants listening to music in a naturalistic setting. Voxel-wise eigenvector centrality mapping was calculat…

Dynamic Functional Connectivity Captures Individuals’ Unique Brain Signatures

Recent neuroimaging evidence suggest that there exists a unique individual-specific functional connectivity (FC) pattern consistent across tasks. The objective of our study is to utilize FC patterns to identify an individual using a supervised machine learning approach. To this end, we use two previously published data sets that comprises resting-state and task-based fMRI responses. We use static FC measures as input to a linear classifier to evaluate its performance. We additionally extend this analysis to capture dynamic FC using two approaches: the common sliding window approach and the more recent phase synchrony-based measure. We found that the classification models using dynamic FC pa…

Acoustic, neural, and perceptual correlates of polyphonic timbre

Dynamic Functional Connectivity in the Musical Brain

Musical training causes structural and functional changes in the brain due to its sensory-motor demands. This leads to differences in how musicians perceive and process music as compared to non-musicians, thereby providing insights into brain adaptations and plasticity. Correlational studies and network analysis investigations have indicated the presence of large-scale brain networks involved in the processing of music and have highlighted differences between musicians and non-musicians. However, studies on functional connectivity in the brain during music listening tasks have thus far focused solely on static network analysis. Dynamic Functional Connectivity (DFC) studies have lately been …

Identifying musical pieces from fMRI data using encoding and decoding models.

AbstractEncoding models can reveal and decode neural representations in the visual and semantic domains. However, a thorough understanding of how distributed information in auditory cortices and temporal evolution of music contribute to model performance is still lacking in the musical domain. We measured fMRI responses during naturalistic music listening and constructed a two-stage approach that first mapped musical features in auditory cortices and then decoded novel musical pieces. We then probed the influence of stimuli duration (number of time points) and spatial extent (number of voxels) on decoding accuracy. Our approach revealed a linear increase in accuracy with duration and a poin…

Effect of Enculturation on the Semantic and Acoustic Correlates of Polyphonic Timbre

polyphonic timbre perception was investigated in a cross-cultural context wherein Indian and Western nonmusicians rated short Indian and Western popular music excerpts (1.5 s, n = 200) on eight bipolar scales. Intrinsic dimensionality estimation revealed a higher number of perceptual dimensions in the timbre space for music from one's own culture. Factor analyses of Indian and Western participants' ratings resulted in highly similar factor solutions. The acoustic features that predicted the perceptual dimensions were similar across the two participant groups. Furthermore, both the perceptual dimensions and their acoustic correlates matched closely with the results of a previous study perfor…

Towards Multimodal MIR: Predicting individual differences from music-induced movement

As the field of Music Information Retrieval grows, it is important to take into consideration the multi-modality of music and how aspects of musical engagement such as movement and gesture might be taken into account. Bodily movement is universally associated with music and reflective of important individual features related to music preference such as personality, mood, and empathy. Future multimodal MIR systems may benefit from taking these aspects into account. The current study addresses this by identifying individual differences, specifically Big Five personality traits, and scores on the Empathy and Systemizing Quotients (EQ/SQ) from participants' free dance movements. Our model succe…

Large-scale brain networks emerge from dynamic processing of musical timbre, key and rhythm

We investigated the neural underpinnings of timbral, tonal, and rhythmic features of a naturalistic musical stimulus. Participants were scanned with functional Magnetic Resonance Imaging (fMRI) while listening to a stimulus with a rich musical structure, a modern tango. We correlated temporal evolutions of timbral, tonal, and rhythmic features of the stimulus, extracted using acoustic feature extraction procedures, with the fMRI time series. Results corroborate those obtained with controlled stimuli in previous studies and highlight additional areas recruited during musical feature processing. While timbral feature processing was associated with activations in cognitive areas of the cerebel…

Diffusion map for clustering fMRI spatial maps extracted by Indipendent Component Analysis

Functional magnetic resonance imaging (fMRI) produces data about activity inside the brain, from which spatial maps can be extracted by independent component analysis (ICA). In datasets, there are n spatial maps that contain p voxels. The number of voxels is very high compared to the number of analyzed spatial maps. Clustering of the spatial maps is usually based on correlation matrices. This usually works well, although such a similarity matrix inherently can explain only a certain amount of the total variance contained in the high-dimensional data where n is relatively small but p is large. For high-dimensional space, it is reasonable to perform dimensionality reduction before clustering.…

Emotions of music listening in Finland and in India : Comparison of an individualistic and a collectivistic culture

Music is appreciated for emotional reasons across cultures, but knowledge on the cross-cultural similarities and differences of music-evoked emotions is still sparse. The current study compared music-evoked emotions in Finland and in India, contextualizing them within the perceived psychological functionality of music in an individualistic versus collectivistic culture. Participants ( N = 230) answered an online survey on music-evoked emotions and related personal meanings. A mixed-method approach using factor analysis and qualitative content analysis was used to identify the concepts for cross-cultural comparison. Results show that both cultures value music for positive emotional experien…

Influence of Musical Expertise on the processing of Musical Features in a Naturalistic Setting

Musical training causes structural and functional changes in the brain due to its sensory-motor demands, but the modulatory effect of musical training on music feature processing in the brain in a continuous music listening paradigm, has not been investigated thus far. In this work, we investigate the differences between musicians and non-musicians in the encoding of musical features encompassing musical timbre, rhythm and tone. 18 musicians and 18 non-musicians were scanned using fMRI while listening to 3 varied stimuli. Acoustic features corresponding to timbre, rhythm and tone were computationally extracted from the stimuli and correlated with brain responses, followed by t-tests on grou…

Event-related brain responses while listening to entire pieces of music

Brain responses to discrete short sounds have been studied intensively using the event-related potential (ERP) method, in which the electroencephalogram (EEG) signal is divided into epochs time-locked to stimuli of interest. Here we introduce and apply a novel technique which enables one to isolate ERPs in human elicited by continuous music. The ERPs were recorded during listening to a Tango Nuevo piece, a deep techno track and an acoustic lullaby. Acoustic features related to timbre, harmony, and dynamics of the audio signal were computationally extracted from the musical pieces. Negative deflation occurring around 100 milliseconds after the stimulus onset (N100) and positive deflation occ…

From Vivaldi to Beatles and back: predicting lateralized brain responses to music.

We aimed at predicting the temporal evolution of brain activity in naturalistic music listening conditions using a combination of neuroimaging and acoustic feature extraction. Participants were scanned using functional Magnetic Resonance Imaging (fMRI) while listening to two musical medleys, including pieces from various genres with and without lyrics. Regression models were built to predict voxel-wise brain activations which were then tested in a cross-validation setting in order to evaluate the robustness of the hence created models across stimuli. To further assess the generalizability of the models we extended the cross-validation procedure by including another dataset, which comprised …

Timbre and affect dimensions: Evidence from affect and similarity ratings and acoustic correlates of isolated instrument sounds

considerable effort has been made towards understanding how acoustic and structural features contribute to emotional expression in music, but relatively little attention has been paid to the role of timbre in this process. Our aim was to investigate the role of timbre in the perception of affect dimensions in isolated musical sounds, by way of three behavioral experiments. In Experiment 1, participants evaluated perceived affects of 110 instrument sounds that were equal in duration, pitch, and dynamics using a three-dimensional affect model (valence, energy arousal, and tension arousal) and preference and emotional intensity. In Experiment 2, an emotional dissimilarity task was applied to a…

Connectivity patterns during music listening: Evidence for action-based processing in musicians

Musical expertise is visible both in the morphology and functionality of the brain. Recent research indicates that functional integration between multi-sensory, somato-motor, default-mode (DMN), and salience (SN) networks of the brain differentiates musicians from non-musicians during resting state. Here, we aimed at determining whether brain networks differentially exchange information in musicians as opposed to non-musicians during naturalistic music listening. Whole-brain graph-theory analyses were performed on participants' fMRI responses. Group-level differences revealed that musicians' primary hubs comprised cerebral and cerebellar sensorimotor regions whereas non-musicians' dominant …