Perception of Segment Boundaries in Musicians and Non-Musicians

In the act of music listening, many people break down musical pieces into chunks such as verses and choruses. Recent work on music segmentation has shown that highly agreed segment boundaries are also considered strong and are described by using multiple cues. However, these studies could not pinpoint the effects of data collection methods and of musicianship on boundary perception. Our study investigated the differences between segmentation tasks performed by musicians in real-time and non real-time listening contexts. Further, we assessed the effect of musical training on the perception of boundaries in real-time listening. We collected perceived boundaries by 18 musicians and 18 non-musi…

Video Visualization of Predictors of Emotions Dynamically Expressed by Music

Music is particularly appreciated for its capacity of evoking a large range of emotions. A growing area of research in music psychology and computer music endeavors to understand and model the role of the different constituents of music in its overall emotional impact. Whereas research so far has mainly focused on global musical and emotional descriptions, our project investigates more in detail the relationship between the dynamic evolution of musical content and the dynamic development of the resulting emotional reaction. We introduce a new method for predicting the dynamic appreciation of emotion based on audio and musical descriptions. We focus on one particular emotion: power. We propo…

Taxonomic categorisation of motivic patterns

The issue of pattern description in computational models for motivic analysis is closely related to the cognitive debate on categorisation, in which are traditionally opposed “well-defined” and “ill-defined” categorisations. The ill-defined conceptualisation has been considered as a suitable framework for the formalisation of musical categorisation as it takes into account motivic variations. It seems that computational models rely rather on well-defined categorisation, due to its better controllability. The computational model we previously presented (Lartillot & Toiviainen, 2007) strikes a balance by developing a new flexible framework allowing the taking into account of unrestricted…

Multi-scale Modelling of Segmentation

While listening to music, people often unwittingly break down musical pieces into constituent chunks such as verses and choruses. Music segmentation studies have suggested that some consensus regarding boundary perception exists, despite individual differences. However, neither the effects of experimental task (i.e., real-time vs. annotated segmentation), nor of musicianship on boundary perception are clear. Our study assesses musicianship effects and differences between segmentation tasks. We conducted a real-time experiment to collect segmentations by musicians and nonmusicians from nine musical pieces. In a second experiment on non-real-time segmentation, musicians indicated boundaries a…

Reflections towards a generative theory of musical parallelism

Parallelism plays a core role in Lerdahl and Jackendoff's (1983) GTTM, as it rules the emergence of motivic, metrical, grouping and even formal structures. Due to the high amount of detail and complexity characterising associational structures, neither explicit model nor systematic methodology of parallelism-based structural inference has been included into the GTTM. This paper develops a methodological and computational answer to this problem founded on a computational modelling of pattern extraction operations. The paper focuses in particular on the methodological interest of the pattern mining formalism, and in particular its application to the formalisation of grouping and metrical str…

Multi-Dimensional motivic pattern extraction founded on adaptive redundancy filtering

Abstract We present a computational model for discovering repeated patterns in symbolic representations of monodic music. Patterns are discovered through an incremental adaptive identification along a multi-dimensional parametric space. The difficulties of pattern discovery mainly come from combinatorial redundancies, that our model is able to control efficiently. A specificity relation is defined between pattern descriptions, unifying suffix and inclusion relations and enabling a filtering of redundant descriptions. Combinatorial proliferation caused by successive repetitions of patterns is managed using cyclic patterns. The modelling of these redundancy control mechanisms enables an autom…

Modelling the relationships between emotional responses to, and musical content of, music therapy improvisations

This article reports a study in which listeners were asked to provide continuous ratings of perceived emotional content of clinical music therapy improvisations. Participants were presented with 20 short excerpts of music therapy improvisations, and had to rate perceived activity, pleasantness and strength using a computer-based slider interface. A total of nine musical features relating to various aspects of the music (timing, register, dynamics, tonality, pulse clarity and sensory dissonance) were extracted from the excerpts, and relationships between these features and participants' emotion ratings were investigated. The data were analysed in three stages. First, inter-dimension correla…

Generalizability and Simplicity as Criteria in Feature Selection: Application to Mood Classification in Music

Classification of musical audio signals according to expressed mood or emotion has evident applications to content-based music retrieval in large databases. Wrapper selection is a dimension reduction method that has been proposed for improving classification performance. However, the technique is prone to lead to overfitting of the training data, which decreases the generalizability of the obtained results. We claim that previous attempts to apply wrapper selection in the field of music information retrieval (MIR) have led to disputable conclusions about the used methods due to inadequate analysis frameworks, indicative of overfitting, and biased results. This paper presents a framework bas…

Modeling musical attributes to characterize ensemble recordings using rhythmic audio features

In this paper, we present the results of a pre-study on music performance analysis of ensemble music. Our aim is to implement a music classification system for the description of live recordings, for instance to help musicologist and musicians to analyze improvised ensemble performances. The main problem we deal with is the extraction of a suitable set of audio features from the recorded instrument tracks. Our approach is to extract rhythm-related audio features and to apply them for regression-based modeling of eight more general musical attributes. The model based on Partial Least-Squares Regression without preceding Principal Component Analysis performed best for all of the eight attribu…

Motivic Pattern Extraction in Symbolic Domain

This chapter offers an overview of computational research in motivic pattern extraction. The central questions underlying the topic, concerning the formalization of the motivic structures, the matching strategies and the filtering of the results, have been addressed in various ways. A detailed analysis of these problems leads to the proposal of a new methodology, which will be developed throughout the study. One main conclusion of this review is that the problems cannot be tackled using purely mathematic or geometric heuristics or classical engineering tools, but require also a detailed understanding of the multiple constraints derived by the underlying cognitive context.

Motivic matching strategies for automated pattern extraction

This article proposes an approach to the problem of automated extraction of motivic patterns in monodies. Different musical dimensions, restricted in current approaches to the most prominent melodic and rhythmic features at the surface level, are defined. The proposed strategy of detection of repeated patterns consists of an exact matching of the successive parameters forming the motives. We suggest a generalization of the multiple-viewpoint approach that allows a variability of the types of parameters (melodic, rhythmic, etc.) defining each successive extension of these motives. This enables us to take into account a more general class of motives, called heterogeneous motives, which inclu…

Multi-Scale Modelling of Segmentation : Effect of Music Training and Experimental Task

While listening to music, people, often unwittingly, break down musical pieces into constituent chunks such as verses and choruses. Music segmentation studies have suggested that some consensus regarding boundary perception exists, despite individual differences. However, neither the effects of experimental task (i.e. realtime vs annotated segmentation), nor of musicianship on boundary perception are clear. Our study assesses musicianship effects and differences between segmentation tasks. We conducted a real-time task experiment to collect segmentations by musicians and non-musicians from 9 musical pieces; in a second experiment on non-realtime segmentation, musicians indicated boundaries …

Predicting Music Therapy Clients’ Type of Mental Disorder Using Computational Feature Extraction and Statistical Modelling Techniques

Background. Previous work has shown that improvisations produced by clients during clinical music therapy sessions are amenable to computational analysis. For example, it has been shown that the perception of emotion in such improvisations is related to certain musical features, such as note density, tonal clarity, and note velocity. Other work has identified relationships between an individual’s level of mental retardation and features such as amount of silence, integration of tempo with the therapist, and amount of dissonance. The present study further develops this work by attempting to predict music therapy clients’ type of mental disorder, as clinically diagnosed, from their improvisat…

Computational Analysis Workshop: Comparing Four Approaches to Melodic Analysis

We compare four computational approaches of melodic analysis according to diverse approach aspects: input type (monophonic or polyphonic), pattern identification type (strict or similar), analysis segmentation, aim of approach, motivic pattern representation, and type of result representations. The considered four computational approaches are the following: a similarity neighbourhood approach by Adiloglu (Adiloglu and Obermayer 2006a, b), a multiple viewpoint representation and discovery approach by Anagnostopoulou (Anagnostopoulou, Share and Conklin 2006), a topological approach by Buteau (2005), and an approach based on multidimensional closed pattern mining by Lartillot (Lartillot and To…

Book Review: Analyse musicale; Sémiologie et cognition des formes temporelles

The Sound of Emotion

What is the effect of performers’ experienced emotions on the auditory characteristics of their performances? By asking performers to play a music phrase in response to three different instructions we attempted to answer this question. Performers were instructed to do the following: 1) play while focusing on the technical aspects of their playing; 2) give an expressive performance; and 3) focus on their experienced emotions, prior to which they were subjected to a sadness-inducing mood induction task. Performers were interviewed after each playing condition. We analyzed the tempo, articulation, dynamics, timbre, and vibrato of the performances obtained as well as the interview data. A focus…

Interaction features for prediction of perceptual segmentation:Effects of musicianship and experimental task

As music unfolds in time, structure is recognised and understood by listeners, regardless of their level of musical expertise. A number of studies have found spectral and tonal changes to quite successfully model boundaries between structural sections. However, the effects of musical expertise and experimental task on computational modelling of structure are not yet well understood. These issues need to be addressed to better understand how listeners perceive the structure of music and to improve automatic segmentation algorithms. In this study, computational prediction of segmentation by listeners was investigated for six musical stimuli via a real-time task and an annotation (non real-tim…

A Musical Pattern Discovery System Founded on a Modeling of Listening Strategies

Music is a domain of expression that conveys a paramount degree of complexity. The musical surface, composed of a multitude of notes, results from the elaboration of numerous structures of different types and sizes. The composer constructs this structural complexity in a more or less explicit way. The listener, faced by such a complex phenomenon, is able to reconstruct only a limited part of it, mostly in a non-explicit way. One particular aim of music analysis is to objectify such complexity, thus offering to the listener a tool for enriching the appreciation of music (Lartillot and SaintJames, 2004). The trouble is, traditional musical analysis, although offering a valuable understanding …

Genre-adaptive Semantic Computing and Audio-based Modelling for Music Mood Annotation

This study investigates whether taking genre into account is beneficial for automatic music mood annotation in terms of core affects valence, arousal, and tension, as well as several other mood scales. Novel techniques employing genre-adaptive semantic computing and audio-based modelling are proposed. A technique called the ACTwg employs genre-adaptive semantic computing of mood-related social tags, whereas ACTwg-SLPwg combines semantic computing and audio-based modelling, both in a genre-adaptive manner. The proposed techniques are experimentally evaluated at predicting listener ratings related to a set of 600 popular music tracks spanning multiple genres. The results show that ACTwg outpe…

Automated Extraction of Motivic Patterns and Application to the Analysis of Debussy’s Syrinx

A methodology for automated extraction of repeated patterns in discrete time series data is presented, dedicated to the discovery of musical motives in symbolic music representations. The basic principle of the approach consists in a search for closed patterns in a multi-dimensional parametric space, comprising various features related to melodic and rhythmic aspects, which can be organized into note-based and interval-based descriptions. The pattern description is further reduced through a lossless pruning of the sequence description. This requires in particular a detailed estimation of the specificity relations between patterns. For instance, a pattern is more specific than its suffix, an…

Event-related brain responses while listening to entire pieces of music

Brain responses to discrete short sounds have been studied intensively using the event-related potential (ERP) method, in which the electroencephalogram (EEG) signal is divided into epochs time-locked to stimuli of interest. Here we introduce and apply a novel technique which enables one to isolate ERPs in human elicited by continuous music. The ERPs were recorded during listening to a Tango Nuevo piece, a deep techno track and an acoustic lullaby. Acoustic features related to timbre, harmony, and dynamics of the audio signal were computationally extracted from the musical pieces. Negative deflation occurring around 100 milliseconds after the stimulus onset (N100) and positive deflation occ…

Exploring relationships between audio features and emotion in music

In this paper, we present an analysis of the associations between emotion categories and audio features automatically extracted from raw audio data. This work is based on 110 excerpts from film soundtracks evaluated by 116 listeners. This data is annotated with 5 basic emotions (fear, anger, happiness, sadness, tenderness) on a 7 points scale. Exploiting state-of-the-art Music Information Retrieval (MIR) techniques, we extract audio features of different kind: timbral, rhythmic and tonal. Among others we also compute estimations of dissonance, mode, onset rate and loudness. We study statistical relations between audio descriptors and emotion categories confirming results from psychological …

A matlab toolbox for music information retrieval

We present MIRToolbox, an integrated set of functions written in Matlab, dedicated to the extraction from audio files of musical features related, among others, to timbre, tonality, rhythm or form. The objective is to offer a state of the art of computational approaches in the area of Music Information Retrieval (MIR). The design is based on a modular framework: the different algorithms are decomposed into stages, formalized using a minimal set of elementary mechanisms, and integrating different variants proposed by alternative approaches — including new strategies we have developed —, that users can select and parametrize. These functions can adapt to a large area of objects as input.