Perception and replication of planar sonic gestures

As tables, boards, and walls become surfaces where interaction can be supported by auditory displays, it becomes important to know how accurately and effectively a spatial gesture can be rendered by means of an array of loudspeakers embedded in the surface. Two experiments were designed and performed to assess: (i) how sequences of sound pulses are perceived as gestures when the pulses are distributed in space and time along a line; (ii) how the timing of pulses affects the perceived and reproduced continuity of sequences; and (iii) how effectively a second parallel row of speakers can extend sonic gestures to a two-dimensional space. Results show that azimuthal trajectories can be effectiv…

A Case of Cooperative Sound Design

In this design case study, protocol and linkographic analysis are applied to a task of cooperative vocal sketching, proposed in the scope of educational research activities. The understanding of the cognitive behaviors involved in sound creation is aimed at setting the ground for the development of rigorous, designerly evaluation practices tailored to sound design, all the way to the final interactive product. Relevant qualitative and quantitative information about the creative process informs the assessment and possibly improvement of sound design methods.

miMic

miMic, a sonic analogue of paper and pencil is proposed: An augmented microphone for vocal and gestural sonic sketching. Vocalizations are classified and interpreted as instances of sound models, which the user can play with by vocal and gestural control. The physical device is based on a modified microphone, with embedded inertial sensors and buttons. Sound models can be selected by vocal imitations that are automatically classified, and each model is mapped to vocal and gestural features for real-time control. With miMic, the sound designer can explore a vast sonic space and quickly produce expressive sonic sketches, which may be turned into sound prototypes by further adjustment of model…

Interaction by ear

Abstract Speech-based interaction is now part of our everyday experiences, in the home and on the move. More subtle is the presence of designed non-speech sounds in human-machine interactions, and far less evident is their importance to create aural affordances and to support human actions. However, new application areas for interactive sound, beyond the domains of speech and music, have been emerging. These range from tele-operation and way-finding, to peripheral process monitoring and augmented environments. Beyond signalling location, presence, and states, future sounding artifacts are expected to be plastic and reconfigurable, and take into account the inherently egocentric nature of so…

Reverberation still in business: Thickening and propagating micro-textures in physics-based sound modeling

Artificial reverberation is usually introduced, as a digital audio effect, to give a sense of enclosing architectural space. In this paper we argue about the effectiveness and usefulness of diffusive reverberators in physically-inspired sound synthesis. Examples are given for the synthesis of textural sounds, as they emerge from solid mechanical interactions, as well as from aerodynamic and liquid phenomena.

Innovative Tools for Sound Sketching Combining Vocalizations and Gestures

International audience; Designers are used to produce a variety of physical and digital representations at different stages of the design process. These intermediary objects (IOs) do support the externalization of ideas and the mediation with the different stakeholders. In the same manner, sound designers deliver several intermediate sounds to their clients, through iteration and refinement. In fact, these preliminary sounds are sound sketches representing the intermediate steps of an evolving creation. In this paper we reflect on the method of sketching sounds through vocalizations and gestures, and how a technological support, grounded in the understanding of the design practice, can fost…

Cooperative sound design: a protocol analysis

Formal protocol analysis and linkographic representations are well-established approaches in design cognition studies, in the visual domain. We introduce the method and tools in the auditory domain, by analysing a case of collaborative sound design. We show how they can provide relevant qualitative and quantitative information about the efficiency of the creative process.

Outils innovants pour la création d’esquisses sonores combinant vocalisations et gestes

Les designers produisent différents types de représentations physiques et/ou digitales lors des différentes phases d'un processus de design. Ces objets intermédiaires de représentation permettent et supportent l'incarnation des idées du designer, de les externaliser, mais aussi la médiation entre les personnes qui sont impliquées dans les différentes phases du design (designers produits, ingénieurs, marketing, ...). Les designers sonores, eux aussi, produisent des sons intermédiaires pour les présenter aux commanditaires par un processus itératif de raffinement de ces propositions. Ainsi ces différents sons intermédiaires sont des esquisses sonores qui représentent les différentes étapes in…

To “Sketch-a-Scratch”

A surface can be harsh and raspy, or smooth and silky, and everything in between. We are used to sense these features with our fingertips as well as with our eyes and ears: the exploration of a surface is a multisensory experience. Tools, too, are often employed in the interaction with surfaces, since they augment our manipulation capabilities. “Sketch-a-Scratch” is a tool for the multisensory exploration and sketching of surface textures. The user’s actions drive a physical sound model of real materials’ response to interactions such as scraping, rubbing or rolling. Moreover, different input signals can be converted into 2D visual surface profiles, thus enabling to experience them visually…

Sketcching sonic interactions

This chapter provides an overview of methods and practices for sketching sound in interactive contexts. The chapter stresses the role and importance of producing and interacting with provisional sound representations in conceptual sound design. The cognitive benefits of embodied sound sketching are discussed and outlined by means of practical examples and basic exercises. In particular, vocalizations and gestures are proposed as primary cognitive artifacts available to sound designers to enact sonic impressions and displays, cooperatively.

Growing the practice of vocal sketching

Sketch-thinking in the design domain is a complex representational activity, emerging from the reflective conversation with the sketch. A recent line of research on computational support for sound design has been focusing on the exploitation of voice, and especially vocal imitations, as effective representation strategy for the early stage of the design process. A set of introductory exercises on vocal sketching, to probe the communication effectiveness of vocal imitations for design purposes, are presented and discussed, in the scope of the research-through-design workshop activities of the EU project SkAT-VG

Embodied sound design

Abstract Embodied sound design is a process of sound creation that involves the designer’s vocal apparatus and gestures. The possibilities of vocal sketching were investigated by means of an art installation. An artist–designer interpreted several vocal self-portraits and rendered the corresponding synthetic sketches by using physics-based and concatenative sound synthesis. Both synthesis techniques afforded a broad range of artificial sound objects, from concrete to abstract, all derived from natural vocalisations. The vocal-to-synthetic transformation process was then automated in SEeD, a tool allowing to set and play interactively with physics- or corpus-based sound models. The voice-dri…

Bauhaus Legacy in Research through Design: The Case of Basic Sonic Interaction Design

Understanding cooperative sound design through linkographic analysis

Protocol and linkographic analysis are applied to a task of cooperative vocal sketching, proposed in the scope of educational research activites. The understanding of the cognitive behaviors involved in sound creation is aimed at setting the ground for the development of rigorous, designerly evaluation practices tailored to sound design. We show how relevant qualitative and quantitative information about the creative process can be used to inform the assessment and possibly improvement of vocal sketching methods.

The Sound Design Toolkit

The Sound Design Toolkit is a collection of physically informed sound synthesis models, specifically designed for practice and research in Sonic Interaction Design. The collection is based on a hierarchical, perceptually founded taxonomy of everyday sound events, and implemented by procedural audio algorithms which emphasize the role of sound as a process rather than a product. The models are intuitive to control – and the resulting sounds easy to predict – as they rely on basic everyday listening experience. Physical descriptions of sound events are intentionally simplified to emphasize the most perceptually relevant timbral features, and to reduce computational requirements as well. Keywo…

To embody or not to embody: A sound design dilemma

Designing with sound is about constructing appropriate sound representations of a concept, from the early ideas to the final product. A survey research on embodied sound sketching is presented, and the problems of early representation in sound design are discussed by analysing the questionnaire results of three workshops on vocal sketching.

Sketching sonic interactions by imitation-driven sound synthesis

Sketching is at the core of every design activity. In visual design, pencil and paper are the preferred tools to produce sketches for their simplicity and immediacy. Analogue tools for sonic sketching do not exist yet, although voice and gesture are embodied abilities commonly exploited to communicate sound concepts. The EU project SkAT-VG aims to support vocal sketching with computeraided technologies that can be easily accessed, understood and controlled through vocal and gestural imitations. This imitation-driven sound synthesis approach is meant to overcome the ephemerality and timbral limitations of human voice and gesture, allowing to produce more refined sonic sketches and to think a…