Search results for "gesture"

Gestural Attributions as Semantics in User Interface Sound Design

2010

This paper proposes a gesture-based approach to user interface sound design, which utilises projections of body movements in sounds as meaningful attributions. The approach is founded on embodied conceptualisation of human cognition and it is justified through a literature review on the subject of interpersonal action understanding. According to the resulting hypothesis, stereotypical gestural cues, which correlate with, e.g., a certain communicative intention, represent specific non-linguistic meanings. Based on this theoretical framework, a model of a process is also outlined where stereotypical gestural cues are implemented in sound design.

2021

Abstract Pantomime has a long tradition in clinical neuropsychology of apraxia. It has been much more used by researchers and clinicians to assess tool-use disorders than real tool use. Nevertheless, it remains incompletely understood and has given rise to controversies, such as the involvement of the left inferior parietal lobe or the nature of the underlying cognitive processes. The present article offers a comprehensive framework, with the aim of specifying the neural and cognitive bases of pantomime. To do so, we conducted a series of meta-analyses of brain-lesion, neuroimaging, and behavioral studies about pantomime and other related tasks (i.e., real tool use, imitation of meaningless…

A musical reading of a contemporary installation and back: mathematical investigations of patterns in Qwalala

2021

Mathematical music theory helps us investigate musical compositions in mathematical terms. Some hints can be extended towards the visual arts. Mathematical approaches can also help formalize a "translation" from the visual domain to the auditory one and vice versa. Thus, a visual artwork can be mathematically investigated, then translated into music. The final, refined musical rendition can be compared to the initial visual idea. Can an artistic idea be preserved through these changes of media? Can a non-trivial pattern be envisaged in an artwork, and then still be identified after the change of medium? Here, we consider a contemporary installation and an ensemble musical piece derived from…

An investigation of conductors' temporal gestures and conductor— musician synchronization, and a first experiment

2007

This article reports an empirical investigation into the characteristics of conductors' temporal gestures and people's ability to synchronize with them. It describes a new approach to the investigation of the characteristics of conductors' gestures, the investigation of conductor—musician synchronization, and a first experiment to demonstrate the feasibility of the approach. As part of this approach, a new computer-based environment for manipulating and presenting conductors' gestures, and recording participants' responses, was developed. This involved the creation of several pieces of new software, all of which were developed under Microsoft® Windows® using Visual C++. Used in combination…

Hand Gestures Alert Auditory Cortices

2017

When acquiring a foreign language, the first challenge is to break into the speech stream to identify basic linguistic units. The present study tested the hypothesis that hand gestures facilitate this process by alerting auditory cortices to attend to and identify meaningful phonemic information. During fMRI data acquisition, participants watched videos of an actor speaking in Russian under three conditions. Sentences were produced with just speech alone or were accompanied by two types of hand gestures: 1) metaphoric gesture and 2) free gesture. The main finding was that there was increased auditory cortex activation when both types of gestures accompanied speech compared to speech alone, …

Exploring relationships between pianists’ body movements, their expressive intentions, and structural elements of the music

2011

Body movements during music performance have been found to be indicative of the performer’s musical intentionality, and contribute to an observer’s perception of expressive playing. This study investigates the effect of structural elements of the score, and the playing of different levels of expression on body movements during a piano performance. Pianists were required to play the same piece in four different performance conditions. Their movements were tracked by an optical motion capture system, and the comparisons that were made between specific parts of the body used, performance condition, and musical score locations were subsequently statistically examined. We found that the head an…

A longitudinal examination of the transition to symbolic communication in the second year of life

2003

Between 10 and 24 months of age, children progress from communicating through conventional signals to communicating through symbols in a variety of situations. The present study investigates this transition analysing mother–child communication frames and the child’s communicative acts, and tracing the developmental changes in both frames and communicative acts. Four children (2 girls, 2 boys) and their mothers were observed longitudinally and extensively, from 10 to 24 months of age, using a multiple case-study method. Through the detailed investigation of these single cases, clear developmental trajectories were found, showing that conventional frames and representational gestures ‘bridge’…

An empirical evaluation of two natural hand interaction systems in augmented reality

2022

AbstractHuman-computer interaction based on hand gesture tracking is not uncommon in Augmented Reality. In fact, the most recent optical Augmented Reality devices include this type of natural interaction. However, due to hardware and system limitations, these devices, more often than not, settle for semi-natural interaction techniques, which may not always be appropriate for some of the tasks needed in Augmented Reality applications. For this reason, we compare two different optical Augmented Reality setups equipped with hand tracking. The first one is based on a Microsoft HoloLens (released in 2016) and the other one is based on a Magic Leap One (released more than two years later). Both d…

Touch or touchless?:Evaluating usability of interactive displays for persons with autistic spectrum disorders

2019

Interactive public displays have been exploited and studied for engaging interaction in several previous studies. In this context, applications have been focused on supporting learning or entertainment activities, specifically designed for people with special needs. This includes, for example, those with Autism Spectrum Disorders (ASD). In this paper, we present a comparison study aimed at understanding the difference in terms of usability, effectiveness, and enjoyment perceived by users with ASD between two interaction modalities usually supported by interactive displays: touch-based and touchless gestural interaction. We present the outcomes of a within-subject setup involving 8 ASD users…

Touchless Interfaces For Public Displays

2016

Public displays have lately become ubiquitous thanks to the decreasing cost of such technology and public policies supporting the development of smart cities. Depending on form factor, those displays might use touchless gestural interfaces that therefore are becoming more often the subject of public and private research. In this paper, we focus on touchless interactions with situated public displays, and introduce a pilot study on comparing two interfaces: an interface based on the Microsoft Human Interface Guidelines (HIG), a de facto standard in the field, and a novel interface, designed by us. Differently from the HIG-based one, our interface displays an avatar, which does not require an…