Pseudo-haptic Controls for Mid-air Finger-based Menu Interaction

2019

Virtual Reality (VR) is more accessible than ever these days. While topics like performance, motion sickness and presence are well investigated, basic topics as VR User Interfaces (UIs) for menu control are lagging far behind. A major issue is the absence of haptic feedback and naturalness, especially when considering mid-air finger-based interaction in VR, when "grabbable" controllers are not available. In this work, we present and compare the following two visual approaches to mid-air finger-based menu control in VR environments: a planar UI similar to common 2D desktop UIs, and a pseudo-haptic UI based on physical metaphors. The results show that the pseudo-haptic UI performs better in t…

Pervasive Displays Research: What's Next?

2019

Reports on the 7th ACM International Symposium on Pervasive Displays that took place from June 6-8 in Munich, Germany.

Touchless gestural interfaces for networked public displays

2015

In the near future, we can easily imagine a significant increment in diffusion of networked public displays, as well as novel interaction modalities used in their applications. In the following, we present two of the main challenges related to networked displays we are dealing with, with a particular focus on touchless gestural interfaces: overcoming interaction blindness (i.e. enable users to immediately guess the interactivity of the display, and the gestural nature of it) and performing evaluations in-the-wild (i.e. outside any controlled environment).

Exploring usability and accessibility of avatar-based touchless gestural interfaces for autistic people

2018

Many prior works investigated the potential of pervasive technologies and interactive applications to increase access capabilities to digital content for people with disability, particularly Neuro-Developmental Disorders (NDDs). In this paper, we present an exploratory study aimed at understanding if an avatar-based touchless gestural interface is able to foster interest towards digital representations of artworks, e.g. paintings or sculptures usually exhibited in museums, and to make them more accessible for such people. In particular, the study involved three autistic people and a therapist, and allowed us to report the potential of an avatar to communicate the interactivity and stimulate…

Real-Time Hand Pose Recognition Based on a Neural Network Using Microsoft Kinect

2013

The Microsoft Kinect sensor is largely used to detect and recognize body gestures and layout with enough reliability, accuracy and precision in a quite simple way. However, the pretty low resolution of the optical sensors does not allow the device to detect gestures of body parts, such as the fingers of a hand, with the same straightforwardness. Given the clear application of this technology to the field of the user interaction within immersive multimedia environments, there is the actual need to have a reliable and effective method to detect the pose of some body parts. In this paper we propose a method based on a neural network to detect in real time the hand pose, to recognize whether it…

Ecological Invitation to Engage with Public Displays

2018

Interactive public displays pose several research issues, which include display blindness and interaction blindness. In this paper, we shortly introduce our idea of a sound-based system to overcome the display blindness, and some experiments that we are carrying out in order to test its effectiveness.

Conveying Audience Emotions Through Humanoid Robot Gestures to an Orchestra During a Live Musical Exhibition

2017

In the last twenty years, robotics have been applied in many heterogeneous contexts. Among them, the use of humanoid robots during musical concerts have been proposed and investigated by many authors. In this paper, we propose a contribution in the area of robotics application in music, consisting of a system for conveying audience emotions during a live musical exhibition, by means of a humanoid robot. In particular, we provide all spectators with a mobile app, by means of which they can select a specific color while listening to a piece of music (act). Each color is mapped to an emotion, and the audience preferences are then processed in order to select the next act to be played. This dec…

A REST-based framework to support non-invasive and early coeliac disease diagnosis

2019

The health sector has traditionally been one of the early adopters of databases, from the most simple Electronic Health Record (formerly Computer-Based Patient Record) systems in use in general practice, hospitals and intensive care units to big data, multidata based systems used to support diagnosis and care decisions. In this paper we present a framework to support non-invasive and early diagnosis of coeliac disease. The proposed framework makes use of well-known technologies and techniques, both hardware and software, put together in a novel way. The main goals of our framework are: (1) providing users with a reliable and fast repository of a large amount of data; (2) to make such reposi…

KIND-DAMA: A modular middleware for Kinect-like device data management

2017

Summary In the last decades, we have witnessed a growing interest toward touchless gestural user interfaces. Among other reasons, this is due to the large availability of different low-cost gesture acquisition hardware (the so-called “Kinect-like devices”). As a consequence, there is a growing need for solutions that allow to easily integrate such devices within actual systems. In this paper, we present KIND-DAMA, an open and modular middleware that helps in the development of interactive applications based on gestural input. We first review the existing middlewares for gestural data management. Then, we describe the proposed architecture and compare its features against the existing simila…

Exploiting Correlation between Body Gestures and Spoken Sentences for Real-time Emotion Recognition

2017

Humans communicate their affective states through different media, both verbal and non-verbal, often used at the same time. The knowledge of the emotional state plays a key role to provide personalized and context-related information and services. This is the main reason why several algorithms have been proposed in the last few years for the automatic emotion recognition. In this work we exploit the correlation between one's affective state and the simultaneous body expressions in terms of speech and gestures. Here we propose a system for real-time emotion recognition from gestures. In a first step, the system builds a trusted dataset of association pairs (motion data -> emotion pattern), a…

Plantxel: Towards a plant-based controllable display

2018

The use of plants as a mean for both visualization and interaction has been already explored in smart environments. In this work, we explore the possibility of constructing a controllable dynamic plant-based display using thigmonastic plants, i.e. plants that change the shape and position of their leaves as a response to external stimuli. As an initial step towards this vision, we first introduce our approach of building a plant-based pixel (plant-pixel, or plantxel), and the principles of composing a plantxel-based public display. We then present the results of a feasibility study conducted with Mimosa spegazzinii plants, showing that our approach can achieve an acceptable contrast ratio, …

A Multimodal Fruition Model for Graphical Contents in Ancient Books

2016

One of the most common and efficient way to preserve ancient books is to digitize them, and make their content someway browsable. However, this process often does not take into account the fruition point of view. Exploiting the currently available technologies allows for new ways of content fruition, to attract more people and help them to better understand the available information. In this paper, we present a solution for an innovative and multimodal exploration of ancient books contents, using both touch and touchless gestures. This allows people for both near and far distance interaction, which results in different interaction models.

Body Gestures and Spoken Sentences: A Novel Approach for Revealing User’s Emotions

2017

In the last decade, there has been a growing interest in emotion analysis research, which has been applied in several areas of computer science. Many authors have con- tributed to the development of emotion recognition algorithms, considering textual or non verbal data as input, such as facial expressions, gestures or, in the case of multi-modal emotion recognition, a combination of them. In this paper, we describe a method to detect emotions from gestures using the skeletal data obtained from Kinect-like devices as input, as well as a textual description of their meaning. The experimental results show that the correlation existing between body movements and spoken user sentence(s) can be u…

Touchless Interfaces For Public Displays

2016

Public displays have lately become ubiquitous thanks to the decreasing cost of such technology and public policies supporting the development of smart cities. Depending on form factor, those displays might use touchless gestural interfaces that therefore are becoming more often the subject of public and private research. In this paper, we focus on touchless interactions with situated public displays, and introduce a pilot study on comparing two interfaces: an interface based on the Microsoft Human Interface Guidelines (HIG), a de facto standard in the field, and a novel interface, designed by us. Differently from the HIG-based one, our interface displays an avatar, which does not require an…

A touchless gestural system for extended information access within a campus

2017

In the last two decades, we have witnessed a growing spread of touchless interfaces, facilitated by higher performances of computational systems, as well as the increased availability of cheaper sensors and devices. Putting the focus on gestural input, several researchers and designers used Kinect-like devices to implement touchless gestural interfaces. The latter extends the possible deployments and usage of public interactive displays. For example, wall-sized displays may become interactive even if they are unreachable by touch. Moreover, billboard-sized displays may be placed in safe cases to avoid vandalism, while still maintaining their interactivity. Finally, people with temporary or …

Real-Time Body Gestures Recognition Using Training Set Constrained Reduction

2017

Gesture recognition is an emerging cross-discipline research field, which aims at interpreting human gestures and associating them to a well-defined meaning. It has been used as a mean for supporting human to machine interaction in several applications of robotics, artificial intelligence, and machine learning. In this paper, we propose a system able to recognize human body gestures which implements a constrained training set reduction technique. This allows the system for a real-time execution. The system has been tested on a publicly available dataset of 7,000 gestures, and experimental results have highlighted that at the cost of a little decrease in the maximum achievable recognition ac…

Elicitation and evaluation of zoom gestures for touchless interaction with desktop displays

2019

Touchless gestural interaction has been widely studied and adopted in many contexts. Furthermore, the growing availability of low-cost enabling devices, such as Kinect or Leap Motion, boosted up the interest in such interaction both for commercial and scientific purposes, both for large public displays and for personal displays. The problem of choosing the right touchless gesture for the right action is thus still an open issue, depending on several aspects, such as context, purpose, users' culture, etc. In this work, we first present the results of a gesture elicitation study that allowed us to identify a set of touchless gestures for performing zoom actions while interacting with desktop …

Human-to-Human Interaction: The Killer Application of Ubiquitous Computing?

2017

Twenty-five years past the Weiser’s vision of Ubiquitous Computing, and there is not a clear understanding of what is or is not a pervasive system. Due to the loose boundaries of such paradigm, almost any kind of remotely ac-cessible networked system is classified as a pervasive system. We think that that is mainly due to the lack of killer applications that could make this vi-sion clearer. Actually, we think that the most promising killer application is already here, but we are so used to it that we do not see it, as a perfect fitting of the Weiser’s vision: the Human-to-Human Interaction mediated by com-puters.

Continuous hand openness detection using a Kinect-like device

2014

This paper presents a novel method to reproduce in real time the opening and closing gestures of a human hand, animating a three-dimensional model of it. In other works, this result can be achieved by mapping a set of significant points of a real hand on the corresponding points of the model to animate. We propose an alternative way to produce the same effect without mapping points, but using a level-based estimation of the degree of opening of the hand. The experiments have been executed using Microsoft KinectTM, but the method would work on any other Kinect-like devices (as defined herein). The results obtained are particularly encouraging and demonstrate real-time performance of the syst…

They are looking at me! Understanding how Audience Presence Impacts on Public Display Users

2017

It is well known from prior work, that people interacting as well as attending to a public display attract further people to interact. This behavior is commonly referred to as the honeypot effect. At the same time, there are often situations where an audience is present in the vicinity of a public display that does not actively engage or pay attention to the display or an approaching user. However, it is largely unknown how such a \textit{passive audience} impacts on users or people who intend to interact. In this paper, we investigate the influence of a passive audience on the engagement of people with a public display. In more detail, we report on the deployment of a display in a public s…

Investigating how user avatar in touchless interfaces affects perceived cognitive load and two-handed interactions

2017

In recent years, touchless-enabling technologies have been more and more adopted for providing public displays with gestural interactivity. This has led to the need for novel visual interfaces aimed at solving issues such as communicating interactivity to users, as well as supporting immediate usability and "natural" interactions. In this paper, we focus our investigation on a visual interface based only on the use of in-air direct manipulations. Our study aims at evaluating whether and how the presence of an Avatar that replays user's movements may decrease the perceived cognitive workload during interactions. Moreover, we conducted a brief evaluation of the relationship between the presen…

Child–display interaction:Lessons learned on touchless avatar-based large display interfaces

2020

AbstractDuring the last decade, touchless gestural interfaces have been widely studied as one of the most promising interaction paradigms in the context of pervasive displays. In particular, avatars and silhouettes have proved to be effective in making the touchless capacity of displays self-evident. In this paper, we focus on a child–display interaction approach to avatar-based touchless gestural interfaces. We believe that large displays offer an opportunity to stimulate children’s experiences and engagement; for instance, learning about art is very engaging for children but can bring a number of challenges. Our study aims to contribute to the literature on both pervasive displays and chi…

Designing Touchless Gestural Interactions for Public Displays In-the-Wild

2015

Public displays, typically equipped with touchscreens, are used for interactions in public spaces, such as streets or fairs. Currently low-cost visual sensing technologies, such as Kinect-like devices and high quality cameras, allow to easily implement touchless interfaces. Nevertheless, the arising interactions have not yet been fully investigated for public displays in-the-wild (i.e. in appropriate social contexts where public displays are typically deployed). Different audiences, cultures and social settings strongly affect users and their interactions. Besides gestures for public displays must be guessable to be easy to use for a wide audience. Issues like these could be solved with use…

Palmprint principal lines extraction

2014

The palmprint recognition has become a focus in biological recognition and image processing fields. In this process, the features extraction (with particular attention to palmprint principal line extraction) is especially important. Although a lot of work has been reported, the representation of palmprint is still an open issue. In this paper we propose a simple, efficient, and accurate palmprint principal lines extraction method. Our approach consists of six simple steps: normalization, median filtering, average filters along four prefixed directions, grayscale bottom-hat filtering, combination of bottom-hat filtering, binarization and post processing. The contribution of our work is a new…

Evaluation of a Visual Tool for Early Patent Infringement Detection During Design

2019

Patent infringement detection usually implies research among documents in different forms, in both natural and unstructured language, often involving a lot of human resources and time. In order to ease this patent check process, we previously presented a visual tool to be used by designers themselves at any stage of the design process, providing them with useful and reliable information for deciding whether to steer their design away from potential patent infringements. In this work, we report on a usability study carried out on such a tool with 21 professional designers from industry in the field of mechanical engineering. The outcome of our study shows that our tool is very well accepted …

Recognizing the Emergent and Submerged Iceberg of the Celiac Disease: ITAMA Project-Global Strategy Protocol.

2022

Coeliac disease (CD) is frequently underdiagnosed with a consequent heavy burden in terms of morbidity and health care costs. Diagnosis of CD is based on the evaluation of symptoms and anti-transglutaminase antibodies IgA (TGA-IgA) levels, with values above a tenfold increase being the basis of the biopsy-free diagnostic approach suggested by present guidelines. This study showcased the largest screening project for CD carried out to date in school children (n=20,000) aimed at assessing the diagnostic accuracy of minimally invasive finger prick point-of-care tests (POCT) which, combined with conventional celiac serology and the aid of an artificial intelligence-based system, may eliminate t…

Modular Middleware for Gestural Data and Devices Management

2017

In the last few years, the use of gestural data has become a key enabler for human-computer interaction (HCI) applications. The growing diffusion of low-cost acquisition devices has thus led to the development of a class of middleware aimed at ensuring a fast and easy integration of such devices within the actual HCI applications. The purpose of this paper is to present a modular middleware for gestural data and devices management. First, we describe a brief review of the state of the art of similar middleware. Then, we discuss the proposed architecture and the motivation behind its design choices. Finally, we present a use case aimed at demonstrating the potential uses as well as the limit…

Supporting tourism with public interactive displays

2019

Interactive displays are common in public and semi-public areas, such as museums, shopping malls, train stations, and even on streets. Especially with the emergence of new ways of interaction, interactive displays could be introduced to popular tourist attractions to support tourism. The aim of this tutorial is to prepare its participants for designing such interactive public displays. The tutorial will first introduce participants to state of the art in interactive public displays. This will be followed by a city tour where a number of local attractions are visited. Finally, the participants will brainstorm and create concepts for novel interactive public displays that aim to improve the e…

Exploiting social comparison using pervasive displays and mobile notifications for reducing energy consumption

2017

Abstract Public displays have been used as a persuasive technology in many prior works. In this paper, we describe a persuasive pervasive system aimed at influencing users’ behavior for reducing energy consumption in buildings. This system exploits a pervasive display that visualizes the energy consumption in different levels (rooms, floors, and buildings), exploiting social comparison techniques. Moreover, our system may be extended by means of a mobile application that sends persuasive personalized messages to the users regarding their energy consumption. We then present our evaluation study design, aimed at comparing the effectiveness of three possible implementation of our system: one b…

Predicting mid-air gestural interaction with public displays based on audience behaviour

2020

Abstract Knowledge about the expected interaction duration and expected distance from which users will interact with public displays can be useful in many ways. For example, knowing upfront that a certain setup will lead to shorter interactions can nudge space owners to alter the setup. If a system can predict that incoming users will interact at a long distance for a short amount of time, it can accordingly show shorter versions of content (e.g., videos/advertisements) and employ at-a-distance interaction modalities (e.g., mid-air gestures). In this work, we propose a method to build models for predicting users’ interaction duration and distance in public display environments, focusing on …

Touch or touchless?:Evaluating usability of interactive displays for persons with autistic spectrum disorders

2019

Interactive public displays have been exploited and studied for engaging interaction in several previous studies. In this context, applications have been focused on supporting learning or entertainment activities, specifically designed for people with special needs. This includes, for example, those with Autism Spectrum Disorders (ASD). In this paper, we present a comparison study aimed at understanding the difference in terms of usability, effectiveness, and enjoyment perceived by users with ASD between two interaction modalities usually supported by interactive displays: touch-based and touchless gestural interaction. We present the outcomes of a within-subject setup involving 8 ASD users…

Child-display interaction: Exploring avatar-based touchless gestural interfaces

2019

During the last decade, touchless gestural interfaces have been widely studied as one of the most promising interaction paradigms in the context of pervasive displays. In particular, avatars and silhouettes have been proved to be effective in communicating the touchless gestural interactivity supported by displays. In the paper, we take a child-display interaction perspective by exploring avatar-based touchless gestural interfaces. We believe that large displays offer an opportunity to stimulate child experience and engagement, for instance when learning about art, as well as bringing a number of challenges. The purpose of this study is twofold: 1) identifying the relevant aspects of childr…

Investigating Avatar Influence on Perceived Cognitive Load and Bimanual Interactions with Touchless Interfaces

2017

In recent years, touchless-enabling technologies have been more and more adopted for providing public displays with gestural interactivity. This has led to the need for novel visual interfaces aimed at solving issues such as communicating interactivity to users, as well as supporting immediate usability and "natural" interactions. In this paper, we focus our investigation on a visual interface based only on the use of in-air direct manipulations. Our study aims at evaluating whether and how the presence of an Avatar that replays user’s movements may decrease the perceived cognitive workload during interactions. Moreover, we conducted a brief evaluation of the relationship between the presen…

KIND-DAMA: A modular middleware for Kinect-like device data management

2018

In the last decades, we have witnessed a growing interest toward touchless gestural user interfaces. Among other reasons, this is due to the large availability of different low-cost gesture acquisition hardware (the so-called “Kinect-like devices”). As a consequence, there is a growing need for solutions that allow to easily integrate such devices within actual systems. In this paper, we present KIND-DAMA, an open and modular middleware that helps in the development of interactive applications based on gestural input. We first review the existing middlewares for gestural data management. Then, we describe the proposed architecture and compare its features against the existing similar so…

Designing Touchless Gestural Interfaces for Public Displays

Nell’ultimo decennio, molti autori hanno studiato la possibilità di utilizzare le interfacce a gesti come strumento innovativo per supportare l’interazione con i computer. Inoltre, le recenti innovazioni tecnologiche hanno permesso di installare display interattivi in ambienti privati e pubblici. Tuttavia, l’interattività di tali display è spesso basata sull’uso di touchscreen, mentre tecnologie come i dispositivi Kinect-like vengono adottate molto più raramente, soprattutto se si considera l’ambito dei display pubblici. Al giorno d’oggi, l’opportunità di studiare le interfacce touchless per i display pubblici è diventata concreta, e rappresenta il campo di studio di diversi ricercatori. L’…