Search results for "multimodal interaction"
showing 9 items of 19 documents
Multimodal virtual navigation of a cultural heritage site: The medieval ceiling of Steri in Palermo
2009
The advance of information technology has enabled in recent years new fruition scenarios for cultural heritage sites. Multidisciplinary approaches integrate survey techniques with multimodal interfaces to allow enhanced fruition for larger group of users. In this paper we propose a multimodal interface to a virtual representation of a medieval ceiling, built in the XIV century, which covers the “Sala Magna” of Steri, the historical headquarters of the University of Palermo, in Italy. This research deals with the definition of a process for the integration of surveying techniques, modelling processes and communication technologies for the documentation of such artifacts. This is a two-stage …
Enabling Multimodal Interaction in XPL – the eXtensible Presentation Language
2007
This paper introduces the multimodal extension of the eXtensible Presentation architecture and Language (XPL), a framework aimed at streamlining multi-channel interface design process and enabling full component reuse. XPL incorporates a presentation language based on design pattern paradigm, which supplies a clear distinction between the presentation layer and the corresponding programming logic, promoting contents aggregation and a variety of event handlers described without relying on a (procedural) scripting language. In this paper, the design pattern concept is extended to voice-based interaction, and two verbal design pattern (VeDP) are introduced along to their visual counterparts. T…
A Multimodal Fruition Model for Graphical Contents in Ancient Books
2016
One of the most common and efficient way to preserve ancient books is to digitize them, and make their content someway browsable. However, this process often does not take into account the fruition point of view. Exploiting the currently available technologies allows for new ways of content fruition, to attract more people and help them to better understand the available information. In this paper, we present a solution for an innovative and multimodal exploration of ancient books contents, using both touch and touchless gestures. This allows people for both near and far distance interaction, which results in different interaction models.
A Multimodal Interaction Guide for Pervasive Services Access
2007
A pervasive, multimodal virtual guide for a cultural heritage site tour is illustrated. The guide is based on the integration of different technologies such as conversational agents, commonsense reasoning knowledge bases, multimodal interfaces and self-location detection systems. The aim of the work is to offer a more natural, context sensitive access to information with respect to traditional audio/visual pre-recorded guides. A prototype has been developed and implemented on a Qtek 9090 with Windows Mobile 2003 in order to deal with the "Museo Archeologico Regionale di Agrigento" domain.
Adaptive voice interaction for 3D representation of Cultural Heritage site
2010
In the area of cultural heritage preservation and fruition the development of electronics and information technologies has opened new scenarios of research in the field of survey, representation and communication of Cultural Heritage. The aim is thus to make the fruition of works of art available to as many users as possible by using survey techniques and multimodal interaction. In this paper we propose a multimodal fruition of a virtual representation of a medieval ceiling, built in the XIV century, which covers the “Sala Magna” in the Steri of Palermo. The research deals with the design of an intelligent relational agent which interacts with the user in a natural way. To address this very…
Developing picture communication for interactional situations at the beginning of the asylum process; mapping interactional practices
2019
The paper reports the initial findings of the first phase of the research and development project PICCORE – Picture Communication in Reception Centres. The goal was to map the use of pictures and other visual modes of communication at reception centres in Finland using an ethnographic, multimodal research approach. The ethnographic data was collected at four reception c entres in Finland. A multimodal viewpoint draws attention to how action and meanings are mediated through pictures. The initial findings mark established practices for enabling and coordinating mutual attention, supporting the use of visual and embodied resources in interactions and – as a consequence – supporting mutual und…
Kilencévesek a lakatlan szigeten : az érvelés szituációs beágyazottságának jelei. Esettanulmány
2015
This case study is a qualitative analysis of a video-recorded reasoning situation in which a pair of nine year old children argues for tools that should be taken to a deserted island. The students’ answers to a sociometric questionnaire are taken into account to establish background data on the children’s social positions in their clas and into their relationship. This information was used to interpret the multimodal data. A creative problemsolving task was the context of the reasoning situation, where the children were asked to choose 7 objects from 17 to be taken to a deserted island. Although there are many studies on children’s reasoning, it’s bodily and socially distributed nature has …
Apperception as a Multisensory Process in Material Experience
2015
Visual perspective has dominated experience research in humantechnology interaction for decades now. The neglect of other sensory modalities is gradually being addressed by scholars and designers, who investigate user experience based on touch, smell, taste, sound and even expressive bodily interactions. In cognitive and affective processes, user experience is always multi-modal, not just regarding perceived multi-sensory information, but also while perceiving through one modality we mentally construct information relevant to the other senses. This article reports the results of an experiment, where participants (N = 52) appraised materials either only by touching them or only by seeing. Th…
Kielikäsitykset vanhustyössä - asukkaiden, hoitajien ja esimiesten käsityksiä hyvästä vuorovaikutuksesta ja kielenkäytöstä
2018
In this article, we apply critical discourse analysis and ethnography to examine conceptionsand experiences of inclusive interaction and language use in the context of eldercare in aretirement home. Our data consist of interviews with residents, nurses and managers intwo bilingual eldercare units, as well as ethnographic observations of communication andlanguage use in these units. Our analysis focuses especially on language ideologies andmetalinguistic aspects of inclusive language in the context of eldercare. On the basis of ouranalysis, we suggest that four elements are particularly central to elderly-centric language: 1)the quality, quantity and tone of interaction; 2) the practices of …