XPL, a Presentation Language based on User Interface Design Pattern

The great diversity of presentations in software applications deals with fulfillment of various type of graphic interface constructions related to different programming Languages. Moreover, in the Internet era html pages play a main role because of the increasing complexity of Web applications. In Software Engineering, the use of design patterns is proven remarkable for the design and reuse of software components. Visual Design. Patterns (ViDP) are useful to define interaction schemas between. user and computer. By the some token, visual design. patterns are useful to incorporate common interfaces of interaction, schemas between user and computer. This paper describes the eXtensible Present…

The eXtensible Dynamic Presentation Manager for content adaptation

Human Computer Interaction studies deals with systems and tools that are able to improve user experience during interaction with computer. For this purpose, modern web application are expected to supply multimodal and multi-channel access, adaptivity and transcoding features. We will present in this work the eXtensible Dynamic Presentation Manager (XDPM) which is a set of innovative tools that support the eXtensible Presentation Language (XPL) in the adaptation of contents to different working contexts. The adaptation is performed according to the delivery context information which have been formalized by means of a profiler system. A profile holds information about the specific access devi…

A Multimodal Guide for Virtual 3D Models of Cultural Heritage Artifacts

The area of cultural heritage preservation and fruition has drawn an ever growing attention of artificial intelligence and human-computer interaction research in the last decades. The common aim is to develop systems that can interact with the user in a variety of modes and in the most natural way. In this paper, a multimodal guide for virtual 3D environment navigation is presented. The proposed system integrates X3D environment with a multimodal interface. The application scenario is to provide a visitor assistance and guidance during the visit of one of the halls in the historical Palazzo Steri, the headquarters of the University of Palermo.

Adaptive voice interaction for 3D representation of Cultural Heritage site

In the area of cultural heritage preservation and fruition the development of electronics and information technologies has opened new scenarios of research in the field of survey, representation and communication of Cultural Heritage. The aim is thus to make the fruition of works of art available to as many users as possible by using survey techniques and multimodal interaction. In this paper we propose a multimodal fruition of a virtual representation of a medieval ceiling, built in the XIV century, which covers the “Sala Magna” in the Steri of Palermo. The research deals with the design of an intelligent relational agent which interacts with the user in a natural way. To address this very…

Exploiting multimodality for intelligent mobile access to pervasive services in cultural heritage sites

In this chapter the role of multimodality in intelligent, mobile guides for cultural heritage environments is discussed. Multimodal access to information contents enables the creation of systems with a higher degree of accessibility and usability. A multimodal interaction may involve several human interaction modes, such as sight, touch and voice to navigate contents, or gestures to activate controls. We first start our discussion by presenting a timeline of cultural heritage system evolution, spanning from 2001 to 2008, which highlights design issues such as intelligence and context-awareness in providing information. Then, multimodal access to contents is discussed, along with problems an…

Human-to-human interfaces: emerging trends and challenges

We present a new research domain, human-to-human interaction (HHI) that describes how today's human interaction is largely indirect and mediated by a wide variety of technologies and devices. We show how this new and exciting field of design originates from the convergence of a few well-established research areas, such as traditional graphical user interfaces (GUIs), tangible user interfaces (TUIs), touchless gesture user interfaces (TGUIs), voice user interfaces (VUIs), and brain computer interfaces (BCIs). We analyse and describe current research in those areas and offer a first-hand view and presentation of its salient aspects for the human-to human interaction domain.

A Multimodal Guide for the Augmented Campus

The use of Personal Digital Assistants (PDAs) with ad-hoc built-in information retrieval and auto-localization functionalities can help people navigating an environment in a more natural manner compared to traditional audio/visual pre-recorded guides. In this work we propose and discuss a user-friendly, multi-modal guide system for pervasive context-aware service provision within augmented environments. The proposed system is adaptable to the user needs of mobility within a given environment; it is usable on different mobile devices and in particular on PDAs, which are used as advanced adaptive HEI (human-environment interaction) interfaces. An information retrieval service is provided that…

An Intelligent Multimodal Site-guide for the “Parco Archeologico della Valle dei Templi” in Agrigento

XPL and the Synchronization of Multimodal User Interfaces based on Design Pattern

The great diversity of presentations in software applications deals with fulfilment of various types of user interface constructions related to different programming languages. Furthermore, the growing interest for multimodal applications entails that their user interfaces have to support multiple access channels within a single development framework. User Interfaces Design Patterns (UIDPs) are helpful to define interaction schemas between user and computer and they provide remarkable tools for the design and reuse of software components. This paper describes the eXtensible Presentation architecture and Language (XPL), a framework aimed at streamlining multi-channel interface design process…

A Multimodal Interaction Guide for Pervasive Services Access

A pervasive, multimodal virtual guide for a cultural heritage site tour is illustrated. The guide is based on the integration of different technologies such as conversational agents, commonsense reasoning knowledge bases, multimodal interfaces and self-location detection systems. The aim of the work is to offer a more natural, context sensitive access to information with respect to traditional audio/visual pre-recorded guides. A prototype has been developed and implemented on a Qtek 9090 with Windows Mobile 2003 in order to deal with the "Museo Archeologico Regionale di Agrigento" domain.

XPL the Extensible Presentation Language

The last decade has witnessed a growing interest in the development of web interfaces enabling both multiple ways to access contents and, at the same time, fruition by multiple modalities of interaction (point-and-click, contents reading, voice commands, gestures, etc.). In this paper we describe a framework aimed at streamlining the design process of multi-channel, multimodal interfaces enabling full reuse of software components. This framework is called the eXtensible Presentation architecture and Language (XPL), a presentation language based on design pattern paradigm that keeps separated the presentation layer from the underlying programming logic. The language supplies a methodology to…

SISTEMA DI NAVIGAZIONE VIRTUALE MULTIMODALE

MAGA: A Mobile Archaeological Guide at Agrigento

Surveying, modeling and communication techniques for the documentation of medieval wooden painted ceilings in the Mediterranean area

Wooden painted ceilings of the Mediterranean area in the middle age have their origin in the islamic culture and were then spread in the countries under the dominion of the Arabs; some of the surviving ceilings are now located in Sicily and Spain. In the historic centre of Palermo two well preserved medieval ceilings are still surviving; the first, built in the XII century, is located in the Palatine chapel; the second one, built in the XIV century covers the “Sala Magna” in the Steri of Palermo. The research, focused on the ceiling in the Steri, deals with the definition of a process for the integration of surveying techniques (photogrammetry, laser scanning), modelling processes and commu…

Interacting with Augmented Environments

Pervasive systems augment environments by integrating information processing into everyday objects and activities. They consist of two parts: a visible part populated by animate (visitors, operators) or inanimate (AI) entities interacting with the environment through digital devices, and an invisible part composed of software objects performing specific tasks in an underlying framework. This paper shows an ongoing work from the University of Palermo''s Department of Computer Science and Engineering that addresses two issues related to simplifying and broadening augmented environment access.

Enabling Multimodal Interaction in XPL – the eXtensible Presentation Language

This paper introduces the multimodal extension of the eXtensible Presentation architecture and Language (XPL), a framework aimed at streamlining multi-channel interface design process and enabling full component reuse. XPL incorporates a presentation language based on design pattern paradigm, which supplies a clear distinction between the presentation layer and the corresponding programming logic, promoting contents aggregation and a variety of event handlers described without relying on a (procedural) scripting language. In this paper, the design pattern concept is extended to voice-based interaction, and two verbal design pattern (VeDP) are introduced along to their visual counterparts. T…

Platforms for Human-Human Interaction in Large Social Events

In this paper we present the evolution of QRouteMe, an information system built to provide people with rich user experiences when attending museums or exhibits. QRouteMe is a platform for indirect, mediated, and facilitated interactions among humans during large social events, by means of a wide variety of concurrent technologies and devices. The system evolution is analyzed according to the new human-to-human interaction (HHI) research domain. In particular, we show how QRouteMe has been adapted for the different events in which it has been used. We analyze and describe social interaction aspects during the events and we discuss some data and results. Finally we outline some possible futur…

Multimodal virtual navigation of a cultural heritage site: The medieval ceiling of Steri in Palermo

The advance of information technology has enabled in recent years new fruition scenarios for cultural heritage sites. Multidisciplinary approaches integrate survey techniques with multimodal interfaces to allow enhanced fruition for larger group of users. In this paper we propose a multimodal interface to a virtual representation of a medieval ceiling, built in the XIV century, which covers the “Sala Magna” of Steri, the historical headquarters of the University of Palermo, in Italy. This research deals with the definition of a process for the integration of surveying techniques, modelling processes and communication technologies for the documentation of such artifacts. This is a two-stage …

An RFID framework for multimodal service provision

In recent years there has been a growing interest toward the development of pervasive and context-aware services, and RFID technology played a relevant role in the context sensing task. We propose the use of RFID technology together with a conversational agent in order to implement a multimodal information retrieval service we call SensorMesh. The information acquired from RFID tags about the nearest point of interest is processed by the conversational agent that carries a more natural interaction with the user, also exploiting a common sense ontology. The service is accessible using a multimodal browser on Personal Digital Assistants (PDAs); the browser allows the user to interact with the…

A Virtual Shopper Customer Assistant in Pervasive Environments

In this work we propose a smart, human-like PDA-based personal shopper assistant. The system is able to understand the user needs through a spoken natural language interaction and then stores the preferences of the potential customer. Subsequently the personal shopper suggests the most suitable items and shops that match the user profile. The interaction is given by automatic speech recognition and text-to-speech technologies; localization is allowed by the use of Wireless technologies, while the interaction is performed by an Alice-based chat-bot endowed with reasoning capabilities. Besides, being implemented on a PDA, the personal shopper satisfies the user needs of mobility and it is als…

Novel Human-to-Human Interactions from the Evolution of HCI

The interaction ways made available by the evolution of the human-computer interfaces, led to novel Human-to-Human Interaction (HHI) modes, enabling people to cooperate for almost any task any time and any where. HHI nowadays is largely indirect and mediated by a wide variety of technologies and devices. This new and exciting field of design originates from the convergence of a few well-established research fields within the HCI area, such as traditional Graphical User Interfaces (GUI), Tangible User Interfaces (TUI), Touchless Gesture User Interface (TGUI), Voice User Interfaces (VUI), and Brain Computer Interfaces (BCI). We analyze and describe the evolution of the HCI in those fields, an…

Multimodal fruition of 3D virtual environment for Cultural Heritage sites

A Virtual Shopper Customer Assistant in Pervasive Environments.

QRouteMe: A Multichannel Information System to Ensure Rich User-Experience in Exhibits and Museums

In this article the QRouteMe system is presented. QRouteMe is a multichannel information system built to ensure rich user experiences in exhibits and museums. The system starts from basic information about a particular exhibit or museum while delivering a wide user experience based on different distribution channels. The organization of the systems’ components allow to build different solutions that can be simultaneously delivered on different media. A wide range of media from touch-screen installations to portable devices like smartphones have been used. The used devices can communicate each others to increase the usability and the user experience for the visitors. Another important featur…

A Multichannel Information System to Build and Deliver Rich User-Experiences in Exhibits and Museums

In this article a multichannel information system to build and deliver rich user experiences in exhibits and museums is presented. The system was designed to use information about a particular exhibit or museum while delivering a wide user experience based on different distribution channels. The overall information is used to build different solutions that can be delivered simultaneously on different media from touch-screen installations to portable devices like smart phones. Moreover, all the devices signed in the environment are able to communicate to each others to increase the level of the usability of the system. A case study and analysis of experimental results are also provided.