Visual Re-Ranking for Multi-Aspect Information Retrieval

2017

We present visual re-ranking, an interactive visualization technique for multi-aspect information retrieval. In multi-aspect search, the information need of the user consists of more than one aspect or query simultaneously. While visualization and interactive search user interface techniques for improving user interpretation of search results have been proposed, the current research lacks understanding on how useful these are for the user: whether they lead to quantifiable benefits in perceiving the result space and allow faster, and more precise retrieval. Our technique visualizes relevance and document density on a two-dimensional map with respect to the query phrases. Pointing to a locat…

Interweaving visual and audio-haptic augmented reality for urban exploration

2018

While ordinary touchscreen-based interfaces on urban explorer applications draw much of a user's attention onto the screen, visual and audio-haptic augmented reality interfaces have emerged as the two main streams for enabling direct focus on the surroundings. However, neither interface alone satisfies users in the highly dynamic urban environment. This research investigates how the two complementary augmentation can coexist on one system and how people adapt to the situation by selecting the more suitable interface. A prototype was deployed in a field experiment in which participants explored points of interest in an urban environment with both interfaces. The engagement with the surroundi…

InspirationWall

2015

Collaborative idea generation leverages social interactions and knowledge sharing to spark diverse associations and produce creative ideas. Information exploration systems expand the current context by suggesting novel but related concepts. In this paper we introduce InspirationWall, an unobtrusive display that leverages speech recognition and information exploration to enhance an ongoing idea generation session with automatically retrieved concepts that relate to the conversation. We evaluated the system in six idea generation sessions of 20 minutes with small groups of two people. Preliminary results suggest that InspirationWall contrasts the decay of idea productivity over time and can t…

Does More Context Help? Effects of Context Window and Application Source on Retrieval Performance

2021

We study the effect of contextual information obtained from a user’s digital trace on Web search performance. Contextual information is modeled using Dirichlet–Hawkes processes (DHP) and used in augmenting Web search queries. The context is captured by monitoring all naturally occurring user behavior using continuous 24/7 recordings of the screen and associating the context with the queries issued by the users. We report a field study in which 13 participants installed a screen recording and digital activity monitoring system on their laptops for 14 days, resulting in data on all Web search queries and the associated context data. A query augmentation (QAug) model was built to expand the or…

Embedded Knowledge-based Speech Detectors for Real-Time Recognition Tasks

2006

Speech recognition has become common in many application domains, from dictation systems for professional practices to vocal user interfaces for people with disabilities or hands-free system control. However, so far the performance of automatic speech recognition (ASR) systems are comparable to human speech recognition (HSR) only under very strict working conditions, and in general much lower. Incorporating acoustic-phonetic knowledge into ASR design has been proven a viable approach to raise ASR accuracy. Manner of articulation attributes such as vowel, stop, fricative, approximant, nasal, and silence are examples of such knowledge. Neural networks have already been used successfully as de…

The Idea Machine: LLM-based Expansion, Rewriting, Combination, and Suggestion of Ideas

2022

We introduce the Idea Machine, a creativity support tool that leverages large language models (LLMs) to empower people engaged in idea generation tasks. The tool includes a number of affordances that can be used to enable various levels of automation and intelligent support. Each idea entered into the system can be expanded, rewritten, or combined with other ideas or concepts. An idea suggestion mode can also be enabled to make the system proactively suggest ideas.

The design of interfaces for multi-robot path planning and control

2014

The field of human-robot interaction has evolved beyond issues concerning the design and development of one person controlling one robot to exploring HRI for groups of robots and teams. Our design research explores biologically-inspired motion that is initiated by a human operator, applied to a single or a small group of robots, and used to affect the motion and path planning of another subset of robots. This exploratory design study first created a taxonomy to categorize individual robot motions, looking at how they could be categorized and used as building blocks. We then combined individual motions with time and velocity as design variables to guide our interaction design. This work led …

Adaptive voice interaction for 3D representation of Cultural Heritage site

2010

In the area of cultural heritage preservation and fruition the development of electronics and information technologies has opened new scenarios of research in the field of survey, representation and communication of Cultural Heritage. The aim is thus to make the fruition of works of art available to as many users as possible by using survey techniques and multimodal interaction. In this paper we propose a multimodal fruition of a virtual representation of a medieval ceiling, built in the XIV century, which covers the “Sala Magna” in the Steri of Palermo. The research deals with the design of an intelligent relational agent which interacts with the user in a natural way. To address this very…

A wearable multimodal interface for exploring urban points of interest

2015

Locating points of interest (POIs) in cities is typically facilitated by visual aids such as paper maps, brochures, and mobile applications. However, these techniques require visual attention, which ideally should be on the surroundings. Non-visual techniques for navigating towards specific POIs typically lack support for free exploration of the city or more detailed guidance. To overcome these issues, we propose a multimodal, wearable system for alerting the user of nearby recommended POIs. The system, built around a tactile glove, provides audio-tactile cues when a new POI is in the vicinity, and more detailed information and guidance if the user expresses interest in this POI. We evaluat…

Exploitation of Mobile Access to Context-Based Information in Cultural Heritage Fruition

2012

More than one billion smartphone users are estimated by 2014. With this in mind, visiting cultural heritage sites and exhibits may offer new level of engagement and entertainment just reaching down in our pockets. With orders of magnitude more computational horsepower than a five years-old desktop machine, stuffed with all sorts of sensors, these modern gizmos have a largely untapped potential to gain us access to personalized and on-demand information wherever it is needed. This paper is exactly about this, exploring with several case studies how these devices may become part of a memorable experience during a visit that one may want to share with friends and relatives. Specifically, the p…

Querytogether : Enabling entity-centric exploration in multi-device collaborative search

2018

Collaborative and co-located information access is becoming increasingly common. However, fairly little attention has been devoted to the design of ubiquitous computing approaches for spontaneous exploration of large information spaces enabling co-located collaboration. We investigate whether an entity-based user interface provides a solution to support co-located search on heterogeneous devices. We present the design and implementation of QueryTogether, a multi-device collaborative search tool through which entities such as people, documents, and keywords can be used to compose queries that can be shared to a public screen or specific users with easy touch enabled interaction. We conducted…

Investigating Proactive Search Support in Conversations

2018

Conversations among people involve solving disputes, building common ground, and reinforce mutual beliefs and assumptions. Conversations often require external information that can support these human activities. In this paper, we study how a spoken conversation can be supported by a proactive search agent that listens to the conversation, detects entities mentioned in the conversation, and proactively retrieves and presents information related to the conversation. A total of 24 participants (12 pairs) were involved in informal conversations, using either the proactive search agent or a control condition that did not support conversational analysis or proactive information retrieval. Data c…

Crowdboard: Augmenting in-person idea generation with real-time crowds

2017

Online crowds can help infuse creativity into the design process, but traditional strategies for leveraging them, such as large-scale ideation platforms, require time and organizational effort in order to obtain results. We propose a new method for crowd-based ideation that simplifies the process by having smaller crowds join in-person ideators during synchronous creative sessions. Our system Crowdboard allows online crowds to provide real-time creative input during early-stage design activities, such as brainstorming or concept mapping. The system enables in-person ideators to develop ideas on a physical or digital whiteboard which is augmented with real-time creative input from online par…

Spoken conversational context improves query auto-completion in web search

2021

Web searches often originate from conversations in which people engage before they perform a search. Therefore, conversations can be a valuable source of context with which to support the search process. We investigate whether spoken input from conversations can be used as a context to improve query auto-completion. We model the temporal dynamics of the spoken conversational context preceding queries and use these models to re-rank the query auto-completion suggestions. Data were collected from a controlled experiment and comprised conversations among 12 participant pairs conversing about movies or traveling. Search query logs during the conversations were recorded and temporally associated…

SearchBot: Supporting voice conversations with proactive search

2018

Searching during conversations and social interactions is becoming increasingly common. Although searching could be helpful for solving arguments, building common ground, and reinforcing mutual assumptions, it can also cause inter-actional problems. Proactive search approaches can enrich conversations with additional information without neglecting the shared and established social norms of being attentive to ongoing interaction. This demo showcases SearchBot, a tool that minimizes the issues associated with the practice of searching during conversations. It accomplishes this by tracking conversational background speech and then providing continuous recommendations of related documents and e…

Designing for Exploratory Search on Touch Devices

2015

Exploratory search confront users with challenges in expressing search intents as the current search interfaces require investigating result listings to identify search directions, iterative typing, and reformulating queries. We present the design of Exploration Wall, a touch-based search user interface that allows incremental exploration and sense-making of large information spaces by combining entity search, flexible use of result entities as query parameters, and spatial configuration of search streams that are visualized for interaction. Entities can be flexibly reused to modify and create new search streams, and manipulated to inspect their relationships with other entities. Data compr…

Entity Recommendation for Everyday Digital Tasks

2021

| openaire: EC/H2020/826266/EU//CO-ADAPT Recommender systems can support everyday digital tasks by retrieving and recommending useful information contextually. This is becoming increasingly relevant in services and operating systems. Previous research often focuses on specific recommendation tasks with data captured from interactions with an individual application. The quality of recommendations is also often evaluated addressing only computational measures of accuracy, without investigating the usefulness of recommendations in realistic tasks. The aim of this work is to synthesize the research in this area through a novel approach by (1) demonstrating comprehensive digital activity monitor…

IntentStreams

2015

The user's understanding of information needs and the information available in the data collection can evolve during an exploratory search session. Search systems tailored for well-defined narrow search tasks may be suboptimal for exploratory search where the user can sequentially refine the expressions of her information needs and explore alternative search directions. A major challenge for exploratory search systems design is how to support such behavior and expose the user to relevant yet novel information that can be difficult to discover by using conventional query formulation techniques. We introduce IntentStreams, a system for exploratory search that provides interactive query refine…

EntityBot: Supporting Everyday Digital Tasks with Entity Recommendations

2021

Everyday digital tasks can highly benefit from systems that recommend the right information to use at the right time. However, existing solutions typically support only specific applications and tasks. In this demo, we showcase EntityBot, a system that captures context across application boundaries and recommends information entities related to the current task. The user’s digital activity is continuously monitored by capturing all content on the computer screen using optical character recognition. This includes all applications and services being used and specific to individuals’ computer usages such as instant messaging, emailing, web browsing, and word processing. A linear model is then …

Platforms for Human-Human Interaction in Large Social Events

2012

In this paper we present the evolution of QRouteMe, an information system built to provide people with rich user experiences when attending museums or exhibits. QRouteMe is a platform for indirect, mediated, and facilitated interactions among humans during large social events, by means of a wide variety of concurrent technologies and devices. The system evolution is analyzed according to the new human-to-human interaction (HHI) research domain. In particular, we show how QRouteMe has been adapted for the different events in which it has been used. We analyze and describe social interaction aspects during the events and we discuss some data and results. Finally we outline some possible futur…

Multimodal virtual navigation of a cultural heritage site: The medieval ceiling of Steri in Palermo

2009

The advance of information technology has enabled in recent years new fruition scenarios for cultural heritage sites. Multidisciplinary approaches integrate survey techniques with multimodal interfaces to allow enhanced fruition for larger group of users. In this paper we propose a multimodal interface to a virtual representation of a medieval ceiling, built in the XIV century, which covers the “Sala Magna” of Steri, the historical headquarters of the University of Palermo, in Italy. This research deals with the definition of a process for the integration of surveying techniques, modelling processes and communication technologies for the documentation of such artifacts. This is a two-stage …

A real-time network architecture for biometric data delivery in Ambient Intelligence

2012

Ambient Intelligent applications involve the deployment of sensors and hardware devices into an intelligent environment surrounding people, meeting users’ requirements and anticipating their needs (Ambi- ent Intelligence-AmI). Biometrics plays a key role in surveillance and security applications. Fingerprint, iris and voice/speech traits can be acquired by contact, contact-less, and at-a-distance sensors embedded in the environment. Biometric traits transmission and delivery is very critical and it needs real-time transmission net- work with guaranteed performance and QoS. Wireless networks become suitable for AmI if they are able to satisfy real-time communication and security system requi…

A two-step, user-centered approach to personalized tourist recommendations

2017

Geo-localized, mobile applications can simplify a tourist visit, making the relevant Point of Interests more easily and promptly discernible to users. At the same time, such solutions must avoid creating unfitting or rigid user profiles that impoverish the users' options instead of refining them. Currently, user profiles in recommender systems rely on dimensions whose relevance to the user is more often presumed than empirically defined. To avoid this drawback, we build our recommendation system in a two-step process, where profile parameters are evaluated preliminarily and separately from the recommendations themselves. We describe this two-step evaluation process including an initial surv…

EntityBot: Actionable Entity Recommendations for Everyday Digital Task

2022

Our everyday digital tasks require access to information from a wide range of applications and systems. Although traditional search systems can help find information, they usually operate within one application (e.g., email client or web browser) and require the user's cognitive effort and attention to formulate proper search queries. In this paper, we demonstrate EntityBot, a system that proactively provides useful and supporting entities across application boundaries without requiring explicit query formulation. Our methodology is to exploit the context from screen frames captured every 2 seconds to recommend relevant entities for the current task. Recommendations are not restricted to on…

Crowdboard: An Augmented Whiteboard to Support Large-Scale Co-Design

2013

Co-design efforts attempt to account for many diverse viewpoints. However, design teams lack support for meaningful real-time interaction with a large community of potential stakeholders. We present Crowdboard, a novel whiteboard system that enables many potential stakeholders to provide real-time input during early-stage design activities, such as concept mapping. Local design teams develop ideas on a standard whiteboard, which is augmented with annotations and comments from online participants. The system makes it possible for design teams to solicit real-time opinions and ideas from a community of people intrinsically motivated to shape the product/service.

A perceptual sound space for auditory displays based on sung-vowel synthesis.

2022

AbstractWhen designing displays for the human senses, perceptual spaces are of great importance to give intuitive access to physical attributes. Similar to how perceptual spaces based on hue, saturation, and lightness were constructed for visual color, research has explored perceptual spaces for sounds of a given timbral family based on timbre, brightness, and pitch. To promote an embodied approach to the design of auditory displays, we introduce the Vowel–Type–Pitch (VTP) space, a cylindrical sound space based on human sung vowels, whose timbres can be synthesized by the composition of acoustic formants and can be categorically labeled. Vowels are arranged along the circular dimension, whi…

Flexible entity search on surfaces

2016

Surface computing allows flexible search interaction where users can manipulate the representation of entities recommended for them to create new queries or augment existing queries by taking advantage of increased screen estate and almost physical tactile interaction. We demonstrate a search system based on 1) Direct Manipulation of Entity Representation on Surfaces and 2) Entity Recommendation and Document Retrieval. Entities are modeled as a knowledge-graph and the relevances of entities are computed using the graph structure. Users can manipulate the representation of entities via spatial grouping and assigning preferences on entities. Our contribution can help to design effective infor…

Developing hand-worn input and haptic support for real-world target finding

2019

Locating places in cities is typically facilitated by handheld mobile devices, which draw the visual attention of the user on the screen of the device instead of the surroundings. In this research, we aim at strengthening the connection between people and their surroundings through enabling mid-air gestural interaction with real-world landmarks and delivering information through audio to retain users' visual attention on the scene. Recent research on gesture-based and haptic techniques for such purposes has mainly considered handheld devices that eventually direct users' attention back to the devices. We contribute a hand-worn, mid-air gestural interaction design with directional vibrotacti…

A Multi-touch Interface for Multi-robot Path Planning and Control

2014

In the last few years, research in human-robot interaction has moved beyond the issues concerning the design of the interaction between a person and a single robot. Today many researchers have shifted their focus toward the problem of how humans can control a multi-robot team. The rising of multi-touch devices provides a new range of opportunities in this sense. Our research seeks to discover new insights and guidelines for the design of multi-touch interfaces for the control of biologically inspired multi-robot teams. We have developed an iPad touch interface that lets users exert partial control over a set of autonomous robots. The interface also serves as an experimental platform to stud…

A Multichannel Information System to Build and Deliver Rich User-Experiences in Exhibits and Museums

2011

In this article a multichannel information system to build and deliver rich user experiences in exhibits and museums is presented. The system was designed to use information about a particular exhibit or museum while delivering a wide user experience based on different distribution channels. The overall information is used to build different solutions that can be delivered simultaneously on different media from touch-screen installations to portable devices like smart phones. Moreover, all the devices signed in the environment are able to communicate to each others to increase the level of the usability of the system. A case study and analysis of experimental results are also provided.

Improving the User Experience in Cultural Heritage Sites

2014

QRouteMe: A Multichannel Information System to Ensure Rich User-Experience in Exhibits and Museums

2012

In this article the QRouteMe system is presented. QRouteMe is a multichannel information system built to ensure rich user experiences in exhibits and museums. The system starts from basic information about a particular exhibit or museum while delivering a wide user experience based on different distribution channels. The organization of the systems’ components allow to build different solutions that can be simultaneously delivered on different media. A wide range of media from touch-screen installations to portable devices like smartphones have been used. The used devices can communicate each others to increase the usability and the user experience for the visitors. Another important featur…