Apperception as a Multisensory Process in Material Experience

Visual perspective has dominated experience research in humantechnology interaction for decades now. The neglect of other sensory modalities is gradually being addressed by scholars and designers, who investigate user experience based on touch, smell, taste, sound and even expressive bodily interactions. In cognitive and affective processes, user experience is always multi-modal, not just regarding perceived multi-sensory information, but also while perceiving through one modality we mentally construct information relevant to the other senses. This article reports the results of an experiment, where participants (N = 52) appraised materials either only by touching them or only by seeing. Th…

Sensory modalities and mental content in product experience

Contemporary research in human-technology interaction emphasises the need to focus on what people experience when they interact with technological artefacts. Understanding how people experience products requires detailed investigation of how physical design properties are mentally represented, and the theorisation of how people represent information obtained through different modalities still needs work. The objective of this study is to investigate how people experience modality-related affective aspects of products, using the psychological concept of mental content. For this purpose, we adopt the framework of user psychology, which is the sub-area of psychology involved with investigating…

Visual-manual in-car tasks decomposed: text entry and kinetic scrolling as the main sources of visual distraction

Distraction effects of in-car tasks with a touch screen based navigation system user interface were studied in a driving simulator experiment with eye tracking. The focus was to examine which particular in-car task components visually distract drivers the most. The results indicate that all of the visual-manual in-car tasks led to increased levels of experienced demands and to lower driving speeds. The most significant finding was that text entry and kinetic scrolling of lists were the main sources of visual distraction whereas simple selection tasks with familiar target locations led to least severe distraction effects.

A Systematic Way of Structuring Real-World Multiobjective Optimization Problems

In recent decades, the benefits of applying multiobjective optimization (MOO) methods in real-world applications have rapidly increased. The MOO literature mostly focuses on problem-solving, typically assuming the problem has already been correctly formulated. The necessity of verifying the MOO problem and the potential impacts of having an incorrect problem formulation on the optimization results are not emphasized enough in the literature. However, verification is crucial since the optimization results will not be meaningful without an accurate problem formulation, not to mention the resources spent in the optimization process being wasted. In this paper, we focus on the MOO problem struc…

Towards essential visual variables in user interface design

This paper focuses on visual variables in user interface design from the user perspective. Visual design of user interfaces is essential to users interacting with different software. The study is conducted with 3E-templates for users to express their impressions by writing and drawing regarding visual website design. The data is analyzed with qualitative content analysis through interpretation framework. The results of this study provide new insights into user-centered visual user interface design. The results indicate which are the most essential visual variables in user interface design and therefore should be emphasized in user-centered design that promotes positive UX and, thus, benefit…

Aesthetic Appeal and Visual Usability in Four Icon Design Eras

Technological artefacts express time periods in their visual design. Due time, visual culture changes and thus affects the design of pictorial representations in technological products, such as icons in user interfaces. Previous research of temporal aspects in human-computer interaction has been focusing on particular interaction situations, but not on the effects of design eras on user experience. The influence of icon design styles of different eras on aesthetic and usability experiences was studied with the method of primed product comparisons. Affective preferences and their processing times were analysed in order to examine visual usability in terms of semantic distance and aesthetic a…

Cross-Cultural Factors in Experiencing Online Video Contents in Product Marketing

Although online shops represent convenient tools to buy and sell products, they do not offer as rich multisensory experiences than physical retailing offers. Audio-visual contents could provide dynamic multisensory information and offer more engaging experiences. However, to be successful, audio-visual contents need to be adjusted to the cultural characteristics of the users. This manuscript presents a study in which Spanish and Finnish participants interacted with audiovisual products depicting videos of the brand design. Through content analysis of participants' verbalizations, the authors identified categories and subcategories that defined the representation of the video elements and th…

Interactionist Approach to Visual Aesthetics in HCI

Visual Aesthetics has gathered interest among scholars in HCI research. The growing interest stems from examinations of the aesthetic-usability effect (“what is beautiful is usable”), and possibly vice versa. Thus, numerous studies focus on understanding how we make sense and experience visual entities in interacting with technology. However, theoretical, and methodological stances vary, which impact conclusions of the studies conducted, and thus, affect design implications. Visual experience research in HCI lacks detailed conceptualizations of the constituents of visual experience and understanding of how these conceptualizations affect the overall research results through implicit methodo…

Beyond MAYA for game-changing multisensory design

With information technology becoming ever more embedded in our surrounding everyday things, the nature of interactions and the way we experience digitalization is becoming increasingly embodied. Thus, growing effort is placed on examining the multisensory nature of interaction experience. From a design perspective, increased knowledge of how people experience materials and how to design to encourage varying material experiences opens new opportunities for the generation of rich multisensory user experience, and accomplishing game-changing results. In particular, the innovation space opened up by understanding people's material expectations of designs is significant. An experiment (N = 78) w…

Affective contents of cross-cultural audiovisual experience

Audiovisual presentation of a product has a direct impact on the mental representation of an individual when interacting with a product. Companies produce audiovisual contents that can be used in different cultural environments as a way of having broader impact at a cheaper cost. But do video contents have the same impact in different countries? We addressed this question by using images and videos of Iittala products (Finnish design) with the goal of finding whether participants from two countries, Finland and Spain, appraised the designs similarly. We performed an experiment in which participants interacted with three audiovisual products depicting images/videos of the brand design. Throu…

Expert Drivers' Prospective Thinking-Aloud to Enhance Automated Driving Technologies - Investigating Uncertainty and Anticipation in Traffic.

Abstract Current automated driving technology cannot cope in numerous conditions that are basic daily driving situations for human drivers. Previous studies show that profound understanding of human drivers’ capability to interpret and anticipate traffic situations is required in order to provide similar capacities for automated driving technologies. There is currently not enough a priori understanding of these anticipatory capacities for safe driving applicable to any given driving situation. To enable the development of safer, more economical, and more comfortable automated driving experience, expert drivers’ anticipations and related uncertainties were studied on public roads. First, dri…

Introducing usability heuristics for mobile map applications

In this paper, a set of heuristics for evaluating the usability of mobile map applications is introduced. We developed the heuristics by exploring the present generic heuristics and then forming new theory-based heuristics. Usability specialists tested the heuristics by evaluating the usability of a mobile map application with both generic and domain-specific heuristics. As a result, more usability problems were found with the proposed domain-specific heuristics. In addition, based on the evaluators’ views the initial domain-specific heuristics were further developed. We conclude by proposing domain-specific usability heuristics for evaluating the mobile map applications. peerReviewed

Neuroaesthetic exploration on the cognitive processing behind repeating graphics

Repeating graphics are common research objects in modern design education. However, we do not exactly know the attentional processes underlying graphic artifacts consisting of repeating rhythms. In this experiment, the event-related potential, a neuroscientific measure, was used to study the neural correlates of repeating graphics within graded orderliness. We simulated the competitive identification process of people recognizing artifacts with graded repeating rhythms from a scattered natural environment with the oddball paradigm. In the earlier attentional processing related to the P2 component around the Fz electrode within the 150−250 ms range, a middle-grade repeating rhythm (Target 1)…

Quick Affective Judgments: Validation of a Method for Primed Product Comparisons

A method for primed product comparisons was developed, based on the methodological considerations of emotional appraisal process and affective mental contents. The method was implemented as a computer tool, which was utilised in two experiments (N = 18 for both). Ten adjectives served as primes, and five drinking glass pictures as stimuli. Participants' task was to choose a preference between two glasses, given the priming adjective. The results validate the method by providing test-retest reliability measures and showing convergence with questionnaires. Further, different evaluation times between the primes and the stimuli reveal the existence of different mental processes associated with …

Emotional Information Space in Designing AI Technologies

Current and future AI design needs to recognize the intertwined nature of cognition and affect to design more human-like intelligent systems. The majority of current AI design focuses on cognitive information processes and knowledge. However, human action and human-like actions must also consider the emotional aspects of the environment. We present the concept of emotional information space, which incorporates all issues within a certain environment with cognitively appraised affective meanings and the ability to encode these information contents into designing emotionally intelligent technologies.

Designing empirical experiments to compare interactive multiobjective optimization methods

Interactive multiobjective optimization methods operate iteratively so that a decision maker directs the solution process by providing preference information, and only solutions of interest are generated. These methods limit the amount of information considered in each iteration and support the decision maker in learning about the trade-offs. Many interactive methods have been developed, and they differ in technical aspects and the type of preference information used. Finding the most appropriate method for a problem to be solved is challenging, and supporting the selection is crucial. Published research lacks information on the conducted experiments’ specifics (e.g. questions asked), makin…

User Psychology Lab, University of Jyväskylä

Surprise as a Design Strategy in Goal-oriented Mobile Applications

Emotions are important in product experiences. Besides utilitarian aspects in interacting with technological products, recent research has focused on hedonic and affective aspects of product interaction. Designing unexpected product features evoking surprise has proven to be advantageous for product user interaction and user experience. Especially, in classical product design surprising the user is an efficient design strategy to create pleasurable products. However, current research has not extensively focused on the effects of surprise in technological interactive products. It is important to understand users´ experiences in relation to unexpected events in digital products in order to de…

Enhancing the user authentication process with colour memory cues

The authentication process is the first line of defence against potential impostors, and therefore is an important concern when protecting personal and organisational data. Although there are many options to authenticate digital users, passwords remain the most common authentication mechanism. However, with password numbers increasing, many users struggle with remembering multiple passwords, which affects their security behaviour. Previous researchers and practitioners have attempted to suggest ways to improve password memorability and security simultaneously. We introduce novel approach that utilises colour as a memory cue to increase password memorability and security. A longitudinal stud…

Apperceiving visual elements in human-technology interaction design

Visual design of technological artefacts is an integral part of peoples’ experiences in technology-interaction. Visual product properties are capable of eliciting affective responses and multisensorial experiences in human- technology interaction. Current research in the field of human-technology interaction focuses on visual, emotional and multisensory aspects of interaction in addition to functionality and usability. However, the focus has not been on how performative aspects of visual elements affect technology-interaction as a cognitive sense making process shaping human experiences. To design technological contact points to be made sense of, the substance of visual representations requ…

The Appraisal Theory of Emotion in Human–Computer Interaction

This chapter reviews the appraisal theory of emotion and how it has been employed in human–computer interaction (HCI) research. This theory views emotion as a process that evaluates the subjective significance of an event. We demonstrate the usefulness of the perspective for HCI, as emotion is defined in terms of the events of the task environment and the goals and knowledge of the subject. Importantly, the appraisal theory ties these factors together in a cognitive appraisal process order to explain the variety of subjective emotional experiences. This is important for two reasons. First, a strong theoretical commitment allows researchers and designers to derive testable hypotheses from th…

Experiencing Commercial Videos for Online Shopping

In recent years online shopping has become a popular and convenient instrument for companies to buy and sell products. However, the design of these web-shops does not offer the rich multisensory experiences than physical retailing offers. In the paper we argue that audio-visual contents could provide dynamic multisensory information to offer more engaging experiences to the consumer, but to achieve this goal, audio-visual contents need to be adjusted to the cultural characteristics of the users. Despite controversies regarding universalism of the emotional experiences induced by perceptual processes, we present evidence that suggests cultural modulations of videos experiences. In the report…

Semantic distance as a critical factor in icon design for in-car infotainment systems

In-car infotainment systems require icons that enable fluent cognitive information processing and safe interaction while driving. An important issue is how to find an optimised set of icons for different functions in terms of semantic distance. In an optimised icon set, every icon needs to be semantically as close as possible to the function it visually represents and semantically as far as possible from the other functions represented concurrently. In three experiments (N = 21 each), semantic distances of 19 icons to four menu functions were studied with preference rankings, verbal protocols, and the primed product comparisons method. The results show that the primed product comparisons me…

Interactive map interface for controlling bridge crane automation A cognitive-affective approach

This study outlines the development process of a new touch screen based user interface for controlling bridge crane automation in industrial environments. A user study of existing situation (n1 = 11) was used to develop an understanding of the cognitive and emotional design goals for crane operation. Conceptualization of these goals in the context of automation produced a set of requirements, which were used to develop a map-based touch screen user interface. Two field tests (n2 = 5, n3 = 5) revealed how bridge crane operators cognise and emotionally experience the increasing automation and how the user interface should be designed to support the operator’s spatial mental representation as …

Appraisals of Salient Visual Elements in Web Page Design

Visual elements in user interfaces elicit emotions in users and are, therefore, essential to users interacting with different software. Although there is research on the relationship between emotional experience and visual user interface design, the focus has been on the overall visual impression and not on visual elements. Additionally, often in a software development process, programming and general usability guidelines are considered as the most important parts of the process. Therefore, knowledge of programmers’ appraisals of visual elements can be utilized to understand the web page designs we interact with. In this study, appraisal theory of emotion is utilized to elaborate the relati…

Simplicity and the art of something more: A cognitive-semiotic approach to simplicity and complexity in human-technology interaction and design experience

In human–technology interaction, the balance between simplicity and complexity has been much discussed. Emphasis is placed on the value of simplicity when designing for usability. Often simplicity is interpreted as reductionism, which compromises both the affective nature of the design and usability itself. This paper takes a cognitive–semiotic approach toward understanding the dynamics between the utilitarian benefits of simplicity in design and the art of something more: considerate complexity. The cognitive–semiotic approach to human–technology design experience is a vehicle for explaining the relationship between simplicity and complexity, and this relationship’s multisensory character …

Aesthetic Appeal and Visual Usability in Four Icon Design Eras

Technological artefacts express time periods in their visual design. Due time, visual culture changes and thus affects the design of pictorial representations in technological products, such as icons in user interfaces. Previous research of temporal aspects in human-computer interaction has been focusing on particular interaction situations, but not on the effects of design eras on user experience. The influence of icon design styles of different eras on aesthetic and usability experiences was studied with the method of primed product comparisons. Affective preferences and their processing times were analysed in order to examine visual usability in terms of semantic distance and aesthetic a…

Creative interpretation in web design experience

Insight into how people mentally represent, and thus, make sense of visual designs is the key to understanding how people interact with technological devices. This paper presents a study in which participants were asked to write their interpretations of two webpage design examples, based on what they thought they would say and what would remain as a thought. The data comprised 80 3E templates (N = 40), a template allowing participants to express experiences through writing and drawing. Inductive data analysis through a phenomenological lens revealed that supposed mental and verbal representations concentrated on the following design properties: colors, themes, interface layout and quality, …

Survey data for the validation of a perceived visual usability (PVU) measurement

This dataset is part of a publication (Differentiating Perceived Usability from Aesthetics in Visual Experience – Validation of a Perceived Visual Usability Measurement). Data concerning confirmatory factor analysis (CFA) utilised in the measurement validation is freely available. Data for the CFA was collected via Prolific, an online survey platform. Thirty-three different surveys were created with the Webropol survey tool, all with different stimulus. Participants (N = 330) answered to the 33 surveys (161 female, 163 male, age range = 18-53, M = 25.8, SD = 7.0). The CFA dataset contains variables of Participant id, Stimulus, Age, Gender, Country, Native-English, and survey scales comprise…