Search results for "Multimodal"
showing 10 items of 330 documents
Feature Extraction and Selection for Pain Recognition Using Peripheral Physiological Signals.
2019
In pattern recognition, the selection of appropriate features is paramount to both the performance and the robustness of the system. Over-reliance on machine learning-based feature selection methods can, therefore, be problematic; especially when conducted using small snapshots of data. The results of these methods, if adopted without proper interpretation, can lead to sub-optimal system design or worse, the abandonment of otherwise viable and important features. In this work, a deep exploration of pain-based emotion classification was conducted to better understand differences in the results of the related literature. In total, 155 different time domain and frequency domain features were e…
Depression Assessment by Fusing High and Low Level Features from Audio, Video, and Text
2016
International audience; Depression is a major cause of disability world-wide. The present paper reports on the results of our participation to the depression sub-challenge of the sixth Audio/Visual Emotion Challenge (AVEC 2016), which was designed to compare feature modalities ( audio, visual, interview transcript-based) in gender-based and gender-independent modes using a variety of classification algorithms. In our approach, both high and low level features were assessed in each modality. Audio features were extracted from the low-level descriptors provided by the challenge organizers. Several visual features were extracted and assessed including dynamic characteristics of facial elements…
Mediated learning materials: visibility checks in telepresence robot mediated classroom interaction
2021
Videoconferencing is increasingly used in education as a way to support distance learning. This article contributes to the emerging interactional literature on video-mediated educational interaction by exploring how a telepresence robot is used to facilitate remote participation in university-level foreign language teaching. A telepresence robot differs from commonly used videoconferencing set-ups in that it allows mobility and remote camera control. A remote student can thus move a classroom-based robot from a distance in order to shift attention between people, objects and environmental structures during classroom activities. Using multimodal conversation analysis, we focus on how partici…
A Low-Cost Multi-Modal Auditory-Visual-Tactile Framework for Remote Touch
2020
International audience; Haptic technology for human augmentation provides gains in ability for different applications, whether the aim is to enhance "disabilities" to "abilities", or "abilities" to "super-abilities". Commercially-available devices are generally expensive and tailored to specific applications and hardware. To give researchers a haptic feedback system that is economical, customisable, and fast to fabricate, our group developed a low-cost immersive haptic, audio, and visual experience built by using off-the-shelf (COTS) components. It is composed of a vibrotactile glove, a Leap Motion sensor, and an head-mounted display, integrated together to provide compelling immersive sens…
The Multimodality of Digital Longform Journalism
2016
Digital longform journalism has recently attracted increased attention among both academics and professionals. This study contributes to the growing body of research by dissecting the multimodal structure of digital longform journalism, that is, how the emerging genre combines written language, photography, short videos, maps and other graphical elements, and joins them together into a seamless narrative using subtle transitions. The data consist of 12 longform articles published in 2012–2013, which have been annotated for their visual and verbal content, their underlying principle of organization and the transitions that hold between them. The annotation is stored into a digital corpus, wh…
E-Learning material presentation and visualization types and schemes
2014
Multimedia and content visualisation provide ability to transform electronic materials into more dynamic format. This can provide positive aspect on learning, but also can overload the limited information processing capacity in human brains. Cognitive load in technology-enhanced learning is closely related to the learning styles of learners. This study examines interactions between learning styles of students and how these are related to studentâs working memory and cognitive traits. To investigate the learning styles of learners the Felder- Soloman questionnaire was chosen. It allows analyse studentsâ learning styles with respect to the Felder-Silverman learning style model, which is t…
Student-generated instructional videos facilitate learning through positive emotions
2016
The central focus of this study is a learning method in which university students produce instructional videos about the content matter as part of their learning process, combined with other learning assignments. The rationale for this is to promote a more multimodal pedagogy, and to provide students opportunities for a more learner-centred, motivating, active, engaging and productive role in their learning process. As such we designed a ‘video course’ where the students needed to produce an instructional video which could be used for university teaching. In addition to producing the video, the students needed to write a literature review of the topic of the video and a learning journal. At…
Co-participatory multimodal intergenerational storytelling : preschool children’s relationship with modality creating elder inclusion
2021
The COVID-19 crisis has highlighted elderly people as a vulnerable and excluded community, and connecting to the younger social media generation requires a shift in intergenerational storytelling performance. Recent research on multimodality has emphasized its benefits for the interactional process in storytelling. This study examines three aspects of storytelling – participation, multimodality, and emotional interaction – and uses co-creation and multimodal discourse analysis to investigate two questions: (1) To what extent can intergenerational storytelling benefit older people’s community engagement? (2) In a globalized world, how do children’s relationships with modalities create new l…
Studying social media as semiotic technology: a social semiotic multimodal framework
2018
How do we study social media technology? While social semiotics provides an extensive toolkit for analysing multimodal texts and semiotic practices, the study of social media as semiotic technology poses a significant challenge to existing research methodologies. In this article, we present a social semiotic framework that allows us to describe in analytical details the multimodal meaning potentials offered by digital social media technology and connect these to multimodal text-making and semiotic practices while underscoring the role of technology. Our framework is organized around seven interrelated and inherently informed dimensions: (1) multimodality, (2) practice, (3) the social, (4) m…
Multimodal approach to foster the Multiliteracies pedagogy in the teaching of EFL through picturebooks: The Snow Lion
2020
espanolEnsenar lenguas en el siglo veintiuno implica prestar atencion a las demandas pedagogicas multimodales de un mundo digital global. Para alcanzar este objetivo, la ensenanza efectiva del ingles requiere una preparacion de los instructores para guiar de manera consciente el desarrollo de la literacidad de los alumnos e integrar las distintas maneras de construir significado que van mucho mas alla del mero uso de la lengua. Este articulo suscribe la nocion de literacidad como un concepto multidimensional y propone un conjunto de herramientas multimodales como medio para que los maestros de ingles como lengua extranjera (ILE) trabajen con elementos literarios y visuales clave. Centrandon…