Search results for "Multi-modal"
showing 10 items of 17 documents
Automatic sleep scoring: A deep learning architecture for multi-modality time series
2020
Background: Sleep scoring is an essential but time-consuming process, and therefore automatic sleep scoring is crucial and urgent to help address the growing unmet needs for sleep research. This paper aims to develop a versatile deep-learning architecture to automate sleep scoring using raw polysomnography recordings. Method: The model adopts a linear function to address different numbers of inputs, thereby extending model applications. Two-dimensional convolution neural networks are used to learn features from multi-modality polysomnographic signals, a “squeeze and excitation” block to recalibrate channel-wise features, together with a long short-term memory module to exploit long-range co…
Simplification Of Painting Images For Tactile Perception By Visually Impaired Persons
2018
The access to artworks by visually impaired people requires a simplified tactile representation of paintings. This paper presents the difficulties of direct transcription of artworks and the test results of simplification of the paintings done by Australian Aborigines which don't have purely visual elements such as shadows or perspective. The implemented methodology is bottom-up: it starts with tactile representation of basic elements relevant to the understanding of the whole painting, then their association into more complex concepts. The context of associations is explained through audio-description. The results of the tests with visually impaired persons are analyzed and explained.
Deep multimodal fusion for semantic image segmentation: A survey
2021
International audience; Recent advances in deep learning have shown excellent performance in various scene understanding tasks. However, in some complex environments or under challenging conditions, it is necessary to employ multiple modalities that provide complementary information on the same scene. A variety of studies have demonstrated that deep multimodal fusion for semantic image segmentation achieves significant performance improvement. These fusion approaches take the benefits of multiple information sources and generate an optimal joint prediction automatically. This paper describes the essential background concepts of deep multimodal fusion and the relevant applications in compute…
Multi-modality of polysomnography signals’ fusion for automatic sleep scoring
2019
Abstract Objective The study aims to develop an automatic sleep scoring method by fusing different polysomnography (PSG) signals and further to investigate PSG signals’ contribution to the scoring result. Methods Eight combinations of four modalities of PSG signals, namely electroencephalogram (EEG), electrooculogram (EOG), electromyogram (EMG), and electrocardiogram (ECG) were considered to find the optimal fusion of PSG signals. A total of 232 features, covering statistical characters, frequency characters, time-frequency characters, fractal characters, entropy characters and nonlinear characters, were derived from these PSG signals. To select the optimal features for each signal fusion, …
Audio-video people recognition system for an intelligent environment
2011
In this paper an audio-video system for intelligent environments with the capability to recognize people is presented. Users are tracked inside the environment and their positions and activities can be logged. Users identities are assessed through a multimodal approach by detecting and recognizing voices and faces through the different cameras and microphones installed in the environment. This approach has been chosen in order to create a flexible and cheap but reliable system, implemented using consumer electronics. Voice features are extracted by a short time cepstrum analysis, and face features are extracted using the eigenfaces technique. The recognition task is solved using the same Su…
An Automatic Sleep Scoring Toolbox : Multi-modality of Polysomnography Signals’ Processing
2019
Sleep scoring is a fundamental but time-consuming process in any sleep laboratory. To speed up the process of sleep scoring without compromising accuracy, this paper develops an automatic sleep scoring toolbox with the capability of multi-signal processing. It allows the user to choose signal types and the number of target classes. Then, an automatic process containing signal pre-processing, feature extraction, classifier training (or prediction) and result correction will be performed. Finally, the application interface displays predicted sleep structure, related sleep parameters and the sleep quality index for reference. To improve the identification accuracy of minority stages, a layer-w…
Powder metallurgy processing and deformation characteristics of bulk multimodal nickel
2014
cited By 7; International audience; Spark plasma sintering was used to process bulk nickel samples from a blend of three powder types. The resulting multimodal microstructure was made of coarse (average size ∼ 135 μm) spherical microcrystalline entities (the core) surrounded by a fine-grained matrix (average grain size ∼ 1.5 μm) or a thick rim (the shell) distinguishable from the matrix. Tensile tests revealed yield strength of ∼ 470 MPa that was accompanied by limited ductility (∼ 2.8% plastic strain). Microstructure observation after testing showed debonding at interfaces between the matrix and the coarse entities, but in many instances, shallow dimples within the rim were observed indica…
Depth Attention for Scene Understanding
2022
Deep learning models can nowadays teach a machine to realize a number of tasks, even with better precision than human beings. Among all the modules of an intelligent machine, perception is the most essential part without which all other action modules have difficulties in safely and precisely realizing the target task under complex scenes. Conventional perception systems are based on RGB images which provide rich texture information about the 3D scene. However, the quality of RGB images highly depends on environmental factors, which further influence the performance of deep learning models. Therefore, in this thesis, we aim to improve the performance and robustness of RGB models with comple…
Analyse et fusion d’images multimodales pour la navigation autonome
2021
Robust semantic scene understanding is challenging due to complex object types, as well as environmental changes caused by varying illumination and weather conditions. This thesis studies the problem of deep semantic segmentation with multimodal image inputs. Multimodal images captured from various sensory modalities provide complementary information for complete scene understanding. We provided effective solutions for fully-supervised multimodal image segmentation and few-shot semantic segmentation of the outdoor road scene. Regarding the former case, we proposed a multi-level fusion network to integrate RGB and polarimetric images. A central fusion framework was also introduced to adaptiv…
Experiences from a wearable-mobile acquisition system for ambulatory assessment of diet and activity
2017
Public health trends are currently monitored and diagnosed based on large studies that often rely on pen-and-paper data methods that tend to require a large collection campaign. With the pervasiveness of smart-phones and -watches throughout the general population, we argue in this paper that such devices and their built-in sensors can be used to capture such data more accurately with less of an effort. We present a system that targets a pan-European and harmonised architecture, using smartphones and wrist-worn activity loggers to enable the collection of data to estimate sedentary behavior and physical activity, plus the consumption of sugar-sweetened beverages. We report on a unified pilot…