0000000001219405
AUTHOR
Yannick Benezeth
Remote Photoplethysmography Based on Implicit Living Skin Tissue Segmentation
International audience; Region of interest selection is an essential part for remote photoplethysmography (rPPG) algorithms. Most of the time, face detection provided by a supervised learning of physical appearance features coupled with skin detection is used for region of interest selection. However, both methods have several limitations and we propose to implicitly select living skin tissue via their particular pulsatility feature. The input video stream is decomposed into several temporal superpixels from which pulse signals are extracted. Pulsatility measure for each temporal superpixel is then used to merge pulse traces and estimate the photoplethysmogram signal. This allows to select …
Remote Photoplethysmography measurement using constrained ICA
Remote Photoplethysmography (rPPG) is a technique that consists in estimating physiological parameters such as heart rate from live or recorded video sequences taken by conventional camera or even webcams. This technique is increasingly used in many application fields thanks to its simplicity and affordability. The basic idea is that the arterial blood flow shows regularity due to the heartbeat. This regularity is manifested by very small periodic variations in the color of the skin, which can be isolated and quantified by signal and image processing methods. In this context, Independent Component Analysis (ICA) is largely used to separate the signal due to arterial flow from signals from o…
Remote heart rate variability for emotional state monitoring
International audience; Several researches have been conducted to recognize emotions using various modalities such as facial expressions , gestures, speech or physiological signals. Among all these modalities, physiological signals are especially interesting because they are mainly controlled by the autonomic nervous system. It has been shown for example that there is an undeniable relationship between emotional state and Heart Rate Variability (HRV). In this paper, we present a methodology to monitor emotional state from physiological signals acquired remotely. The method is based on a remote photoplethysmography (rPPG) algorithm that estimates remote Heart Rate Variability (rHRV) using a …
Meta-Tracking for Video Scene Understanding
International audience; This paper presents a novel method to extract dominant motion patterns (MPs) and the main entry/exit areas from a surveillance video. The method first computes motion histograms for each pixel and then converts it into orientation distribution functions (ODFs). Given these ODFs, a novel particle meta-tracking procedure is launched which produces meta-tracks, i.e. particle trajectories. As opposed to conventional tracking which focuses on individual moving objects, meta-tracking uses particles to follow the dominant flow of the traffic. In a last step, a novel method is used to simultaneously identify the main entry/exit areas and recover the predominant MPs. The meta…
Parameter-free adaptive step-size multiobjective optimization applied to remote photoplethysmography
International audience; In this work, we propose to reformulate the objective function of Independent Component Analysis (ICA) to make it a better posed problem in the context of Remote photoplethysmography (rPPG). In recent previous works, linear combination coefficients of RGB channels are estimated maximizing the non-Gaussianity of ICA output components. However, in the context of rPPG a priori knowledge of the pulse signal can be incorporated into the component extraction algorithm. To this end, the contrast function of regular ICA is extended with a measure of periodicity formulated using autocorrelation. This novel semi-blind source extraction method for measuring rPPG has the interes…
Learning Bag of Spatio-Temporal Features for Human Interaction Recognition
Bag of Visual Words Model (BoVW) has achieved impressive performance on human activity recognition. However, it is extremely difficult to capture high-level semantic meanings behind video features with this method as the spatiotemporal distribution of visual words is ignored, preventing localization of the interactions within a video. In this paper, we propose a supervised learning framework that automatically recognizes high-level human interaction based on a bag of spatiotemporal visual features. At first, a representative baseline keyframe that captures the major body parts of the interacting persons is selected and the bounding boxes containing persons are extracted to parse the poses o…
Background subtraction with multispectral video sequences
International audience; Motion analysis of moving targets is an important issue in several applications such as video surveillance or robotics. Background subtraction is one of the simplest and widely used techniques for moving target detection in video sequences. In this paper, we investigate the advantages of using a multispectral video acquisition system of more than three bands for background subtraction over the use of trichromatic or monochromatic video sequences. To this end, we have established a dataset of multispectral videos with a manual annotation of moving objects. To the best of our knowledge, this is the first publicly available dataset of multispectral video sequences. Expe…
Estimating intrinsic image from successive images by solving underdetermined and overdetermined systems of the dichromatic model
International audience; Estimating an intrinsic image from a sequence of successive images taken from an object at different angles of illumination can be used in various applications such as objects recognition, color classification, and the like; because, in so doing, it can provide more visual information. Meanwhile, according to the well-known dichromatic model, each image can be considered a linear combination of three components, including intrinsic image, shading factor, and specularity. In this study, at first, two simple independent constrained and parallelized quadratic programming steps were used for computing values of the shading factor and the specularity of each successive of…
Quadratic Objective Functions for Dichromatic Model Parameters Estimation
International audience; In this paper, we present a novel method to estimate dichromatic model parameters from a single color image. Estimation of reflectance, shading and specularity has many applications such as shape recovery, specularity removal and facilitates classical image processing and computer vision tasks such as segmentation or classification. Our method is based on two successive and independent constrained quadratic programming steps to recover the parameters of the model. Compared to recent methods, our approach has the advantage to transform a complex inverse problem into two parralelizable optimization steps that are much easier to solve. We have compared our method with r…
Sketch of an automatic image based pollen detection system
The ability of measuring accurately airborne pollen concentration in the environment is an important goal for palynology. It has been unsatisfactory for agile usage to date. Huge volumes of airborne particles prevent palynologists from opportunely processing statistically suitable information. Additionally, measurements from stationary pollen monitors cannot be accurately associated to individuals. In the context of computer vision, this paper presents the outline for the structure of an image based pollen detection system, under the framework of the Personalized Pollen Profiling and Geospatial Mapping project based on individual information of allergic patient profile measured at multiple …
Comparison of region of interest segmentation methods for video-based heart rate measurements
International audience; Conventional contact photoplethysmography (PPG) sensors are not suitable in situations of skin damage or when unconstrained movement is required. As a consequence, remote photoplethysmography (rPPG) has recently emerged because it provides remote physiological measurements without expensive hardware and improves comfort for long-term monitoring. RPPG estimation methods use the spatially averaged RGB values of pixels in a Region Of Interest (ROI) to generate a temporal RGB signal. The selection of ROI is a critical first step to obtain reliable pulse signals and must contain as many skin pixels as possible with a low percentage of non-skin pixels. In this paper, we ex…
The Bag of Words Approach for the Classification of Head and Neck Cancers Using Diffuse Reflectance Spectroscopy
Diffuse Reflectance Spectroscopy (DRS) is a leading technique for the detection of head and neck cancers. It can capture information regarding tissue absorption and scattering. In this research work, we propose a novel method for the identification of normal and Squamous Cell Carcinoma (SCC) mucosa tissues using the Bag Of Words (BOW) approach. The study included 70 spectra from normal mucosa tissue sites and 70 spectra from SCC mucosa tissue sites. First, the spectra are preprocessed by extracting the useful wavelength range, denoising and reducing the inter and intra patient variability. Subsequently, features are extracted from each spectrum by continuously sliding a window with a predef…
WiseEye: A Platform to Manage and Experiment on Smart Camera Networks
International audience; Embedded vision is probably at the edge of phenomenal expansion. The smart cameras are embedding some processing units which are more and more powerful. Last decade, high-speed image processing can be implemented on specifically designed architectures [1] nevertheless the designing time of such systems was quite high and time to market therefore as well. Since, powerful chips (i.e System On Chip) and quick prototyping methodologies are contently emerging [2],[3],[4] and enable more complex algorithms to be implemented faster. Moreover, smart cameras which are embedding flexible and powerful multi-core processors or Graphic Processors Unit (GPU) are now available and …
Automatic spatial and temporal organization of long range video sequences from low level motion features
International audience; In this paper, we address the analysis of activities from long range video sequences. We present a method to automatically extract spatial and temporal structure from a video sequence from low level motion features. The scene layout is first extracted, with a set of regions that have homogeneous activities called Motion Patterns. These regions are then analyzed and the recurrent temporal motifs are extracted for each Motion Patterns. Preliminary results show that our method can accurately extract important temporal motifs from video surveillance sequences.
Periodic Variance Maximization using Generalized Eigenvalue Decomposition applied to Remote Photoplethysmography estimation
International audience; A generic periodic variance maximization algorithm to extract periodic or quasi-periodic signals of unknown periods embedded into multi-channel temporal signal recordings is described in this paper. The algorithm combines the notion of maximizing a periodicity metric combined with the global optimization scheme to estimate the source periodic signal of an unknown period. The periodicity maximization is performed using Generalized Eigenvalue Decomposition (GEVD) and the global optimization is performed using tabu search. A case study of remote photoplethysmography signal estimation has been utilized to assess the performance of the method using videos from public data…
Development of a multispectral endoscope to improve the detection of precancerous lesions in digestive gastroendoscopy (Orale)
International audience
Bag-of-word based brand recognition using Markov Clustering Algorithm for codebook generation
International audience; In order to address the issue of counterfeiting online, it is necessary to use automatic tools that analyze the large amount of information available over the Internet. Analysis methods that extract information about the content of the images are very promising for this purpose. In this paper, a method that automatically extract the brand of objects in images is proposed. The method does not explicitly search for text or logos. This information is implicitly included in the Bag-of-Words representation. In the Bag-of-Words paradigm, visual features are clustered to create the visual words. Despite its shortcomings, k-means is the most widely used algorithm. With k-mea…
[Regular Paper] Detection of H. pylori Induced Gastric Inflammation by Diffuse Reflectance Analysis
Spectral acquisitions contain rich information and thus, are promising modalities for early detection of gastric diseases. In this study, we analyze the diffuse reflectance of the gastric inflammatory lesions induced by the bacterium H. pylori in the mouse stomach. A pipeline has been designed to characterize and classify spectra acquired on mice. The pipeline is based on a band clustering algorithm followed by the computation of meaningful division and subtraction features and by classification with a linear SVM classifier. Currently, the pipeline is able to recognize inflamed stomach's spectra with an accuracy of 98%. These results are promising and the same pipeline could be adapted for …
Modular Method of Detection, Localization and Counting of Mutliple-Taxon Pollen Apertures Using Bag of Words
International audience; Accurate recognition of airborne pollen taxa is crucial for understanding and treating allergic diseases, which affect an important proportion of the world population. Modern computer vision techniques enables the detection of discriminant characteristics. Apertures is one of these characteristic that has been little explored up to now. In this paper, a flexible method of detection, localization and counting of apertures of different pollen taxa with varying appearances is proposed. Apertures are described based by primitive images following the Bag-of-Words strat-egy. A confidence map is estimated based on the classification of sampled regions. The method is designe…
Classification of Pollen Apertures Using Bag of Words
International audience; The taxonomical recognition of microscopic biological parti- cles such as pollen and spores is relevant for medical and aerobiological applications. Focusing on an accurate and automatic vision-based pollen recognition system, we propose a method for classification of pollen aper- tures based on bag-of-words strategy, with the ability of learning new types from different taxa without the need of new algorithms. Results demonstrate suitable performance and ability to add new taxa.
Human interaction recognition based on the co-occurence of visual words
International audience; This paper describes a novel methodology for automated recognition of high-level activities. A key aspect of our framework relies on the concept of co-occurring visual words for describing interactions between several persons. Motivated by the numerous success of human activity recognition methods using bag-of-words, this paradigm is extended. A 3-D XYT spatio-temporal volume is generated for each interacting person and a set of visual words is extracted to represent his activity. The interaction is then represented by the frequency of co-occurring visual words between persons. For our experiments, we used the UT-interaction dataset which contains several complex hum…
CDnet 2014: An Expanded Change Detection Benchmark Dataset
International audience; Change detection is one of the most important low-level tasks in video analytics. In 2012, we introduced the changedetection.net (CDnet) benchmark, a video dataset devoted to the evalaution of change and motion detection approaches. Here, we present the latest release of the CDnet dataset, which includes 22 additional videos (~70,000 pixel-wise annotated frames) spanning 5 new categories that incorporate challenges encountered in many surveillance settings. We describe these categories in detail and provide an overview of the results of more than a dozen methods submitted to the IEEE Change Detection Workshop 2014. We highlight strengths and weaknesses of these metho…
Real-Time Temporal Superpixels for Unsupervised Remote Photoplethysmography
International audience; Segmentation is a critical step for many computer vision applications. Among them, the remote photoplethys-mography technique is significantly impacted by the quality of region of interest segmentation. With the heart-rate estimation accuracy, the processing time is obviously a key issue for real-time monitoring. Recent face detection algorithms can perform real-time processing, however for unsupervised algorithms, i.e. without any subject detection based on supervised learning, existing methods are not able to achieve real-time on regular platform. In this paper, we propose a new method to perform real-time un-supervised remote photoplethysmograhy based on efficient…
Skin Parameter Map Retrieval from a Dedicated Multispectral Imaging System Applied to Dermatology/Cosmetology
International audience; In vivo quantitative assessment of skin lesions is an important step in the evaluation of skin condition. An objective measurement device can help as a valuable tool for skin analysis. We propose an explorative new multispectral camera specifically developed for dermatology/cosmetology applications. The multispectral imaging system provides images of skin reflectance at different wavebands covering visible and near-infrared domain. It is coupled with a neural network-based algorithm for the reconstruction of reflectance cube of cutaneous data. This cube contains only skin optical reflectance spectrum in each pixel of the bidimensional spatial information. The reflect…
UBFC-Phys: A Multimodal Database For Psychophysiological Studies of Social Stress
As humans, we experience social stress in countless everyday-life situations. Giving a speech in front of an audience, passing a job interview, and similar experiences all lead us to go through stress states that impact both our psychological and physiological states. Therefore, studying the link between stress and physiological responses had become a critical societal issue, and recently, research in this field has grown in popularity. However, publicly available datasets have limitations. In this article, we propose a new dataset, UBFC-Phys, collected with and without contact from participants living social stress situations. A wristband was used to measure contact blood volume pulse (BVP…
L’analyse spectrale de la lumière réfléchie permet de différencier les différentes étapes des lésions inflammatoires de la muqueuse gastrique dans un modèle expérimental d’infection à H. pylori chez la souris (Orale)
International audience
An Embedded Solution for Multispectral Palmprint Recognition
Palmprint based identification has attracted much attention in the past decades. In some real-life applications, portable personal authentication systems with high accuracy and speed efficiency are required. This paper presents an embedded palmprint recognition solution based on the multispectral image modality. We first develop an effective recognition algorithm by using partial least squares regression, then a FPGA prototype is implemented and optimized through high-level synthesis technique. The evaluation experiments demonstrate that the proposed system can achieve a higher recognition rate at a lower running cost comparing to the reference implementations.
Detection of H. pylori induced gastric inflammation by diffuse reflectance analysis
International audience; Spectral acquisitions contain rich information and thus, are promising modalities for early detection of gastric diseases. In this study, we analyze the diffuse reflectance of the gastric inflammatory lesions induced by the bacterium H. pylori in the mouse stomach. A pipeline has been designed to characterize and classify spectra acquired on mice. The pipeline is based on a band clustering algorithm followed by the computation of meaningful division and subtraction features and by classification with a linear SVM classifier. Currently, the pipeline is able to recognize inflamed stomachs spectra with an accuracy of 98%. These results are promising and the same pipelin…
Unmixing of human skin optical reflectance maps by Non-negative Matrix Factorization algorithm
International audience; We present in this paper the decomposition of human skin absorption spectra with a Non-negative Matrix Factorization method. In doing so, we are able to quantify the relative proportion of the main chromophores present in the epidermis and the dermis. We present experimental results showing that we obtain a good estimate of melanin and hemoglobin concentrations. Our approach has been validated by analyzing the human skin absorption spectra in areas of healthy skin and areas affected by melasma on eight patients.
Real-Time Temporal Superpixels for Unsupervised remote photopletysmography
International audience