Attitude estimation from polarimetric cameras

International audience; In the robotic field, navigation and path planning applications benefit from a wide range of visual systems (e.g. perspective cameras, depth cameras, catadioptric cameras, etc.). In outdoor conditions, these systems capture information in which sky regions cover a major segment of the images acquired. However, sky regions are discarded and are not considered as visual cue in vision applications. In this paper, we propose to estimate attitude of Unmanned Aerial Vehicle (UAV) from sky information using a polarimetric camera. Theoretically , we provide a framework estimating the attitude from the skylight polarized patterns. We showcase this formulation on both simulate…

Mirror-based matching of catadioptric images

International audience

Incorporating depth information into few-shot semantic segmentation

International audience; Few-shot segmentation presents a significant challengefor semantic scene understanding under limited supervision.Namely, this task targets at generalizing the segmentationability of the model to new categories given a few samples.In order to obtain complete scene information, we extend theRGB-centric methods to take advantage of complementary depthinformation. In this paper, we propose a two-stream deep neuralnetwork based on metric learning. Our method, known as RDNet,learns class-specific prototype representations within RGB anddepth embedding spaces, respectively. The learned prototypesprovide effective semantic guidance on the corresponding RGBand depth query ima…

Range image binarization: applications to wooden stamps analysis

This paper deals with the analysis of ancient wooden stamps. The aim is to extract a binary image from the stamp. This image must be the closer to the image produced by inking and using a printing press with the stamps. A range image based method is proposed to extract a stamped image from the stamps. The range image acquisition from a 3D laser scanner is presented. Pre-filtering for range image enhancement is detailed. The range image binarization method is based on an adaptive thresholding. Few simple processes applied on the range image enable a final binarized image computing. The proposed method provides here a very efficient way to perform "virtual" stampings with ancient wooden stamp…

Extract information of polarization imaging from local matching stereo

Since polarization of light was used in the field of computer vision, the research of polarization vision is rapidly growing. Polarization vision has been shown to simplify some important image understanding tasks that can be more difficult to be performed with intensity vision. Furthermore, it has computational efficiency because it only needs grayscale images and can be easily applied by a simple optical setup. Nowadays, we can find various types of polarization cameras in the market. However, they are very expensive. In our work, we will study and develop a low price polarization camera setup with parallel acquisition using a stereo system. This system requires only two general cameras e…

Cardiovascular effects of air pollution

Air pollution is composed of particulate matter (PM) and gaseous pollutants, such as nitrogen dioxide and ozone. PM is classified according to size into coarse particles (PM 10), fine particles (PM 2.5) and ultrafine particles. We aim to provide an original review of the scientific evidence from epidemiological and experimental studies examining the cardiovascular effects of outdoor air pollution. Pooled epidemiological studies reported that a 10 μg/m 3 increase in long-term exposure to PM 2.5 was associated with an 11% increase in cardiovascular mortality. Increased cardiovascular mortality was also related to long-term and short-term exposure to nitrogen dioxide. Exposure to air pollution…

Characterization of the 3D information of Calyptogena shells

We propose in this paper an application of multiresolution analysis techniques to extract information contained in the growth increments of a bivalve mollusk called: Calyptogena . The first stage consists in extracting a range image of the mollusk’s shell using a 3-D scanner. Applying a multiresolution analysis enables us to localize precisely those growth increments by preserving relevant details. Moreover, interesting spatial and frequency properties of the multiresolution analysis underline information contained on the shell. Intra-individual variation and inter-individual variations are compared to assume some conclusions as for the ontogenetic evolution of the animal such as periodicit…

Automatic differentiation of melanoma from dysplastic nevi.

International audience; Malignant melanoma causes the majority of deaths related to skin cancer. Nevertheless, it is the most treatable one, depending on its early diagnosis. The early prognosis is a challenging task for both clinicians and dermatologist, due to the characteristic similarities of melanoma with other skin lesions such as dysplastic nevi. In the past decades, several computerized lesion analysis algorithms have been proposed by the research community for detection of melanoma. These algorithms mostly focus on differentiating melanoma from benign lesions and few have considered the case of melanoma against dysplastic nevi. In this paper, we consider the most challenging task a…

Design and Calibration of an Omni-RGB plus D Camera

International audience; In this paper, we present the design of a new camera combining both predator-like and prey-like vision features. This setup provides both a spherical RGB-view and a directional depth-view of the environment. The model and calibration of the full set-up are described. A few examples will be given to demonstrate the interest and the versatility of such camera for robotics and video surveillance at the oral presentation.

Adapted processing of catadioptric images using polarization imaging

A non parametric method that defines a pixel neighborhood within catadioptric images is presented in this paper. It is based on an accurate modeling of the mirror shape by using polarization imaging. Unlike the most of current processing methods in the literature, this method is non-parametric and can deal with the deformation of catadioptric images. This paper demonstrates how an appropriate neighborhood can be derived from the polarization parameters by estimation of the degree of polarization and the angle of polarization which in return directly provide an adapted neighborhood of each pixel that can be used to perform image derivation, edge detection, interest point detection and namely…

“Scanning from Heating” and “Shape from Fluorescence”

3D surface acquisition is a subject which has been studied to a large extent. A significant number of techniques for acquiring shape have been proposed, and a wide range of commercial solutions are available. Nevertheless, today’s systems still have difficulties when digitizing objects with non-Lambertian surfaces in the visible light spectrum, as is the case of transparent, semi-transparent or highly reflective materials (e.g. glass, crystals, some plastics and shiny metals). In this chapter, some of the issues of traditional scanning systems are addressed by considering various approaches using the radioactive properties of materials, the polarization information of the reflected light as…

Tackling the Problem of Data Imbalancing for Melanoma Classification

Comunicació de congrés presentada a: 3rd International Conference on Bioimaging, BIOIMAGING 2016 - Part of 9th International Joint Conference on Biomedical Engineering Systems and Technologies, BIOSTEC 2016, Roma, Italy Malignant melanoma is the most dangerous type of skin cancer, yet melanoma is the most treatable kind of cancer when diagnosed at an early stage. In this regard, Computer-Aided Diagnosis systems based on machine learning have been developed to discern melanoma lesions from benign and dysplastic nevi in dermoscopic images. Similar to a large range of real world applications encountered in machine learning, melanoma classification faces the challenge of imbalanced data, where …

Multiscale Attention-Based Prototypical Network For Few-Shot Semantic Segmentation

International audience; Deep learning-based image understanding techniques require a large number of labeled images for training. Few-shot semantic segmentation, on the contrary, aims at generalizing the segmentation ability of the model to new categories given only a few labeled samples. To tackle this problem, we propose a novel prototypical network (MAPnet) with multiscale feature attention. To fully exploit the representative features of target classes, we firstly extract rich contextual information of labeled support images via a multiscale feature enhancement module. The learned prototypes from support features provide further semantic guidance on the query image. Then we adaptively i…

Measurement of three-dimensional mirror parameters by polarization imaging applied to catadioptric camera calibration

International audience; We present a new efficient method for calibration of cata- dioptric sensors. The method is based on an accurate measurement of the three-dimensional parameters of the mirror through polariza- tion imaging. While inserting a rotating polarizer between the cam- era and the mirror, the system is automatically calibrated without any calibration patterns. Moreover, this method permits most of the constraints related to the calibration of catadioptric systems to be relaxed. We show that, contrary to our system, the traditional meth- ods of calibration are very sensitive to misalignment of the camera axis and the symmetry axis of the mirror. From the measurement of three-di…

From Nowhere to Everywhere

International audience; This paper presents a synthetic view of a variety of projects built upon an Erasmuss Mundus Master Course. It highlights double degree programs, European credits transfer, joint PhDs, research collaborations as well as few other related European projects going from Thematic Networks to another Erasmus Mundus Course.

Procede de commande de pulverisation, dispositif et programme correspondant

Multimodal 2D Image to 3D Model Registration via a Mutual Alignment of Sparse and Dense Visual Features

International audience; Many fields of application could benefit from an accurate registration of measurements of different modalities over a known 3D model. However, aligning a 2D image to a 3D model is a challenging task and is even more complex when the two have a different modality. Most of the 2D/3D registration methods are based on either geometric or dense visual features. Both have their own advantages and their own drawbacks. We propose, in this paper, to mutually exploit the advantages of one feature type to reduce the drawbacks of the other one. For this, an hybrid registration framework has been designed to mutually align geometrical and dense visual features in order to obtain …

Contributions à l’imagerie polarimétrique et à ses applications en vision pour la robotique

Ce mémoire présente le bilan de l'ensemble de mes travaux de recherche effectués de septembre 2006 à juin 2019 au sein de l'équipe Creusotine du laboratoire Le2i devenue équipe Vibot ERL CNRS 6000 depuis janvier 2018. Les principales activités de recherche que j'ai menées au cours de ces 10 dernières années autour de l'imagerie polarimétrique, de la vision omnidirectionnelle et de leurs applications en vision pour la robotique seront particulièrement détaillées dans ce document. Elles seront présentées selon deux grandes parties : la première concernera plutôt l'aspect mise en œuvre et calibrage des caméras et la seconde se concentrera sur les applications potentielles en vision pour la rob…

Classification of Melanoma Lesions Using Sparse Coded Features and Random Forests

International audience; Malignant melanoma is the most dangerous type of skin cancer, yet it is the most treatable kind of cancer, conditioned by its early diagnosis which is a challenging task for clinicians and dermatologists. In this regard, CAD systems based on machine learning and image processing techniques are developed to differentiate melanoma lesions from benign and dysplastic nevi using dermoscopic images. Generally, these frameworks are composed of sequential processes: pre-processing, segmentation, and classification. This architecture faces mainly two challenges: (i) each process is complex with the need to tune a set of parameters, and is specific to a given dataset; (ii) the…

Autonomous patrol and surveillance system using unmanned aerial vehicles

Unmanned aerial vehicles (UAVs) are increasingly popular for civil applications due to their flight capabilities and mobility. This paper proposes an indoor autonomous patrol and surveillance system using unmanned aerial vehicles. The system consists of six components: (1) a low cost vision-based pose estimation of UAVs, (2) vision-based state estimation, (3) UAV patrol path planning, (4) UAV controller to desired waypoints, (5) manual joystick controller and (6) assignment of priority hierarchy to multiple control inputs of a robot. The components were integrated successfully and the designed system was experimented in an indoor setup. Results has shown that the proposed system is suitable…

Active lighting applied to three-dimensional reconstruction of specular metallic surfaces by polarization imaging

International audience; In the field of industrial vision, the three-dimensional inspection of highly reflective metallic objects is still a delicate task. We deal with a new automated three-dimensional inspection system based on polarization analysis. We first present an extension of the shape-from-polarization method for dielectric surfaces to metallic surfaces. Then, we describe what we believe to be a new way of solving the ambiguity concerning the normal orientation with an active lighting system. Finally, applications to shape-defect detection are discussed, and the efficiency of the system to discriminate defects on specular metallic objects made by stamping and polishing is presente…

Active lighting applied to three dimensional reconstruction of specular metallic surfaces by polarization imaging

This thesis deals with a new automated tridimensional inspection system for specular metallic surfaces, based on polarization analysis. We first present an extension of the ‘Shape from Polarization' method for dielectric to metallic surfaces. After being reflected, an unpolarized light wave becomes partially linearly polarized, depending on the surface normal and on the refractive index of the media it impinges on. Hence, by measuring the polarization parameters of the reflected light, the surface normals are computed. The ambiguity concerning the normals orientation is solved here with a new active lighting system. Then, we describe and compare numerical methods for reconstructing the trid…

Choosing local matching score method for stereo matching based-on polarization imaging

Polarization imaging is a powerful tool to observe hidden information from an observed object. It has significant advantages, such as computational efficiency (it only needs gray scale images) and can be easily applied by adding a polarizer in front of a camera. Many researchers used polarization in various areas of computer vision, such as object recognition, segmentation and so on. However, there is very little research in stereo vision based on polarization. Stereo vision is a well known technique for obtaining depth information from pairs of stereo digital images. One of the main focuses of research in this area is to get accurate stereo correspondences. In our work, we will study and d…

(Omni-)Pola-Catadioptric Vision: Ideas and Experiments

International audience

Outdoor Scenes Pixel-wise Semantic Segmentation using Polarimetry and Fully Convolutional Network

International audience; In this paper, we propose a novel method for pixel-wise scene segmentation application using polarimetry. To address the difficulty of detecting highly reflective areas such as water and windows, we use the angle and degree of polarization of these areas, obtained by processing images from a polarimetric camera. A deep learning framework, based on encoder-decoder architecture, is used for the segmentation of regions of interest. Different methods of augmentation have been developed to obtain a sufficient amount of data, while preserving the physical properties of the polarimetric images. Moreover, we introduce a new dataset comprising both RGB and polarimetric images…

Temporal Trends in Transcatheter Aortic Valve Replacement in France

Background - Transcatheter aortic valve replacement (TAVR) is standard therapy for patients with severe aortic stenosis who are at high surgical risk. However, national data regarding procedural characteristics and clinical outcomes over time are limited. Objectives - The aim of this study was to assess nationwide performance trends and clinical outcomes of TAVR during a 6-year period. Methods - TAVRs performed in 48 centers across France between January 2013 and December 2015 were prospectively included in the FRANCE TAVI (French Transcatheter Aortic Valve Implantation) registry. Findings were further compared with those reported from the FRANCE 2 (French Aortic National CoreValve and Edwa…

Effects of alirocumab on types of myocardial infarction : insights from the ODYSSEY OUTCOMES trial

Gislason, Gunnar H/0000-0002-0548-402X; Malynovsky, Yaroslav V/0000-0002-9118-1104; Bhatt, Deepak L./0000-0002-1278-6245; Nikolaev, Konstantin/0000-0003-4601-6203; Sherwood, Matthew/0000-0002-4305-5883; Chumakova, Galina A/0000-0002-2810-6531; Raffel, Owen C/0000-0001-5470-7050; Leonardi, Sergio/0000-0002-4800-6132; Tse, Hung Fat/0000-0002-9578-7808; Reshetko, Olga/0000-0003-3107-7636; Pereira, Helder/0000-0001-8656-4883; Racca, Vittorio/0000-0002-4465-3789; Podoleanu, Cristian/0000-0001-9987-2519; Ersanli, Murat/0000-0003-1847-3087; Muenzel, Thomas/0000-0001-5503-4150; Sandhu, Manjinder/0000-0003-2538-2079; Taskinen, Marja-Riitta/0000-0002-6229-3588; bastos, jose/0000-0002-9526-3123; Manak…

Polarization-based Robot Orientation and Navigation

From insects in your garden to creatures in the sea, inspiration can be drawn from nature to design a whole new class of smart robotic devices. These smart machines may move like living creatures. They can be launched toward a specific target for a pre-defined task. Bio-inspiration is developing to meet the needs of many challenges particularly in machine vision. Some species in the animal kingdom like cephalopods, crustaceans and insects are distinguished with their visual capabilities which are strongly improved by means of polarization. This work surveys the most recent research in the area of bio-inspired polarization based robot orientation and navigation. Firstly, the authors will bri…

Polarimetric image augmentation

Robotics applications in urban environments are subject to obstacles that exhibit specular reflections hampering autonomous navigation. On the other hand, these reflections are highly polarized and this extra information can successfully be used to segment the specular areas. In nature, polarized light is obtained by reflection or scattering. Deep Convolutional Neural Networks (DCNNs) have shown excellent segmentation results, but require a significant amount of data to achieve best performances. The lack of data is usually overcomed by using augmentation methods. However, unlike RGB images, polarization images are not only scalar (intensity) images and standard augmentation techniques cann…

Visual Behaviour Based Bio-Inspired Polarization Techniques in Computer Vision and Robotics

For long time, it was thought that the sensing of polarization by animals is invariably related to their behavior, such as navigation and orientation. Recently, it was found that polarization can be part of a high-level visual perception, permitting a wide area of vision applications. Polarization vision can be used for most tasks of color vision including object recognition, contrast enhancement, camouflage breaking, and signal detection and discrimination. The polarization based visual behavior found in the animal kingdom is briefly covered. Then, the authors go in depth with the bio-inspired applications based on polarization in computer vision and robotics. The aim is to have a comprehe…

P2D: a self-supervised method for depth estimation from polarimetry

Monocular depth estimation is a recurring subject in the field of computer vision. Its ability to describe scenes via a depth map while reducing the constraints related to the formulation of perspective geometry tends to favor its use. However, despite the constant improvement of algorithms, most methods exploit only colorimetric information. Consequently, robustness to events to which the modality is not sensitive to, like specularity or transparency, is neglected. In response to this phenomenon, we propose using polarimetry as an input for a self-supervised monodepth network. Therefore, we propose exploiting polarization cues to encourage accurate reconstruction of scenes. Furthermore, we…

Promotion et Développement d'un Master Erasmus Mundus - L'Exemple du VIBOT

Cet article decrit l’offre de formation a l’internationale proposee au Centre Universitaire Condorcet du Creusot (Universite de Bourgogne) dans le domaine de la vision par ordinateur et de la robotique. Il presente l’organisation particuliere de ces formations et les actions de support mises en place pour en assurer la perennite.

Design and calibration of an omni-RGB+D camera

International audience; In this paper, we present the design of a new camera combining both predator-like and prey-like vision features. This setup provides both a spherical RGB-view and a directional depth-view of the environment. The model and calibration of the full setup are described. A few examples will be given to demonstrate the interest and the versatility of such camera for robotics and video surveillance at the oral presentation.

Design and Calibration of a Specialized Polydioptric Camera Rig

International audience; It has been observed in the nature that all creatures have evolved highly exclusive sensory organs depending on their habitat and the form of resources availability for their survival. In this project, a novel omnidirectional camera rig, inspired from natural vision sensors, is proposed. It is exclusively designed to operate for highly specified tasks in the field of mobile robotics. Navigation problems on uneven terrains and detection of the moving objects while the robot is itself in motion are the core problems that omnidirectional systems tackle. The proposed omnidirectional system is a compact and a rigid vision system with dioptric cameras that provide a 360° f…

Introducing the Concept of Hyperactivity in Multi Agent Systems

International audience; Software Agents are no longer the simple communication gateways for devices to interconnect using one or more networks. With Multi Agent Systems contributing in a wide spectrum of intelligent systems, the Agents are in a more proactive role than just being responsible for passing messages between their respective base systems. Agent Relation Charts and the Hyperactive Transaction Model in general is one of the recent attempts of developing a multi-view design model for Multi Agent Systems. The model has made a clear distinction in the regular and intelligent activities of an agent. Based on these differences, the agents are classified into three main categories named…

Éclairage structuré et vision active pour le contrôle qualité de surfaces métalliques réfléchissantes

Dans le domaine de la vision artificielle, l’inspection de surfaces parfaitement reflechissantes est une tâche particulierement delicate, car elle implique de maitriser completement l’eclairage de la scene. Nous proposons dans ce papier, deux methodes complementaires permettant le controle qualite de surfaces metalliques reflechissantes. La premiere methode est un systeme d’acquisition utilisant un eclairage dynamique a franges. Elle permet de reveler les defauts d’aspect sur les surfaces a faible courbure. Dans des zones locales a plus forte courbure, il devient necessaire d’avoir acces a des informations tridimensionnelles afin de pouvoir segmenter les defauts. Nous proposons ainsi une me…

Active Lighting Applied to Shape from Polarization

We present a new way of solving the ambiguity that appears in the shape from polarization method. This ambiguity, concerning the normals orientation of the surface, was previously solved thanks to a propagation algorithm. In this paper, the ambiguity is solved thanks to a special active lighting. This method is less time computing and is more robust to noise acquisition. We first demonstrate how to compute the right normals on the whole surface. Then, we show an application on the 3D reconstruction of a specular metallic object made by stamping and polishing.

Bio-Inspired Polarization Vision Techniques for Robotics Applications

Researchers have been inspired by nature to build the next generation of smart robots. Based on the mechanisms adopted by the animal kingdom, research teams have developed solutions to common problems that autonomous robots faced while performing basic tasks. Polarization-based behaviour is one of the most distinctive features of some species of the animal kingdom. Light polarization parameters significantly expand visual capabilities of autonomous robots. Polarization vision can be used for most tasks of color vision, like object recognition, contrast enhancement, camouflage breaking, and signal detection and discrimination. In this chapter, the authors briefly cover polarization-based vis…

Deep multimodal fusion for semantic image segmentation: A survey

International audience; Recent advances in deep learning have shown excellent performance in various scene understanding tasks. However, in some complex environments or under challenging conditions, it is necessary to employ multiple modalities that provide complementary information on the same scene. A variety of studies have demonstrated that deep multimodal fusion for semantic image segmentation achieves significant performance improvement. These fusion approaches take the benefits of multiple information sources and generate an optimal joint prediction automatically. This paper describes the essential background concepts of deep multimodal fusion and the relevant applications in compute…

N-QGN: Navigation Map from a Monocular Camera using Quadtree Generating Networks

Monocular depth estimation has been a popular area of research for several years, especially since self-supervised networks have shown increasingly good results in bridging the gap with supervised and stereo methods. However, these approaches focus their interest on dense 3D reconstruction and sometimes on tiny details that are superfluous for autonomous navigation. In this paper, we propose to address this issue by estimating the navigation map under a quadtree representation. The objective is to create an adaptive depth map prediction that only extract details that are essential for the obstacle avoidance. Other 3D space which leaves large room for navigation will be provided with approxi…

Sub-optimal waypoints, UAV path planning and mosaicing application

International audience; Create a complete system of video surveillance using camera mounted on a robot like UAV to maintain optimized vast area coverage and reconstruct an image by using mosaicing techniques. This paper demonstrated the efficiency of using one UAV to cover vast area using optimized positions.