Attitude estimation from polarimetric cameras

International audience; In the robotic field, navigation and path planning applications benefit from a wide range of visual systems (e.g. perspective cameras, depth cameras, catadioptric cameras, etc.). In outdoor conditions, these systems capture information in which sky regions cover a major segment of the images acquired. However, sky regions are discarded and are not considered as visual cue in vision applications. In this paper, we propose to estimate attitude of Unmanned Aerial Vehicle (UAV) from sky information using a polarimetric camera. Theoretically , we provide a framework estimating the attitude from the skylight polarized patterns. We showcase this formulation on both simulate…

Range image binarization: applications to wooden stamps analysis

This paper deals with the analysis of ancient wooden stamps. The aim is to extract a binary image from the stamp. This image must be the closer to the image produced by inking and using a printing press with the stamps. A range image based method is proposed to extract a stamped image from the stamps. The range image acquisition from a 3D laser scanner is presented. Pre-filtering for range image enhancement is detailed. The range image binarization method is based on an adaptive thresholding. Few simple processes applied on the range image enable a final binarized image computing. The proposed method provides here a very efficient way to perform "virtual" stampings with ancient wooden stamp…

An efficient method for fully automatic 3D digitization of unknown objects

Our goal is to develop a complete and automatic scanning strategy with minimum prior information about the object shape. We aim to establish a methodology for the automation of the 3D digitization process. The paper presents a novel approach to determine the Next Best View (NBV) for an efficient reconstruction of highly accurate 3D models. Our method is based on the classification of the acquired surfaces into Well Visible and Barely Visible combined with a best view selection algorithm based on mean shift, which avoids unreachable positions. Our approach is applicable to all kinds of range sensors. To prove the efficiency and the robustness of our method, test objects are first scanned man…

Design and Calibration of an Omni-RGB plus D Camera

International audience; In this paper, we present the design of a new camera combining both predator-like and prey-like vision features. This setup provides both a spherical RGB-view and a directional depth-view of the environment. The model and calibration of the full set-up are described. A few examples will be given to demonstrate the interest and the versatility of such camera for robotics and video surveillance at the oral presentation.

Trois algorithmes intelligents pour la numérisation 3D automatique d'objets inconnus

National audience; Ce papier propose trois approches itératives et intelligentes de planification de vue pour la numérisation 3D d'objets sans connaissance a priori de leurs formes. La première méthode est une approche simple et naïve basée sur la génération d'un ensemble de points de vues par échantillonnage régulier de l'enveloppe englobante des données acquises. La deuxième méthode est basée sur une analyse de l'orientation des différentes parties acquises. La troisième méthode vise à explorer les parties de l'objet qui figurent dans la limite du champ de visibilité et est basée sur un couplage de la visibilité angulaire avec la visibilité réelle par lancer de rayons. Les résultats de nu…

Développement, optimisation et utilisation d'un simulateur de mastication "bouche artificielle"

National audience

Measurement of three-dimensional mirror parameters by polarization imaging applied to catadioptric camera calibration

International audience; We present a new efficient method for calibration of cata- dioptric sensors. The method is based on an accurate measurement of the three-dimensional parameters of the mirror through polariza- tion imaging. While inserting a rotating polarizer between the cam- era and the mirror, the system is automatically calibrated without any calibration patterns. Moreover, this method permits most of the constraints related to the calibration of catadioptric systems to be relaxed. We show that, contrary to our system, the traditional meth- ods of calibration are very sensitive to misalignment of the camera axis and the symmetry axis of the mirror. From the measurement of three-di…

Scale invariant line matching on the sphere

International audience; This paper proposes a novel approach of line matching across images captured by different types of cameras, from perspective to omnidirectional ones. Based on the spherical mapping, this method utilizes spherical SIFT point features to boost line matching and searches line correspondences using an affine invariant measure of similarity. It permits to unify the commonest cameras and to process heterogeneous images with the least distortion of visual information.

Procede de commande de pulverisation, dispositif et programme correspondant

Multimodal 2D Image to 3D Model Registration via a Mutual Alignment of Sparse and Dense Visual Features

International audience; Many fields of application could benefit from an accurate registration of measurements of different modalities over a known 3D model. However, aligning a 2D image to a 3D model is a challenging task and is even more complex when the two have a different modality. Most of the 2D/3D registration methods are based on either geometric or dense visual features. Both have their own advantages and their own drawbacks. We propose, in this paper, to mutually exploit the advantages of one feature type to reduce the drawbacks of the other one. For this, an hybrid registration framework has been designed to mutually align geometrical and dense visual features in order to obtain …

Real-time Multispectral Image Processing and Registration on 3D Point Cloud for Vineyard Analysis

International audience; Nowadays, precision agriculture and precision viticulture are under strong development. In order to accomplish effective actions, robots require robust perception of the culture and the surrounding environment. Computer vision systems have to identify plant parts (branches, stems, leaves, flowers, fruits, vegetables, etc.) and their respective health status. Moreover, they must merge various plant information, to measure agronomic indices, to classify them and finally to extract data to enable the agriculturist or expert to make a relevant decision. We propose a real-time method to acquire, process and register multispectral images fused to 3D. The sensors system, co…

Autonomous patrol and surveillance system using unmanned aerial vehicles

Unmanned aerial vehicles (UAVs) are increasingly popular for civil applications due to their flight capabilities and mobility. This paper proposes an indoor autonomous patrol and surveillance system using unmanned aerial vehicles. The system consists of six components: (1) a low cost vision-based pose estimation of UAVs, (2) vision-based state estimation, (3) UAV patrol path planning, (4) UAV controller to desired waypoints, (5) manual joystick controller and (6) assignment of priority hierarchy to multiple control inputs of a robot. The components were integrated successfully and the designed system was experimented in an indoor setup. Results has shown that the proposed system is suitable…

PROCEDE DE NUMERISATION TRIDIMENSIONNELLE AUTOMATIQUE

Study of the imaging conditions and processing for the aspect control of specular surfaces

A vision system capable of imaging and detecting defects on reflective nonplanar surfaces in the production line at a high cadence is presented in this paper. Defects are typically dust located under the metallic layer of packaging products used in cosmetic industries. To realize this processing, structured lighting which reveals the defects in the image is proposed. Defects appear clearly in the images like a set of brilliant pixels in dark zones. The signature of the defect is then obtained. The size of this signature does not depend linearly on the size of the defect. It is a function of the observation angle. In order to realize a precise and robust process, the necessity of acquiring s…

Outdoor Scenes Pixel-wise Semantic Segmentation using Polarimetry and Fully Convolutional Network

International audience; In this paper, we propose a novel method for pixel-wise scene segmentation application using polarimetry. To address the difficulty of detecting highly reflective areas such as water and windows, we use the angle and degree of polarization of these areas, obtained by processing images from a polarimetric camera. A deep learning framework, based on encoder-decoder architecture, is used for the segmentation of regions of interest. Different methods of augmentation have been developed to obtain a sufficient amount of data, while preserving the physical properties of the polarimetric images. Moreover, we introduce a new dataset comprising both RGB and polarimetric images…

3D reconstruction of hot metallic surfaces for industrial part characterization

During industrial forging of big hot metallic shells, it is necessary to regularly measure the dimensions of the parts, especially the inner and outer diameters and the thickness of the walls, in order to decide when to stop the forging process. The inner and outer diameters of the shells range from 4 to 6 meters and to measure them a large ruler is placed horizontally at the end of the shell. Two blacksmiths standing on each side of the ruler at about ten meters from it visually reads the graduations on the ruler in order to determine the inner and outer diameters from which the thickness of the wall is determined. This operation is carried out several times during a forging process and it…

Polarimetric image augmentation

Robotics applications in urban environments are subject to obstacles that exhibit specular reflections hampering autonomous navigation. On the other hand, these reflections are highly polarized and this extra information can successfully be used to segment the specular areas. In nature, polarized light is obtained by reflection or scattering. Deep Convolutional Neural Networks (DCNNs) have shown excellent segmentation results, but require a significant amount of data to achieve best performances. The lack of data is usually overcomed by using augmentation methods. However, unlike RGB images, polarization images are not only scalar (intensity) images and standard augmentation techniques cann…

P2D: a self-supervised method for depth estimation from polarimetry

Monocular depth estimation is a recurring subject in the field of computer vision. Its ability to describe scenes via a depth map while reducing the constraints related to the formulation of perspective geometry tends to favor its use. However, despite the constant improvement of algorithms, most methods exploit only colorimetric information. Consequently, robustness to events to which the modality is not sensitive to, like specularity or transparency, is neglected. In response to this phenomenon, we propose using polarimetry as an input for a self-supervised monodepth network. Therefore, we propose exploiting polarization cues to encourage accurate reconstruction of scenes. Furthermore, we…

3D dimensional measurement of large hot metallic shells

Design and calibration of an omni-RGB+D camera

International audience; In this paper, we present the design of a new camera combining both predator-like and prey-like vision features. This setup provides both a spherical RGB-view and a directional depth-view of the environment. The model and calibration of the full setup are described. A few examples will be given to demonstrate the interest and the versatility of such camera for robotics and video surveillance at the oral presentation.

Design and Calibration of a Specialized Polydioptric Camera Rig

International audience; It has been observed in the nature that all creatures have evolved highly exclusive sensory organs depending on their habitat and the form of resources availability for their survival. In this project, a novel omnidirectional camera rig, inspired from natural vision sensors, is proposed. It is exclusively designed to operate for highly specified tasks in the field of mobile robotics. Navigation problems on uneven terrains and detection of the moving objects while the robot is itself in motion are the core problems that omnidirectional systems tackle. The proposed omnidirectional system is a compact and a rigid vision system with dioptric cameras that provide a 360° f…

View planning algorithms for fully automatic 3D acquisition of unknown objects

International audience; This paper addresses the view planning problem for the digitization of 3D objects without prior knowledge on their shape and presents a novel method called Orientation, Angle and Covering (OAC). The proposed method is based on a combination of two concepts: the Mass Vector Chains (MVC) and the Measurability Matrix. The MVC allows to define the global orientation of the scanned part. All of the view points are sorted using an orientation criterion to define a first set of candidates for the Next Best View (NBV). The Measurability Matrix allows to determine the coverage rate for each candidate. The covering criterion leads to reduce the number of view points of the fir…

Éclairage structuré et vision active pour le contrôle qualité de surfaces métalliques réfléchissantes

Dans le domaine de la vision artificielle, l’inspection de surfaces parfaitement reflechissantes est une tâche particulierement delicate, car elle implique de maitriser completement l’eclairage de la scene. Nous proposons dans ce papier, deux methodes complementaires permettant le controle qualite de surfaces metalliques reflechissantes. La premiere methode est un systeme d’acquisition utilisant un eclairage dynamique a franges. Elle permet de reveler les defauts d’aspect sur les surfaces a faible courbure. Dans des zones locales a plus forte courbure, il devient necessaire d’avoir acces a des informations tridimensionnelles afin de pouvoir segmenter les defauts. Nous proposons ainsi une me…

Near-infrared imaging and structured light ranging for automatic catheter insertion

Vein localization and catheter insertion constitute the first and perhaps most important phase of many medical procedures. Currently, catheterization is performed manually by trained personnel. This process can prove problematic, however, depending upon various physiological factors of the patient. We present in this paper initial work for localizing surface veins via near-infrared (NIR) imaging and structured light ranging. The eventual goal of the system is to serve as the guidance for a fully automatic (i.e., robotic) catheterization device. Our proposed system is based upon near-infrared (NIR) imaging, which has previously been shown effective in enhancing the visibility of surface vein…

3D laser system for shell dimension measurement during forging

View Planning Approach for Automatic 3D Digitization of Unknown Objects

International audience; This paper addresses the view planning problem for the digitization of 3D objects without prior knowledge on their shape and presents a novel surface approach for the Next Best View (NBV) computation. The proposed method uses the concept of Mass Vector Chains (MVC) to define the global orientation of the scanned part. All of the viewpoints satisfying an orientation constraint are clustered using the Mean Shift technique to construct a first set of candidates for the NBV. Then, a weight is assigned to each mode according to the elementary orientations of its different descriptors. The NBV is chosen among the modes with the highest weights and which comply with the rob…