0000000000970050
AUTHOR
Pascal Vasseur
Homography based egomotion estimation with a common direction
International audience; In this paper, we explore the different minimal solutions for egomotion estimation of a camera based on homography knowing the gravity vector between calibrated images. These solutions depend on the prior knowledge about the reference plane used by the homography. We then demonstrate that the number of matched points can vary from two to three and that a direct closed-form solution or a Gröbner basis based solution can be derived according to this plane. Many experimental results on synthetic and real sequences in indoor and outdoor environments show the efficiency and the robustness of our approach compared to standard methods.
Summarizing Large Scale 3D Point Cloud for Navigation Tasks
International audience; Democratization of 3D sensor devices makes 3D maps building easier especially in long term mapping and autonomous navigation. In this paper we present a new method for summarizing a 3D map (dense cloud of 3D points). This method aims to extract a summary map facilitating the use of this map by navigation systems with limited resources (smartphones, cars, robots...). This Vision-based summarizing process is applied in a fully automatic way using the photometric, geometric and semantic information of the studied environment.
LMI-based 2D-3D Registration: from Uncalibrated Images to Euclidean Scene
International audience; This paper investigates the problem of registering a scanned scene, represented by 3D Euclidean point coordinates , and two or more uncalibrated cameras. An unknown subset of the scanned points have their image projections detected and matched across images. The proposed approach assumes the cameras only known in some arbitrary projective frame and no calibration or autocalibration is required. The devised solution is based on a Linear Matrix Inequality (LMI) framework that allows simultaneously estimating the projective transformation relating the cameras to the scene and establishing 2D-3D correspondences without triangulating image points. The proposed LMI framewo…
A homography formulation to the 3pt plus a common direction relative pose problem
International audience; In this paper we present an alternative formulation for the minimal solution to the 3pt plus a common direction relative pose prob-lem. Instead of the commonly used epipolar constraint we use the homog-raphy constraint to derive a novel formulation for the 3pt problem. This formulation allows the computation of the normal vector of the plane defined by the three input points without any additional computation in addition to the standard motion parameters of the camera. We show the working of the method on synthetic and real data sets and compare it to the standard 3pt method and the 5pt method for relative pose estima-tion. In addition we analyze the degenerate condi…
Short baseline line matching for central imaging systems
We develop a generic line matching method especially applicable to omnidirectional images taken from constructed scenes with short baseline motion where the motion of the imaging system between two views is mainly an arbitrary rotation and the translation of the camera between two views with respect to its distance to the imaged scene is negligible. We start by studying the relationship between images of lines on unitary sphere followed by proposing a simple algorithm for simultaneously matching vanishing points and lines. The developed algorithm is very simple, yet it works on images captured by all types of central imaging systems, including perspective, fish-eye and catadioptric images. …
Local path planning in a complex environment for self-driving car
This paper introduces an local path planning algorithm for the self-driving car in a complex environment. The proposed algorithm is composed of three parts: the novel path representation, the collision detection and the path modification using a voronoi cell. The novel path representation provides convenience for checking the collision and modifying the path and continuous control input for steering wheel rather than way point navigation. The proposed algorithm were applied to the self-driving car, EureCar(KAIST) and its applicability and feasibility of real time use were validated.
Motion estimation and reconstruction of piecewise planar scenes from two views
The task of recovering the camera motion relative to the environment (motion estimation) is fundamental to many computer vision applications. We present an algorithm for reconstruction of piece-wise planar scenes from only two views and based on minimum line correspondences. We first recover camera rotation by matching vanishing points based on the methods already exist in the literature and then recover the camera translation by searching among a family of hypothesized planes passing through one line. Unlike algorithms based on line segments, the presented algorithm does not require an overlap between two line segments or more than one line correspondence across more than two views to reco…
Omnidirectional vision for UAV: applications to attitude, motion and altitude estimation for day and night conditions
International audience; This paper presents the combined applications of omnidirectional vision featuring on its application to aerial robotics. Omnidirectional vision is first used to compute the attitude, altitude and motion not only in rural environment but also in the urban space. Secondly, a combination of omnidirectional and perspective cameras permits to estimate the altitude. Finally we present a stereo system consisting of an omnidirectional camera with a laser pattern projector enables to calculate the altitude and attitude during the improperly illuminated conditions to dark environments. We demonstrate that omnidirectional camera in conjunction with other sensors is suitable cho…
Localization of 2D Cameras in a Known Environment Using Direct 2D-3D Registration
International audience; In this paper we propose a robust and direct 2D-to- 3D registration method for localizing 2D cameras in a known 3D environment. Although the 3D environment is known, localizing the cameras remains a challenging problem that is particularly undermined by the unknown 2D-3D correspondences, outliers, scale ambiguities and occlusions. Once the cameras are localized, the Structure-from-Motion reconstruction obtained from image correspondences is refined by means of a constrained nonlinear optimization that benefits from the knowledge of the scene. We also propose a common optimization framework for both localization and refinement steps in which projection errors in one v…
A Generic Method of Line Matching for Central Imaging Systems under Short-Baseline Motion
Line matching across images taken by a central imaging system (perspective or catadioptric) with focus on short baseline motion of the system is proposed. The relationship between images of lines on unitary sphere is studied and a simple algorithm for matching lines are proposed assuming the rotation of the system is known apriori or it can be estimated from some correspondences in two views. Two methods are discussed for retrieving R in the case it is not known apriori. Experimental results on both synthetic and real images are also presented.
A Geometrical Approach for Vision Based Attitude and Altitude Estimation for UAVs in Dark Environments
International audience; This paper presents a single camera and laser system dedicated to the realtime estimation of attitude and altitude for unmanned aerial vehicles (UAV) under low illumination conditions to dark environments. The fisheye camera allows to cover a large field of view (FOV). The approach, close to structured light systems, uses the geometrical information obtained by the projection of a laser circle onto the ground plane and perceived by the camera. We propose some experiments based on simulated data and real sequences. The results show good agreement with the ground truth values from the commercial sensors in terms of its accuracy and correctness. The results also prove i…
Estimation de mouvement d'un drone à partir d'un capteur stéréo hybride
Motion and velocity are two of the most important parameters to be known for an Unmanned Aerial Vehicle (UAV) especially during critical maneuvers such as landing or steady flight. In this paper, we present mixed stereoscopic vision system made of a fish-eye camera and a perspective camera for motion estimation. Contrary to classical stereoscopic systems based on feature matching between cameras, we propose an algorithm which tracks and exploits points in each camera independently. The omnidirectional view estimates the orientation of the motion while the perspective view contribute to estimate the scale of the translation and brings accuracy. By fusing points tracked in each camera and kno…
MOISST: Multimodal Optimization of Implicit Scene for SpatioTemporal calibration
With the recent advances in autonomous driving and the decreasing cost of LiDARs, the use of multimodal sensor systems is on the rise. However, in order to make use of the information provided by a variety of complimentary sensors, it is necessary to accurately calibrate them. We take advantage of recent advances in computer graphics and implicit volumetric scene representation to tackle the problem of multi-sensor spatial and temporal calibration. Thanks to a new formulation of the Neural Radiance Field (NeRF) optimization, we are able to jointly optimize calibration parameters along with scene representation based on radiometric and geometric measurements. Our method enables accurate and …
Estimation de la pose d'une caméra dans un environnement connu à partir d'un recalage 2D-3D
National audience; Nous proposons une méthode directe de recalage robuste 2D-3D permettant de localiser une caméra dans un environnement 3D connu. Il s'agit d'un problème rendu particulièrement difficile par l'absence de correspondances entre les points 3D du nuage et les points 2D. A cette difficulté, s'ajoute la différence d'échelle entre le nuage 3D connu et le nuage 3D reconstruit à partir d'images qui, de plus, peut contenir des points aberrants et des occultations. Notre méthode consiste en l'optimisation d'une fonctionnelle de manière itérative en deux étapes : estimation de la pose de la caméra et mise en correspondance 2D-3D. Ainsi, nous obtenons une méthode d'estimation conjointe …
Line based motion estimation and reconstruction of piece-wise planar scenes
We present an algorithm for reconstruction of piece-wise planar scenes from only two views and based on minimum line correspondences. We first recover camera rotation by matching vanishing points based on the methods already exist in the literature and then recover the camera translation by searching among a family of hypothesized planes passing through one line. Unlike algorithms based on line segments, the presented algorithm does not require an overlap between two line segments or more that one line correspondence across more than two views to recover the translation and achieves the goal by exploiting photometric constraints of the surface around the line. Experimental results on real i…
Analysis of Low-Altitude Aerial Sequences for Road Traffic Diagnosis using Graph Partitioning and Markov Hierarchical Models
International audience; This article focuses on an original approach aiming the processing of low-altitude aerial sequences taken from an helicopter (or drone) and presenting a road traffic. Proposed system attempts to extract vehicles from acquired sequences. Our approach begins with detecting the primitives of sequence images. At the time of this step of segmentation, the system computes dominant motion for each pair of images. This motion is computed using wavelets analysis on optical flow equation and robust techniques. Interesting areas (areas not affected by the dominant motion) are detected thanks to a Markov hierarchical model. Primitives stemming from segmentation and interesting a…
Central catadioptric image processing with geodesic metric
International audience; Because of the distortions produced by the insertion of a mirror, catadioptric images cannot be processed similarly to classical perspective images. Now, although the equivalence between such images and spherical images is well known, the use of spherical harmonic analysis often leads to image processing methods which are more difficult to implement. In this paper, we propose to define catadioptric image processing from the geodesic metric on the unitary sphere. We show that this definition allows to adapt very simply classical image processing methods. We focus more particularly on image gradient estimation, interest point detection, and matching. More generally, th…
Vision based attitude and altitude estimation for UAVs in dark environments
This paper presents a system dedicated to the real-time estimation of attitude and altitude for unmanned aerial vehicles (UAV) under low light and dark environment. This system consists in a fisheye camera, which allows to cover a large field of view (FOV), and a laser circle projector mounted on a fixed baseline. The approach, close to structured light systems, uses the geometrical information obtained by the projection of the laser circle onto the ground plane and perceived by the camera. We present a theoretical study of the system in which the camera is modelled as a sphere and show that the estimation of a conic on this sphere allows to obtain the attitude and the altitude of the robot…
Line Segment Based Structure and Motion from Two Views: a Practical Issue
International audience
Rotation estimation and vanishing point extraction by omnidirectional vision in urban environment
International audience; Rotation estimation is a fundamental step for various robotic applications such as automatic control of ground/aerial vehicles, motion estimation and 3D reconstruction. However it is now well established that traditional navigation equipments, such as global positioning systems (GPSs) or inertial measurement units (IMUs), suffer from several disadvantages. Hence, some vision-based works have been proposed recently. Whereas interesting results can be obtained, the existing methods have non-negligible limitations such as a difficult feature matching (e.g. repeated textures, blur or illumination changes) and a high computational cost (e.g. analyze in the frequency domai…
Estimation de mouvement d'un système stéréoscopique hybride à partir des droites
We present a motion estimation approach for hybrid stereo rigs using line images. The proposed method can be applied to a hybrid system built up from any single view point (SVP) cameras such as perspective, central catadioptric and fisheye cameras. Such configuration combines advantageous characteristics of different types of cameras. Images captured by SVP imaging devices may be mapped to spherical images using the unified projection model. It is possible to recover the camera orientations using vanishing points of parallel line sets. We then estimate the translations from known rotations and line images on the spheres. The algorithm has been validated on simulated data and real images tak…
Two View Line-Based Motion and Structure Estimation for Planar Scenes
We present an algorithm for reconstruction of piece-wise planar scenes from only two views and based on minimum line correspondences. We first recover camera rotation by matching vanishing points based on the methods already exist in the literature and then recover the camera translation by searching among a family of hypothesized planes passing through one line. Unlike algorithms based on line segments, the presented algorithm does not require an overlap between two line segments or more that one line correspon- dence across more than two views to recover the translation and achieves the goal by exploiting photometric constraints of the surface around the line. Experimental results on real…
2D-3D Camera Fusion for Visual Odometry in Outdoor Environments
International audience; Accurate estimation of camera motion is very important for many robotics applications involving SfM and visual SLAM. Such accuracy is attempted by refining the estimated motion through nonlinear optimization. As many modern robots are equipped with both 2D and 3D cameras, it is both highly desirable and challenging to exploit data acquired from both modalities to achieve a better localization. Existing refinement methods, such as Bundle adjustment and loop closing, may be employed only when precise 2D-to-3D correspondences across frames are available. In this paper, we propose a framework for robot localization that benefits from both 2D and 3D information without re…
Real time UAV altitude, attitude and motion estimation from hybrid stereovision
International audience; Knowledge of altitude, attitude and motion is essential for an Unmanned Aerial Vehicle during crit- ical maneuvers such as landing and take-off. In this paper we present a hybrid stereoscopic rig composed of a fisheye and a perspective camera for vision-based navigation. In contrast to classical stereoscopic systems based on feature matching, we propose methods which avoid matching between hybrid views. A plane-sweeping approach is proposed for estimating altitude and de- tecting the ground plane. Rotation and translation are then estimated by decoupling: the fisheye camera con- tributes to evaluating attitude, while the perspective camera contributes to estimating t…
Extrinsic calibration of heterogeneous cameras by line images
International audience; The extrinsic calibration refers to determining the relative pose of cameras. Most of the approaches for cameras with non-overlapping fields of view (FOV) are based on mirror reflection, object tracking or rigidity constraint of stereo systems whereas cameras with overlapping FOV can be calibrated using structure from motion solutions. We propose an extrinsic calibration method within structure from motion framework for cameras with overlapping FOV and its extension to cameras with partially non-overlapping FOV. Recently, omnidirectional vision has become a popular topic in computer vision as an omnidirectional camera can cover large FOV in one image. Combining the g…
N-QGN: Navigation Map from a Monocular Camera using Quadtree Generating Networks
Monocular depth estimation has been a popular area of research for several years, especially since self-supervised networks have shown increasingly good results in bridging the gap with supervised and stereo methods. However, these approaches focus their interest on dense 3D reconstruction and sometimes on tiny details that are superfluous for autonomous navigation. In this paper, we propose to address this issue by estimating the navigation map under a quadtree representation. The objective is to create an adaptive depth map prediction that only extract details that are essential for the obstacle avoidance. Other 3D space which leaves large room for navigation will be provided with approxi…
Extraction d'un graphe de navigabilité à partir d'un nuage de points 3D enrichis
International audience; Ce travail se place dans le cadre général du projet ANR pLaTINUM lié à la navigation autonome et plus parti-culièrement à la génération de cartes pour la navigation basée perception. Il consiste à développer une nouvelle méthode pour résumer une carte 3D (un nuage dense de points 3D) et extraire un graphe de navigabilité facilitant l'utilisation de cette carte par des systèmes de navigation à ressources matérielles limitées (smart-phones, voitures, robots.. .). Cette méthode vise à ex-traire les régions les plus saillantes de l'environnement étudié afin de construire une carte récapitulative. Ce processus de résumé de carte basé sur la vision est appliqué d'une façon…
Globally Optimal Line Clustering and Vanishing Point Estimation in Manhattan World
The projections of world parallel lines in an image intersect at a single point called the vanishing point (VP). VPs are a key ingredient for various vision tasks including rotation estimation and 3D reconstruction. Urban environments generally exhibit some dominant orthogonal VPs. Given a set of lines extracted from a calibrated image, this paper aims to (1) determine the line clustering, i.e. find which line belongs to which VP, and (2) estimate the associated orthogonal VPs. None of the existing methods is fully satisfactory because of the inherent difficulties of the problem, such as the local minima and the chicken-and-egg aspect. In this paper, we present a new algorithm that solves t…
Summarizing Large Scale 3D Mesh
International audience; Recent progress in 3D sensor devices and in semantic mapping allows to build very rich HD 3D maps very useful for autonomous navigation and localization. However , these maps are particularly huge and require important memory capabilities as well computational resources. In this paper, we propose a new method for summarizing a 3D map (Mesh) as a set of compact spheres in order to facilitate its use by systems with limited resources (smartphones, robots, UAVs, ...). This vision-based summarizing process is applied in a fully automatic way using jointly photometric, geometric and semantic information of the studied environment. The main contribution of this research is…