Super-resolution-based magnification of endothelium cells from biomicroscope videos of the cornea
We present a practical, robust, and effective pipeline to compute a high-resolution (HR) image of the corneal endothelium starting from a low-resolution (LR) video sequence obtained with a general purpose slit lamp biomicroscope. An image quality typical of dedicated and more expensive confocal microscopes is achieved via software magnification by exploiting information redundancy in the video sequence. In particular, the HR image is generated from the best LR frames, obtained by identifying the most suitable endothelium video subsequence using a support vector machine-based learning approach, followed by a robust graph-based frame registration. Results on long, real sequences show that the…
Accurate keyframe selection and keypoint tracking for robust visual odometry
This paper presents a novel stereo visual odometry (VO) framework based on structure from motion, where a robust keypoint tracking and matching is combined with an effective keyframe selection strategy. In order to track and find correct feature correspondences a robust loop chain matching scheme on two consecutive stereo pairs is introduced. Keyframe selection is based on the proportion of features with high temporal disparity. This criterion relies on the observation that the error in the pose estimation propagates from the uncertainty of 3D points—higher for distant points, that have low 2D motion. Comparative results based on three VO datasets show that the proposed solution is remarkab…
Restoration and Enhancement of Historical Stereo Photos
Restoration of digital visual media acquired from repositories of historical photographic and cinematographic material is of key importance for the preservation, study and transmission of the legacy of past cultures to the coming generations. In this paper, a fully automatic approach to the digital restoration of historical stereo photographs is proposed, referred to as Stacked Median Restoration plus (SMR+). The approach exploits the content redundancy in stereo pairs for detecting and fixing scratches, dust, dirt spots and many other defects in the original images, as well as improving contrast and illumination. This is done by estimating the optical flow between the images, and using it …
Fast adaptive frame preprocessing for 3D reconstruction
Abstract: This paper presents a new online preprocessing strategy to detect and discard ongoing bad frames in video sequences. These include frames where an accurate localization between corresponding points is difficult, such as for blurred frames, or which do not provide relevant information with respect to the previous frames in terms of texture, image contrast and non-flat areas. Unlike keyframe selectors and deblurring methods, the proposed approach is a fast preprocessing working on a simple gradient statistic, that does not require to compute complex time-consuming image processing, such as the computation of image feature keypoints, previous poses and 3D structure, or to know a prio…
Color Correction for Image Stitching by Monotone Cubic Spline Interpolation
This paper proposes a novel color correction scheme for image stitching where the color map transfer is modelled by a monotone Hermite cubic spline and smoothly propagated into the target image. A three-segments monotone cubic spline minimizing color distribution statistics and gradient differences with respect to both the source and target images is used. While the spline model can handle non-linear color maps, the minimization over the gradient differences limits strong alterations on the image structure. Adaptive heuristics are introduced to reduce the minimization search space and thus computational time. Experimental comparisons with respect to the state-of-the-art linear mapping model…
The ARROWS project: adapting and developing robotics technologies for underwater archaeology
4th IFAC Workshop on Navigation, Guidance and Control of Underwater Vehicles, NGCUV 2015; Girona; Spain; 28 April 2015 through 30 April 2015
A vision-based fully automated approach to robust image cropping detection
Abstract The definition of valid and robust methodologies for assessing the authenticity of digital information is nowadays critical to contrast social manipulation through the media. A key research topic in multimedia forensics is the development of methods for detecting tampered content in large image collections without any human intervention. This paper introduces AMARCORD (Automatic Manhattan-scene AsymmetRically CrOpped imageRy Detector), a fully automated detector for exposing evidences of asymmetrical image cropping on Manhattan-World scenes. The proposed solution estimates and exploits the camera principal point, i.e., a physical feature extracted directly from the image content th…
Which Is Which? Evaluation of Local Descriptors for Image Matching in Real-World Scenarios
Matching with local image descriptors is a fundamental task in many computer vision applications. This paper describes the WISW contest held within the framework of the CAIP 2019 conference, aimed at benchmarking recent descriptors in challenging planar and non-planar real image matching scenarios. According to the contest results, the descriptors submitted to the competition, most of which based on deep learning, perform significantly better than the current state-of-the-art in image matching. Nonetheless, there is still room for improvement, especially in the case of non-planar scenes.
RootsGLOH2: embedding RootSIFT 'square rooting' in sGLOH2
This study introduces an extension of the shifting gradient local orientation histogram doubled (sGLOH2) local image descriptor inspired by RootSIFT ‘square rooting’ as a way to indirectly alter the matching distance used to compare the descriptor vectors. The extended descriptor, named RootsGLOH2, achieved the best results in terms of matching accuracy and robustness among the latest state-of-the-art non-deep descriptors in recent evaluation contests dealing with both planar and non-planar scenes. RootsGLOH2 also achieves a matching accuracy very close to that obtained by the best deep descriptors to date. Beside confirming that ‘square rooting’ has beneficial effects on sGLOH2 as it happe…
Robust Selective Stereo SLAM without Loop Closure and Bundle Adjustment
This paper presents a novel stereo SLAM framework, where a robust loop chain matching scheme for tracking keypoints is combined with an effective frame selection strategy. The proposed approach, referred to as selective SLAM (SSLAM), relies on the observation that the error in the pose estimation propagates from the uncertainty of the three-dimensional points. This is higher for distant points, corresponding to matches with low temporal flow disparity in the images. Comparative results based on the reference KITTI evaluation framework show that SSLAM is effective and can be implemented efficiently, as it does not require any loop closure or bundle adjustment.
Restoration and Enhancement of Historical Stereo Photos Through Optical Flow
Restoration of digital visual media acquired from repositories of historical photographic and cinematographic material is of key importance for the preservation, study and transmission of the legacy of past cultures to the coming generations. In this paper, a fully automatic approach to the digital restoration of historical stereo photographs is proposed. The approach exploits the content redundancy in stereo pairs for detecting and fixing scratches, dust, dirt spots and many other defects in the original images, as well as improving contrast and illumination. This is done by estimating the optical flow between the images, and using it to register one view onto the other both geometrically …
Estimating the Best Reference Homography for Planar Mosaics From Videos
This paper proposes a novel strategy to find the best reference homography in mosaics from video sequences. The reference homography globally minimizes the distortions induced on each image frame by the mosaic homography itself. This method is designed for planar mosaics on which a bad choice of the first reference image frame can lead to severe distortions after concatenating several successive homographies. This often happens in the case of underwater mosaics with non-flat seabed and no georeferential information available. Given a video sequence of an almost planar surface, sub-mosaics with low distortions of temporally close image frames are computed and successively merged according to…
Dissecting and Reassembling Color Correction Algorithms for Image Stitching
This paper introduces a new compositional framework for classifying color correction methods according to their two main computational units. The framework was used to dissect fifteen among the best color correction algorithms and the computational units so derived, with the addition of four new units specifically designed for this work, were then reassembled in a combinatorial way to originate about one hundred distinct color correction methods, most of which never considered before. The above color correction methods were tested on three different existing datasets, including both real and artificial color transformations, plus a novel dataset of real image pairs categorized according to …
SAMSLAM: Simulated Annealing Monocular SLAM
This paper proposes a novel monocular SLAM approach. For a triplet of successive keyframes, the approach inteleaves the registration of the three 3D maps associated to each image pair in the triplet and the refinement of the corresponding poses, by progressively limiting the allowable reprojection error according to a simulated annealing scheme. This approach computes only local overlapping maps of almost constant size, thus avoiding problems of 3D map growth. It does not require global optimization, loop closure and back-correction of the poses.
Piecewise planar underwater mosaicing
A commonly ignored problem in planar mosaics, yet often present in practice, is the selection of a reference homography reprojection frame where to attach the successive image frames of the mosaic. A bad choice for the reference frame can lead to severe distortions in the mosaic and can degenerate in incorrect configurations after some sequential frame concatenations. This problem is accentuated in uncontrolled underwater acquisition setups as those provided by AUVs or ROVs due to both the noisy trajectory of the acquisition vehicle — with roll and pitch shakes — and to the non-flat nature of the seabed which tends to break the planarity assumption implicit in the mosaic construction. These…
Experiencing with electronic image stabilization and PRNU through scene content image registration
Abstract This paper explores content-based image registration as a means of dealing with and understanding better Electronic Image Stabilization (EIS) in the context of Photo Response Non-Uniformity (PRNU) alignment. A novel and robust solution to extrapolate the transformation relating the different image output formats for a given device model is proposed. This general approach can be adapted to specifically extract the scale factor (and, when appropriate, the translation) so as to align native resolution images to video frames, with or without EIS on, and proceed to compare PRNU patterns. Comparative evaluations show that the proposed approach outperforms those based on brute-force and p…
Rethinking the sGLOH Descriptor
sGLOH (shifting GLOH) is a histogram-based keypoint descriptor that can be associated to multiple quantized rotations of the keypoint patch without any recomputation. This property can be exploited to define the best distance between two descriptor vectors, thus avoiding computing the dominant orientation. In addition, sGLOH can reject incongruous correspondences by adding a global constraint on the rotations either as an a priori knowledge or based on the data. This paper thoroughly reconsiders sGLOH and improves it in terms of robustness, speed and descriptor dimension. The revised sGLOH embeds more quantized rotations, thus yielding more correct matches. A novel fast matching scheme is a…
Extending the sGLOH descriptor
This paper proposes an extension of the sGLOH keypoint descriptor [3] which improves its robustness and discriminability. The sGLOH descriptor can handle discrete rotations by a cyclic shift of its elements thanks to its circular structure, but its performance can decrease when the keypoint relative rotation is in between two sGLOH discrete rotations. The proposed extension couples together two sGLOH descriptors for the same patch with different rotations in order to cope with this issue and it can be also applied straightly to the sCOr and sGOr matching strategies of sGLOH. Experimental results show a consistent improvement of the descriptor discriminability, while different setups can be …
Is There Anything New to Say About SIFT Matching?
SIFT is a classical hand-crafted, histogram-based descriptor that has deeply influenced research on image matching for more than a decade. In this paper, a critical review of the aspects that affect SIFT matching performance is carried out, and novel descriptor design strategies are introduced and individually evaluated. These encompass quantization, binarization and hierarchical cascade filtering as means to reduce data storage and increase matching efficiency, with no significant loss of accuracy. An original contextual matching strategy based on a symmetrical variant of the usual nearest-neighbor ratio is discussed as well, that can increase the discriminative power of any descriptor. Th…
Prnu Pattern Alignment for Images and Videos Based on Scene Content
This paper proposes a novel approach for registering the PRNU pattern between different camera acquisition modes by relying on the imaged scene content. First, images are aligned by establishing correspondences between local descriptors: The result can then optionally be refined by maximizing the PRNU correlation. Comparative evaluations show that this approach outperforms those based on brute-force and particle swarm optimization in terms of reliability, accuracy and speed. The proposed scene-based approach for PRNU pattern alignment is suitable for video source identification in multimedia forensics applications.
Selective visual odometry for accurate AUV localization
In this paper we present a stereo visual odometry system developed for autonomous underwater vehicle localization tasks. The main idea is to make use of only highly reliable data in the estimation process, employing a robust keypoint tracking approach and an effective keyframe selection strategy, so that camera movements are estimated with high accuracy even for long paths. Furthermore, in order to limit the drift error, camera pose estimation is referred to the last keyframe, selected by analyzing the feature temporal flow. The proposed system was tested on the KITTI evaluation framework and on the New Tsukuba stereo dataset to assess its effectiveness on long tracks and different illumina…
An evaluation of recent local image descriptors for real-world applications of image matching
This paper discusses and compares the best and most recent local descriptors, evaluating them on increasingly complex image matching tasks, encompassing planar and non-planar scenarios under severe viewpoint changes. This evaluation, aimed at assessing descriptor suitability for real-world applications, leverages the concept of approximated overlap error as a means to naturally extend to non-planar scenes the standard metric used for planar scenes. According to the evaluation results, most descriptors exhibit a gradual performance degradation in the transition from planar to non-planar scenes. The best descriptors are those capable of capturing well not only the local image context, but als…
FISH: Face Intensity-Shape Histogram representation for automatic face splicing detection
Abstract Tampered images spread nowadays over any visual media influencing our judgement in many aspects of our life. This is particularly critical for face splicing manipulations, where recognizable identities are put out of context. To contrast these activities on a large scale, automatic detectors are required. In this paper, we present a novel method for automatic face splicing detection, based on computer vision, that exploits inconsistencies in the lighting environment estimated from different faces in the scene. Differently from previous approaches, we do not rely on an ideal mathematical model of the lighting environment. Instead, our solution, built upon the concept of histogram-ba…
3D Map Computation from Historical Stereo Photographs of Florence
The analysis of early photographic sources is fundamental for documenting and understanding the evolution of a city so rich in history and art as Florence. Indeed, by the 1860s several photographers used to work in town, and their images (often obtained through stereoscopic set-ups) can help us to reconstruct Florence in 3D as it was by the time of the Italian unification. The first and most delicate part of such reconstruction process is the computation of disparity maps from the historical stereo pairs. This is a very challenging task for fully-automatic computer vision algorithms, since XIX century photographs are affected by several problems—ranging from superficial damages to asynchron…
Design of a modular Autonomous Underwater Vehicle for archaeological investigations
MARTA (MARine Tool for Archaeology) is a modular AUV (Autonomous Underwater Vehicle) designed and developed by the University of Florence in the framework of the ARROWS (ARchaeological RObot systems for the World's Seas) FP7 European project. The ARROWS project challenge is to provide the underwater archaeologists with technological tools for cost affordable campaigns: i.e. ARROWS adapts and develops low cost AUV technologies to significantly reduce the cost of archaeological operations, covering the full extent of an archaeological campaign (underwater mapping, diagnosis and cleaning tasks). The tools and methodologies developed within ARROWS comply with the "Annex" of the 2001 UNESCO Conv…