0000000000447647
AUTHOR
Devis Tuia
Discovering single classes in remote sensing images with active learning
When dealing with supervised target detection, the acquisition of labeled samples is one of the most critical phases: the samples must be yet representative of the class of interest, but must also be found among a vast majority of non-target examples. Moreover, the efficiency of the search is also an issue, since the samples labeled as background are not used by target detectors such as the support vector data description (SVDD). In this work we propose a competitive and effective approach to identify the most relevant training samples for one-class classification based on the use of an active learning strategy. The SVDD classifier is first trained with insufficient target examples. It is t…
PRINCIPAL POLYNOMIAL ANALYSIS
© 2014 World Scientific Publishing Company. This paper presents a new framework for manifold learning based on a sequence of principal polynomials that capture the possibly nonlinear nature of the data. The proposed Principal Polynomial Analysis (PPA) generalizes PCA by modeling the directions of maximal variance by means of curves instead of straight lines. Contrarily to previous approaches PPA reduces to performing simple univariate regressions which makes it computationally feasible and robust. Moreover PPA shows a number of interesting analytical properties. First PPA is a volume preserving map which in turn guarantees the existence of the inverse. Second such an inverse can be obtained…
Putting the user into the active learning loop : Towards realistic but efficient photointerpretation
In recent years, several studies have been published about the smart definition of training set using active learning algorithms. However, none of these works consider the contradiction between the active learning methods, which rank the pixels according to their uncertainty, and the confidence of the user in labeling, which is related both to the homogeneity of the pixel context and to the knowledge of the user of the scene. In this paper, we propose a two-steps procedure based on a filtering scheme to learn the confidence of the user in labeling. This way, candidate training pixels are ranked according both to their uncertainty and to the chances of being labeled correctly by the user. In…
Advances in Kernel Machines for Image Classification and Biophysical Parameter Retrieval
Remote sensing data analysis is knowing an unprecedented upswing fostered by the activities of the public and private sectors of geospatial and environmental data analysis. Modern imaging sensors offer the necessary spatial and spectral information to tackle a wide range problems through Earth Observation, such as land cover and use updating, urban dynamics, or vegetation and crop monitoring. In the upcoming years even richer information will be available: more sophisticated hyperspectral sensors with high spectral resolution, multispectral sensors with sub-metric spatial detail or drones that can be deployed in very short time lapses. Besides such opportunities, these new and wealthy infor…
Structured Output SVM for Remote Sensing Image Classification
Traditional kernel classifiers assume independence among the classification outputs. As a consequence, each misclassification receives the same weight in the loss function. Moreover, the kernel function only takes into account the similarity between input values and ignores possible relationships between the classes to be predicted. These assumptions are not consistent for most of real-life problems. In the particular case of remote sensing data, this is not a good assumption either. Segmentation of images acquired by airborne or satellite sensors is a very active field of research in which one tries to classify a pixel into a predefined set of classes of interest (e.g. water, grass, trees,…
Improving active learning methods using spatial information
Active learning process represents an interesting solution to the problem of training sample collection for the classification of remote sensing images. In this work, we propose a criterion based on the spatial information that can be used in combination with a spectral criterion in order to improve the selection of training samples. Experimental results obtained on a very high resolution image show the effectiveness of regularization in spatial domain and open challenging perspectives for terrain campaigns planning. © 2011 IEEE.
Remote sensing image segmentation by active queries
Active learning deals with developing methods that select examples that may express data characteristics in a compact way. For remote sensing image segmentation, the selected samples are the most informative pixels in the image so that classifiers trained with reduced active datasets become faster and more robust. Strategies for intelligent sampling have been proposed with model-based heuristics aiming at the search of the most informative pixels to optimize model's performance. Unlike standard methods that concentrate on model optimization, here we propose a method inspired in the cluster assumption that holds in most of the remote sensing data. Starting from a complete hierarchical descri…
Estimating biophysical variable dependences with kernels
This paper introduces a nonlinear measure of dependence between random variables in the context of remote sensing data analysis. The Hilbert-Schmidt Independence Criterion (HSIC) is a kernel method for evaluating statistical dependence. HSIC is based on computing the Hilbert-Schmidt norm of the cross-covariance operator of mapped samples in the corresponding Hilbert spaces. The HSIC empirical estimator is very easy to compute and has good theoretical and practical properties. We exploit the capabilities of HSIC to explain nonlinear dependences in two remote sensing problems: temperature estimation and chlorophyll concentration prediction from spectra. Results show that, when the relationshi…
Kernel manifold alignment for domain adaptation
The wealth of sensory data coming from different modalities has opened numerous opportu- nities for data analysis. The data are of increasing volume, complexity and dimensionality, thus calling for new methodological innovations towards multimodal data processing. How- ever, multimodal architectures must rely on models able to adapt to changes in the data dis- tribution. Differences in the density functions can be due to changes in acquisition conditions (pose, illumination), sensors characteristics (number of channels, resolution) or different views (e.g. street level vs. aerial views of a same building). We call these different acquisition modes domains, and refer to the adaptation proble…
Unsupervised change detection with kernels
In this paper an unsupervised approach to change detection relying on kernels is introduced. Kernel based clustering is used to partition a selected subset of pixels representing both changed and unchanged areas. Once the optimal clustering is obtained the estimated representatives (centroids) of each group are used to assign the class membership to all others pixels composing the multitemporal scenes. Different approaches of considering the multitemporal information are considered with accent on the computation of the difference image directly in the feature spaces. For this purpose a difference kernel approach is successfully adopted. Finally an effective way to cope with the estimation o…
Nonlinear data description with Principal Polynomial Analysis
Principal Component Analysis (PCA) has been widely used for manifold description and dimensionality reduction. Performance of PCA is however hampered when data exhibits nonlinear feature relations. In this work, we propose a new framework for manifold learning based on the use of a sequence of Principal Polynomials that capture the eventually nonlinear nature of the data. The proposed Principal Polynomial Analysis (PPA) is shown to generalize PCA. Unlike recently proposed nonlinear methods (e.g. spectral/kernel methods and projection pursuit techniques, neural networks), PPA features are easily interpretable and the method leads to a fully invertible transform, which is a desirable property…
Recent advances in remote sensing image processing
Remote sensing image processing is nowadays a mature research area. The techniques developed in the field allow many real-life applications with great societal value. For instance, urban monitoring, fire detection or flood prediction can have a great impact on economical and environmental issues. To attain such objectives, the remote sensing community has turned into a multidisciplinary field of science that embraces physics, signal theory, computer science, electronics, and communications. From a machine learning and signal/image processing point of view, all the applications are tackled under specific formalisms, such as classification and clustering, regression and function approximation…
Multi-temporal and Multi-source Remote Sensing Image Classification by Nonlinear Relative Normalization
Remote sensing image classification exploiting multiple sensors is a very challenging problem: data from different modalities are affected by spectral distortions and mis-alignments of all kinds, and this hampers re-using models built for one image to be used successfully in other scenes. In order to adapt and transfer models across image acquisitions, one must be able to cope with datasets that are not co-registered, acquired under different illumination and atmospheric conditions, by different sensors, and with scarce ground references. Traditionally, methods based on histogram matching have been used. However, they fail when densities have very different shapes or when there is no corres…
Graph matching for efficient classifiers adaptation
In this work we present an adaptation algorithm focused on the description of the measurement changes under different acquisition conditions. The adaptation is carried out by transforming the manifold in the first observation conditions into the corresponding manifold in the second. The eventually non-linear transform is based on vector quantization and graph matching. The transfer learning mapping is defined in an unsupervised manner. Once this mapping has been defined, the labeled samples in the first are projected into the second domain, thus allowing the application of any classifier in the transformed domain. Experiments on VHR series of images show the validity of the proposed method …
Explicit Recursive and Adaptive Filtering in Reproducing Kernel Hilbert Spaces
This brief presents a methodology to develop recursive filters in reproducing kernel Hilbert spaces. Unlike previous approaches that exploit the kernel trick on filtered and then mapped samples, we explicitly define the model recursivity in the Hilbert space. For that, we exploit some properties of functional analysis and recursive computation of dot products without the need of preimaging or a training dataset. We illustrate the feasibility of the methodology in the particular case of the $\gamma$ -filter, which is an infinite impulse response filter with controlled stability and memory depth. Different algorithmic formulations emerge from the signal model. Experiments in chaotic and elect…
Explicit recursivity into reproducing kernel Hilbert spaces
This paper presents a methodology to develop recursive filters in reproducing kernel Hilbert spaces (RKHS). Unlike previous approaches that exploit the kernel trick on filtered and then mapped samples, we explicitly define model recursivity in the Hilbert space. The method exploits some properties of functional analysis and recursive computation of dot products without the need of pre-imaging. We illustrate the feasibility of the methodology in the particular case of the gamma-filter, an infinite impulse response (IIR) filter with controlled stability and memory depth. Different algorithmic formulations emerge from the signal model. Experiments in chaotic and electroencephalographic time se…
Large scale semi-supervised image segmentation with active queries
A semiautomatic procedure to generate classification maps of remote sensing images is proposed. Starting from a hierarchical unsupervised classification, the algorithm exploits the few available labeled pixels to assign each cluster to the most probable class. For a given amount of labeled pixels, the algorithm returns a classified segmentation map, along with confidence levels of class membership for each pixel. Active learning methods are used to select the most informative samples to increase confidence in the class membership. Experiments on a AVIRIS hyperspectral image confirm the effectiveness of the method, especially when used with active learning query functions and spatial regular…
Using active learning to adapt remote sensing image classifiers
The validity of training samples collected in field campaigns is crucial for the success of land use classification models. However, such samples often suffer from a sample selection bias and do not represent the variability of spectra that can be encountered in the entire image. Therefore, to maximize classification performance, one must perform adaptation of the first model to the new data distribution. In this paper, we propose to perform adaptation by sampling new training examples in unknown areas of the image. Our goal is to select these pixels in an intelligent fashion that minimizes their number and maximizes their information content. Two strategies based on uncertainty and cluster…
Learning spatial filters for multispectral image segmentation.
International audience; We present a novel filtering method for multispectral satel- lite image classification. The proposed method learns a set of spatial filters that maximize class separability of binary support vector machine (SVM) through a gradient descent approach. Regularization issues are discussed in detail and a Frobenius-norm regularization is proposed to efficiently exclude uninformative filters coefficients. Experiments car- ried out on multiclass one-against-all classification and tar- get detection show the capabilities of the learned spatial fil- ters.
Learning the relevant image features with multiple kernels
This paper proposes to learn the relevant features of remote sensing images for automatic spatio-spectral classification with the automatic optimization of multiple kernels. The method consists of building dedicated kernels for different sets of bands, contextual or textural features. The optimal linear combination of kernels is optimized through gradient descent on the support vector machine (SVM) objective function. Since a na¨ive implementation is computationally demanding, we propose an efficient model selection procedure based on kernel alignment. The result is a weight — learned from the data — for each kernel where both relevant and meaningless image features emerge after training. E…
Spectral alignment of multi-temporal cross-sensor images with automated kernel canonical correlation analysis
In this paper we present an approach to perform relative spectral alignment between optical cross-sensor acquisitions. The proposed method aims at projecting the images from two different and possibly disjoint input spaces into a common latent space, in which standard change detection algorithms can be applied. The system relies on the regularized kernel canonical correlation analysis transformation (kCCA), which can accommodate nonlinear dependencies between pixels by means of kernel functions. To learn the projections, the method employs a subset of samples belonging to the unchanged areas or to uninteresting radiometric differences. Since the availability of ground truth information to p…
Toward a Collective Agenda on AI for Earth Science Data Analysis
In the last years we have witnessed the fields of geosciences and remote sensing and artificial intelligence to become closer. Thanks to both the massive availability of observational data, improved simulations, and algorithmic advances, these disciplines have found common objectives and challenges to advance the modeling and understanding of the Earth system. Despite such great opportunities, we also observed a worrying tendency to remain in disciplinary comfort zones applying recent advances from artificial intelligence on well resolved remote sensing problems. Here we take a position on research directions where we think the interface between these fields will have the most impact and be…
Advancing Deep Learning for Earth Sciences: From Hybrid Modeling to Interpretability
Machine learning and deep learning in particular have made a huge impact in many fields of science and engineering. In the last decade, advanced deep learning methods have been developed and applied to remote sensing and geoscientific data problems extensively. Applications on classification and parameter retrieval are making a difference: methods are very accurate, can handle large amounts of data, and can deal with spatial and temporal data structures efficiently. Nevertheless, several important challenges need still to be addressed. First, current standard deep architectures cannot deal with long-range dependencies so distant driving processes (in space or time) are not captured, and the…
Cluster-based active learning for compact image classification
In this paper, we consider active sampling to label pixels grouped with hierarchical clustering. The objective of the method is to match the data relationships discovered by the clustering algorithm with the user's desired class semantics. The first is represented as a complete tree to be pruned and the second is iteratively provided by the user. The active learning algorithm proposed searches the pruning of the tree that best matches the labels of the sampled points. By choosing the part of the tree to sample from according to current pruning's uncertainty, sampling is focused on most uncertain clusters. This way, large clusters for which the class membership is already fixed are no longer…
Domain separation for efficient adaptive active learning
This paper proposes a procedure aimed at efficiently adapting a classifier trained on a source image to a similar target image. The adaptation is carried out through active queries in the target domain following a strategy particularly designed for the case where class distributions have shifted between the two images. We first suggest a pre-selection of candidate pixels issued from the target image by keeping only those samples appearing to be lying in a region of the input space not yet covered by the existing ground truth (source domain pixels). Then, exploiting a classifier integrating instance weights, active queries are performed on the target image. As the inclusion to the training s…
Learning User's Confidence for Active Learning
In this paper, we study the applicability of active learning in operative scenarios: more particularly, we consider the well-known contradiction between the active learning heuristics, which rank the pixels according to their uncertainty, and the user's confidence in labeling, which is related to both the homogeneity of the pixel context and user's knowledge of the scene. We propose a filtering scheme based on a classifier that learns the confidence of the user in labeling, thus minimizing the queries where the user would not be able to provide a class for the pixel. The capacity of a model to learn the user's confidence is studied in detail, also showing the effect of resolution is such a …
Dataset shift adaptation with active queries
In remote sensing image classification, it is commonly assumed that the distribution of the classes is stable over the entire image. This way, training pixels labeled by photointerpretation are assumed to be representative of the whole image. However, differences in distribution of the classes throughout the image make this assumption weak and a model built on a single area may be suboptimal when applied to the rest of the image. In this paper, we investigate the use of active learning to correct the shifts that may appear when training and test data do not come from the same distribution. Experiments are carried out on a VHR remote sensing classification scenario showing that active learni…
Large-scale random features for kernel regression
Kernel methods constitute a family of powerful machine learning algorithms, which have found wide use in remote sensing and geosciences. However, kernel methods are still not widely adopted because of the high computational cost when dealing with large scale problems, such as the inversion of radiative transfer models. This paper introduces the method of random kitchen sinks (RKS) for fast statistical retrieval of bio-geo-physical parameters. The RKS method allows to approximate a kernel matrix with a set of random bases sampled from the Fourier domain. We extend their use to other bases, such as wavelets, stumps, and Walsh expansions. We show that kernel regression is now possible for data…
Weakly supervised alignment of multisensor images
Manifold alignment has become very popular in recent literature. Aligning data distributions prior to product generation is an appealing strategy, since it allows to provide data spaces that are more similar to each other, regardless of the subsequent use of the transformed data. We propose a methodology that finds a common representation among data spaces from different sensors using geographic image correspondences, or semantic ties. To cope with the strong deformations between the data spaces considered, we propose to add nonlineari-ties by expanding the input space with Gaussian Radial Basis Function (RBF) features with respect to the centroids of a partitioning of the data. Such featur…
A Deep Network Approach to Multitemporal Cloud Detection
We present a deep learning model with temporal memory to detect clouds in image time series acquired by the Seviri imager mounted on the Meteosat Second Generation (MSG) satellite. The model provides pixel-level cloud maps with related confidence and propagates information in time via a recurrent neural network structure. With a single model, we are able to outline clouds along all year and during day and night with high accuracy.
Spectral adaptation of hyperspectral flight lines using VHR contextual information
Abstract: Due to technological constraints, hyperspectral earth observation imagery are often a mosaic of overlapping flight lines collected in different passes over the area of interest. This causes variations in aqcuisition conditions such that the reflected spectrum can vary significantly between these flight lines. Partly, this problem is solved by atmospherical correction, but residual spectral differences often remain. A probabilistic domain adaptation framework based on graph matching using Hidden Markov Random Fields was recently proposed for transforming hyperspectral data from one image to better correspond to the other. This paper investigates the use of scale and angle invariant…
A survey of active learning algorithms for supervised remote sensing image classification
Defining an efficient training set is one of the most delicate phases for the success of remote sensing image classification routines. The complexity of the problem, the limited temporal and financial resources, as well as the high intraclass variance can make an algorithm fail if it is trained with a suboptimal dataset. Active learning aims at building efficient training sets by iteratively improving the model performance through sampling. A user-defined heuristic ranks the unlabeled pixels according to a function of the uncertainty of their class membership and then the user is asked to provide labels for the most uncertain pixels. This paper reviews and tests the main families of active …
Active Learning for Monitoring Network Optimization
Kernel-based active learning strategies were studied for the optimization of environmental monitoring networks. This chapter introduces the basic machine learning algorithms originated in the statistical learning theory of Vapnik (1998). Active learning is closer to an optimization done using sequential Gaussian simulations. The chapter presents the general ideas of statistical learning from data. It derives the basics of kernel-based support vector algorithms. The active learning framework is presented and machine learning extensions for active learning are described in the chapter. Kernel-based active learning strategies are tested on real case studies. The chapter explores the use of a c…
Cluster kernels for semisupervised classification of VHR urban images
In this paper, we present and apply a semisupervised support vector machine based on cluster kernels for the problem of very high resolution image classification. In the proposed setting, a base kernel working with labeled samples only is deformed by a likelihood kernel encoding similarities between unlabeled examples. The resulting kernel is used to train a standard support vector machine (SVM) classifier. Experiments carried out on very high resolution (VHR) multispectral and hyperspectral images using very few labeled examples show the relevancy of the method in the context of urban image classification. Its simplicity and the small number of parameters involved make it versatile and wor…
Traitement de données RGB et Lidar à extrêmement haute résolution: retombées de la compétition de fusion de données 2015 de l'IEEE GRSS - Partie A / compétition 2D
International audience; In this paper, we discuss the scientific outcomes of the 2015 data fusion contest organized by the Image Analysis and Data Fusion Technical Committee (IADF TC) of the IEEE Geoscience and Remote Sensing Society (IEEE GRSS). As for previous years, the IADF TC organized a data fusion contest aiming at fostering new ideas and solutions for multisource studies. The 2015 edition of the contest proposed a multiresolution and multisensorial challenge involving extremely high-resolution RGB images and a three-dimensional (3-D) LiDAR point cloud. The competition was framed in two parallel tracks, considering 2-D and 3-D products, respectively. In this paper, we discuss the sci…