The Reprocessed Proba-V Collection 2: Product Validation

With the objective to improve data quality in terms of cloud detection, absolute radiometric calibration and atmospheric correction, the PRoject for On-Board Autonomy-Vegetation (PROBA-V) data archive (October 2013 - June 2020) will be reprocessed to Collection 2 (C2). The product validation is organized in three phases and focuses on the intercomparison with PROBA-V Collection 1 (C1), but also consistency analysis with SPOT-VGT, Sentinel-3 SYN-VGT, Terra-MODIS and METOP-AVHRR is foreseen. First preliminary results show the better performance of cloud and snow/ice masking, and indicate that statistical consistency between PROBA-V C2 and C1 are in line with expectations. PROBA-V C2 data are …

Multitemporal fusion of Landsat and MERIS images

Monitoring Earth dynamics from current and future observation satellites is one of the most important objectives for the remote sensing community. In this regard, the exploitation of image time series from sensors with different characteristics provides an opportunity to increase the knowledge about environmental changes, which are needed in many operational applications, such as monitoring vegetation dynamics and land cover/use changes. Many studies in the literature have proven that high spatial resolution sensors like Landsat are very useful for monitoring land cover changes. However, the cloud cover probability of many areas and the 15-days temporal resolution restrict its use to monito…

Design of a configurable multispectral imaging system based on an AOTF.

In this paper, we present a configurable multispectral imaging system based on an acousto-optic tunable filter (AOTF). Typically, AOTFs are used to filter a single wavelength at a time, but thanks to the use of a versatile sweeping frequency generator implemented with a direct digital synthesizer, the imager may capture a configurable spectral range. Experimental results show a good spectral and imaging response of the system for spectral bandwidth up to a 50 nm.

Semi-Supervised Classification Method for Hyperspectral Remote Sensing Images

A new approach to the classification of hyperspectral images is proposed. The main problem with supervised methods is that the learning process heavily depends on the quality of the training data set. In remote sensing, the training set is useful only for simultaneous images or for images with the same classes taken under the same conditions; and, even worse, the training set is frequently not available. On the other hand, unsupervised methods are not sensitive to the number of labelled samples since they work on the whole image. Nevertheless, relationship between clusters and classes is not ensured. In this context, we propose a combined strategy of supervised and unsupervised learning met…

Multi-Temporal Image Classification with Kernels

Modelling spatial and spectral systematic noise patterns on CHRIS/PROBA hyperspectral data

In addition to typical random noise, remote sensing hyperspectral images are generally affected by non-periodic partially deterministic disturbance patterns due to the image formation process and characterized by a high degree of spatial and spectral coherence. This paper presents a new technique that faces the problem of removing the spatial coherent noise known as vertical stripping (VS) usually found in images acquired by push-broom sensors, in particular for the Compact High Resolution Imaging Spectrometer (CHRIS). The correction is based on the hypothesis that the vertical disturbance presents higher spatial frequencies than the surface radiance. The proposed method introduces a way to…

Explicit signal to noise ratio in reproducing kernel Hilbert spaces

This paper introduces a nonlinear feature extraction method based on kernels for remote sensing data analysis. The proposed approach is based on the minimum noise fraction (MNF) transform, which maximizes the signal variance while also minimizing the estimated noise variance. We here propose an alternative kernel MNF (KMNF) in which the noise is explicitly estimated in the reproducing kernel Hilbert space. This enables KMNF dealing with non-linear relations between the noise and the signal features jointly. Results show that the proposed KMNF provides the most noise-free features when confronted with PCA, MNF, KPCA, and the previous version of KMNF. Extracted features with the explicit KMNF…

Urban monitoring using multi-temporal SAR and multi-spectral data

In some key operational domains, the joint use of synthetic aperture radar (SAR) and multi-spectral sensors has shown to be a powerful tool for Earth observation. In this paper, we analyze the potentialities of combining interferometric SAR and multi-spectral data for urban area characterization and monitoring. This study is carried out following a standard multi-source processing chain. First, a pre-processing stage is performed taking into account the underlying physics, geometry, and statistical models for the data from each sensor. Second, two different methodologies, one for supervised and another for unsupervised approaches, are followed to obtain features that optimize the urban rela…

Statistical biophysical parameter retrieval and emulation with Gaussian processes

Abstract Earth observation from satellites poses challenging problems where machine learning is being widely adopted as a key player. Perhaps the most challenging scenario that we are facing nowadays is to provide accurate estimates of particular variables of interest characterizing the Earth's surface. This chapter introduces some recent advances in statistical bio-geophysical parameter retrieval from satellite data. In particular, we will focus on Gaussian process regression (GPR) that has excelled in parameter estimation as well as in modeling complex radiative transfer processes. GPR is based on solid Bayesian statistics and generally yields efficient and accurate parameter estimates, a…

Optimal implementation of neural activation functions in programmable logic using fuzzy logic

Abstract This work presents a methodology for implementing neural activation function in programmable logic using tools from fuzzy logic. This methodology will allow implementing these intrinsic non-linear functions using comparators and simple linear modellers, easily implemented in programmable logic. This work is particularized to the case of a hyperbolic tangent, the most common function in neural models, showing the excellent results yielded with the proposed approximation.

Multitemporal Unmixing of Medium-Spatial-Resolution Satellite Images: A Case Study Using MERIS Images for Land-Cover Mapping

Data from current medium-spatial-resolution imaging spectroradiometers are used for land-cover mapping and land-cover change detection at regional to global scales. However, few landscapes are homogeneous at these scales, and this creates the so-called mixed-pixel problem. In this context, this study explores the use of the linear spectral mixture model to extract subpixel land-cover composition from medium-spatial-resolution data. In particular, a time series of MEdium Resolution Imaging Spectrometer (MERIS) full-resolution (FR; pixel size of 300 m) images acquired over The Netherlands is used to illustrate this study. The Netherlands was selected because of the following: 1) the fragmenta…

Convolutional Neural Networks for Cloud Screening: Transfer Learning from Landsat-8 to Proba-V

Cloud detection is a key issue for exploiting the information from Earth observation satellites multispectral sensors. For Proba-V, cloud detection is challenging due to the limited number of spectral bands. Advanced machine learning methods, such as convolutional neural networks (CNN), have shown to work well on this problem provided enough labeled data. However, simultaneous collocated information about the presence of clouds is usually not available or requires a great amount of manual labor. In this work, we propose to learn from the available Landsat −8 cloud masks datasets and transfer this learning to solve the Proba-V cloud detection problem. CNN are trained with Landsat images adap…

Diurnal Cycle Relationships between Passive Fluorescence, PRI and NPQ of Vegetation in a Controlled Stress Experiment

In order to estimate vegetation photosynthesis from remote sensing observations; some critical parameters need to be quantified. From all absorbed light; the plant needs to release any excess that is not used for photosynthesis; by non-photochemical quenching; by fluorescence emission and unregulated thermal dissipation. Non-photochemical quenching (NPQ) processes are controlled photoprotective mechanisms which; once activated; strongly control the dynamics of photochemical efficiency. With illumination conditions increasing and decreasing during a diurnal cycle; photoprotection mechanisms needs to change accordingly. The goal of this work is to quantify dynamic NPQ; measured from active fl…

Randomized kernels for large scale Earth observation applications

Abstract Current remote sensing applications of bio-geophysical parameter estimation and image classification have to deal with an unprecedented big amount of heterogeneous and complex data sources. New satellite sensors involving a high number of improved time, space and wavelength resolutions give rise to challenging computational problems. Standard physical inversion techniques cannot cope efficiently with this new scenario. Dealing with land cover classification of the new image sources has also turned to be a complex problem requiring large amount of memory and processing time. In order to cope with these problems, statistical learning has greatly helped in the last years to develop st…

Hyperspectral Image Classification with Kernels

The information contained in hyperspectral images allows the characterization, identification, and classification of land covers with improved accuracy and robustness. However, several critical problems should be considered in the classification of hyperspectral images, among which are (a) the high number of spectral channels, (b) the spatial variability of the spectral signature, (c) the high cost of true sample labeling, and (d) the quality of data. Recently, kernel methods have offered excellent results in this context. This chapter reviews the state-of-the-art hyperspectral image classifiers, presents two recently proposed kernel-based approaches, and systematically discusses the specif…

A Cloud masking algorithm for the XBAER aerosol retrieval using MERIS data

Abstract To determine aerosol optical thickness, AOT, and other geophysical parameters describing conditions in the atmosphere and at the earth's surface by inversion of remote sensing measurements from space based instrumentation, it is necessary to separate ground scenes into cloud free and cloudy or cloud contaminated. Identifying the presence of cloud in a ground scene and establishing an accurate and adequate cloud mask is a challenging task. In this study, measurements by the European Space Agency (ESA) MEdium Resolution Imaging Spectrometer (MERIS) have been used to develop a cloud identification and cloud mask algorithm for preprocessing prior to application of the new algorithm cal…

A kernel regression approach to cloud and shadow detection in multitemporal images

Earth observation satellites will provide in the next years time series with enough revisit time allowing a better surface monitoring. In this work, we propose a cloud screening and a cloud shadow detection method based on detecting abrupt changes in the temporal domain. It is considered that the time series follows smooth variations and abrupt changes in certain spectral features will be mainly due to the presence of clouds or cloud shadows. The method is based on linear and nonlinear regression analysis; in particular we focus on the regularized least squares and kernel regression methods. Experiments are carried out using Landsat 5 TM time series acquired over Albacete (Spain), and compa…

Optimizing Kernel Ridge Regression for Remote Sensing Problems

Kernel methods have been very successful in remote sensing problems because of their ability to deal with high dimensional non-linear data. However, they are computationally expensive to train when a large amount of samples are used. In this context, while the amount of available remote sensing data has constantly increased, the size of training sets in kernel methods is usually restricted to few thousand samples. In this work, we modified the kernel ridge regression (KRR) training procedure to deal with large scale datasets. In addition, the basis functions in the reproducing kernel Hilbert space are defined as parameters to be also optimized during the training process. This extends the n…

Pattern Recognition Scheme for Large-Scale Cloud Detection over Landmarks

Landmark recognition and matching is a critical step in many Image Navigation and Registration (INR) models for geostationary satellite services, as well as to maintain the geometric quality assessment (GQA) in the instrument data processing chain of Earth observation satellites. Matching the landmark accurately is of paramount relevance, and the process can be strongly impacted by the cloud contamination of a given landmark. This paper introduces a complete pattern recognition methodology able to detect the presence of clouds over landmarks using Meteosat Second Generation (MSG) data. The methodology is based on the ensemble combination of dedicated support vector machines (SVMs) dependent…

Analysis of acousto-optic tunable filter performance for imaging applications

Acousto-optic tunable filters (AOTFs) can be used as spec- tral filters in multispectral imaging applications. Acousto-optic crystals diffract a single wavelength from a broadband light beam, depending on the applied radio frequency signal. However, experimental measurements show that the actual performance is far from the expected behavior. We present an experimental characterization of several commercial off-the- shelf AOTFs for the implementation of multispectral imaging instruments. The diffraction performance of three bare crystals is compared, while a fourth AOTF crystal is mounted on the optical path of a multispectral im- ager to evaluate its performance. The experiments show that t…

400– to 1000–nm imaging spectrometer based on acousto-optic tunable filters

An imaging spectrometer covering the 400-1000 nm band has been conceived and developed. The system is based on an Acousto-Optic Tunable Filter (AOTF) attached to a high performance digital camera. The AOTF permits the selection of spectral bands with an RF signal in the range of 70-210 MHz. The range is covered using two transducers attached to a single crystal. Although the idea is not new it covers a broader spectrum than previous systems. It includes a telecentric optical system that enhances system efficiency, by ensuring that the chief ray of each light cone emerges out of this doublet parallel to the optical axes. Additionally, an smart choice of integration time reduces the dependenc…

Domain Adaptation of Landsat-8 and Proba-V Data Using Generative Adversarial Networks for Cloud Detection

Training machine learning algorithms for new satellites requires collecting new data. This is a critical drawback for most remote sensing applications and specially for cloud detection. A sensible strategy to mitigate this problem is to exploit available data from a similar sensor, which involves transforming this data to resemble the new sensor data. However, even taking into account the technical characteristics of both sensors to transform the images, statistical differences between data distributions still remain. This results in a poor performance of the methods trained on one sensor and applied to the new one. In this this work, we propose to use the generative adversarial networks (G…

Developments for vegetation fluorescence retrieval from spaceborne high-resolution spectrometry in the O2-A and O2-B absorption bands

Solar-induced chlorophyll fluorescence is a weak electromagnetic signal emitted in the red and far-red spectral regions by vegetation chlorophyll under excitation by solar radiation. Chlorophyll fluorescence has been demonstrated to be a close proxy to vegetation physiological functioning. The basis for fluorescence retrieval from passive space measurements is the exploitation of the O2-A and O2-B atmospheric absorption features to isolate the fluorescence signal from the solar radiation reflected by the surface and the atmosphere. High spectral resolution measurements and a precise modeling of the atmospheric radiative transfer in the visible and near-infrared regions are mandatory. Recent…

Convolutional Neural Networks for Multispectral Image Cloud Masking

Convolutional neural networks (CNN) have proven to be state of the art methods for many image classification tasks and their use is rapidly increasing in remote sensing problems. One of their major strengths is that, when enough data is available, CNN perform an end-to-end learning without the need of custom feature extraction methods. In this work, we study the use of different CNN architectures for cloud masking of Proba-V multispectral images. We compare such methods with the more classical machine learning approach based on feature extraction plus supervised classification. Experimental results suggest that CNN are a promising alternative for solving cloud masking problems.

Kernel-based retrieval of atmospheric profiles from IASI data

This paper proposes the use of kernel ridge regression (KRR) to derive surface and atmospheric properties from hyperspectral infrared sounding spectra. We focus on the retrieval of temperature and humidity atmospheric profiles from Infrared Atmospheric Sounding Interferometer (MetOp-IASI) data, and provide confidence maps on the predictions. In addition, we propose a scheme for the identification of anomalies by supervised classification of discrepancies with the ECMWF estimates. For the retrieval, we observed that KRR clearly outperformed linear regression. Looking at the confidence maps, we observed that big discrepancies are mainly due to the presence of clouds and low emissivities in de…

Evaluation of remote sensing of vegetation fluorescence by the analysis of diurnal cycles

Chlorophyll fluorescence (ChF) emission is a direct indicator of the photosynthetic activity of vegetation, which is a key parameter of the carbon cycle. This paper analyses chlorophyll fluorescence evolution at leaf level during a complete diurnal cycle in simulated and natural conditions, for two species under different stress conditions. Absolute spectral radiance of the ChF emission is obtained allowing a quantitative derivation of the fluorescence yield of the ChF, which correlates well with established fluorescence instruments. The studied cases show that the ChF emission is mainly driven by the photosynthetic active radiation during the whole cycle, but the fluorescence yield is seve…

Methodology for the retrieval of vegetation chlorophyll fluorescence from space in the frame of the flex mission preparatory

FLEX (FLuorescence EXperiment) is a candidate mission for the European Space Agency (ESA) Earth Explorer program. The main objective of the mission is the measurement the chlorophyll fluorescence signal emitted by vegetation at the red and far-red spectral regions (roughly 630-770 nm). The current FLEX mission design includes different instruments intended to provide the appropriate characterization of those atmospheric and surface parameters necessary for the retrieval and interpretation of the fluorescence signal. The complete processing chain for the derivation of fluorescence and reflectance products from the radiance data acquired by the different instruments included in the FLEX paylo…

Semi-Supervised Remote Sensing Image Classification based on Clustering and the Mean Map Kernel

This paper presents a semi-supervised classifier based on the combination of the expectation-maximization (EM) algorithm for Gaussian mixture models (GMM) and the mean map kernel. The proposed method uses the most reliable samples in terms of maximum likelihood to compute a kernel function that accurately reflects the similarity between clusters in the kernel space. The proposed method improves classification accuracy in situations where the available labeled information does not properly describe the classes in the test image.

Multiset Kernel CCA for multitemporal image classification

The analysis of multitemporal remote sensing images is becoming an increasingly important problem because of the upcoming scenario of multispectral satellite constellations monitoring our Planet. Algorithms that can analyze such amount of heterogeneous information are necessary. While linear techniques have been extensively deployed, this work considers a kernel method that finds nonlinear correlations between all image sources and the class labels. We introduce in this context the Kernel Canonical Correlation Analysis (KCCA) to exploit the wealth of temporal image information and to handle nonlinear relations in a natural way via kernels. To achieve this goal, we use the generalization of …

Regularized RBF Networks for Hyperspectral Data Classification

In this paper, we analyze several regularized types of Radial Basis Function (RBF) Networks for crop classification using hyperspectral images. We compare the regularized RBF neural network with Support Vector Machines (SVM) using the RBF kernel, and AdaBoost Regularized (ABR) algorithm using RBF bases, in terms of accuracy and robustness. Several scenarios of increasing input space dimensionality are tested for six images containing six crop classes. Also, regularization, sparseness, and knowledge extraction are paid attention.

Sensitivity analysis of the fraunhofer line discrimination method for the measurement of chlorophyll fluorescence using a field spectroradiometer

The Fraunhofer Line Discrimination (FLD) principle is established as a good method for remote sensing of solar induced chlorophyll fluorescence. Some improvements to the method are analysed in order to determine and reduce the sources of error in the estimation of the fluorescence emission. A sensitivity analysis has been performed over simulated data generated from real diurnal cycle measurements.

Proba-V cloud detection Round Robin: Validation results and recommendations

This paper discusses results from 12 months of a Round Robin exercise aimed at the inter-comparison of different cloud detection algorithms for Proba-V. Clouds detection is a critical issue for satellite optical remote sensing, since potential errors in cloud masking directly translates into significant uncertainty in the retrieved downstream geophysical products. Cloud detection is particularly challenging for Proba-V due to the presence of a limited number of spectral bands and the lack of thermal infrared bands. The main objective of the project was the inter-comparison of several cloud detection algorithms for Proba-V over a wide range of surface types and environmental conditions. Prob…

A Review of Kernel Methods in Remote Sensing Data Analysis

Kernel methods have proven effective in the analysis of images of the Earth acquired by airborne and satellite sensors. Kernel methods provide a consistent and well-founded theoretical framework for developing nonlinear techniques and have useful properties when dealing with low number of (potentially high dimensional) training samples, the presence of heterogenous multimodalities, and different noise sources in the data. These properties are particularly appropriate for remote sensing data analysis. In fact, kernel methods have improved results of parametric linear methods and neural networks in applications such as natural resource control, detection and monitoring of anthropic infrastruc…

Cloud screening with combined MERIS and AATSR images

This paper presents a cloud screening algorithm based on ensemble methods that exploits the combined information from both MERIS and AATSR instruments on board ENVISAT in order to improve current cloud masking products for both sensors. The first step is to analyze the synergistic use of MERIS and AATSR images in order to extract some physically-based features increasing the separability of clouds and surface. Then, several artificial neural networks are trained using different sets of input features and different sets of training samples depending on acquisition and surface conditions. Finally, outputs of the trained neural networks are combined at the decision level to construct a more ac…

Fair Kernel Learning

New social and economic activities massively exploit big data and machine learning algorithms to do inference on people’s lives. Applications include automatic curricula evaluation, wage determination, and risk assessment for credits and loans. Recently, many governments and institutions have raised concerns about the lack of fairness, equity and ethics in machine learning to treat these problems. It has been shown that not including sensitive features that bias fairness, such as gender or race, is not enough to mitigate the discrimination when other related features are included. Instead, including fairness in the objective function has been shown to be more efficient.

Cloud detection machine learning algorithms for PROBA-V

This paper presents the development and implementation of a cloud detection algorithm for Proba-V. Accurate and automatic detection of clouds in satellite scenes is a key issue for a wide range of remote sensing applications. With no accurate cloud masking, undetected clouds are one of the most significant sources of error in both sea and land cover biophysical parameter retrieval. The objective of the algorithms presented in this paper is to detect clouds accurately providing a cloud flag per pixel. For this purpose, the method exploits the information of Proba-V using statistical machine learning techniques to identify the clouds present in Proba-V products. The effectiveness of the propo…

Multitemporal fusion of Landsat/TM and ENVISAT/MERIS for crop monitoring

Abstract Monitoring Earth dynamics using current and future satellites is one of the most important objectives of the remote sensing community. The exploitation of image time series from sensors with different characteristics provides new opportunities to increase the knowledge about environmental changes and to support many operational applications. This paper presents an image fusion approach based on multiresolution and multisensor regularized spatial unmixing. The approach yields a composite image with the spatial resolution of the high spatial resolution image while retaining the spectral and temporal characteristics of the medium spatial resolution image. The approach is tested using …

Mean Kernels for Semi-Supervised Remote Sensing Image Classification

Multitemporal unmixing of MERIS FR data

Cross-Sensor Adversarial Domain Adaptation of Landsat-8 and Proba-V images for Cloud Detection

The number of Earth observation satellites carrying optical sensors with similar characteristics is constantly growing. Despite their similarities and the potential synergies among them, derived satellite products are often developed for each sensor independently. Differences in retrieved radiances lead to significant drops in accuracy, which hampers knowledge and information sharing across sensors. This is particularly harmful for machine learning algorithms, since gathering new ground truth data to train models for each sensor is costly and requires experienced manpower. In this work, we propose a domain adaptation transformation to reduce the statistical differences between images of two…

Signal-to-noise ratio in reproducing kernel Hilbert spaces

This paper introduces the kernel signal-to-noise ratio (kSNR) for different machine learning and signal processing applications}. The kSNR seeks to maximize the signal variance while minimizing the estimated noise variance explicitly in a reproducing kernel Hilbert space (rkHs). The kSNR gives rise to considering complex signal-to-noise relations beyond additive noise models, and can be seen as a useful signal-to-noise regularizer for feature extraction and dimensionality reduction. We show that the kSNR generalizes kernel PCA (and other spectral dimensionality reduction methods), least squares SVM, and kernel ridge regression to deal with cases where signal and noise cannot be assumed inde…

Learning with the kernel signal to noise ratio

This paper presents the application of the kernel signal to noise ratio (KSNR) in the context of feature extraction to general machine learning and signal processing domains. The proposed approach maximizes the signal variance while minimizes the estimated noise variance in a reproducing kernel Hilbert space (RKHS). The KSNR can be used in any kernel method to deal with correlated (possibly non-Gaussian) noise. We illustrate the method in nonlinear regression examples, dependence estimation and causal inference, nonlinear channel equalization, and nonlinear feature extraction from high-dimensional satellite images. Results show that the proposed KSNR yields more fitted solutions and extract…

Segmentation of Hyperspectral Images for the Detection of Rotten Mandarins

The detection of rotten citrus in packing lines is carried out manually under ultraviolet illumination, which is dangerous for workers. Light emitted by the rotten region of the fruit due to the ultraviolet-induced fluorescence is used by the operator to detect the damages. This procedure is required because the low contrast between the damaged and sound skin under visible illumination difficult their detection. We study a set of techniques aimed to detect rottenness in citrususing visible and near infrared lighting trough an hyperspectral imaging system. Methods for selecting a proper set of wavelengths are investigated such as correlation analysis, mutual information, stepwise or genetic …

Support Vector Machines for Crop Classification Using Hyperspectral Data

In this communication, we propose the use of Support Vector Machines (SVM) for crop classification using hyperspectral images. SVM are benchmarked to well–known neural networks such as multilayer perceptrons (MLP), Radial Basis Functions (RBF) and Co-Active Neural Fuzzy Inference Systems (CANFIS). Models are analyzed in terms of efficiency and robustness, which is tested according to their suitability to real–time working conditions whenever a preprocessing stage is not possible. This can be simulated by considering models with and without a preprocessing stage. Four scenarios (128, 6, 3 and 2 bands) are thus evaluated. Several conclusions are drawn: (1) SVM yield better outcomes than neura…

Spectro-temporal reflectance surfaces: a new conceptual framework for the integration of remote-sensing data from multiple different sensors

The conflict between spatial and temporal resolution of satellite systems, as well as the frequent presence of clouds in the images, has been a traditional limitation of remote sensing in the optical domain. Nevertheless, most of the conceptual tools and algorithms developed classically in remote sensing are based on the input of a series of cloud-free images from identical sensors. In this study, we propose a conceptual framework that is able to ingest data from several different sensors, make them homogeneous, eliminate clouds virtually, and make them usable in a flexible, efficient, and transparent way. The methodology is based on previous developments such as spatial ‘downscaling’, temp…

Convolutional Long Short-Term Memory Network for Multitemporal Cloud Detection Over Landmarks

In this work, we propose to exploit both the temporal and spatial correlations in Earth observation satellite images through deep learning methods. In particular, the combination of a U-Net convolutional neural network together with a convolutional long short-term memory (LSTM) layer is proposed. This model is applied for cloud detection on MSG/SEVIRI image time series over selected landmarks. Implementation details are provided and our proposal is compared against a standard SVM and a U-Net without the convolutional LSTM layer but including temporal information too. Experimental results show that this combination of networks exploits both the spatial and temporal dependence and provides st…

Spectral clustering with the probabilistic cluster kernel

Abstract This letter introduces a probabilistic cluster kernel for data clustering. The proposed kernel is computed with the composition of dot products between the posterior probabilities obtained via GMM clustering. The kernel is directly learned from the data, is parameter-free, and captures the data manifold structure at different scales. The projections in the kernel space induced by this kernel are useful for general feature extraction purposes and are here exploited in spectral clustering with the canonical k-means. The kernel structure, informative content and optimality are studied. Analysis and performance are illustrated in several real datasets.

Configurable Passband Imaging Spectrometer Based on Acousto-optic Tunable Filter

This work presents a new configurable imaging spectrometer called Autonomous Tunable Filtering System (ATFS). The system can be configured to acquire a single narrow spectral band, a composite multispectral image, or a broad pass-band. This flexibility is given by the use of an Acousto-Optic Tunable Filter (AOTF) driven by a programmable radio frequency (rf) signal generator. The AOTF acts as a light-diffraction element which output wavelength is selected by the frequency of an rf signal applied to it. The designed rf driver is based on a high-speed Digital-to-Analog converter, which can synthesize any composite rf waveform formed by a combination of sine signals. The images are formed thro…

Cloud detection on the Google Earth engine platform

The vast amount of data acquired by current high resolution Earth observation satellites implies some technical challenges to be faced. Google Earth Engine (GEE) platform provides a framework for the development of algorithms and products built over this data in an easy and scalable manner. In this paper, we take advantage of the GEE platform capabilities to exploit the wealth of information in the temporal dimension by processing a long time series of satellite images. A cloud detection algorithm for Landsat-8, which uses previous images of the same location to detect clouds, is implemented and tested on the GEE platform.

Gridding artifacts on medium-resolution satellite image time series: MERIS case study

Earth observation satellites provide a valuable source of data which when conveniently processed can be used to better understand the Earth system dynamics. In this regard, one of the prerequisites for the analysis of satellite image time series is that the images are spatially coregistered so that the resulting multitemporal pixel entities offer a true temporal view of the area under study. This implies that all the observations must be mapped to a common system of grid cells. This process is known as gridding and, in practice, two common grids can be used as a reference: 1) a grid defined by some kind of external data set (e.g., an existing land-cover map) or 2) a grid defined by one of t…

Semisupervised kernel orthonormalized partial least squares

This paper presents a semisupervised kernel orthonormalized partial least squares (SS-KOPLS) algorithm for non-linear feature extraction. The proposed method finds projections that minimize the least squares regression error in Hilbert spaces and incorporates the wealth of unlabeled information to deal with small size labeled datasets. The method relies on combining a standard RBF kernel using labeled information, and a generative kernel learned by clustering all available data. The positive definiteness of the kernels is proven, and the structure and information content of the derived kernels is studied. The effectiveness of the proposed method is successfully illustrated in standard UCI d…

Study of the diurnal cycle of stressed vegetation for the improvement of fluorescence remote sensing

Chlorophyll fluorescence (Chf) emission allows estimating the photosynthetic activity of vegetation - a key parameter for the carbon cycle models - in a quite direct way. However, measuring Chf is difficult because it represents a small fraction of the radiance to be measured by the sensor. This paper analyzes the relationship between the solar induced Chf emission and the photosynthetically active radiation (PAR) in plants under water stress condition. The solar induced fluorescence emission is measured at leaf level by means of three different methodologies. Firstly, an active modulated light fluorometer gives the relative fluorescence yield. Secondly, a quantitative measurement of the Ch…

Estimating crop primary productivity with Sentinel-2 and Landsat 8 using machine learning methods trained with radiative transfer simulations

Abstract Satellite remote sensing has been widely used in the last decades for agricultural applications, both for assessing vegetation condition and for subsequent yield prediction. Existing remote sensing-based methods to estimate gross primary productivity (GPP), which is an important variable to indicate crop photosynthetic function and stress, typically rely on empirical or semi-empirical approaches, which tend to over-simplify photosynthetic mechanisms. In this work, we take advantage of all parallel developments in mechanistic photosynthesis modeling and satellite data availability for an advanced monitoring of crop productivity. In particular, we combine process-based modeling with …

Towards a novel approach for Sentinel-3 synergistic OLCI/SLSTR cloud and cloud shadow detection based on stereo cloud-top height estimation

Abstract Sentinel-3 is an Earth observation satellite constellation launched by the European Space Agency. Each satellite carries two optical multispectral instruments: the Ocean and Land Colour Instrument (OLCI) and the Sea and Land Surface Temperature Radiometer (SLSTR). OLCI and SLSTR sensors produce images covering the visible and infrared spectrum that can be collocated in order to generate synergistic products. In Earth observation, a particular weakness of optical sensors is their high sensitivity to clouds and their shadows. An incorrect cloud and cloud shadow detection leads to mistakes in both land and ocean retrievals of biophysical parameters. In order to exploit both OLCI and S…

Remote sensing of chlorophyll fluorescence for estimation of stress in vegetation. Recommendations for future missions

Vegetation monitoring is a key issue in Earth Observation due to its relation with the global CO2 cycle. Chlorophyll fluorescence (ChF) emitted by the vegetation is an accurate indicator of the plant status and their photosynthetic activity. This work analyses the diurnal evolution of the ChF emission spectrum and the fluorescence yield in order to determine the best conditions for remote sensing of ChF from a satellite platform. The ChF evolution is studied at leaf level during several diurnal cycles, in simulated conditions, for two species under different stress conditions. The analysis of the signal levels gives an estimation of the values of ChF emission which could be observed from a …

Biophysical parameter estimation with adaptive Gaussian Processes

We evaluate Gaussian Processes (GPs) for the estimation of biophysical parameters from acquired multispectral data. The standard GP formulation is used, and all hyperparameters (kernel parameters and noise variance) are optimized by maximizing the marginal likelihood. This gives rise to a fully-adaptive GP to data characteristics, both in terms of signal and noise properties. The good numerical results in the estimation of oceanic chlorophyll concentration and leaf membrane state confirm GPs as adequate, alternative non-parametric methods for biophysical parameter estimation. GPs are also analyzed by scrutinizing the predictive variance, the estimated noise variance, and the relevance of ea…

Configurable bandwidth imaging spectrometer based on acousto-optic tunable filter

This paper presents a new portable instrument called Autonomous Tunable Filtering System (ATFS), developed for highly customisable imaging spectrometry in the VIS-NIR range. The ATFS instrument consists of an Acousto-Optic Tunable Filter (AOTF), an optical system, a Radio Frequency (RF) driver based on a Direct Digital Synthesiser (DDS) and control software. The ATFS can be attached to a variety of high-performance monochrome cameras. The system works as a spectral bandpass filter whose wavelength can be selected between 400nm and 1000nm and whose bandwidth can be adjusted between 4nm and 50nm. The filter can be tuned electronically at a very high speed and accuracy, thanks to the DDS versa…

Kernels for Remote Sensing Image Classification

Classification of images acquired by airborne and satellite sensors is a very challenging problem. These remotely sensed images usually acquire information from the scene at different wavelengths or spectral channels. The main problems involved are related to the high dimensionality of the data to be classified and the very few existing labeled samples, the diverse noise sources involved in the acquisition process, the intrinsic nonlinearity and non-Gaussianity of the data distribution in feature spaces, and the high computational cost involved to process big data cubes in near real time. The framework of statistical learning in general, and of kernel methods in particular, has gained popul…

Cloud-screening algorithm for ENVISAT/MERIS multispectral images

This paper presents a methodology for cloud screening of multispectral images acquired with the Medium Resolution Imaging Spectrometer (MERIS) instrument on-board the Environmental Satellite (ENVISAT). The method yields both a discrete cloud mask and a cloud-abundance product from MERIS level-lb data on a per-pixel basis. The cloud-screening method relies on the extraction of meaningful physical features (e.g., brightness and whiteness), which are combined with atmospheric-absorption features at specific MERIS-band locations (oxygen and watervapor absorptions) to increase the cloud-detection accuracy. All these features are inputs to an unsupervised classification algorithm; the cloud-proba…

Operational cloud screening service for Sentinel-2 image time series

This paper deals with the development and implementation of a cloud screening algorithm for image time series, with the focus on the forthcoming Sentinel-2 satellites to be launched under the ESA Copernicus Programme. The proposed methodology is based on kernel ridge regression and exploits the temporal information to detect anomalous changes that correspond to cloud covers. The huge data volumes to be processed when dealing with high temporal, spatial, and spectral resolution datasets motivate the implementation of the algorithm within distributed computer resources. In consequence, an operational cloud screening service has been specifically designed and implemented in the frame of the Se…

Cloud screening and multitemporal unmixing of MERIS FR data

The operational use of MERIS images can be hampered by the presence of clouds. This work presents a cloud screening algorithm that takes advantage of the high spectral and radiometric resolutions of MERIS and the specific location of some of its bands to increase the cloud detection accuracy. Moreover, the proposed algorithm provides a per-pixel probabilistic map of cloud abundance rather than a binary cloud presence flag. In order to test the proposed algorithm we propose a cloud screening validation method based on temporal series. In addition, we evaluate the impact of the cloud screening in a multitemporal unmixing application, where a temporal series of MERIS FR images acquired over Th…

Regularized multiresolution spatial unmixing for ENVISAT/MERIS and landsat/TM image fusion

Earth observation satellites currently provide a large volume of images at different scales. Most of these satellites provide global coverage with a revisit time that usually depends on the instrument characteristics and performance. Typically, medium-spatial-resolution instruments provide better spectral and temporal resolutions than mapping-oriented high-spatial-resolution multispectral sensors. However, in order to monitor a given area of interest, users demand images with the best resolution available, which cannot be reached using a single sensor. In this context, image fusion may be effective to merge information from different data sources. In this letter, an image fusion approach ba…

Kernel-Based Framework for Multitemporal and Multisource Remote Sensing Data Classification and Change Detection

The multitemporal classification of remote sensing images is a challenging problem, in which the efficient combination of different sources of information (e.g., temporal, contextual, or multisensor) can improve the results. In this paper, we present a general framework based on kernel methods for the integration of heterogeneous sources of information. Using the theoretical principles in this framework, three main contributions are presented. First, a novel family of kernel-based methods for multitemporal classification of remote sensing images is presented. The second contribution is the development of nonlinear kernel classifiers for the well-known difference and ratioing change detectio…

Cloud masking and removal in remote sensing image time series

Automatic cloud masking of Earth observation images is one of the first required steps in optical remote sensing data processing since the operational use and product generation from satellite image time series might be hampered by undetected clouds. The high temporal revisit of current and forthcoming missions and the scarcity of labeled data force us to cast cloud screening as an unsupervised change detection problem in the temporal domain. We introduce a cloud screening method based on detecting abrupt changes along the time dimension. The main assumption is that image time series follow smooth variations over land (background) and abrupt changes will be mainly due to the presence of clo…

CHRIS/PROBA toolbox for hyperspectral and multiangular data exploitations

The project CHRIS/Proba Toolbox for BEAM (CHRIS-Box) has been developed in order to support users of data from the CHRIS sensor onboard of ESA's Proba platform. BEAM and the CHRIS-Box are user tools which ESA/ESRTN are providing free of charge to the Earth Observation Community. The CHRIS-Box software provides extensions for BEAM that allows accomplishing the following tasks: a) Noise reduction to remove the vertical striping and other noise present in CHRIS response-corrected images; b) Cloud screening to mark cloudy pixels in CHRIS noise-corrected images; the cloud screening algorithm provides cloud probability and abundances for each pixel; c) Atmospheric correction that provides surface…

Retrieval of oceanic chlorophyll concentration with relevance vector machines

Abstract In this communication, we evaluate the performance of the relevance vector machine (RVM) for the estimation of biophysical parameters from remote sensing data. For illustration purposes, we focus on the estimation of chlorophyll-a concentrations from remote sensing reflectance just above the ocean surface. A variety of bio-optical algorithms have been developed to relate measurements of ocean radiance to in situ concentrations of phytoplankton pigments, and ultimately most of these algorithms demonstrate the potential of quantifying chlorophyll-a concentrations accurately from multispectral satellite ocean color data. Both satellite-derived data and in situ measurements are subject…

Nonlinear statistical retrieval of surface emissivity from IASI data

Emissivity is one of the most important parameters to improve the determination of the troposphere properties (thermodynamic properties, aerosols and trace gases concentration) and it is essential to estimate the radiative budget. With the second generation of infrared sounders, we can estimate emissivity spectra at high spectral resolution, which gives us a global view and long-term monitoring of continental surfaces. Statistically, this is an ill-posed retrieval problem, with as many output variables as inputs. We here propose nonlinear multi-output statistical regression based on kernel methods to estimate spectral emissivity given the radiances. Kernel methods can cope with high-dimensi…

Cloud detection for CHRIS/Proba hyperspectral images

Accurate and automatic detection of clouds in satellite scenes is a key issue for a wide range of remote sensing applications. With no accurate cloud masking, undetected clouds are one of the most significant source of error in both sea and land cover biophysical parameter retrieval. Sensors with spectral channels beyond 1 um have demonstrated good capabilities to perform cloud masking. This spectral range can not be exploited by recently developed hyperspectral sensors that work in the spectral range between 400- 1000 nm. However, one can take advantage of their high number of channels and spectral resolution to increase the cloud detection accuracy, and to describe properly the detected c…

Semisupervised Kernel Feature Extraction for Remote Sensing Image Analysis

This paper presents a novel semisupervised kernel partial least squares (KPLS) algorithm for nonlinear feature extraction to tackle both land-cover classification and biophysical parameter retrieval problems. The proposed method finds projections of the original input data that align with the target variable (labels) and incorporates the wealth of unlabeled information to deal with low-sized or underrepresented data sets. The method relies on combining two kernel functions: the standard radial-basis-function kernel based on labeled information and a generative, i.e., probabilistic, kernel directly learned by clustering the data many times and at different scales across the data manifold. Th…

Encoding Invariances in Remote Sensing Image Classification With SVM

This letter introduces a simple method for including invariances in support-vector-machine (SVM) remote sensing image classification. We design explicit invariant SVMs to deal with the particular characteristics of remote sensing images. The problem of including data invariances can be viewed as a problem of encoding prior knowledge, which translates into incorporating informative support vectors (SVs) that better describe the classification problem. The proposed method essentially generates new (synthetic) SVs from the obtained by training a standard SVM with the available labeled samples. Then, original and transformed SVs are used for training the virtual SVM introduced in this letter. W…

Improving the performance of acousto-optic tunable filters in imaging applications

Acousto-optic tunable filters (AOTFs) can be used as spectral filters for the implementation of multispectral imaging systems. However, obtaining quality images is challenging. In this work, we propose several improvements that enable the use of these systems in quantitative spectroscopic imaging applications. The improvements are based on three pillars: 1. a finer spectral bandpass shaping by dynamically optimizing the radio frequency (rf) driving signal, 2. an extensive calibration process, and 3. careful image preprocessing that uses calibration data to correct some well known AOTF issues in imaging applications. A novel multispectral imaging instrument is built using commercial off-the-…

Estimation of solar-induced vegetation fluorescence from space measurements

[1] A characteristic spectral emission is observed in vegetation chlorophyll under excitation by solar radiation. This emission, known as solar-induced chlorophyll fluorescence, occurs in the red and near infra-red spectral regions. In this paper a new methodology for the estimation of solar-induced chlorophyll fluorescence from spaceborne and airborne sensors is presented. The fluorescence signal is included in an atmospheric radiative transfer scheme so that chlorophyll fluorescence and surface reflectance are retrieved consistently from the measured at-sensor radiance. This methodology is tested on images acquired by the Medium Resolution Imaging Spectrometer (MERIS) on board the ENVIron…

Fair Kernel Learning

New social and economic activities massively exploit big data and machine learning algorithms to do inference on people's lives. Applications include automatic curricula evaluation, wage determination, and risk assessment for credits and loans. Recently, many governments and institutions have raised concerns about the lack of fairness, equity and ethics in machine learning to treat these problems. It has been shown that not including sensitive features that bias fairness, such as gender or race, is not enough to mitigate the discrimination when other related features are included. Instead, including fairness in the objective function has been shown to be more efficient. We present novel fai…

Transferring deep learning models for cloud detection between Landsat-8 and Proba-V

Abstract Accurate cloud detection algorithms are mandatory to analyze the large streams of data coming from the different optical Earth observation satellites. Deep learning (DL) based cloud detection schemes provide very accurate cloud detection models. However, training these models for a given sensor requires large datasets of manually labeled samples, which are very costly or even impossible to create when the satellite has not been launched yet. In this work, we present an approach that exploits manually labeled datasets from one satellite to train deep learning models for cloud detection that can be applied (or transferred) to other satellites. We take into account the physical proper…

Coupled retrieval of aerosol optical thickness, columnar water vapor and surface reflectance maps from ENVISAT/MERIS data over land

An algorithm for the derivation of atmospheric parameters and surface reflectance data from MEdium Resolution Imaging Specrometer Instrument (MERIS) on board ENVIronmental SATellite (ENVISAT) images has been developed. Geo-rectified aerosol optical thickness (AOT), columnar water vapor (CWV) and spectral surface reflectance maps are generated from MERIS Level-1b data over land. The algorithm has been implemented so that AOT, CWV and reflectance products are provided on an operational manner, making no use of ancillary parameters apart from those attached to MERIS products. For this reason, it has been named Self-Contained Atmospheric Parameters Estimation from MERIS data (SCAPE-M). The fund…

Correction of systematic spatial noise in push-broom hyperspectral sensors: application to CHRIS/PROBA images

Hyperspectral remote sensing images are affected by different types of noise. In addition to typical random noise, nonperiodic partially deterministic disturbance patterns generally appear in the data. These patterns, which are intrinsic to the image formation process, are characterized by a high degree of spatial and spectral coherence. We present a new technique that faces the problem of removing the spatially coherent noise known as vertical striping, usually found in images acquired by push-broom sensors. The developed methodology is tested on data acquired by the Compact High Resolution Imaging Spectrometer (CHRIS) onboard the Project for On-board Autonomy (PROBA) orbital platform, whi…

Including invariances in SVM remote sensing image classification

This paper introduces a simple method to include invariances in support vector machine (SVM) for remote sensing image classification. We rely on the concept of virtual support vectors, by which the SVM is trained with both the selected support vectors and synthetic examples encoding the invariance of interest. The algorithm is very simple and effective, as demonstrated in two particularly interesting examples: invariance to the presence of shadows and to rotations in patchbased image segmentation. The improved accuracy (around +6% both in OA and Cohen's κ statistic), along with the simplicity of the approach encourage its use and extension to encode other invariances and other remote sensin…

Semisupervised nonlinear feature extraction for image classification

Feature extraction is of paramount importance for an accurate classification of remote sensing images. Techniques based on data transformations are widely used in this context. However, linear feature extraction algorithms, such as the principal component analysis and partial least squares, can address this problem in a suboptimal way because the data relations are often nonlinear. Kernel methods may alleviate this problem only when the structure of the data manifold is properly captured. However, this is difficult to achieve when small-size training sets are available. In these cases, exploiting the information contained in unlabeled samples together with the available training data can si…

Hyperspectral system for early detection of rottenness caused by Penicillium digitatum in mandarins

Abstract Nowadays, the detection of fruit infected with Penicillium sp. fungi on packing lines is carried out manually under ultraviolet illumination. Ultraviolet sources induce visible fluorescence of essential oils, present in the skin of citrus and which are released by the action of fungi, thus increasing the contrast between sound and rotten skin. This work analyses a set of techniques aimed at detecting rotten citrus without the use of UV lighting. The techniques used include hyperspectral image acquisition, pre-processing and calibration, feature selection and segmentation using linear and non-linear methods for classification of fruits. Different methods such as correlation analysis…

Semi-Supervised Support Vector Biophysical Parameter Estimation

Two kernel-based methods for semi-supervised regression are presented. The methods rely on building a graph or hypergraph Laplacian with both the labeled and unlabeled data, which is further used to deform the training kernel matrix. The deformed kernel is then used for support vector regression (SVR). The semi-supervised SVR methods are sucessfully tested in LAI estimation and ocean chlorophyll concentration prediction from remotely sensed images.

Automatic correction of the effects of the light source on spherical objects. An application to the analysis of hyperspectral images of citrus fruits

This study proposes a method for correcting the adverse effects produced by the curvature of spherical objects in acquiring images with a computer vision system. Its suitability has been illustrated in a specific case of citrus fruits. The images of this kind of fruit are darker in areas nearer the edge than in the centre, and this makes them more difficult to analyse. This methodology considers the fruit as being a Lambertian ellipsoidal surface and produces a 3D model of the fruit. By doing it becomes possible to calculate the part of the radiation that should really reach the camera and to make the intensity of the radiation uniform over the whole of the fruit surface captured by the cam…

Multitemporal Cloud Masking in the Google Earth Engine

The exploitation of Earth observation satellite images acquired by optical instruments requires an automatic and accurate cloud detection. Multitemporal approaches to cloud detection are usually more powerful than their single scene counterparts since the presence of clouds varies greatly from one acquisition to another whereas surface can be assumed stationary in a broad sense. However, two practical limitations usually hamper their operational use: the access to the complete satellite image archive and the required computational power. This work presents a cloud detection and removal methodology implemented in the Google Earth Engine (GEE) cloud computing platform in order to meet these r…

HyperLabelMe : A Web Platform for Benchmarking Remote-Sensing Image Classifiers

HyperLabelMe is a web platform that allows the automatic benchmarking of remote-sensing image classifiers. To demonstrate this platform's attributes, we collected and harmonized a large data set of labeled multispectral and hyperspectral images with different numbers of classes, dimensionality, noise sources, and levels. The registered user can download training data pairs (spectra and land cover/use labels) and submit the predictions for unseen testing spectra. The system then evaluates the accuracy and robustness of the classifier, and it reports different scores as well as a ranked list of the best methods and users. The system is modular, scalable, and ever-growing in data sets and clas…