Search results for "dimensionality"
showing 10 items of 231 documents
Forward-backward equations for nonlinear propagation in axially invariant optical systems
2004
We present a novel general framework to deal with forward and backward components of the electromagnetic field in axially-invariant nonlinear optical systems, which include those having any type of linear or nonlinear transverse inhomogeneities. With a minimum amount of approximations, we obtain a system of two first-order equations for forward and backward components explicitly showing the nonlinear couplings among them. The modal approach used allows for an effective reduction of the dimensionality of the original problem from 3+1 (three spatial dimensions plus one time dimension) to 1+1 (one spatial dimension plus one frequency dimension). The new equations can be written in a spinor Dir…
Systematic Assessment of MODTRAN Emulators for Atmospheric Correction
2021
Atmospheric radiative transfer models (RTMs) simulate the light propagation in the Earth's atmosphere. With the evolution of RTMs, their increase in complexity makes them impractical in routine processing such as atmospheric correction. To overcome their computational burden, standard practice is to interpolate a multidimensional lookup table (LUT) of prestored simulations. However, accurate interpolation relies on large LUTs, which still implies large computation times for their generation and interpolation. In recent years, emulation has been proposed as an alternative to LUT interpolation. Emulation approximates the RTM outputs by a statistical regression model trained with a low number …
Variable Selection in Predictive MIDAS Models
2014
In short-term forecasting, it is essential to take into account all available information on the current state of the economic activity. Yet, the fact that various time series are sampled at different frequencies prevents an efficient use of available data. In this respect, the Mixed-Data Sampling (MIDAS) model has proved to outperform existing tools by combining data series of different frequencies. However, major issues remain regarding the choice of explanatory variables. The paper first addresses this point by developing MIDAS based dimension reduction techniques and by introducing two novel approaches based on either a method of penalized variable selection or Bayesian stochastic searc…
Assessment of computational methods for the analysis of single-cell ATAC-seq data
2019
Abstract Background Recent innovations in single-cell Assay for Transposase Accessible Chromatin using sequencing (scATAC-seq) enable profiling of the epigenetic landscape of thousands of individual cells. scATAC-seq data analysis presents unique methodological challenges. scATAC-seq experiments sample DNA, which, due to low copy numbers (diploid in humans), lead to inherent data sparsity (1–10% of peaks detected per cell) compared to transcriptomic (scRNA-seq) data (10–45% of expressed genes detected per cell). Such challenges in data generation emphasize the need for informative features to assess cell heterogeneity at the chromatin level. Results We present a benchmarking framework that …
Fair Kernel Learning
2017
New social and economic activities massively exploit big data and machine learning algorithms to do inference on people’s lives. Applications include automatic curricula evaluation, wage determination, and risk assessment for credits and loans. Recently, many governments and institutions have raised concerns about the lack of fairness, equity and ethics in machine learning to treat these problems. It has been shown that not including sensitive features that bias fairness, such as gender or race, is not enough to mitigate the discrimination when other related features are included. Instead, including fairness in the objective function has been shown to be more efficient.
Functional Linear Regression
2018
This article presents a selected bibliography on functional linear regression (FLR) and highlights the key contributions from both applied and theoretical points of view. It first defines FLR in the case of a scalar response and shows how its modelization can also be extended to the case of a functional response. It then considers two kinds of estimation procedures for this slope parameter: projection-based estimators in which regularization is performed through dimension reduction, such as functional principal component regression, and penalized least squares estimators that take into account a penalized least squares minimization problem. The article proceeds by discussing the main asympt…
Nonlinearities and Adaptation of Color Vision from Sequential Principal Curves Analysis
2016
Mechanisms of human color vision are characterized by two phenomenological aspects: the system is nonlinear and adaptive to changing environments. Conventional attempts to derive these features from statistics use separate arguments for each aspect. The few statistical explanations that do consider both phenomena simultaneously follow parametric formulations based on empirical models. Therefore, it may be argued that the behavior does not come directly from the color statistics but from the convenient functional form adopted. In addition, many times the whole statistical analysis is based on simplified databases that disregard relevant physical effects in the input signal, as, for instance…
Transfer Learning with Convolutional Networks for Atmospheric Parameter Retrieval
2018
The Infrared Atmospheric Sounding Interferometer (IASI) on board the MetOp satellite series provides important measurements for Numerical Weather Prediction (NWP). Retrieving accurate atmospheric parameters from the raw data provided by IASI is a large challenge, but necessary in order to use the data in NWP models. Statistical models performance is compromised because of the extremely high spectral dimensionality and the high number of variables to be predicted simultaneously across the atmospheric column. All this poses a challenge for selecting and studying optimal models and processing schemes. Earlier work has shown non-linear models such as kernel methods and neural networks perform w…
Unsupervised Anomaly and Change Detection With Multivariate Gaussianization
2022
Anomaly detection (AD) is a field of intense research in remote sensing (RS) image processing. Identifying low probability events in RS images is a challenging problem given the high dimensionality of the data, especially when no (or little) information about the anomaly is available a priori. While a plenty of methods are available, the vast majority of them do not scale well to large datasets and require the choice of some (very often critical) hyperparameters. Therefore, unsupervised and computationally efficient detection methods become strictly necessary, especially now with the data deluge problem. In this article, we propose an unsupervised method for detecting anomalies and changes …
Diffusion map for clustering fMRI spatial maps extracted by Indipendent Component Analysis
2013
Functional magnetic resonance imaging (fMRI) produces data about activity inside the brain, from which spatial maps can be extracted by independent component analysis (ICA). In datasets, there are n spatial maps that contain p voxels. The number of voxels is very high compared to the number of analyzed spatial maps. Clustering of the spatial maps is usually based on correlation matrices. This usually works well, although such a similarity matrix inherently can explain only a certain amount of the total variance contained in the high-dimensional data where n is relatively small but p is large. For high-dimensional space, it is reasonable to perform dimensionality reduction before clustering.…