Search results for "Entropy"
showing 10 items of 496 documents
Kernel methods and their derivatives: Concept and perspectives for the earth system sciences.
2020
Kernel methods are powerful machine learning techniques which implement generic non-linear functions to solve complex tasks in a simple way. They Have a solid mathematical background and exhibit excellent performance in practice. However, kernel machines are still considered black-box models as the feature mapping is not directly accessible and difficult to interpret.The aim of this work is to show that it is indeed possible to interpret the functions learned by various kernel methods is intuitive despite their complexity. Specifically, we show that derivatives of these functions have a simple mathematical formulation, are easy to compute, and can be applied to many different problems. We n…
A survey of active learning algorithms for supervised remote sensing image classification
2011
Defining an efficient training set is one of the most delicate phases for the success of remote sensing image classification routines. The complexity of the problem, the limited temporal and financial resources, as well as the high intraclass variance can make an algorithm fail if it is trained with a suboptimal dataset. Active learning aims at building efficient training sets by iteratively improving the model performance through sampling. A user-defined heuristic ranks the unlabeled pixels according to a function of the uncertainty of their class membership and then the user is asked to provide labels for the most uncertain pixels. This paper reviews and tests the main families of active …
Fractional generalized cumulative entropy and its dynamic version
2021
Following the theory of information measures based on the cumulative distribution function, we propose the fractional generalized cumulative entropy, and its dynamic version. These entropies are particularly suitable to deal with distributions satisfying the proportional reversed hazard model. We study the connection with fractional integrals, and some bounds and comparisons based on stochastic orderings, that allow to show that the proposed measure is actually a variability measure. The investigation also involves various notions of reliability theory, since the considered dynamic measure is a suitable extension of the mean inactivity time. We also introduce the empirical generalized fract…
Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes
2017
Exploiting the theory of state space models, we derive the exact expressions of the information transfer, as well as redundant and synergistic transfer, for coupled Gaussian processes observed at multiple temporal scales. All of the terms, constituting the frameworks known as interaction information decomposition and partial information decomposition, can thus be analytically obtained for different time scales from the parameters of the VAR model that fits the processes. We report the application of the proposed methodology firstly to benchmark Gaussian systems, showing that this class of systems may generate patterns of information decomposition characterized by prevalently redundant or sy…
Local Granger causality
2021
Granger causality is a statistical notion of causal influence based on prediction via vector autoregression. For Gaussian variables it is equivalent to transfer entropy, an information-theoretic measure of time-directed information transfer between jointly dependent processes. We exploit such equivalence and calculate exactly the 'local Granger causality', i.e. the profile of the information transfer at each discrete time point in Gaussian processes; in this frame Granger causality is the average of its local version. Our approach offers a robust and computationally fast method to follow the information transfer along the time history of linear stochastic processes, as well as of nonlinear …
Multiscale analysis of information dynamics for linear multivariate processes.
2016
In the study of complex physical and physiological systems represented by multivariate time series, an issue of great interest is the description of the system dynamics over a range of different temporal scales. While information-theoretic approaches to the multiscale analysis of complex dynamics are being increasingly used, the theoretical properties of the applied measures are poorly understood. This study introduces for the first time a framework for the analytical computation of information dynamics for linear multivariate stochastic processes explored at different time scales. After showing that the multiscale processing of a vector autoregressive (VAR) process introduces a moving aver…
Gaussianizing the Earth: Multidimensional Information Measures for Earth Data Analysis
2021
Information theory is an excellent framework for analyzing Earth system data because it allows us to characterize uncertainty and redundancy, and is universally interpretable. However, accurately estimating information content is challenging because spatio-temporal data is high-dimensional, heterogeneous and has non-linear characteristics. In this paper, we apply multivariate Gaussianization for probability density estimation which is robust to dimensionality, comes with statistical guarantees, and is easy to apply. In addition, this methodology allows us to estimate information-theoretic measures to characterize multivariate densities: information, entropy, total correlation, and mutual in…
Multiscale partial information decomposition of dynamic processes with short and long-range correlations: theory and application to cardiovascular co…
2022
Abstract Objective. In this work, an analytical framework for the multiscale analysis of multivariate Gaussian processes is presented, whereby the computation of Partial Information Decomposition measures is achieved accounting for the simultaneous presence of short-term dynamics and long-range correlations. Approach. We consider physiological time series mapping the activity of the cardiac, vascular and respiratory systems in the field of Network Physiology. In this context, the multiscale representation of transfer entropy within the network of interactions among Systolic arterial pressure (S), respiration (R) and heart period (H), as well as the decomposition into unique, redundant and s…
Fractional quantum Hall effect in the interacting Hofstadter model via tensor networks
2017
We show via tensor network methods that the Harper-Hofstadter Hamiltonian for hard-core bosons on a square geometry supports a topological phase realizing the $\nu=1/2$ fractional quantum Hall effect on the lattice. We address the robustness of the ground state degeneracy and of the energy gap, measure the many-body Chern number, and characterize the system using Green functions, showing that they decay algebraically at the edges of open geometries, indicating the presence of gapless edge modes. Moreover, we estimate the topological entanglement entropy by taking a combination of lattice bipartitions that reproduces the topological structure of the original proposals by Kitaev and Preskill,…
Risk Assessment and Analysis
2014
Once threats are identified, they must be assessed or evaluated, which is the objective of this chapter. There is usually a large number of threats, making it impossible or unprofitable to analyze them all, meaning selection of the threats that will be addressed is important. It is a decision-making process; an example is proposed and solved by one of the many techniques available. The chapter proposes a very standard requested study, which is the assessment of the economic and financial risks of a project. This is done through a real-life-example, followed by another appraisal, this time devoted to economic issues, as well as another for transportation, introducing the important concept of…