Search results for "Markov"
showing 10 items of 628 documents
A simplified predictive control of constrained Markov jump system with mixed uncertainties
2014
Published version of an article in the journal: Abstract and Applied Analysis. Also available from the publisher at: http://dx.doi.org/10.1155/2014/475808 Open Access A simplified model predictive control algorithm is designed for discrete-time Markov jump systems with mixed uncertainties. The mixed uncertainties include model polytope uncertainty and partly unknown transition probability. The simplified algorithm involves finite steps. Firstly, in the previous steps, a simplified mode-dependent predictive controller is presented to drive the state to the neighbor area around the origin. Then the trajectory of states is driven as expected to the origin by the final-step mode-independent pre…
Least-squares temporal difference learning based on an extreme learning machine
2014
Abstract Reinforcement learning (RL) is a general class of algorithms for solving decision-making problems, which are usually modeled using the Markov decision process (MDP) framework. RL can find exact solutions only when the MDP state space is discrete and small enough. Due to the fact that many real-world problems are described by continuous variables, approximation is essential in practical applications of RL. This paper is focused on learning the value function of a fixed policy in continuous MPDs. This is an important subproblem of several RL algorithms. We propose a least-squares temporal difference (LSTD) algorithm based on the extreme learning machine. LSTD is typically combined wi…
Bayesian estimation of edge orientations in junctions
1999
Abstract Junctions, defined as those points of an image where two or more edges meet, play a significant role in many computer vision applications. Junction detection is a widely treated problem, and some detectors can provide even the directions of the edges that meet in a junction. The main objective of this paper is the precise estimation of such directions. It is supposed that the junction point has been previously found by some detector. Also, it is assumed that samples, possibly noisy, of orientations of the edges found in a circular window surrounding the point are available. A mixture of von Mises distributions is assumed for these data, and then a Bayesian methodology is applied to…
A new strategy for effective learning in population Monte Carlo sampling
2016
In this work, we focus on advancing the theory and practice of a class of Monte Carlo methods, population Monte Carlo (PMC) sampling, for dealing with inference problems with static parameters. We devise a new method for efficient adaptive learning from past samples and weights to construct improved proposal functions. It is based on assuming that, at each iteration, there is an intermediate target and that this target is gradually getting closer to the true one. Computer simulations show and confirm the improvement of the proposed strategy compared to the traditional PMC method on a simple considered scenario.
Performance modeling of epidemic routing
2006
In this paper, we develop a rigorous, unified framework based on ordinary differential equations (ODEs) to study epidemic routing and its variations. These ODEs can be derived as limits of Markovian models under a natural scaling as the number of nodes increases. While an analytical study of Markovian models is quite complex and numerical solution impractical for large networks, the corresponding ODE models yield closed-form expressions for several performance metrics of interest, and a numerical solution complexity that does not increase with the number of nodes. Using this ODE approach, we investigate how resources such as buffer space and the number of copies made for a packet can be tra…
Bayesian adaptive estimation: The next dimension
2006
Abstract We propose a new psychometric model for two-dimensional stimuli, such as color differences, based on parameterizing the threshold of a one-dimensional psychometric function as an ellipse. The Ψ Bayesian adaptive estimation method applied to this model yields trials that vary in multiple stimulus dimensions simultaneously. Simulations indicate that this new procedure can be much more efficient than the more conventional procedure of estimating the psychometric function on one-dimensional lines independently, requiring only one-fourth or less the number of trials for equivalent performance in typical situations. In a real psychophysical experiment with a yes–no task, as few as 22 tri…
Hydrological post-processing based on approximate Bayesian computation (ABC)
2019
[EN] This study introduces a method to quantify the conditional predictive uncertainty in hydrological post-processing contexts when it is cumbersome to calculate the likelihood (intractable likelihood). Sometimes, it can be difficult to calculate the likelihood itself in hydrological modelling, specially working with complex models or with ungauged catchments. Therefore, we propose the ABC post-processor that exchanges the requirement of calculating the likelihood function by the use of some sufficient summary statistics and synthetic datasets. The aim is to show that the conditional predictive distribution is qualitatively similar produced by the exact predictive (MCMC post-processor) or …
Cross-entropy-based adaptive optimization of simulation parameters for Markovian-driven service systems
2005
Abstract Markov fluid models represent a general description of the process of service request arrivals to service systems. The solution of performance analysis problems incorporating them often calls for a simulation approach, for which a reference methodology is Importance Sampling. However, in this case the appropriate choice of the biasing conditions is a problem in itself. In this paper an iterative method based on the cross-entropy is proposed for this choice. The equations are given that allow to derive the biasing conditions from the simulation itself. The application of the proposed method to three different sample cases, referring to one transient scenario (finite time horizon and…
Hydropower Optimization Using Deep Learning
2019
This paper demonstrates how deep learning can be used to find optimal reservoir operating policies in hydropower river systems. The method that we propose is based on the implicit stochastic optimization (ISO) framework, using direct policy search methods combined with deep neural networks (DNN). The findings from a real-world two-reservoir hydropower system in southern Norway suggest that DNNs can learn how to map input (price, inflow, starting reservoir levels) to the optimal production pattern directly. Due to the speed of evaluating the DNN, this approach is from an operational standpoint computationally inexpensive and may potentially address the long-standing problem of high dimension…
A Quick Simulation Technique for a Fluid Information Storage Problem
2001
Summary In this paper we present an application of Importance Sampling (IS) for quick simulation of buffer overflow probability in a statistical multiplexer loaded with a number of independent Markov modulated fluid sources. Runtime improvement is deducible from NMCσ2(p) and NISσ2(p*) that characterize the trade-offs between sample size and variance of the estimators of buffer overflow probability experienced in Monte Carlo (MC) and Importance Sampling simulations. By assuming that the same precision is achieved for the two kinds of simulations if σ2(p)=σ2(p*), an approximate closed form expression for the ratio NIS/NMC is derived, and it is minimized with respect to the load of the multipl…