Deep CNN-ELM Hybrid Models for Fire Detection in Images
In this paper, we propose a hybrid model consisting of a Deep Convolutional feature extractor followed by a fast and accurate classifier, the Extreme Learning Machine, for the purpose of fire detection in images. The reason behind using such a model is that Deep CNNs used for image classification take a very long time to train. Even with pre-trained models, the fully connected layers need to be trained with backpropagation, which can be very slow. In contrast, we propose to employ the Extreme Learning Machine (ELM) as the final classifier trained on pre-trained Deep CNN feature extractor. We apply this hybrid model on the problem of fire detection in images. We use state of the art Deep CNN…
The Bayesian Pursuit Algorithm: A New Family of Estimator Learning Automata
Published version of a chapter in the book: Modern Approaches in Applied Intelligence. Also available from the publisher at http://dx.doi.org/10.1007/978-3-642-21827-9_53 The fastest Learning Automata (LA) algorithms currently available come from the family of estimator algorithms. The Pursuit algorithm (PST), a pioneering scheme in the estimator family, obtains its superior learning speed by using Maximum Likelihood (ML) estimates to pursue the action currently perceived as being optimal. Recently, a Bayesian LA (BLA) was introduced, and empirical results that demonstrated its advantages over established top performers, including the PST scheme, were reported. The BLA scheme is inherently …
Optimizing channel selection for cognitive radio networks using a distributed Bayesian learning automata-based approach
Consider a multi-channel Cognitive Radio Network (CRN) with multiple Primary Users (PUs), and multiple Secondary Users (SUs) competing for access to the channels. In this scenario, it is essential for SUs to avoid collision among one another while maintaining efficient usage of the available transmission opportunities. We investigate two channel access schemes. In the first model, an SU selects a channel and sends a packet directly without Carrier Sensing (CS) whenever the PU is absent on this channel. In the second model, an SU invokes CS in order to avoid collision among co-channel SUs. For each model, we analyze the channel selection problem and prove that it is a so-called "Exact Potent…
Solving Graph Coloring Problems Using Learning Automata
The graph coloring problem (GCP) is a widely studied combinatorial optimization problem with numerous applications, including time tabling, frequency assignment, and register allocation. The growing need for more efficient algorithms has led to the development of several GCP solvers. In this paper, we introduce the first GCP solver that is based on Learning Automata (LA). We enhance traditional Random Walk with LA-based learning capability, encoding the GCP as a Boolean satisfiability problem (SAT). Extensive experiments demonstrate that the LA significantly improve the performance of RW, thus laying the foundation for novel LA-based solutions to the GCP.
A Stochastic Search on the Line-Based Solution to Discretized Estimation
Published version of a chapter in the book: Advanced Research in Applied Artificial Intelligence. Also available from the publisher at: http://dx.doi.org/10.1007/978-3-642-31087-4_77 Recently, Oommen and Rueda [11] presented a strategy by which the parameters of a binomial/multinomial distribution can be estimated when the underlying distribution is nonstationary. The method has been referred to as the Stochastic Learning Weak Estimator (SLWE), and is based on the principles of continuous stochastic Learning Automata (LA). In this paper, we consider a new family of stochastic discretized weak estimators pertinent to tracking time-varying binomial distributions. As opposed to the SLWE, our p…
Crowd Models for Emergency Evacuation: A Review Targeting Human-Centered Sensing
Emergency evacuation of crowds is a fascinating phenomenon that has attracted researchers from various fields. Better understanding of this class of crowd behavior opens up for improving evacuation policies and smarter design of buildings, increasing safety. Recently, a new class of disruptive technology has appeared: Human-centered sensing which allows crowd behavior to be monitored in real-time, and provides the basis for real-time crowd control. The question then becomes: to what degree can previous crowd models incorporate this development, and what areas need further research? In this paper, we provide a survey that describes some widely used crowd models and discuss their advantages a…
A methodology for fire data analysis based on pattern recognition towards the disaster management
The aim of this paper is to investigate a proposed strategy for fire disaster analysis that is implemented based on pattern recognition technique in order to achieve a methodology for disaster management. Since the fire hazard has severe effects onto human and properties, it is essential to predict and possibly prevent it. Almost every fire produces some issues, such as heat, smoke, gas, and flame, which are sensible and measurable via devices or detection systems. The fire behavior is relevant to these issues. In this research, temperature, heat radiation, and visibility (smoke) data of fire that have been obtained from Fire Dynamics Simulator (FDS) are used for analysis. The location of t…
An intelligent architecture for service provisioning in pervasive environments
Accepted version of an article from the conference: 2011 International Symposium on Innovations in Intelligent Systems and Applications (INISTA). Definitive published version available from IEEE: http://dx.doi.org/10.1109/INISTA.2011.5946134 The vision of pervasive environments is being realized more than ever with the proliferation of services and computing resources located in our surrounding environments. Identifying those services that deserve the attention of the user is becoming an increasingly-challenging task. In this paper, we present an adaptive multi-criteria decision making mechanism for recommending relevant services to the mobile user. In this context Relevance is determined b…
A Bayesian Learning Automata-Based Distributed Channel Selection Scheme for Cognitive Radio Networks
We consider a scenario where multiple Secondary Users SUs operate within a Cognitive Radio Network CRN which involves a set of channels, where each channel is associated with a Primary User PU. We investigate two channel access strategies for SU transmissions. In the first strategy, the SUs will send a packet directly without operating Carrier Sensing Medium Access/Collision Avoidance CSMA/CA whenever a PU is absent in the selected channel. In the second strategy, the SUs implement CSMA/CA to further reduce the probability of collisions among co-channel SUs. For each strategy, the channel selection problem is formulated and demonstrated to be a so-called "Potential" game, and a Bayesian Lea…
Towards a Deep Reinforcement Learning Approach for Tower Line Wars
There have been numerous breakthroughs with reinforcement learning in the recent years, perhaps most notably on Deep Reinforcement Learning successfully playing and winning relatively advanced computer games. There is undoubtedly an anticipation that Deep Reinforcement Learning will play a major role when the first AI masters the complicated game plays needed to beat a professional Real-Time Strategy game player. For this to be possible, there needs to be a game environment that targets and fosters AI research, and specifically Deep Reinforcement Learning. Some game environments already exist, however, these are either overly simplistic such as Atari 2600 or complex such as Starcraft II fro…
Learning Automata-Based Solutions to Stochastic Nonlinear Resource Allocation Problems
“Computational Intelligence” is an extremely wide-ranging and all-encompassing area. However, it is fair to say that the strength of a system that possesses “Computational Intelligence” can be quantified by its ability to solve problems that are intrinsically hard. One such class of NP-Hard problems concerns the so-called family of Knapsack Problems, and in this Chapter, we shall explain how a sub-field of Artificial Intelligence, namely that which involves “Learning Automata”, can be used to produce fast and accurate solutions to “difficult” and randomized versions of the Knapsack problem (KP).
A Novel Tsetlin Automata Scheme to Forecast Dengue Outbreaks in the Philippines
Being capable of online learning in unknown stochastic environments, Tsetlin Automata (TA) have gained considerable interest. As a model of biological systems, teams of TA have been used for solving complex problems in a decentralized manner, with low computational complexity. For many domains, decentralized problem solving is an advantage, however, also may lead to coordination difficulties and unstable learning. To combat this negative effect, this paper proposes a novel TA coordination scheme designed for learning problems with continuous input and output. By saving and updating the best solution that has been chosen so far, we can avoid having the overall system being led astray by spur…
The Dreaming Variational Autoencoder for Reinforcement Learning Environments
Reinforcement learning has shown great potential in generalizing over raw sensory data using only a single neural network for value optimization. There are several challenges in the current state-of-the-art reinforcement learning algorithms that prevent them from converging towards the global optima. It is likely that the solution to these problems lies in short- and long-term planning, exploration and memory management for reinforcement learning algorithms. Games are often used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible, and easy to control environment. Regardless, few games feature a state-space where results in exploration, memory, and plannin…
A Novel Bayesian Network Based Scheme for Finding the Optimal Solution to Stochastic Online Equi-partitioning Problems
A number of intriguing decision scenarios, such as order picking, revolve around partitioning a collection of objects so as to optimize some application specific objective function. In its general form, this problem is referred to as the Object Partitioning Problem (OOP), known to be NP-hard. We here consider a variant of OPP, namely the Stochastic Online Equi-Partitioning Problem (SO-EPP). In SO-EPP, objects arrive sequentially, in pairs. The relationship between the arriving object pairs is stochastic: They belong to the same partition with probability p. From a history of object arrivals, the goal is to predict which objects will appear together in future arrivals. As an additional compl…
Discretized Bayesian Pursuit – A New Scheme for Reinforcement Learning
Published version of a chapter in the book: Advanced Research in Applied Artificial Intelligence. Also available from the publisher at: http://dx.doi.org/10.1007/978-3-642-31087-4_79 The success of Learning Automata (LA)-based estimator algorithms over the classical, Linear Reward-Inaction ( L RI )-like schemes, can be explained by their ability to pursue the actions with the highest reward probability estimates. Without access to reward probability estimates, it makes sense for schemes like the L RI to first make large exploring steps, and then to gradually turn exploration into exploitation by making progressively smaller learning steps. However, this behavior becomes counter-intuitive wh…
A formal proof of the ε-optimality of absorbing continuous pursuit algorithms using the theory of regular functions
Published version of an article from the journal: Applied Intelligence. Also available on Springerlink: http://dx.doi.org/10.1007/s10489-014-0541-1 The most difficult part in the design and analysis of Learning Automata (LA) consists of the formal proofs of their convergence accuracies. The mathematical techniques used for the different families (Fixed Structure, Variable Structure, Discretized etc.) are quite distinct. Among the families of LA, Estimator Algorithms (EAs) are certainly the fastest, and within this family, the set of Pursuit algorithms have been considered to be the pioneering schemes. Informally, if the environment is stationary, their ε-optimality is defined as their abili…
Learning Automaton Based On-Line Discovery and Tracking of Spatio-temporal Event Patterns
Published version of an article from the book: Lecture Notes in Computer Science, 2010, Volume 6230/2010, 327-338. The original publication is available at Springerlink. http://dx.doi.org/10.1007/978-3-642-15246-7_31 Discovering and tracking of spatio-temporal patterns in noisy sequences of events is a difficult task that has become increasingly pertinent due to recent advances in ubiquitous computing, such as community-based social networking applications. The core activities for applications of this class include the sharing and notification of events, and the importance and usefulness of these functionalites increases as event-sharing expands into larger areas of one’s life. Ironically, …
A novel Stochastic Discretized Weak Estimator operating in non-stationary environments
The task of designing estimators that are able to track time-varying distributions has found promising applications in many real-life problems. A particularly interesting family of distributions are the binomial/multiomial distributions. Existing approaches resort to sliding windows that track changes by discarding old observations. In this paper, we report a novel estimator referred to as the Stochastic Discretized Weak Estimator (SDWE), that is based on the principles of Learning Automata (LA). In brief, the estimator is able to estimate the parameters of a time varying binomial distribution using finite memory. The estimator tracks changes in the distribution by operating on a controlled…
Learning automata-based solutions to the optimal web polling problem modelled as a nonlinear fractional knapsack problem
We consider the problem of polling web pages as a strategy for monitoring the world wide web. The problem consists of repeatedly polling a selection of web pages so that changes that occur over time are detected. In particular, we consider the case where we are constrained to poll a maximum number of web pages per unit of time, and this constraint is typically dictated by the governing communication bandwidth, and by the speed limitations associated with the processing. Since only a fraction of the web pages can be polled within a given unit of time, the issue at stake is one of determining which web pages are to be polled, and we attempt to do it in a manner that maximizes the number of ch…
A Spatio-temporal Probabilistic Model of Hazard and Crowd Dynamics in Disasters for Evacuation Planning
Published version of a chapter in the book: Recent Trends in Applied Artificial Intelligence. Also available from the publisher at: http://dx.doi.org/10.1007/978-3-642-38577-3_7 Managing the uncertainties that arise in disasters – such as ship fire – can be extremely challenging. Previous work has typically focused either on modeling crowd behavior or hazard dynamics, targeting fully known environments. However, when a disaster strikes, uncertainty about the nature, extent and further development of the hazard is the rule rather than the exception. Additionally, crowd and hazard dynamics are both intertwined and uncertain, making evacuation planning extremely difficult. To address this chal…
On the analysis of a new Markov chain which has applications in AI and machine learning
Accepted version of an article from the conference: 2011 24th Canadian Conference on Electrical and Computer Engineering. Published version available from IEEE: http://dx.doi.org/10.1109/CCECE.2011.6030727 In this paper, we consider the analysis of a fascinating Random Walk (RW) that contains interleaving random steps and random "jumps". The characterizing aspect of such a chain is that every step is paired with its counterpart random jump. RWs of this sort have applications in testing of entities, where the entity is never allowed to make more than a pre-specified number of consecutive failures. This paper contains the analysis of the chain, some fascinating limiting properties, and some i…
Tracking the Preferences of Users Using Weak Estimators
Published version of am article from the book:AI 2011: Advances in Artificial Intelligence. Also available from the publisher on SpringerLink:http://dx.doi.org/10.1007/978-3-642-25832-9_81 Since a social network, by definition, is so diverse, the problem of estimating the preferences of its users is becoming increasingly essential for personalized applications which range from service recommender systems to the targeted advertising of services. However, unlike traditional estimation problems where the underlying target distribution is stationary, estimating a user’s interests, typically, involves non-stationary distributions. The consequent time varying nature of the distribution to be trac…
An adaptive approach to learning the preferences of users in a social network using weak estimators
Published version of an article in the journal: Journal of Information Processing Systems. Also available from the publisher at: http://dx.doi.org/10.3745/JIPS.2012.8.2.191 - Open Access Since a social network by definition is so diverse, the problem of estimating the preferences of its users is becoming increasingly essential for personalized applications, which range from service recommender systems to the targeted advertising of services. However, unlike traditional estimation problems where the underlying target distribution is stationary; estimating a user's interests typically involves non-stationary distributions. The consequent time varying nature of the distribution to be tracked i…
A two-armed bandit collective for examplar based mining of frequent itemsets with applications to intrusion detection
Chapter from the book: Computational Collective Intelligence. Technologies and Applications. Also available from the publisher at SpringerLink: http://dx.doi.org/10.1007/978-3-642-23935-9_7 Over the last decades, frequent itemset mining has become a major area of research, with applications including indexing and similarity search, as well as mining of data streams, web, and software bugs. Although several efficient techniques for generating frequent itemsets with a minimum support (frequency) have been proposed, the number of itemsets produced is in many cases too large for effective usage in real-life applications. Indeed, the problem of deriving frequent itemsets that are both compact an…
Modeling Snow Dynamics Using a Bayesian Network
In this paper we propose a novel snow accumulation and melt model, formulated as a Dynamic Bayesian Network DBN. We encode uncertainty explicitly and train the DBN using Monte Carlo analysis, carried out with a deterministic hydrology model under a wide range of plausible parameter configurations. The trained DBN was tested against field observations of snow water equivalents SWE. The results indicate that our DBN can be used to reason about uncertainty, without doing resampling from the deterministic model. In all brevity, the DBN's ability to reproduce the mean of the observations was similar to what could be obtained with the deterministic hydrology model, but with a more realistic repre…
The design of absorbing Bayesian pursuit algorithms and the formal analyses of their ε-optimality
The fundamental phenomenon that has been used to enhance the convergence speed of learning automata (LA) is that of incorporating the running maximum likelihood (ML) estimates of the action reward probabilities into the probability updating rules for selecting the actions. The frontiers of this field have been recently expanded by replacing the ML estimates with their corresponding Bayesian counterparts that incorporate the properties of the conjugate priors. These constitute the Bayesian pursuit algorithm (BPA), and the discretized Bayesian pursuit algorithm. Although these algorithms have been designed and efficiently implemented, and are, arguably, the fastest and most accurate LA report…
Novel Distance Estimation Methods Using 'Stochastic Learning on the Line' Strategies
In this paper, we consider the problem of Distance Estimation (DE) when the inputs are the $x$ and $y$ coordinates (or equivalently, the latitudinal and longitudinal positions) of the points under consideration. The aim of the problem is to yield an accurate value for the real (road) distance between the points specified by the latter coordinates. 1 This problem has, typically, been tackled by utilizing parametric functions called the “Distance Estimation Functions” (DEFs). The parameters are learned from the training data (i.e., the true road distances) between a subset of the points under consideration. We propose to use Learning Automata (LA)-based strategies to solve the problem. In par…
A Scheme for Continuous Input to the Tsetlin Machine with Applications to Forecasting Disease Outbreaks
In this paper, we apply a new promising tool for pattern classification, namely, the Tsetlin Machine (TM), to the field of disease forecasting. The TM is interpretable because it is based on manipulating expressions in propositional logic, leveraging a large team of Tsetlin Automata (TA). Apart from being interpretable, this approach is attractive due to its low computational cost and its capacity to handle noise. To attack the problem of forecasting, we introduce a preprocessing method that extends the TM so that it can handle continuous input. Briefly stated, we convert continuous input into a binary representation based on thresholding. The resulting extended TM is evaluated and analyzed…
Thompson Sampling for Dynamic Multi-armed Bandits
The importance of multi-armed bandit (MAB) problems is on the rise due to their recent application in a large variety of areas such as online advertising, news article selection, wireless networks, and medicinal trials, to name a few. The most common assumption made when solving such MAB problems is that the unknown reward probability theta k of each bandit arm k is fixed. However, this assumption rarely holds in practice simply because real-life problems often involve underlying processes that are dynamically evolving. In this paper, we model problems where reward probabilities theta k are drifting, and introduce a new method called Dynamic Thompson Sampling (DTS) that facilitates Order St…
Thompson Sampling Guided Stochastic Searching on the Line for Adversarial Learning
The multi-armed bandit problem has been studied for decades. In brief, a gambler repeatedly pulls one out of N slot machine arms, randomly receiving a reward or a penalty from each pull. The aim of the gambler is to maximize the expected number of rewards received, when the probabilities of receiving rewards are unknown. Thus, the gambler must, as quickly as possible, identify the arm with the largest probability of producing rewards, compactly capturing the exploration-exploitation dilemma in reinforcement learning. In this paper we introduce a particular challenging variant of the multi-armed bandit problem, inspired by the so-called N-Door Puzzle. In this variant, the gambler is only tol…
A New Tool for the Modeling of AI and Machine Learning Applications: Random Walk-Jump Processes
Published version of an article from the book: Hybrid artificial intelligent systems, Lecture notes in computer science. The original publication is available at www.springerlink.com, http://dx.doi.org/10.1007/978-3-642-21219-2_2 There are numerous applications in Artificial Intelligence (AI) and Machine Learning (ML) where the criteria for decisions are based on testing procedures. The most common tools used in such random phenomena involve Random Walks (RWs). The theory of RWs and its applications have gained an increasing research interest since the start of the last century. [1]. In this context, we note that a RW is, usually, defined as a trajectory involving a series of successive ran…
On Using “Stochastic Learning on the Line” to Design Novel Distance Estimation Methods
In this paper, we consider the problem of Distance Estimation (DE) when the inputs are the x and y coordinates of the points under consideration. The aim of the problem is to yield an accurate value for the real (road) distance between the points specified by the latter coordinates. This problem has, typically, been tackled by utilizing parametric functions called Distance Estimation Functions (DEFs). The parameters are learned from the training data (i.e., the true road distances) between a subset of the points under consideration. We propose to use Learning Automata (LA)-based strategies to solve the problem. In particular, we resort to the Adaptive Tertiary Search (ATS) strategy, propose…
THE USE OF WEAK ESTIMATORS TO ACHIEVE LANGUAGE DETECTION AND TRACKING IN MULTILINGUAL DOCUMENTS
This paper deals with the problems of language detection and tracking in multilingual online short word-of-mouth (WoM) discussions. This problem is particularly unusual and difficult from a pattern recognition perspective because, in these discussions, the participants and content involve the opinions of users from all over the world. The nature of these discussions, consisting of multiple topics in different languages, presents us with a problem of finding training and classification strategies when the class-conditional distributions are nonstationary. The difficulties in solving the problem are many-fold. First of all, the analyst has no knowledge of when one language stops and when the…
Achieving Unbounded Resolution inFinitePlayer Goore Games Using Stochastic Automata, and Its Applications
Abstract This article concerns the sequential solution to a distributed stochastic optimization problem using learning automata and the Goore game (also referred to as the Gur game in the related literature). The amazing thing about our solution is that, unlike traditional methods, which need N automata (where N determines the degree of accuracy), in this article, we show that we can obtain arbitrary accuracy by recursively using only three automata. To be more specific, the Goore game (GG) introduced in Tsetlin (1973) has the fascinating property that it can be resolved in a completely distributed manner with no inter-communication between the players. The game has recently found applicati…
Using the Theory of Regular Functions to Formally Prove the ε-Optimality of Discretized Pursuit Learning Algorithms
Learning Automata LA can be reckoned to be the founding algorithms on which the field of Reinforcement Learning has been built. Among the families of LA, Estimator Algorithms EAs are certainly the fastest, and of these, the family of Pursuit Algorithms PAs are the pioneering work. It has recently been reported that the previous proofs for e-optimality for all the reported algorithms in the family of PAs have been flawed. We applaud the researchers who discovered this flaw, and who further proceeded to rectify the proof for the Continuous Pursuit Algorithm CPA. The latter proof, though requires the learning parameter to be continuously changing, is, to the best of our knowledge, the current …
A two-armed bandit based scheme for accelerated decentralized learning
Published version of a chapter from the book: Modern Approaches in Applied Intelligence. Also available at SpringerLink: http://dx.doi.org/10.1007/978-3-642-21827-9_54 The two-armed bandit problem is a classical optimization problem where a decision maker sequentially pulls one of two arms attached to a gambling machine, with each pull resulting in a random reward. The reward distributions are unknown, and thus, one must balance between exploiting existing knowledge about the arms, and obtaining new information. Bandit problems are particularly fascinating because a large class of real world problems, including routing, QoS control, game playing, and resource allocation, can be solved in a …
A Hierarchy of Twofold Resource Allocation Automata Supporting Optimal Sampling
We consider the problem of allocating limited sampling resources in a "real-time" manner with the purpose of estimating multiple binomial proportions. More specifically, the user is presented with `n ' sets of data points, S 1 , S 2 , ..., S n , where the set S i has N i points drawn from two classes {*** 1 , *** 2 }. A random sample in set S i belongs to *** 1 with probability u i and to *** 2 with probability 1 *** u i , with {u i }. i = 1, 2, ...n , being the quantities to be learnt. The problem is both interesting and non-trivial because while both n and each N i are large, the number of samples that can be drawn is bounded by a constant, c . We solve the problem by first modelling it a…
On the Analysis of a Random Interleaving Walk–Jump Process with Applications to Testing
Abstract Although random walks (RWs) with single-step transitions have been extensively studied for almost a century as seen in Feller (1968), problems involving the analysis of RWs that contain interleaving random steps and random “jumps” are intrinsically hard. In this article, we consider the analysis of one such fascinating RW, where every step is paired with its counterpart random jump. In addition to this RW being conceptually interesting, it has applications in testing of entities (components or personnel), where the entity is never allowed to make more than a prespecified number of consecutive failures. The article contains the analysis of the chain, some fascinating limiting proper…
Stochastic discretized learning-based weak estimation: a novel estimation method for non-stationary environments
The task of designing estimators that are able to track time-varying distributions has found promising applications in many real-life problems.Existing approaches resort to sliding windows that track changes by discarding old observations. In this paper, we report a novel estimator referred to as the Stochastic Discretized Weak Estimator (SDWE), that is based on the principles of discretized Learning Automata (LA). In brief, the estimator is able to estimate the parameters of a time varying binomial distribution using finite memory. The estimator tracks changes in the distribution by operating a controlled random walk in a discretized probability space. The steps of the estimator are discre…
Arm Space Decomposition as a Strategy for Tackling Large Scale Multi-armed Bandit Problems
Recent multi-armed bandit based optimization schemes provide near-optimal balancing of arm exploration against arm exploitation, allowing the optimal arm to be identified with probability arbitrarily close to unity. However, the convergence speed drops dramatically as the number of bandit arms grows large, simply because singling out the optimal arm requires experimentation with all of the available arms. Furthermore, effective exploration and exploitation typically demands computational resources that grow linearly with the number of arms. Although the former problem can be remedied to some degree when prior knowledge about arm correlation is available, the latter problem persists. In this…
Vector representation of non-standard spellings using dynamic time warping and a denoising autoencoder
The presence of non-standard spellings in Twitter causes challenges for many natural language processing tasks. Traditional approaches mainly regard the problem as a translation, spell checking, or speech recognition problem. This paper proposes a method that represents the stochastic relationship between words and their non-standard versions in real vectors. The method uses dynamic time warping to preprocess the non-standard spellings and autoencoder to derive the vector representation. The derived vectors encode word patterns and the Euclidean distance between the vectors represents a distance in the word space that challenges the prevailing edit distance. After training the autoencoder o…
Solving Non-Stationary Bandit Problems by Random Sampling from Sibling Kalman Filters
Published version of an article from Lecture Notes in Computer Science. Also available at SpringerLink: http://dx.doi.org/10.1007/978-3-642-13033-5_21 The multi-armed bandit problem is a classical optimization problem where an agent sequentially pulls one of multiple arms attached to a gambling machine, with each pull resulting in a random reward. The reward distributions are unknown, and thus, one must balance between exploiting existing knowledge about the arms, and obtaining new information. Dynamically changing (non-stationary) bandit problems are particularly challenging because each change of the reward distributions may progressively degrade the performance of any fixed strategy. Alt…
A Bayesian Network Model for Fire Assessment and Prediction
Smartphones and other wearable computers with modern sensor technologies are becoming more advanced and widespread. This paper proposes exploiting those devices to help the firefighting operation. It introduces a Bayesian network model that infers the state of the fire and predicts its future development based on smartphone sensor data gathered within the fire area. The model provides a prediction accuracy of 84.79i¾?% and an area under the curve of 0.83. This solution had also been tested in the context of a fire drill and proved to help firefighters assess the fire situation and speed up their work.
A Bayesian Learning Automaton for Solving Two-Armed Bernoulli Bandit Problems
The two-armed Bernoulli bandit (TABB) problem is a classical optimization problem where an agent sequentially pulls one of two arms attached to a gambling machine, with each pull resulting either in a reward or a penalty. The reward probabilities of each arm are unknown, and thus one must balance between exploiting existing knowledge about the arms, and obtaining new information. In the last decades, several computationally efficient algorithms for tackling this problem have emerged, with learning automata (LA) being known for their ?-optimality, and confidence interval based for logarithmically growing regret. Applications include treatment selection in clinical trials, route selection in …
Combining finite learning automata with GSAT for the satisfiability problem
A large number of problems that occur in knowledge-representation, learning, very large scale integration technology (VLSI-design), and other areas of artificial intelligence, are essentially satisfiability problems. The satisfiability problem refers to the task of finding a satisfying assignment that makes a Boolean expression evaluate to True. The growing need for more efficient and scalable algorithms has led to the development of a large number of SAT solvers. This paper reports the first approach that combines finite learning automata with the greedy satisfiability algorithm (GSAT). In brief, we introduce a new algorithm that integrates finite learning automata and traditional GSAT use…
Information Abstraction from Crises Related Tweets Using Recurrent Neural Network
Social media has become an important open communication medium during crises. The information shared about a crisis in social media is massive, complex, informal and heterogeneous, which makes extracting useful information a difficult task. This paper presents a first step towards an approach for information extraction from large Twitter data. In brief, we propose a Recurrent Neural Network based model for text generation able to produce a unique text capturing the general consensus of a large collection of twitter messages. The generated text is able to capture information about different crises from tens of thousand of tweets summarized only in a 2000 characters text.
Deep RTS: A Game Environment for Deep Reinforcement Learning in Real-Time Strategy Games
Reinforcement learning (RL) is an area of research that has blossomed tremendously in recent years and has shown remarkable potential for artificial intelligence based opponents in computer games. This success is primarily due to the vast capabilities of convolutional neural networks, that can extract useful features from noisy and complex data. Games are excellent tools to test and push the boundaries of novel RL algorithms because they give valuable insight into how well an algorithm can perform in isolated environments without the real-life consequences. Real-time strategy games (RTS) is a genre that has tremendous complexity and challenges the player in short and long-term planning. The…
On Utilizing Stochastic Non-linear Fractional Bin Packing to Resolve Distributed Web Crawling
This paper deals with the extremely pertinent problem of web crawling, which is far from trivial considering the magnitude and all-pervasive nature of the World-Wide Web. While numerous AI tools can be used to deal with this task, in this paper we map the problem onto the combinatorially-hard stochastic non-linear fractional knapsack problem, which, in turn, is then solved using Learning Automata (LA). Such LA-based solutions have been recently shown to outperform previous state-of-the-art approaches to resource allocation in Web monitoring. However, the ever growing deployment of distributed systems raises the need for solutions that cope with a distributed setting. In this paper, we prese…
A Learning Automata Based Solution to Service Selection in Stochastic Environments
Published version of a paper published in the book: Trends in Applied Intelligent Systems. Also available on SpringerLink: http://dx.doi.org/10.1007/978-3-642-13033-5_22 With the abundance of services available in today’s world, identifying those of high quality is becoming increasingly difficult. Reputation systems can offer generic recommendations by aggregating user provided opinions about service quality, however, are prone to ballot stuffing and badmouthing . In general, unfair ratings may degrade the trustworthiness of reputation systems, and changes in service quality over time render previous ratings unreliable. In this paper, we provide a novel solution to the above problems based …
A Bayesian network model for evacuation time analysis during a ship fire
We present an evacuation model for ships while a fire happens onboard. The model is designed by utilizing Bayesian networks (BN) and then simulated in GeNIe software. In our proposed model, the most important factors that have significant influence on a rescue process and evacuation time are identified and analyzed. By applying the probability distribution of the considered factors collected from the literature including IMO, real empirical data and practical experiences, the trend of the rescue process and evacuation time can be evaluated and predicted using the proposed model. The results of this paper help understanding about possible consequences of influential factors on the security o…
Generalized Bayesian Pursuit: A Novel Scheme for Multi-Armed Bernoulli Bandit Problems
In the last decades, a myriad of approaches to the multi-armed bandit problem have appeared in several different fields. The current top performing algorithms from the field of Learning Automata reside in the Pursuit family, while UCB-Tuned and the e-greedy class of algorithms can be seen as state-of-the-art regret minimizing algorithms. Recently, however, the Bayesian Learning Automaton (BLA) outperformed all of these, and other schemes, in a wide range of experiments. Although seemingly incompatible, in this paper we integrate the foundational learning principles motivating the design of the BLA, with the principles of the so-called Generalized Pursuit algorithm (GPST), leading to the Gen…
Hydropower Optimization Using Deep Learning
This paper demonstrates how deep learning can be used to find optimal reservoir operating policies in hydropower river systems. The method that we propose is based on the implicit stochastic optimization (ISO) framework, using direct policy search methods combined with deep neural networks (DNN). The findings from a real-world two-reservoir hydropower system in southern Norway suggest that DNNs can learn how to map input (price, inflow, starting reservoir levels) to the optimal production pattern directly. Due to the speed of evaluating the DNN, this approach is from an operational standpoint computationally inexpensive and may potentially address the long-standing problem of high dimension…
A formal proof of the e-optimality of discretized pursuit algorithms
Learning Automata (LA) can be reckoned to be the founding algorithms on which the field of Reinforcement Learning has been built. Among the families of LA, Estimator Algorithms (EAs) are certainly the fastest, and of these, the family of discretized algorithms are proven to converge even faster than their continuous counterparts. However, it has recently been reported that the previous proofs for ??-optimality for all the reported algorithms for the past three decades have been flawed. We applaud the researchers who discovered this flaw, and who further proceeded to rectify the proof for the Continuous Pursuit Algorithm (CPA). The latter proof examines the monotonicity property of the proba…
Ant colony optimisation for planning safe escape routes
Published version of a chapter from the volume: Recent Trends in Applied Artificial Intelligence. Also available on SpringerLink: http://dx.doi.org/10.1007/978-3-642-38577-3_6 An emergency requiring evacuation is a chaotic event filled with uncertainties both for the people affected and rescuers. The evacuees are often left to themselves for navigation to the escape area. The chaotic situation increases when a predefined escape route is blocked by a hazard, and there is a need to re-think which escape route is safest. This paper addresses automatically finding the safest escape route in emergency situations in large buildings or ships with imperfect knowledge of the hazards. The proposed so…
Solving two‐armed Bernoulli bandit problems using a Bayesian learning automaton
PurposeThe two‐armed Bernoulli bandit (TABB) problem is a classical optimization problem where an agent sequentially pulls one of two arms attached to a gambling machine, with each pull resulting either in a reward or a penalty. The reward probabilities of each arm are unknown, and thus one must balance between exploiting existing knowledge about the arms, and obtaining new information. The purpose of this paper is to report research into a completely new family of solution schemes for the TABB problem: the Bayesian learning automaton (BLA) family.Design/methodology/approachAlthough computationally intractable in many cases, Bayesian methods provide a standard for optimal decision making. B…
Towards Open Domain Chatbots—A GRU Architecture for Data Driven Conversations
Understanding of textual content, such as topic and intent recognition, is a critical part of chatbots, allowing the chatbot to provide relevant responses. Although successful in several narrow domains, the potential diversity of content in broader and more open domains renders traditional pattern recognition techniques inaccurate. In this paper, we propose a novel deep learning architecture for content recognition that consists of multiple levels of gated recurrent units (GRUs). The architecture is designed to capture complex sentence structure at multiple levels of abstraction, seeking content recognition for very wide domains, through a distributed scalable representation of content. To …
On Using “Stochastic Learning on the Line” to Design Novel Distance Estimation Methods for Three-Dimensional Environments
We consider the unsolved problem of Distance Estimation (DE) when the inputs are the x and y coordinates (i.e., the latitudinal and longitudinal positions) of the points under consideration, and the elevation/altitudes of the points specified, for example, in terms of their z coordinates (3DDE). The aim of the problem is to yield an accurate value for the real (road) distance between the points specified by all the three coordinates of the cities in question (This is a typical problem encountered in a GISs and GPSs.). In our setting, the distance between any pair of cities is assumed to be computed by merely having access to the coordinates and known inter-city distances of a small subset o…
A Learning Automata Local Contribution Sampling Applied to Hydropower Production Optimisation
Learning Automata (LA) is a powerful approach for solving complex, non-linear and stochastic optimisation problems. However, existing solutions struggle with high-dimensional problems due to slow convergence, arguably caused by the global nature of feedback. In this paper we introduce a novel Learning Automata (LA) scheme to attack this challenge. The scheme is based on a parallel form of Local Contribution Sampling (LCS), which means that the LA receive individually directed feedback, designed to speed up convergence. Furthermore, our scheme is highly decentralized, allowing parallel execution on GPU architectures. To demonstrate the power of our scheme, the LA LCS is applied to hydropower…
Deep Convolutional Neural Networks for Fire Detection in Images
Detecting fire in images using image processing and computer vision techniques has gained a lot of attention from researchers during the past few years. Indeed, with sufficient accuracy, such systems may outperform traditional fire detection equipment. One of the most promising techniques used in this area is Convolutional Neural Networks (CNNs). However, the previous research on fire detection with CNNs has only been evaluated on balanced datasets, which may give misleading information on real-world performance, where fire is a rare event. Actually, as demonstrated in this paper, it turns out that a traditional CNN performs relatively poorly when evaluated on the more realistically balance…
Channel selection in Cognitive Radio Networks: A Switchable Bayesian Learning Automata approach
We consider the problem of a user operating within a Cognitive Radio Network (CRN) which involves N channels each associated with a Primary User (PU). The problem consists of allocating a channel which, at any given time instant is not being used by a PU, to a Secondary User (SU). Within our study, we assume that a SU is allowed to perform “channel switching”, i.e., to choose an alternate channel S times (where S +1 ≤ N) if the previous choice does not lead to a channel which is vacant. The paper first presents a formal probabilistic model for the problem itself, referred to as the Formal Secondary Channel Selection (FSCS) problem, and the characteristics of the FSCS are then analyzed. Ther…
Towards an integrated approach to emergency management: interdisciplinary challenges for research and practice
This article presents an interdisciplinary vision for large-scale integrated emergency management that has been inspired by the transition from platform centric to integrated operations in the oil and gas fields, which uses remote emergency control centres collaborating virtually with local responders. The article discusses some of the most salient research challenges for integrated emergency management, including the role of mobile technology, human-centred sensing, citizen participation and social media, and the socio-cultural determinants of disaster management. The purpose of this article is to frame an integrated emergency management approach that adopts a multi-disciplinary approach, …
Thompson Sampling Based Active Learning in Probabilistic Programs with Application to Travel Time Estimation
The pertinent problem of Traveling Time Estimation (TTE) is to estimate the travel time, given a start location and a destination, solely based on the coordinates of the points under consideration. This is typically solved by fitting a function based on a sequence of observations. However, it can be expensive or slow to obtain labeled data or measurements to calibrate the estimation function. Active Learning tries to alleviate this problem by actively selecting samples that minimize the total number of samples needed to do accurate inference. Probabilistic Programming Languages (PPL) give us the opportunities to apply powerful Bayesian inference to model problems that involve uncertainties.…
A novel strategy for solving the stochastic point location problem using a hierarchical searching scheme
Stochastic point location (SPL) deals with the problem of a learning mechanism (LM) determining the optimal point on the line when the only input it receives are stochastic signals about the direction in which it should move. One can differentiate the SPL from the traditional class of optimization problems by the fact that the former considers the case where the directional information, for example, as inferred from an Oracle (which possibly computes the derivatives), suffices to achieve the optimization-without actually explicitly computing any derivatives. The SPL can be described in terms of a LM (algorithm) attempting to locate a point on a line. The LM interacts with a random environme…
Solving Stochastic Nonlinear Resource Allocation Problems Using a Hierarchy of Twofold Resource Allocation Automata
In a multitude of real-world situations, resources must be allocated based on incomplete and noisy information. However, in many cases, incomplete and noisy information render traditional resource allocation techniques ineffective. The decentralized Learning Automata Knapsack Game (LAKG) was recently proposed for solving one such class of problems, namely the class of Stochastic Nonlinear Fractional Knapsack Problems. Empirically, the LAKG was shown to yield a superior performance when compared to methods which are based on traditional parameter estimation schemes. This paper presents a completely new online Learning Automata (LA) system, namely the Hierarchy of Twofold Resource Allocation …
On Using the Theory of Regular Functions to Prove the ε-Optimality of the Continuous Pursuit Learning Automaton
Published version of a chapter in the book: Recent Trends in Applied Artificial Intelligence. Also available from the publisher at: http://dx.doi.org/10.1007/978-3-642-38577-3_27 There are various families of Learning Automata (LA) such as Fixed Structure, Variable Structure, Discretized etc. Informally, if the environment is stationary, their ε-optimality is defined as their ability to converge to the optimal action with an arbitrarily large probability, if the learning parameter is sufficiently small/large. Of these LA families, Estimator Algorithms (EAs) are certainly the fastest, and within this family, the set of Pursuit algorithms have been considered to be the pioneering schemes. The…
Thompson Sampling Guided Stochastic Searching on the Line for Non-stationary Adversarial Learning
This paper reports the first known solution to the N-Door puzzle when the environment is both non-stationary and deceptive (adversarial learning). The Multi-Armed-Bandit (MAB) problem is the iconic representation of the exploration versus exploitation dilemma. In brief, a gambler repeatedly selects and play, one out of N possible slot machines or arms and either receives a reward or a penalty. The objective of the gambler is then to locate the most rewarding arm to play, while in the process maximize his winnings. In this paper we investigate a challenging variant of the MAB problem, namely the non-stationary N-Door puzzle. Here, instead of directly observing the reward, the gambler is only…
Escape planning in realistic fire scenarios with Ant Colony Optimisation
Published version of an article from the journal:Applied Intelligence Also available on Springerlink: http://dx.doi.org/10.1007/s10489-014-0538-9 An emergency requiring evacuation is a chaotic event, filled with uncertainties both for the people affected and rescuers. The evacuees are often left to themselves for navigation to the escape area. The chaotic situation increases when predefined escape routes are blocked by a hazard, and there is a need to re-think which escape route is safest. This paper addresses automatically finding the safest escape routes in emergency situations in large buildings or ships with imperfect knowledge of the hazards. The proposed solution, based on Ant Colony …
Language Detection and Tracking in Multilingual Documents Using Weak Estimators
Published version of an article from the book: Structural, Syntactic, and Statistical Pattern Recognition . The original publication is available at Spingerlink. http://dx.doi.org/DOI: 10.1007/978-3-642-14980-1_59 This paper deals with the extremely complicated problem of language detection and tracking in real-life electronic (for example, in Word-of-Mouth (WoM)) applications, where various segments of the text are written in different languages. The difficulties in solving the problem are many-fold. First of all, the analyst has no knowledge of when one language stops and when the next starts. Further, the features which one uses for any one language (for example, the n-grams) will not be…
Accelerated Bayesian learning for decentralized two-armed bandit based decision making with applications to the Goore Game
Published version of an article in the journal: Applied Intelligence. Also available from the publisher at: http://dx.doi.org/10.1007/s10489-012-0346-z The two-armed bandit problem is a classical optimization problem where a decision maker sequentially pulls one of two arms attached to a gambling machine, with each pull resulting in a random reward. The reward distributions are unknown, and thus, one must balance between exploiting existing knowledge about the arms, and obtaining new information. Bandit problems are particularly fascinating because a large class of real world problems, including routing, Quality of Service (QoS) control, game playing, and resource allocation, can be solved …
A Novel Multidimensional Scaling Technique for Mapping Word-Of-Mouth Discussions
The techniques which utilize Multidimensional Scaling (MDS) as a fundamental statistical tool have been well developed since the late 1970’s. In this paper we show how anMDS scheme can be enhanced by incorporating into it a Stochastic Point Location (SPL) strategy (one which optimizes the former’s gradient descent learning phase) and a new Stress function. The enhanced method, referred to as MDS SPL, has been used in conjunction with a combination of the TF-IDF and Cosine Similarities on a very noisy Word-Of-Mouth (WoM) discussion set consisting of postings concerning mobile phones, yielding extremely satisfying results.
Fire simulation-based adaptation of SmartRescue App for serious game: Design, setup and user experience
Managing the crisis by embracing game and simulation elements and human participation into an interactive system is a mean to learn about responding to unexpected events. This so-called serious game approach is adopted in a summer school for crisis management attended by doctoral students and practitioners, as a part of its learning curriculum. The participants took part in the Disaster in my Backyard serious game, designed as a realistic crisis environment. A smartphone app encompassing fire simulations of a five-story apartment, showing how the flame, smoke and temperature of the fire developed over time from floor to floor, was tested in this serious game scenario. The color-coding of sm…
A Hierarchical Learning Scheme for Solving the Stochastic Point Location Problem
Published version of a chapter in the book: Advanced Research in Applied Artificial Intelligence. Also available from the publisher at: http://dx.doi.org/10.1007/978-3-642-31087-4_78 This paper deals with the Stochastic-Point Location (SPL) problem. It presents a solution which is novel in both philosophy and strategy to all the reported related learning algorithms. The SPL problem concerns the task of a Learning Mechanism attempting to locate a point on a line. The mechanism interacts with a random environment which essentially informs it, possibly erroneously, if the unknown parameter is on the left or the right of a given point which also is the current guess. The first pioneering work […
On incorporating the paradigms of discretization and Bayesian estimation to create a new family of pursuit learning automata
Published version of an article in the journal: Applied Intelligence. Also available from the publisher at: http://dx.doi.org/10.1007/s10489-013-0424-x There are currently two fundamental paradigms that have been used to enhance the convergence speed of Learning Automata (LA). The first involves the concept of utilizing the estimates of the reward probabilities, while the second involves discretizing the probability space in which the LA operates. This paper demonstrates how both of these can be simultaneously utilized, and in particular, by using the family of Bayesian estimates that have been proven to have distinct advantages over their maximum likelihood counterparts. The success of LA-…