Achieving Intelligent Traffic-aware Consolidation of Virtual Machines in a Data Center Using Learning Automata

2016

Cloud Computing (CC) is becoming increasingly pertinent and popular. A natural consequence of this is that many modern-day data centers experience very high internal traffic within the data centers themselves. The VMs with high mutual traffic often end up being far apart in the data center network, forcing them to communicate over unnecessarily long distances. The consequent traffic bottlenecks negatively affect both the performance of the application and the network in its entirety, posing nontrivial challenges for the administrators of these cloudbased data centers. The problem can, quite naturally, be compartmentalized into two phases which follow each other. First of all, the VMs are co…

On achieving intelligent traffic-aware consolidation of virtual machines in a data center using Learning Automata

2018

Unlike the computational mechanisms of the past many decades, that involved individual (extremely powerful) computers or clusters of machines, cloud computing (CC) is becoming increasingly pertinent and popular. Computing resources such as CPU and storage are becoming cheaper, and the servers themselves are becoming more powerful. This enables clouds to host more virtual machines (VMs). A natural consequence ofthis is that many modern-day data centers experience very high internaltraffic within the data centers themselves. This is, of course, due to the occurrence of servers that belong to the same tenant, communicating between themselves. The problem is accentuated when the VM deployment t…

Higher-Fidelity Frugal and Accurate Quantile Estimation Using a Novel Incremental <italic>Discretized</italic> Paradigm

2018

Traditional pattern classification works with the moments of the distributions of the features and involves the estimation of the means and variances. As opposed to this, more recently, research has indicated the power of using the quantiles of the distributions because they are more robust and applicable for non-parametric methods. The estimation of the quantiles is even more pertinent when one is mining data streams. However, the complexity of quantile estimation is much higher than the corresponding estimation of the mean and variance, and this increased complexity is more relevant as the size of the data increases. Clearly, in the context of infinite data streams, a computational and sp…

Scheduling Domestic Shiftable Loads in Smart Grids: A Learning Automata-Based Scheme

2017

In this paper, we consider the problem of scheduling shiftable loads, over multiple users, in smart grids. We approach the problem, which is becoming increasingly pertinent in our present energy-thirsty society, using a novel distributed game-theoretic framework. From a modeling perspective, the distributed scheduling problem is formulated as a game, and in particular, a so-called “Potential” game. This game has at least one pure strategy Nash Equilibrium (NE), and we demonstrate that the NE point is a global optimal point. The solution that we propose, which is the pioneering solution that incorporates the theory of Learning Automata (LA), permits the total supplied loads to approach the p…

A User-Centric Approach for Personalized Service Provisioning in Pervasive Environments

2011

Published version of an article published in Wireless Personal Communications (2011). Also available from the publisher at http://dx.doi.org/10.1007/s11277-011-0387-3 The vision of pervasive environments is being realized more than ever with the proliferation of services and computing resources located in our surrounding environments. Identifying those services that deserve the attention of the user is becoming an increasingly-challenging task. In this paper, we present an adaptive multi-criteria decision making mechanism for recommending relevant services to the mobile user. In this context, "Relevance" is determined based on a user-centric approach that combines both the reputation of the…

Concept Drift Detection Using Online Histogram-Based Bayesian Classifiers

2016

In this paper, we present a novel algorithm that performs online histogram-based classification, i.e., specifically designed for the case when the data is dynamic and its distribution is non-stationary. Our method, called the Online Histogram-based Naïve Bayes Classifier (OHNBC) involves a statistical classifier based on the well-established Bayesian theory, but which makes some assumptions with respect to the independence of the attributes. Moreover, this classifier generates a prediction model using uni-dimensional histograms, whose segments or buckets are fixed in terms of their cardinalities but dynamic in terms of their widths. Additionally, our algorithm invokes the principles of info…

Solving Two-Person Zero-Sum Stochastic Games With Incomplete Information Using Learning Automata With Artificial Barriers

2021

Learning automata (LA) with artificially absorbing barriers was a completely new horizon of research in the 1980s (Oommen, 1986). These new machines yielded properties that were previously unknown. More recently, absorbing barriers have been introduced in continuous estimator algorithms so that the proofs could follow a martingale property, as opposed to monotonicity (Zhang et al., 2014), (Zhang et al., 2015). However, the applications of LA with artificial barriers are almost nonexistent. In that regard, this article is pioneering in that it provides effective and accurate solutions to an extremely complex application domain, namely that of solving two-person zero-sum stochastic games that…

A Stochastic Search on the Line-Based Solution to Discretized Estimation

2012

Published version of a chapter in the book: Advanced Research in Applied Artificial Intelligence. Also available from the publisher at: http://dx.doi.org/10.1007/978-3-642-31087-4_77 Recently, Oommen and Rueda [11] presented a strategy by which the parameters of a binomial/multinomial distribution can be estimated when the underlying distribution is nonstationary. The method has been referred to as the Stochastic Learning Weak Estimator (SLWE), and is based on the principles of continuous stochastic Learning Automata (LA). In this paper, we consider a new family of stochastic discretized weak estimators pertinent to tracking time-varying binomial distributions. As opposed to the SLWE, our p…

Crowd Models for Emergency Evacuation: A Review Targeting Human-Centered Sensing

2013

Emergency evacuation of crowds is a fascinating phenomenon that has attracted researchers from various fields. Better understanding of this class of crowd behavior opens up for improving evacuation policies and smarter design of buildings, increasing safety. Recently, a new class of disruptive technology has appeared: Human-centered sensing which allows crowd behavior to be monitored in real-time, and provides the basis for real-time crowd control. The question then becomes: to what degree can previous crowd models incorporate this development, and what areas need further research? In this paper, we provide a survey that describes some widely used crowd models and discuss their advantages a…

An intelligent architecture for service provisioning in pervasive environments

2011

Accepted version of an article from the conference: 2011 International Symposium on Innovations in Intelligent Systems and Applications (INISTA). Definitive published version available from IEEE: http://dx.doi.org/10.1109/INISTA.2011.5946134 The vision of pervasive environments is being realized more than ever with the proliferation of services and computing resources located in our surrounding environments. Identifying those services that deserve the attention of the user is becoming an increasingly-challenging task. In this paper, we present an adaptive multi-criteria decision making mechanism for recommending relevant services to the mobile user. In this context Relevance is determined b…

Improving Classification of Tweets Using Linguistic Information from a Large External Corpus

2016

The bag of words representation of documents is often unsatisfactory as it ignores relationships between important terms that do not co-occur literally. Improvements might be achieved by expanding the vocabulary with other relevant word, like synonyms.

Learning Automaton Based On-Line Discovery and Tracking of Spatio-temporal Event Patterns

2010

Published version of an article from the book: Lecture Notes in Computer Science, 2010, Volume 6230/2010, 327-338. The original publication is available at Springerlink. http://dx.doi.org/10.1007/978-3-642-15246-7_31 Discovering and tracking of spatio-temporal patterns in noisy sequences of events is a difficult task that has become increasingly pertinent due to recent advances in ubiquitous computing, such as community-based social networking applications. The core activities for applications of this class include the sharing and notification of events, and the importance and usefulness of these functionalites increases as event-sharing expands into larger areas of one’s life. Ironically, …

A novel Stochastic Discretized Weak Estimator operating in non-stationary environments

2012

The task of designing estimators that are able to track time-varying distributions has found promising applications in many real-life problems. A particularly interesting family of distributions are the binomial/multiomial distributions. Existing approaches resort to sliding windows that track changes by discarding old observations. In this paper, we report a novel estimator referred to as the Stochastic Discretized Weak Estimator (SDWE), that is based on the principles of Learning Automata (LA). In brief, the estimator is able to estimate the parameters of a time varying binomial distribution using finite memory. The estimator tracks changes in the distribution by operating on a controlled…

On the analysis of a new Markov chain which has applications in AI and machine learning

2011

Accepted version of an article from the conference: 2011 24th Canadian Conference on Electrical and Computer Engineering. Published version available from IEEE: http://dx.doi.org/10.1109/CCECE.2011.6030727 In this paper, we consider the analysis of a fascinating Random Walk (RW) that contains interleaving random steps and random "jumps". The characterizing aspect of such a chain is that every step is paired with its counterpart random jump. RWs of this sort have applications in testing of entities, where the entity is never allowed to make more than a pre-specified number of consecutive failures. This paper contains the analysis of the chain, some fascinating limiting properties, and some i…

Tracking the Preferences of Users Using Weak Estimators

2011

Published version of am article from the book:AI 2011: Advances in Artificial Intelligence. Also available from the publisher on SpringerLink:http://dx.doi.org/10.1007/978-3-642-25832-9_81 Since a social network, by definition, is so diverse, the problem of estimating the preferences of its users is becoming increasingly essential for personalized applications which range from service recommender systems to the targeted advertising of services. However, unlike traditional estimation problems where the underlying target distribution is stationary, estimating a user’s interests, typically, involves non-stationary distributions. The consequent time varying nature of the distribution to be trac…

Identifying Unreliable Sensors Without a Knowledge of the Ground Truth in Deceptive Environments

2017

This paper deals with the extremely fascinating area of “fusing” the outputs of sensors without any knowledge of the ground truth. In an earlier paper, the present authors had recently pioneered a solution, by mapping it onto the fascinating paradox of trying to identify stochastic liars without any additional information about the truth. Even though that work was significant, it was constrained by the model in which we are living in a world where “the truth prevails over lying”. Couched in the terminology of Learning Automata (LA), this corresponds to the Environment (Since the Environment is treated as an entity in its own right, we choose to capitalize it, rather than refer to it as an “…

An adaptive approach to learning the preferences of users in a social network using weak estimators

2012

Published version of an article in the journal: Journal of Information Processing Systems. Also available from the publisher at: http://dx.doi.org/10.3745/JIPS.2012.8.2.191 - Open Access Since a social network by definition is so diverse, the problem of estimating the preferences of its users is becoming increasingly essential for personalized applications, which range from service recommender systems to the targeted advertising of services. However, unlike traditional estimation problems where the underlying target distribution is stationary; estimating a user's interests typically involves non-stationary distributions. The consequent time varying nature of the distribution to be tracked i…

On the classification of dynamical data streams using novel “Anti-Bayesian” techniques

2018

Abstract The classification of dynamical data streams is among the most complex problems encountered in classification. This is, firstly, because the distribution of the data streams is non-stationary, and it changes without any prior “warning”. Secondly, the manner in which it changes is also unknown. Thirdly, and more interestingly, the model operates with the assumption that the correct classes of previously-classified patterns become available at a juncture after their appearance. This paper pioneers the use of unreported novel schemes that can classify such dynamical data streams by invoking the recently-introduced “Anti-Bayesian” (AB) techniques. Contrary to the Bayesian paradigm, tha…

A New Tool for the Modeling of AI and Machine Learning Applications: Random Walk-Jump Processes

2011

Published version of an article from the book: Hybrid artificial intelligent systems, Lecture notes in computer science. The original publication is available at www.springerlink.com, http://dx.doi.org/10.1007/978-3-642-21219-2_2 There are numerous applications in Artificial Intelligence (AI) and Machine Learning (ML) where the criteria for decisions are based on testing procedures. The most common tools used in such random phenomena involve Random Walks (RWs). The theory of RWs and its applications have gained an increasing research interest since the start of the last century. [1]. In this context, we note that a RW is, usually, defined as a trajectory involving a series of successive ran…

On Distinguishing between Reliable and Unreliable Sensors Without a Knowledge of the Ground Truth

2015

In many applications, data from different sensors are aggregated in order to obtain more reliable information about the process that the sensors are monitoring. However, the quality of the aggregated information is intricately dependent on the reliability of the individual sensors. In fact, unreliable sensors will tend to report erroneous values of the ground truth, and thus degrade the quality of the fused information. Finding strategies to identify unreliable sensors can assist in having a counter-effect on their respective detrimental influences on the fusion process, and this has has been a focal concern in the literature. The purpose of this paper is to propose a solution to an extreme…

Towards a relation oriented service architecture

2010

Over the past three decades, the Internet has evolved from a point to point, open, academic network to an applications and services oriented critical infrastructure. The Internet has become a vital component of society today, from its simple origin as an academic research project. During this transition, numerous applications and usages of the network emerged that cannot be efficiently implemented by adhering to the original design tenets of the Internet. Some of the tenets have been broken, others diluted and new ones are emerging to accommodate new paradigms. Moreover, applications and services have been moving slowly but consistently towards a uniform model based on Service Oriented Appr…

On Solving the Problem of Identifying Unreliable Sensors Without a Knowledge of the Ground Truth: The Case of Stochastic Environments.

2017

The purpose of this paper is to propose a solution to an extremely pertinent problem, namely, that of identifying unreliable sensors (in a domain of reliable and unreliable ones) without any knowledge of the ground truth. This fascinating paradox can be formulated in simple terms as trying to identify stochastic liars without any additional information about the truth. Though apparently impossible, we will show that it is feasible to solve the problem, a claim that is counterintuitive in and of itself. One aspect of our contribution is to show how redundancy can be introduced, and how it can be effectively utilized in resolving this paradox. Legacy work and the reported literature (for exam…

On the Analysis of a Random Interleaving Walk–Jump Process with Applications to Testing

2011

Abstract Although random walks (RWs) with single-step transitions have been extensively studied for almost a century as seen in Feller (1968), problems involving the analysis of RWs that contain interleaving random steps and random “jumps” are intrinsically hard. In this article, we consider the analysis of one such fascinating RW, where every step is paired with its counterpart random jump. In addition to this RW being conceptually interesting, it has applications in testing of entities (components or personnel), where the entity is never allowed to make more than a prespecified number of consecutive failures. The article contains the analysis of the chain, some fascinating limiting proper…

Stochastic discretized learning-based weak estimation: a novel estimation method for non-stationary environments

2016

The task of designing estimators that are able to track time-varying distributions has found promising applications in many real-life problems.Existing approaches resort to sliding windows that track changes by discarding old observations. In this paper, we report a novel estimator referred to as the Stochastic Discretized Weak Estimator (SDWE), that is based on the principles of discretized Learning Automata (LA). In brief, the estimator is able to estimate the parameters of a time varying binomial distribution using finite memory. The estimator tracks changes in the distribution by operating a controlled random walk in a discretized probability space. The steps of the estimator are discre…

A pattern recognition approach for peak prediction of electrical consumption

2016

Predicting and mitigating demand peaks in electrical networks has become a prevalent research topic. Demand peaks pose a particular challenge to energy companies because these are difficult to foresee and require the net to support abnormally high consumption levels. In smart energy grids, time-differentiated pricing policies that increase the energy cost for the consumers during peak periods, and load balancing are examples of simple techniques for peak regulation. In this paper, we tackle the task of predicting power peaks prior to their actual occurrence in the context of a pilot Norwegian smart grid network.

Distributed Learning Automata-based S-learning scheme for classification

2019

This paper proposes a novel classifier based on the theory of Learning Automata (LA), reckoned to as PolyLA. The essence of our scheme is to search for a separator in the feature space by imposing an LA-based random walk in a grid system. To each node in the grid, we attach an LA whose actions are the choices of the edges forming a separator. The walk is self-enclosing, and a new random walk is started whenever the walker returns to the starting node forming a closed classification path yielding a many-edged polygon. In our approach, the different LA attached to the different nodes search for a polygon that best encircles and separates each class. Based on the obtained polygons, we perform …

On the analysis of a random walk-jump chain with tree-based transitions and its applications to faulty dichotomous search

2018

Random Walks (RWs) have been extensively studied for more than a century [1]. These walks have traditionally been on a line, and the generalizations for two and three dimensions, have been by extending the random steps to the corresponding neighboring positions in one or many of the dimensions. Among the most popular RWs on a line are the various models for birth and death processes, renewal processes and the gambler’s ruin problem. All of these RWs operate “on a discretized line”, and the walk is achieved by performing small steps to the current-state’s neighbor states. Indeed, it is this neighbor-step motion that renders their analyses tractable. When some of the transitions are to non-ne…

A Novel Clustering Algorithm based on a Non-parametric "Anti-Bayesian" Paradigm

2015

The problem of clustering, or unsupervised classification, has been solved by a myriad of techniques, all of which depend, either directly or implicitly, on the Bayesian principle of optimal classification. To be more specific, within a Bayesian paradigm, if one is to compare the testing sample with only a single point in the feature space from each class, the optimal Bayesian strategy would be to achieve this based on the distance from the corresponding means or central points in the respective distributions. When this principle is applied in clustering, one would assign an unassigned sample into the cluster whose mean is the closest, and this can be done in either a bottom-up or a top-dow…

The Hierarchical Discrete Pursuit Learning Automaton: A Novel Scheme With Fast Convergence and Epsilon-Optimality

2022

Author's accepted manuscript © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Since the early 1960s, the paradigm of learning automata (LA) has experienced abundant interest. Arguably, it has also served as the foundation for the phenomenon and field of reinforcement learning (RL). Over the decades, new concepts and fundamental principles have been introduced t…

Adaptive Task Assignment in Online Learning Environments

2016

With the increasing popularity of online learning, intelligent tutoring systems are regaining increased attention. In this paper, we introduce adaptive algorithms for personalized assignment of learning tasks to student so that to improve his performance in online learning environments. As main contribution of this paper, we propose a a novel Skill-Based Task Selector (SBTS) algorithm which is able to approximate a student's skill level based on his performance and consequently suggest adequate assignments. The SBTS is inspired by the class of multi-armed bandit algorithms. However, in contrast to standard multi-armed bandit approaches, the SBTS aims at acquiring two criteria related to stu…

Mitigating DDoS using weight‐based geographical clustering

2020

Distributed denial of service (DDoS) attacks have for the last two decades been among the greatest threats facing the internet infrastructure. Mitigating DDoS attacks is a particularly challenging task as an attacker tries to conceal a huge amount of traffic inside a legitimate traffic flow. This article proposes to use data mining approaches to find unique hidden data structures which are able to characterize the normal traffic flow. This will serve as a mean for filtering illegitimate traffic under DDoS attacks. In this endeavor, we devise three algorithms built on previously uncharted areas within mitigation techniques where clustering techniques are used to create geographical clusters …

PolyACO+: a multi-level polygon-based ant colony optimisation classifier

2017

Ant Colony Optimisation for classification has mostly been limited to rule based approaches where artificial ants walk on datasets in order to extract rules from the trends in the data, and hybrid approaches which attempt to boost the performance of existing classifiers through guided feature reductions or parameter optimisations. A recent notable example that is distinct from the mainstream approaches is PolyACO, which is a proof of concept polygon-based classifier that resorts to ant colony optimisation as a technique to create multi-edged polygons as class separators. Despite possessing some promise, PolyACO has some significant limitations, most notably, the fact of supporting classific…

On using novel “Anti-Bayesian” techniques for the classification of dynamical data streams

2017

The classification of dynamical data streams is among the most complex problems encountered in classification. This is, firstly, because the distribution of the data streams is non-stationary, and it changes without any prior “warning”. Secondly, the manner in which it changes is also unknown. Thirdly, and more interestingly, the model operates with the assumption that the correct classes of previously-classified patterns become available at a juncture after their appearance. This paper pioneers the use of unreported novel schemes that can classify such dynamical data streams by invoking the recently-introduced “Anti-Bayesian” (AB) techniques. Contrary to the Bayesian paradigm, that compare…

A Learning Automata Based Solution to Service Selection in Stochastic Environments

2010

Published version of a paper published in the book: Trends in Applied Intelligent Systems. Also available on SpringerLink: http://dx.doi.org/10.1007/978-3-642-13033-5_22 With the abundance of services available in today’s world, identifying those of high quality is becoming increasingly difficult. Reputation systems can offer generic recommendations by aggregating user provided opinions about service quality, however, are prone to ballot stuffing and badmouthing . In general, unfair ratings may degrade the trustworthiness of reputation systems, and changes in service quality over time render previous ratings unreliable. In this paper, we provide a novel solution to the above problems based …

Achieving Fair Load Balancing by Invoking a Learning Automata-Based Two-Time-Scale Separation Paradigm.

2020

Author's accepted manuscript. © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. In this article, we consider the problem of load balancing (LB), but, unlike the approaches that have been proposed earlier, we attempt to resolve the problem in a fair manner (or rather, it would probably be more appropriate to describe it as an ε-fair manner because, although the LB…

“Anti-Bayesian” flat and hierarchical clustering using symmetric quantiloids

2017

A myriad of works has been published for achieving data clustering based on the Bayesian paradigm, where the clustering sometimes resorts to Naive-Bayes decisions. Within the domain of clustering, the Bayesian principle corresponds to assigning the unlabelled samples to the cluster whose mean (or centroid) is the closest. Recently, Oommen and his co-authors have proposed a novel, counter-intuitive and pioneering PR scheme that is radically opposed to the Bayesian principle. The rational for this paradigm, referred to as the “Anti-Bayesian” (AB) paradigm, involves classification based on the non-central quantiles of the distributions. The first-reported work to achieve clustering using the A…

The Hierarchical Continuous Pursuit Learning Automation for Large Numbers of Actions

2018

Part 10: Learning - Intelligence; International audience; Although the field of Learning Automata (LA) has made significant progress in the last four decades, the LA-based methods to tackle problems involving environments with a large number of actions are, in reality, relatively unresolved. The extension of the traditional LA (fixed structure, variable structure, discretized, and pursuit) to problems within this domain cannot be easily established when the number of actions is very large. This is because the dimensionality of the action probability vector is correspondingly large, and consequently, most components of the vector will, after a relatively short time, have values that are smal…

Ant Colony Optimisation-Based Classification Using Two-Dimensional Polygons

2016

The application of Ant Colony Optimization to the field of classification has mostly been limited to hybrid approaches which attempt at boosting the performance of existing classifiers (such as Decision Trees and Support Vector Machines (SVM)) — often through guided feature reductions or parameter optimizations.

Distributed learning automata-based scheme for classification using novel pursuit scheme

2020

Learning Automata (LA) is a popular decision making mechanism to “determine the optimal action out of a set of allowable actions” (Agache and Oommen, IEEE Trans Syst Man Cybern-Part B Cybern 2002(6): 738–749, 2002). The distinguishing characteristic of automata-based learning is that the search for the optimising parameter vector is conducted in the space of probability distributions defined over the parameter space, rather than in the parameter space itself (Thathachar and Sastry, IEEE Trans Syst Man Cybern-Part B Cybern 32(6): 711–722, 2002). Recently, Goodwin and Yazidi pioneered the use of Ant Colony Optimisation (ACO) for solving classification problems (Goodwin and Yazidi 2016). In th…

Learning-automaton-based online discovery and tracking of spatiotemporal event patterns.

2013

Discovering and tracking of spatiotemporal patterns in noisy sequences of events are difficult tasks that have become increasingly pertinent due to recent advances in ubiquitous computing, such as community-based social networking applications. The core activities for applications of this class include the sharing and notification of events, and the importance and usefulness of these functionalities increase as event sharing expands into larger areas of one's life. Ironically, instead of being helpful, an excessive number of event notifications can quickly render the functionality of event sharing to be obtrusive. Indeed, any notification of events that provides redundant information to the…

Distributed learning automata for solving a classification task

2016

In this paper, we propose a novel classifier in two-dimensional feature spaces based on the theory of Learning Automata (LA). The essence of our scheme is to search for a separator in the feature space by imposing a LA based random walk in a grid system. To each node in the gird we attach an LA, whose actions are the choice of the edges forming the separator. The walk is self-enclosing, i.e, a new random walk is started whenever the walker returns to starting node forming a closed classification path yielding a many edged polygon. In our approach, the different LA attached at the different nodes search for a polygon that best encircles and separates each class. Based on the obtained polygon…

Improving the Diversity of Bootstrapped DQN by Replacing Priors With Noise

2022

Authors accepted manuscript Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Q-learning is one of the most well-known Reinforcement Learning algorithms. There have been tremendous efforts to develop this algorithm using neural networks. Bootstrapped Deep Q-Learning Network is amongst them. It utilizes multiple neural network heads to introduce diversity into Q-learning. Dive…

A Pattern Recognition Approach for Peak Prediction of Electrical Consumption

2014

Predicting and mitigating demand peaks in electrical networks has become a prevalent research topic. Demand peaks pose a particular challenge to energy companies because these are difficult to foresee and require the net to support abnormally high consumption levels. In smart energy grids, time-differentiated pricing policies that increase the energy cost for the consumers during peak periods, and load balancing are examples of simple techniques for peak regulation. In this paper, we tackle the task of predicting power peaks prior to their actual occurrence in the context of a pilot Norwegian smart grid network.

Dynamic Ordering of Firewall Rules Using a Novel Swapping Window-based Paradigm

2016

Designing and implementing efficient firewall strategies in the age of the Internet of Things (IoT) is far from trivial. This is because, as time proceeds, an increasing number of devices will be connected, accessed and controlled on the Internet. Additionally, an ever-increasingly amount of sensitive information will be stored on various networks. A good and effi- cient firewall strategy will attempt to secure this information, and to also manage the large amount of inevitable network traffic that these devices create. The goal of this paper is to propose a framework for designing optimized firewalls for the IoT. This paper deals with two fundamental challenges/problems encountered in such…

On the Online Classification of Data Streams Using Weak Estimators

2016

In this paper, we propose a novel online classifier for complex data streams which are generated from non-stationary stochastic properties. Instead of using a single training model and counters to keep important data statistics, the introduced online classifier scheme provides a real-time self-adjusting learning model. The learning model utilizes the multiplication-based update algorithm of the Stochastic Learning Weak Estimator (SLWE) at each time instant as a new labeled instance arrives. In this way, the data statistics are updated every time a new element is inserted, without requiring that we have to rebuild its model when changes occur in the data distributions. Finally, and most impo…

A novel strategy for solving the stochastic point location problem using a hierarchical searching scheme

2014

Stochastic point location (SPL) deals with the problem of a learning mechanism (LM) determining the optimal point on the line when the only input it receives are stochastic signals about the direction in which it should move. One can differentiate the SPL from the traditional class of optimization problems by the fact that the former considers the case where the directional information, for example, as inferred from an Oracle (which possibly computes the derivatives), suffices to achieve the optimization-without actually explicitly computing any derivatives. The SPL can be described in terms of a LM (algorithm) attempting to locate a point on a line. The LM interacts with a random environme…

The Hierarchical Continuous Pursuit Learning Automation: A Novel Scheme for Environments With Large Numbers of Actions.

2019

Although the field of learning automata (LA) has made significant progress in the past four decades, the LA-based methods to tackle problems involving environments with a large number of actions is, in reality, relatively unresolved. The extension of the traditional LA to problems within this domain cannot be easily established when the number of actions is very large. This is because the dimensionality of the action probability vector is correspondingly large, and so, most components of the vector will soon have values that are smaller than the machine accuracy permits, implying that they will never be chosen . This paper presents a solution that extends the continuous pursuit paradigm to …

A Learning Automaton-based Scheme for Scheduling Domestic Shiftable Loads in Smart Grids

2017

In this paper, we consider the problem of scheduling shiftable loads, over multiple users, in smart electrical grids. We approach the problem, which is becoming increasingly pertinent in our present energy-thirsty society, using a novel distributed game-theoretic framework. In our specific instantiation, we consider the scenario when the power system has a local-area Smart Grid subnet comprising of a single power source and multiple customers. The objective of the exercise is to tacitly control the total power consumption of the customers’ shiftable loads, so to approach the rigid power budget determined by the power source, but to simultaneously not exceed this threshold. As opposed to the…

A novel technique for stochastic root-finding: Enhancing the search with adaptive d-ary search

2017

The most fundamental problem encountered in the field of stochastic optimization, is the Stochastic Root Finding (SRF) problem where the task is to locate an unknown point x∗ for which g(x∗) = 0 for a given function g that can only be observed in the presence of noise [15]. The vast majority of the state-of-the-art solutions to the SRF problem involve the theory of stochastic approximation. The premise of the latter family of algorithms is to oper ate by means of so-called “small-step”processesthat explorethe search space in a conservative manner. Using this paradigm, the point investigated at any time instant is in the proximity of the point investigated at the previous time instant, render…

A Hierarchical Learning Scheme for Solving the Stochastic Point Location Problem

2012

Published version of a chapter in the book: Advanced Research in Applied Artificial Intelligence. Also available from the publisher at: http://dx.doi.org/10.1007/978-3-642-31087-4_78 This paper deals with the Stochastic-Point Location (SPL) problem. It presents a solution which is novel in both philosophy and strategy to all the reported related learning algorithms. The SPL problem concerns the task of a Learning Mechanism attempting to locate a point on a line. The mechanism interacts with a random environment which essentially informs it, possibly erroneously, if the unknown parameter is on the left or the right of a given point which also is the current guess. The first pioneering work […

On the Classification of Dynamical Data Streams Using Novel “Anti–Bayesian” Techniques

2018

The classification of dynamical data streams is among the most complex problems encountered in classification. This is, firstly, because the distribution of the data streams is non-stationary, and it changes without any prior “warning”. Secondly, the manner in which it changes is also unknown. Thirdly, and more interestingly, the model operates with the assumption that the correct classes of previously-classified patterns become available at a juncture after their appearance. This paper pioneers the use of unreported novel schemes that can classify such dynamical data streams by invoking the recently-introduced “Anti- Bayesian” (AB) techniques. Contrary to the Bayesian paradigm, that compar…

Intelligent Learning Automata-based Strategies Applied to Personalized Service Provisioning in Pervasive Environments

2011

Doktorgradsavhandling i informasjons- og kommunikasjonsteknologi, Universitetet i Agder, Grimstad, 2011

Solving Stochastic Root-Finding with adaptive d-ary search

2015

-

The Hierarchical Discrete Learning Automaton Suitable for Environments with Many Actions and High Accuracy Requirements

2022

Author's accepted manuscript Since its early beginning, the paradigm of Learning Automata (LA), has attracted much interest. Over the last decades, new concepts and various improvements have been introduced to increase the LA’s speed and accuracy, including employing probability updating functions, discretizing the probability space, and implementing the “Pursuit” concept. The concept of incorporating “structure” into the ordering of the LA’s actions is one of the latest advancements to the field, leading to the ϵ-optimal Hierarchical Continuous Pursuit LA (HCPA) that has superior performance to other LA variants when the number of actions is large. Although the previously proposed HCPA is …

Identifying unreliable sensors without a knowledge of the ground truth in deceptive environments

2017

This paper deals with the extremely fascinating area of “fusing” the outputs of sensors without any knowledge of the ground truth. In an earlier paper, the present authors had recently pioneered a solution, by mapping it onto the fascinating paradox of trying to identify stochastic liars without any additional information about the truth. Even though that work was significant, it was constrained by the model in which we are living in a world where “the truth prevails over lying”. Couched in the terminology of Learning Automata (LA), this corresponds to the Environment (Since the Environment is treated as an entity in its own right, we choose to capitalize it, rather than refer to it as an “…

Balanced difficulty task finder: an adaptive recommendation method for learning tasks based on the concept of state of flow

2020

An adaptive task difficulty assignment method which we reckon as balanced difficulty task finder (BDTF) is proposed in this paper. The aim is to recommend tasks to a learner using a trade-off between skills of the learner and difficulty of the tasks such that the learner experiences a state of flow during the learning. Flow is a mental state that psychologists refer to when someone is completely immersed in an activity. Flow state is a multidisciplinary field of research and has been studied not only in psychology, but also neuroscience, education, sport, and games. The idea behind this paper is to try to achieve a flow state in a similar way as Elo’s chess skill rating (Glickman in Am Ches…

Service selection in stochastic environments: a learning-automaton based solution

2011

Published version of an article from the journal: Applied Intelligence. Also available from the publisher on SpringerLink: http://dx.doi.org/10.1007/s10489-011-0280-5 In this paper, we propose a novel solution to the problem of identifying services of high quality. The reported solutions to this problem have, in one way or the other, resorted to using so-called “Reputation Systems” (RSs). Although these systems can offer generic recommendations by aggregating user-provided opinions about the quality of the services under consideration, they are, understandably, prone to “ballot stuffing” and “badmouthing” in a competitive marketplace. In general, unfair ratings may degrade the trustworthine…

Ant colony optimisation-based classification using two-dimensional polygons

2016

Higher-Fidelity Frugal and Accurate Quantile Estimation Using a Novel Incremental Discretized Paradigm

2018

Nivå1

Expert Q-learning: Deep Reinforcement Learning with Coarse State Values from Offline Expert Examples

2022

In this article, we propose a novel algorithm for deep reinforcement learning named Expert Q-learning. Expert Q-learning is inspired by Dueling Q-learning and aims at incorporating semi-supervised learning into reinforcement learning through splitting Q-values into state values and action advantages. We require that an offline expert assesses the value of a state in a coarse manner using three discrete values. An expert network is designed in addition to the Q-network, which updates each time following the regular offline minibatch update whenever the expert example buffer is not empty. Using the board game Othello, we compare our algorithm with the baseline Q-learning algorithm, which is a…

On optimizing firewall performance in dynamic networks by invoking a novel swapping window-based paradigm

2018

Designing and implementing efficient firewall strategies in the age of the Internet of Things (IoT) is far from trivial. This is because, as time proceeds, an increasing number of devices will be connected, accessed and controlled on the Internet. Additionally, an everincreasingly amount of sensitive information will be stored on various networks. A good and efficient firewall strategy will attempt to secure this information, and to also manage the large amount of inevitable network traffic that these devices create. The goal of this paper is to propose a framework for designing optimized firewalls for the IoT. This paper deals with two fundamental challenges/problems encountered in such firewalls…

Process Improvement Solution for Mobile Platform Customer SW Development

2008

Masteroppgave i informasjons- og kommunikasjonsteknologi 2008 – Universitetet i Agder, Grimstad Time to market is becoming an increasingly important topic in software industry. In this trend, handling customer change requests is of a paramount importance. In the current thesis, we investigate reducing the lead time of handling customer requests at EMP Grimstad. Problems were identified and an extensive solution that covers all the aspects of these problems are presented. An experience was conducted and the first results are promising.

Stochastic discretized learning-based weak estimation: a novel estimation method for non-stationary environments

2016

- Nivå2