Consensus in inventory games

This paper studies design, convergence, stability and optimality of a distributed consensus protocol for n-player repeated non cooperative games under incomplete information. Information available to each player concerning the other players' strategies evolves in time. At each stage (time period), the players select myopically their best binary strategy on the basis of a payoff, defined on a single stage, monotonically decreasing with the number of active players. The game is specialized to an inventory application, where fixed costs are shared among all retailers, interested in reordering or not from a common warehouse. As information evolves in time, the number of active players changes t…

Distributed Consensus in Noncooperative Inventory Games

This paper deals with repeated nonsymmetric congestion games in which the players cannot observe their payoffs at each stage. Examples of applications come from sharing facilities by multiple users. We show that these games present a unique Pareto optimal Nash equilibrium that dominates all other Nash equilibria and consequently it is also the social optimum among all equilibria, as it minimizes the sum of all the players’ costs. We assume that the players adopt a best response strategy. At each stage, they construct their belief concerning others probable behavior, and then, simultaneously make a decision by optimizing their payoff based on their beliefs. Within this context, we provide a …

AQM Stability in Multiple Bottleneck Networks

In this paper, we highlight that multiple bottlenecks can affect the performance of active queue management controllers, which are usually configured on a single bottleneck basis, as if each controller were the only element regulating the TCP traffic along its path. To see this, we consider a network scenario where RED is configured at each router, according to previously developed control theoretic techniques. These configuration rules assure stability in a single bottleneck scenario. Yet, we show that instability may arise when two link become congested. We justify this result through a multiple bottleneck model and give guidelines for new cooperative AQM controllers.

Robust control of networks under discrete disturbances and controls

We consider dynamic networks where the disturbances and control actions take discrete values. We briefly survey some of our recent results establishing necessary and sufficient conditions for the existence of robustly globally invariant (hyper box) sets, as well as sufficient conditions for global attractivity of such sets.We then establish connections between these results and existing results in the literature for the setup where all the inputs are analog. Finally, we derive tight upper and lower bounds on the smallest such set in the special case of a degenerate network.

Robust dynamic cooperative games

Classical cooperative game theory is no longer a suitable tool for those situations where the values of coalitions are not known with certainty. Recent works address situations where the values of coalitions are modelled by random variables. In this work we still consider the values of coalitions as uncertain, but model them as unknown but bounded disturbances. We do not focus on solving a specific game, but rather consider a family of games described by a polyhedron: each point in the polyhedron is a vector of coalitions’ values and corresponds to a specific game. We consider a dynamic context where while we know with certainty the average value of each coalition on the long run, at each t…

Optimal Impulse Control Problems and Linear Programming

Optimal impulse control problems are, in general, difficult to solve. A current research goal is to isolate those problems that lead to tractable solutions. In this paper, we identify a special class of optimal impulse control problems which are easy to solve. Easy to solve means that solution algorithms are polynomial in time and therefore suitable to the on-line implementation in real-time problems. We do this by using a paradigm borrowed from the Operations Research field. As main result, we present a solution algorithm that converges to the exact solution in polynomial time. Our approach consists in approximating the optimal impulse control problem via a binary linear programming proble…

Robust control of uncertain multi-inventory systems via linear matrix inequality

We consider a continuous time linear multi inventory system with unknown demands bounded within ellipsoids and controls bounded within ellipsoids or polytopes. We address the problem of "-stabilizing the inventory since this implies some reduction of the inventory costs. The main results are certain conditions under which "-stabilizability is possible through a saturated linear state feedback control. All the results are based on a Linear Matrix Inequalities (LMIs) approach and on some recent techniques for the modeling and analysis of polytopic systems with saturations.

Crowd-Averse Cyber-Physical Systems: The Paradigm of Robust Mean-Field Games

For a networked controlled system, we illustrate the paradigm of robust mean-field games. This is a modeling framework at the interface of differential game theory, mathematical physics, and $H_{\infty}$ - optimal control that tries to capture the mutual influence between a crowd and its individuals. First, we establish a mean-field system for such games including the effects of adversarial disturbances. Second, we identify the optimal response of the individuals for a given population behavior. Third, we provide an analysis of equilibria and their stability.

Quantized Dissensus in switching networks with nodes death and duplication* *Research supported by MURST-PRIN “Robust Techniques for uncertain systems control” and by MURST-PRIN 2007ZMZK5T “Decisional model for the design and the management of logistics networks characterized by high interoperability and information integration”

Abstract In this paper we discuss agents exchanging quantized flows to diverge one from the others according to a dissensus protocol. A Quantized Gossip algorithm is considered. Evolutions of the states during switching intervals and at switching instants and their property are described and analyzed. The modeling of switching systems describing networks where death and duplication processes occur is described. Some properties of the topology reached by the network when different rules of duplication and inheritance are implemented.

Robust Mean Field Games with Application to Production of an Exhaustible Resource

International audience; In this paper, we study mean field games under uncertainty. We consider a population of players with individual states driven by a standard Brownian motion and a disturbance term. The contribution is three-fold: First, we establish a mean field system for such robust games. Second, we apply the methodology to an exhaustible resource production. Third, we show that the dimension of the mean field system can be significantly reduced by considering a functional of the first moment of the mean field process.

Noncooperative dynamic games for inventory applications: A consensus approach

We focus on a finite horizon noncooperative dynamic game where the stage cost of a single player associated to a decision is a monotonically nonincreasing function of the total number of players making the same decision. For the single-stage version of the game, we characterize Nash equilibria and derive a consensus protocol that makes the players converge to the unique Pareto optimal Nash equilibrium. Such an equilibrium guarantees the interests of the players and is also social optimal in the set of Nash equilibria. For the multi-stage version of the game, we present an algorithm that converges to Nash equilibria, unfortunately not necessarily Pareto optimal. The algorithm returns a seque…

Non-linear protocols for optimal distributed consensus in networks of dynamic agents

We consider stationary consensus protocols for networks of dynamic agents with fixed topologies. At each time instant, each agent knows only its and its neighbors'' state, but must reach consensus on a group decision value that is function of all the agents'' initial state. We show that the agents can reach consensus if the value of such a function is time-invariant when computed over the agents'' state trajectories. We use this basic result to introduce a non-linear protocol design rule allowing consensus on a quite general set of values. Such a set includes, e.g., any generalized mean of order p of the agents'' initial states. As a second contribution we show that our protocol design is t…

Objective function design for robust optimality of linear control under state-constraints and uncertainty

We consider a model for the control of a linear network flow system with unknown but bounded demand and polytopic bounds on controlled flows. We are interested in the problem of finding a suitable objective function that makes robust optimal the policy represented by the so-called linear saturated feedback control. We regard the problem as a suitable differential game with switching cost and study it in the framework of the viscosity solutions theory for Bellman and Isaacs equations. © 2009 EDP Sciences, SMAI.

Team Theory and Person-by-Person Optimization with Binary Decisions

In this paper, we extend the notion of person-by-person (pbp) optimization to binary decision spaces. The novelty of our approach is the adaptation to a dynamic team context of notions borrowed from the pseudo-boolean optimization field as completely local-global or unimodal functions and submodularity. We also generalize the concept of pbp optimization to the case where groups of $m$ decisions makers make joint decisions sequentially, which we refer to as $m$b$m$ optimization. The main contribution is a description of sufficient conditions, verifiable in polynomial time, under which a pbp or an $m$b$m$ optimization algorithm converges to the team-optimum. As a second contribution, we prese…

Game-Theoretic Learning and Allocations in Robust Dynamic Coalitional Games

The problem of allocation in coalitional games with noisy observations and dynamic environments is considered. The evolution of the excess is modeled by a stochastic differential inclusion involvin...

On robustness and dynamics in (un)balanced coalitional games

In this paper we investigate robustness and dynamics for coalitional games with transferable utilities (TU games). In particular we study sequences of TU games. These sequences model dynamic situations in which the values of coalitions of players are not known beforehand, and are subject to changes over time. An allocation rule assigns a payoff to each player in each time period. This payoff is bounded by external restrictions, for example due to contractual agreements. Our main questions are: (i) under which conditions do the allocations converge to a core-element of the game, and (ii) when do the allocations converge to some specific allocation, the so-called nominal allocation? The main …

Nonlinear network dynamics for interconnected micro-grids

Abstract This paper deals with transient stability in interconnected micro-grids. The main challenge is to understand the impact of the connectivity of the graph and model nonlinearities on transient and steady-state behavior of the system as a whole. The contribution of this paper is three-fold. First, we provide a robust classification of transient dynamics for different intervals of the parameters for a single micro-grid. We prove that underdamped dynamics and oscillations arise when the damping coefficient is below a certain threshold which we calculate explicitly as function of the product between the inertia coefficient and the synchronization parameter. Second, for interconnected mic…

Average flow constraints and stabilizability in uncertain production-distribution systems

We consider a multi-inventory system with controlled flows and uncertain demands (disturbances) bounded within assigned compact sets. The system is modelled as a first-order one integrating the discrepancy between controlled flows and demands at different sites/nodes. Thus, the buffer levels at the nodes represent the system state. Given a long-term average demand, we are interested in a control strategy that satisfies just one of two requirements: (i) meeting any possible demand at each time (worst case stability) or (ii) achieving a predefined flow in the average (average flow constraints). Necessary and sufficient conditions for the achievement of both goals have been proposed by the aut…

Consensus in opinion dynamics as a repeated game

Abstract We study an n -agent averaging process with dynamics subject to controls and adversarial disturbances. The model arises in multi-population opinion dynamics with macroscopic and microscopic intertwined dynamics. The averaging process describes the influence from neighbouring populations, whereas the input term indicates how the distribution of opinions in the population changes as a result of dynamical evolutions at a microscopic level (individuals’ changing opinions). The input term is obtained as the vector payoff of a two player repeated game. We study conditions under which the agents achieve robust consensus to some predefined target set. Such conditions build upon the approac…

A saturated strategy robustly ensures stability of the cooperative equilibrium for Prisoner's dilemma

We study diffusion of cooperation in a two-population game in continuous time. At each instant, the game involves two random individuals, one from each population. The game has the structure of a Prisoner's dilemma where each player can choose either to cooperate (c) or to defect (d), and is reframed within the field of approachability in two-player repeated game with vector payoffs. We turn the game into a dynamical system, which is positive, and propose a saturated strategy that ensures local asymptotic stability of the equilibrium (c, c) for any possible choice of the payoff matrix. We show that there exists a rectangle, in the space of payoffs, which is positively invariant for the syst…

Multiple UAV cooperative path planning via neuro-dynamic programming

In this paper, a team of n unmanned air-vehicles (UAVs) in cooperative path planning is given the task of reaching the assigned target while i) avoiding threat zones ii) synchronizing minimum time arrivals on the target, and iii) ensuring arrivals coming from different directions. We highlight three main contributions. First we develop a novel hybrid model and suit it to the problem at hand. Second, we design consensus protocols for the management of information. Third, we synthesize local predictive controllers through a distributed, scalable and suboptimal neuro-dynamic programming (NDP) algorithm.

ROBUST CONTROL STRATEGIES FOR MULTI—INVENTORY SYSTEMS WITH AVERAGE FLOW CONSTRAINTS

Abstract In this paper we consider multi—inventory systems in presence of uncertain demand. We assume that i) demand is unknown but bounded in an assigned compact set and ii) the control inputs (controlled flows) are subject to assigned constraints. Given a long—term average demand, we select a nominal flow that feeds such a demand. In this context, we are interested in a control strategy that meets at each time all possible current demands and achieves the nominal flow in the average. We provide necessary and sufficient conditions for such a strategy to exist and we characterize the set of achievable flows. Such conditions are based on linear programming and thus they are constructive. In …

Challenging aspects in Consensus protocols for networks

Results on consensus protocols for networks are presented. The basic tools and the main contribution available in the literature are considered, together with some of the related challenging aspects: estimation in networks and how to deal with disturbances is considered. Motivated by applications to sensor, peer-to- peer, and ad hoc networks, many papers have considered the problem of estimation in a consensus fashion. Here, the unknown but bounded (UBB) noise affecting the network is addressed in details. Because of the presence of UBB disturbances convergence to equilibria with all equal components is, in general, not possible. The solution of the epsiv-consensus problem, where the states…

Approximate-Dynamic Programming for Multi-Retailers Inventory Control

Optimal Switches in Multi–inventory Systems

Given a switched multi-inventory system we wish to find the optimal schedule of the resets to maintain the system in a safe operating interval, while minimizing a function related to the cost of the resets. We discuss a family of instances that can be solved in polynomial time by linear programming. We do this by introducing a set-covering formulation with a totally unimodular constraint matrix.

Mixed integer optimal compensation: Decompositions and mean-field approximations

Mixed integer optimal compensation deals with optimizing integer- and real-valued control variables to compensate disturbances in dynamic systems. The mixed integer nature of controls might be a cause of intractability for instances of larger dimensions. To tackle this issue, we propose a decomposition method which turns the original n-dimensional problem into n independent scalar problems of lot sizing form. Each scalar problem is then reformulated as a shortest path one and solved through linear programming over a receding horizon. This last reformulation step mirrors a standard procedure in mixed integer programming. We apply the decomposition method to a mean-field coupled multi-agent s…

Distributed consensus protocols for coordinating buyers

In this paper, we introduce a distributed consensus protocol for coordinating orders of a network of buyers also called agents/decision makers. Each buyer chooses a different threshold strategy, defining its intention to place an order only if at least other l buyers will do the same. We prove that consensus is reached asymptotically globally and coordination is the same that if the decision making process would be centralized, namely, any decision maker (DM) has access to the thresholds of all other DMs and chooses to order or not. The proposed distributed protocol has the advantage that buyers do not have to communicate their threshold strategy in advance, and consensus is reached without…

Robust Allocation Rules in Dynamical Cooperative TU Games

Robust dynamic coalitional TU games are repeated TU games where the values of the coalitions are unknown but bounded variables. We set up the game supposing that the Game Designer uses a vague measure of the extra reward that each coalition has received up to the current time to re-adjust the allocations among the players. As main result, we provide a constructive method for designing allocation rules that converge to the core of the average game. Both the set up and the solution approach also provide an insight on commonalities between coalitional games and stability theory.

About the Stability of Active Queue Management Mechanism

In this paper, we discuss the influence of multiple bottlenecks on the stability of Active Queue Management (AQM) controllers, usually configured on a single bottleneck basis. To see this, we consider a network scenario where RED is configured at each router according to previously developed control theoretic techniques. These configuration rules assure stability in a single bottleneck scenario. Yet, we show that instability may arise when two links become congested. We justify this result through a multiple bottleneck model.

Online Pricing via Stackelberg and Incentive Games in a Micro-Grid

This paper deals with the analysis and design of online pricing mechanisms in micro-grids. Two cases are studied in which the market layer is modeled as an open-loop and closed-loop dynamical system respectively. In the case of open-loop market dynamics, the price is generated as equilibrium price of a Stackelberg game with an incentive strategy. In such Stackelberg game, the leader is the energy supplier, the follower is the consumer, and the leader plays an incentive strategy. In the case of closed-loop market dynamics, the price is obtained as a function of the power supplied and the demand. A stability analysis is provided for both cases, which sheds light on the transient and steady-st…

Mean Field Linear Quadratic Games with Set Up Costs

This paper studies linear quadratic games with set up costs monotonic on the number of active players, namely, players whose action is non-null. Such games arise naturally in joint replenishment inventory systems. Building upon a preliminary analysis of the properties of the best response strategies and Nash equilibria for the given game, the main contribution is the study of the same game under large population. We also analyze the influence of an additional disturbance in the spirit of the literature on H∞ control. Numerical illustrations are provided. © 2012 Springer Science+Business Media New York.

AQM Generalized Nyquist stability in multiple bottlenecks networks

The influence of multiple bottlenecks on the stability of Active Queue Management (AQM) controllers, usually configured on a single bottleneck basis is discussed. We consider a network scenario where RED is configured at each router according to previously developed control theoretic techniques. These configuration rules assure stability in a single bottleneck scenario. We show that instability may arise when two links become congested. We justify this result through a multiple bottleneck model using the Generalized Nyquist stability criterion.

Dissensus, death and division

The modeling of switching systems describing networks where death and duplication processes occur is described. A dissensus protocol, complementary to consensus protocol, is introduced and the convergence or divergence of the agents' state evolution is studied. We discuss some properties of the topology reached by the network when different rules of duplication and inheritance are implemented.

Cooperative Inventory control

In multi-retailer inventory control the possibility of sharing setup costs motivates communication and coordination among the retailers. We solve the problem of finding suboptimal distributed reordering policies that minimize setup, ordering, storage, and shortage costs incurred by the retailers over a finite horizon. Neuro-dynamic programming (NDP) reduces the computational complexity of the solution algorithm from exponential to polynomial on the number of retailers.

Strategic Thinking under social influence: Scalability, stability and robustness of allocations

This paper studies the strategic behavior of a large number of game designers and studies the scalability, stability and robustness of their allocations in a large number of homogeneous coalitional games with transferable utilities (TU). For each TU game, the characteristic function is a continuous-time stochastic process. In each game, a game designer allocates revenues based on the extra reward that a coalition has received up to the current time and the extra reward that the same coalition has received in the other games. The approach is based on the theory of mean-field games with heterogeneous groups in a multi-population regime.

Optimization of Long-Run Average-Flow Cost in Networks With Time-Varying Unknown Demand

We consider continuous-time robust network flows with capacity constraints and unknown but bounded time-varying demand. The problem of interest is to design a control strategy off-line with no knowledge of the demand realization. Such a control strategy regulates the flow on-line as a function of the realized demand. We address both the case of systems without and with buffers. The main novelty in this work is that we consider a convex cost which is a function of the long-run average-flow and average-demand. We distinguish a worst-case scenario where the demand is the worst-one from a deterministic scenario where the demand has a neutral behavior. The resulting strategies are called min-max…

MECHANISM DESIGN FOR OPTIMAL CONSENSUS PROBLEMS

We consider stationary consensus protocols for networks of dynamic agents with fixed and switching topologies. At each time instant, each agent knows only its and its neighbors’ state, but must reach consensus on a group decision value that is function of all the agents’ initial state.We show that our protocol design is the solution of individual optimizations performed by the agents. This notion suggests a game theoretic interpretation of consensus problems as mechanism design problems. Under this perspective a supervisor entails the agents to reach a consensus by imposing individual objectives. We prove that such objectives can be chosen so that rational agents have a unique optimal proto…

AQM generalized nyquist stability in multiple bottleneck networks

Abstract The influence of multiple bottlenecks on the stability of Active Queue Management (AQM) controllers, usually configured on a single bottleneck basis is discussed. We consider a network scenario where RED is configured at each router according to previously developed control theoretic techniques. These configuration rules assure stability in a single bottleneck scenario. We show that instability may arise when two links become congested. We justify this result through a multiple bottleneck model using the Generalized Nyquist stability criterion.

Boolean-controlled systems via receding horizon and linear programing

We consider dynamic systems controlled by boolean signals or decisions. We show that in a number of cases, the receding horizon formulation of the control problem can be solved via linear programing by relaxing the binary constraints on the control. The idea behind our approach is conceptually easy: a feasible control can be forced by imposing that the boolean signal is set to one at least one time over the horizon. We translate this idea into constraints on the controls and analyze the polyhedron of all feasible controls. We specialize the approach to the stabilizability of switched and impulsively controlled systems.

Mean-field games and dynamic demand management in power grids

This paper applies mean-field game theory to dynamic demand management. For a large population of electrical heating or cooling appliances (called agents), we provide a mean-field game that guarantees desynchronization of the agents thus improving the power network resilience. Second, for the game at hand, we exhibit a mean-field equilibrium, where each agent adopts a bang-bang switching control with threshold placed at a nominal temperature. At equilibrium, through an opportune design of the terminal penalty, the switching control regulates the mean temperature (computed over the population) and the mains frequency around the nominal value. To overcome Zeno phenomena we also adjust the ban…

LPV Model Identification For The Stall And Surge Control of a Jet Engine

Abstract The problem of identifying discrete-time Linear Parameter Varying (LPV) models of non-linear or time-varying systems for gain scheduling control is considered assuming that inputs, outputs and the scheduling parameters are measured, and a form of the functional dependence of the coefficients on the parameters is known. The identification procedure is applied to the controlled model of compressors for jet engines. The model is controlled in order to avoid rotating stall and surge. Aim of the present paper is to identify the LPV model based on the nonlinear model of compressors in order to design a robust gain scheduling predictive controller.

Distributed <inline-formula> <tex-math notation="TeX">$n$</tex-math></inline-formula>-Player Approachability and Consensus in Coalitional Games

We study a distributed allocation process where, at each time, every player: i) proposes a new bid based on the average utilities produced up to that time, ii) adjusts such allocations based on the inputs received from its neighbors, and iii) generates and allocates new utilities. The average allocations evolve according to a doubly (over time and space) averaging algorithm. We study conditions under which the average allocations reach consensus to any point within a predefined target set even in the presence of adversarial disturbances. Motivations arise in the context of coalitional games with transferable utilities (TU) where the target set is any set of allocations that makes the grand …

Bio-inspired evolutionary dynamics on complex networks under uncertain cross-inhibitory signals

Given a large population of agents, each agent has three possiblechoices between option 1 or 2 or no option. The two options are equally favorable and the population has to reach consensus on one of the two options quickly and in a distributed way. The more popular an option is, the more likely it is to be chosen by uncommitted agents. Agents committed to one option can be attracted by those committed to the other option through a cross-inhibitory signal. This model originates in the context of honeybee swarms, and we generalize it to duopolistic competition and opinion dynamics. The contributions of this work include (i) the formulation of a model to explain the behavioral traits of the ho…

Dealing with uncertainty in consensus protocol

Recent results on Consensus protocols for networks are presented. The basic tools and the main contribution available in the literature are considered, together with some of the related challenging aspects: estimation in networks and how to deal with disturbances is considered. Motivated by applications to sensor, peer-to-peer, and ad hoc networks, many papers have considered the problem of estimation in a consensus fashion. Here, the Unknown But Bounded (UBB) noise affecting the network is addressed in details. Because of the presence of UBB disturbances convergence to equilibria with all equal components is, in general, not possible. The solution of the $\epsilon$-consensus problem, where…

Dealing with uncertainty in consensus protocols

Recent results on consensus protocols for networks are presented. The basic tools and the main contribution available in the literature are considered, together with some of the related challenging aspects: estimation in networks and how to deal with disturbances is considered. Motivated by applications to sensor, peer-to-peer, and ad hoc networks, many papers have considered the problem of estimation in a consensus fashion. Here, the Unknown But Bounded (UBB) noise affecting the network is addressed in details. Because of the presence of UBB disturbances convergence to equilibria with all equal components is, in general, not possible. The solution of the e-consensus problem, where the stat…

Finite Alphabet Control of Logistic Networks with Discrete Uncertainty

We consider logistic networks in which the control and disturbance inputs take values in finite sets. We derive a necessary and sufficient condition for the existence of robustly control invariant (hyperbox) sets. We show that a stronger version of this condition is sufficient to guarantee robust global attractivity, and we construct a counterexample demonstrating that it is not necessary. Being constructive, our proofs of sufficiency allow us to extract the corresponding robust control laws and to establish the invariance of certain sets. Finally, we highlight parallels between our results and existing results in the literature, and we conclude our study with two simple illustrative exampl…

Approachability in Population Games

This paper reframes approachability theory within the context of population games. Thus, whilst one player aims at driving her average payoff to a predefined set, her opponent is not malevolent but rather extracted randomly from a population of individuals with given distribution on actions. First, convergence conditions are revisited based on the common prior on the population distribution, and we define the notion of \emph{1st-moment approachability}. Second, we develop a model of two coupled partial differential equations (PDEs) in the spirit of mean-field game theory: one describing the best-response of every player given the population distribution (this is a \emph{Hamilton-Jacobi-Bell…

Learning of Cooperative Behaviour in Robot Populations

This paper addresses convergence and equilibrium properties of game theoretic learning algorithms in robot populations using simple and broadly applicable reward/cost models of cooperation between robotic agents. New models for robot cooperation are proposed by combining regret based learning methods and network evolution models. Results of mean-field game theory are employed in order to show the asymptotic second moment boundedness in the variation of cooperative behaviour. The behaviour of the proposed models are tested in simulation results, which are based on sample networks and a single lane traffic flow case study.

Distributed Consensus in Networks of Dynamic Agents

Stationary and distributed consensus protocols for a network of n dynamic agents under local information is considered. Consensus must be reached on a group decision value returned by a function of the agents' initial state values. As a main contribution we show that the agents can reach consensus if the value of such a function computed over the agents' state trajectories is time invariant. We use this basic result to introduce a protocol design rule allowing consensus on a quite general set of values. Such a set includes, e.g., any generalized mean of order p of the agents' initial states. We demonstrate that the asymptotical consensus is reached via a Lyapunov approach. Finally we perfor…

Online pricing for demand-side management in a low-voltage resistive micro-grid via a Stackelberg game with incentive strategies

It has been demonstrated that online pricing mechanisms are a viable solution for demand side management in power systems. This study deals with the analysis and design of a droop-controlled low-voltage resistive AC micro-grid network system. Such a system is subjected to a dynamic demand obtained from an online pricing mechanism, which is proposed as a novelty in the study of micro-grids. This mechanism is derived from a variation of the Stackelberg game, which includes the use of incentive strategies. First, a configuration in which a supplier announces an incentive function and (Formula presented.) -consumers’ reaction to the resulting personalised price is presented. Then, a detailed st…

Adaptation, coordination, and local interactions via distributed approachability

This paper investigates the relation between cooperation, competition, and local interactions in large distributed multi-agent\ud systems. The main contribution is the game-theoretic problem formulation and solution approach based on the new framework\ud of distributed approachability, and the study of the convergence properties of the resulting game model. Approachability\ud theory is the theory of two-player repeated games with vector payoffs, and distributed approachability is here presented for\ud the first time as an extension to the case where we have a team of agents cooperating against a team of adversaries under local\ud information and interaction structure. The game model turns i…

Lazy consensus for network with unknown but bounded noise

Decomposition and Mean-Field Approach to Mixed Integer Optimal Compensation Problems

Mixed integer optimal compensation deals with optimization problems with integer- and real-valued control variables to compensate disturbances in dynamic systems. The mixed integer nature of controls could lead to intractability in problems of large dimensions. To address this challenge, we introduce a decomposition method which turns the original n-dimensional optimization problem into n independent scalar problems of lot sizing form. Each of these problems can be viewed as a two-player zero-sum game, which introduces some element of conservatism. Each scalar problem is then reformulated as a shortest path one and solved through linear programming over a receding horizon, a step that mirro…

Existence and Optimality of Nash Equilibria in Inventory Games

Abstract This paper studies the stability and optimality of a distributed consensus protocol for n -player repeated non cooperative games under incomplete information. At each stage, the players choose binary strategies and incur in a payoff monotonically decreasing with the number of active players. The game is specialized to an inventory application, where fixed costs are shared among all retailers, interested in whether reordering or not from a common warehouse. The authors focus on Pareto optimality as a measure of coordination of reordering strategies, proving that there exists a unique Pareto optimal Nash equilibrium that verifies certain stability conditions.

Learning for allocations in the long-run average core of dynamical cooperative TU games

We consider repeated coalitional TU games characterized by unknown but bounded and time-varying coalitions' values. We build upon the assumption that the Game Designer uses a vague measure of the extra reward that each coalition has received up to the current time to learn on how to re-adjust the allocations among the players. As main result, we present an allocation rule based on the extra reward variable that converges with probability one to the core of the long-run average game. Analogies with stochastic stability theory are put in evidence.

Networked Bio-Inspired Evolutionary Dynamics on a Multi-Population

We consider a multi-population, represented by a network of groups of individuals. Every player of each group can choose between two options, and we study the problem of reaching consensus. The dynamics not only depend on the dynamics within the group, but they also depend on the topology of the network, so neighboring groups influence individuals as well. First, we develop a mathematical model of this networked bio-inspired evolutionary behavior and we study its steady-state. We look at the special case where the underlying network topology is a regular and unweighted graph and show that the steady-state is a consensus equilibrium. A sufficient condition for exponential stability is given.…

Constrained consensus for bargaining in dynamic coalitional TU games

We consider a sequence of transferable utility (TU) games where, at each time, the characteristic function is a random vector with realizations restricted to some set of values. We assume that the players in the game interact only with their neighbors, where the neighbors may vary over time. The main contributions of the paper are the definition of a robust (coalitional) TU game and the development of a distributed bargaining protocol. We prove the convergence with probability 1 of the bargaining protocol to a random allocation that lies in the core of the robust game under some mild conditions on the players' communication graphs.

About the stability of active queue management mechanisms

In this paper, we discuss the influence of multiple bottlenecks on the stability of active queue management (AQM) controllers, usually configured on a single bottleneck basis. To see this, we consider a network scenario where RED is configured at each router according to previously developed control theoretic techniques. These configuration rules assure stability in a single bottleneck scenario. Yet, we show that instability may arise when two links become congested. We justify this result through a multiple bottleneck model.

Robust optimality of linear saturated control in uncertain linear network flows

We propose a novel approach that, given a linear saturated feedback control policy, asks for the objective function that makes robust optimal such a policy. The approach is specialized to a linear network flow system with unknown but bounded demand and politopic bounds on controlled flows. All results are derived via the Hamilton-Jacobi-Isaacs and viscosity theory.

The Role of Asymptomatic Individuals in the COVID-19 Pandemic <i>via</i> Complex Networks

Background: Recent seroprevalence studies have tried to estimate the real number of asymptomatic cases affected by COVID-19. It is of paramount importance to understand the impact of these infections in order to prevent a second wave. This study aims to model the interactions in the population by means of a complex network and to shed some light on the effectiveness of localised control measures in Italy in relation to the school opening in mid-September.}  Methods: The formulation of an epidemiological predictive model is given: the advantage of using this model lies in that it discriminates between asymptomatic and symptomatic cases of COVID-19 as the interactions with these two categorie…

Bandwagon effect in mean field games

Distributed n-player approachability via time and space average consensus

Abstract In this paper we consider repeated coalitional games with transferable utilities (TU) over networks. Namely, we consider a set of n players that have to distribute among themselves a vector of rewards (one for each player). In our network version there is no coordinator allocating the rewards, but the agents have to agree on a common time-averaged vector by updating the local estimates of the reward vector. The common time-averaged reward vector has to approach a suitable constraint set, called core of the game, that guarantees that no agents benefit from quitting the grand coalition. We propose a doubly (over time and space) averaging distributed algorithm. At every iteration, eac…

Mean-field games and two-point boundary value problems

A large population of agents seeking to regulate their state to values characterized by a low density is considered. The problem is posed as a mean-field game, for which solutions depend on two partial differential equations, namely the Hamilton-Jacobi-Bellman equation and the Fokker-Plank-Kolmogorov equation. The case in which the distribution of agents is a sum of polynomials and the value function is quadratic is considered. It is shown that a set of ordinary differential equations, with two-point boundary value conditions, can be solved in place of the more complicated partial differential equations associated with the problem. The theory is illustrated by a numerical example.

Active queue management stability in multiple bottleneck networks

In this paper, we show that the active queue management (AQM) controllers, usually configured on a single bottleneck basis, may not prevent instability in the presence of multiple bottlenecks. We justify this result through a multiple bottleneck model.

LPV model identification for gain scheduling control: An application to rotating stall and surge control problem

Abstract We approach the problem of identifying a nonlinear plant by parameterizing its dynamics as a linear parameter varying (LPV) model. The system under consideration is the Moore–Greitzer model which captures surge and stall phenomena in compressors. The control task is formulated as a problem of output regulation at various set points (stable and unstable) of the system under inputs and states constraints. We assume that inputs, outputs and scheduling parameters are measurable. It is worth pointing out that the adopted technique allows for identification of an LPV model's coefficients without the requirements of slow variations amongst set points. An example of combined identification…

Opinion dynamics, stubbornness and mean-field games

This paper studies opinion dynamics and stubbornness using mean-field game theory. Assuming an initial exponential density function and affine control policies we analyze under what conditions the Fokker-Planck equation returns an exponential density function over the horizon. Consensus and clusters formation are also studied.

Opinion dynamics in coalitional games with transferable utilities

This paper studies opinion dynamics in a large number of homogeneous coalitional games with transferable utilities (TU), where the characteristic function is a continuous-time stochastic process. For each game, which we can see as a “small world”, the players share opinions on how to allocate revenues based on the mean-field interactions with the other small worlds. As a result of such mean-field interactions among small worlds, in each game, a central planner allocates revenues based on the extra reward that a coalition has received up to the current time and the extra reward that the same coalition has received in the other games. The paper also studies the convergence and stability of op…

Robust linear quadratic mean-field games in crowd-seeking social networks.

We consider a social network where opinions evolve following a stochastic averaging process under the influence of adversarial disturbances. We provide a robust mean-field game model in the spirit of H∞-optimal control, establish existence of a mean-field equilibrium, and analyze its stochastic stability.

Game Theoretic Decentralized Feedback Controls in Markov Jump Processes

This paper studies a decentralized routing problem over a network, using the paradigm of mean-field games with large number of players. Building on a state-space extension technique, we turn the problem into an optimal control one for each single player. The main contribution is an explicit expression of the optimal decentralized control which guarantees the convergence both to local and to global equilibrium points. Furthermore, we study the stability of the system also in the presence of a delay which we model using an hysteresis operator. As a result of the hysteresis, we prove existence of multiple equilibrium points and analyze convergence conditions. The stability of the system is ill…

Consensus for networks with unknown but bounded disturbances

We consider stationary consensus protocols for networks of dynamic agents. The measure of the neighbors' states is affected by unknown but bounded disturbances. Here the main contribution is the formulation and solution of what we call the $\epsilon$-consensus problem, where the states are required to converge in a target set of radius $\epsilon$ asymptotically or in finite time. We introduce as a solution a dead-zone policy that we denote as the lazy rule.

Optimal switches in multi-inventory systems

Given a switched multi-inventory system we wish to find the optimal schedule of the resets to maintain the system in a safe operating interval, while minimizing a function related to the cost of the resets. We discuss a family of instances that can be solved in polynomial time by linear programming. We do this by introducing a set-covering formulation with a totally unimodular constraint matrix.

Dynamic Coalitional TU Games: Distributed Bargaining among Players' Neighbors

We consider a sequence of transferable utility (TU) games where, at each time, the characteristic function is a random vector with realizations restricted to some set of values. The game differs from other ones in the literature on dynamic, stochastic or interval valued TU games as it combines dynamics of the game with an allocation protocol for the players that dynamically interact with each other. The protocol is an iterative and decentralized algorithm that offers a paradigmatic mathematical description of negotiation and bargaining processes. The first part of the paper contributes to the definition of a robust (coalitional) TU game and the development of a distributed bargaining protoc…

Density flow over networks: A mean-field game theoretic approach

A distributed routing control algorithm for dynamic networks has recently been presented in the literature. The networks were modeled using time evolution of density at network edges and the routing control algorithm allowed edge density to converge to a Wardrop equilibrium, which was characterized by an equal traffic density on all used paths. We borrow the idea and rearrange the density model to recast the problem within the framework of mean-field games. The contribution of this paper is three-fold. First, we provide a mean-field game formulation of the problem at hand. Second, we illustrate an extended state space solution approach. Third, we study the stochastic case where the density …

Opinion dynamics and stubbornness through mean-field games

This paper provides a mean field game theoretic interpretation of opinion dynamics and stubbornness. The model describes a crowd-seeking homogeneous population of agents, under the influence of one stubborn agent. The game takes on the form of two partial differential equations, the Hamilton-Jacobi-Bellman equation and the Kolmogorov-Fokker-Planck equation for the individual optimal response and the population evolution, respectively. For the game of interest, we establish a mean field equilibrium where all agents reach epsilon-consensus in a neighborhood of the stubborn agent's opinion.

Opinion dynamics in social networks through mean field games

Emulation, mimicry, and herding behaviors are phenomena that are observed when multiple social groups interact. To study such phenomena, we consider in this paper a large population of homogeneous social networks. Each such network is characterized by a vector state, a vector-valued controlled input, and a vector-valued exogenous disturbance. The controlled input of each network aims to align its state to the mean distribution of other networks' states in spite of the actions of the disturbance. One of the contributions of this paper is a detailed analysis of the resulting mean-field game for the cases of both polytopic and $mathcal L_2$ bounds on controls and disturbances. A second contrib…

Distributed consensus for switched networks with unknown but bounded noise

Control of Production-Distribution Systems under Discrete Disturbances and Control Actions

This paper deals with the robust control and optimization of production-distribution systems. The model used in our problem formulation is a general network flow model that describes production, logistics, and transportation applications. The novelty in our formulation is in the discrete nature of the control and disturbance inputs. We highlight three main contributions: First, we derive a necessary and sufficient condition for the existence of robustly control invariant hyperboxes. Second, we show that a stricter version of the same condition is sufficient for global convergence to an invariant set. Third, for the scalar case, we show that these results parallel existing results in the set…

Consensus via multi-population robust mean-field games

In less prescriptive environments where individuals are told ‘what to do’\ud but not ‘how to do’, synchronization can be a byproduct of strategic thinking,\ud prediction, and local interactions. We prove this in the context of multipopulation\ud robust mean-field games. The model sheds light on a multi-scale\ud phenomenon involving fast synchronization within the same population and\ud slow inter-cluster oscillation between different populations.

Density Flow in Dynamical Networks via Mean-Field Games

Current distributed routing control algorithms for dynamic networks model networks using the time evolution of density at network edges, while the routing control algorithm ensures edge density to converge to a Wardrop equilibrium, which was characterized by an equal traffic density on all used paths. We rearrange the density model to recast the problem within the framework of mean-field games. In doing that, we illustrate an extended state-space solution approach and we study the stochastic case where the density evolution is driven by a Brownian motion. Further, we investigate the case where the density evolution is perturbed by a bounded adversarial disturbance. For both the stochastic a…

Evolutionary Game Dynamics for Collective Decision Making in Structured and Unstructured Environments

Abstract For a large population of players we consider a collective decision making process with three possible choices: option A or B or no option. The more popular option is more likely to be chosen by uncommitted players and cross-inhibitory signals can be sent to attract players committed to a different option. This model originates in the context of honeybees swarms, and we generalise it to accommodate other applications such as duopolistic competition and opinion dynamics. The first contribution is an evolutionary game model and a corresponding new game dynamics called expected gain pairwise comparison dynamics explaining how the strategic behaviour of the players may lead to deadlock…

The stall and surge control problem for jet engines: LPV Model Identification and Gain Scheduling Predictive Control.

Robust Mean Field Games

Recently there has been renewed interest in large-scale games in several research disciplines, with diverse application domains as in the smart grid, cloud computing, financial markets, biochemical reaction networks, transportation science, and molecular biology. Prior works have provided rich mathematical foundations and equilibrium concepts but relatively little in terms of robustness in the presence of uncertainties. In this paper, we study mean field games with uncertainty in both states and payoffs. We consider a population of players with individual states driven by a standard Brownian motion and a disturbance term. The contribution is threefold: First, we establish a mean field syste…

Robust control in uncertain multi-inventory systems and consensus problems

Abstract We consider a continuous time linear multi–inventory system with unknown demands bounded within ellipsoids and controls bounded within polytopes. We address the problem of ∈-stabilizing the inventory since this implies some reduction of the inventory costs. The main results are certain conditions under which ∈-stabilizability is possible through a saturated linear state feedback control. The idea of this approach is similar to the consensus problem solution for a network of continuous time dynamic agents, where each agent evolves according to a first order dynamics has bounded control and it is subject to unknown but bounded disturbances. In this context, we derive conditions under…

Large Networks of Dynamic Agents: Consensus under Adversarial Disturbances

This paper studies interactions among homogeneous social groups within the framework of large population games. Each group is represented by a network and the behavior described by a two-player repeated game. The contribution is three-fold. Beyond the idea of providing a novel two-level model with repeated games at a lower level and population games at a higher level, we also establish a mean field equilibrium and study state feedback best-response strategies as well as worst-case adversarial disturbances in that context.

Impulsively-controlled systems and reverse dwell time: A linear programming approach

We present a receding horizon algorithm that converges to the exact solution in polynomial time for a class of optimal impulse control problems with uniformly distributed impulse instants and governed by so-called reverse dwell time conditions. The cost has two separate terms, one depending on time and the second monotonically decreasing on the state norm. The obtained results have both theoretical and practical relevance. From a theoretical perspective we prove certain geometrical properties of the discrete set of feasible solutions. From a practical standpoint, such properties reduce the computational burden and speed up the search for the optimum thus making the algorithm suitable for th…

Population Games with Vector Payoff and Approachability

This paper studies population games with vector payoffs. It provides a new perspective on approachability based on mean-field game theory. The model involves a Hamilton-Jacobi-Bellman equation which describes the best-response of every player given the population distribution and an advection equation, capturing the macroscopic evolution of average payoffs if every player plays its best response.

Quantized Dissensus in Networks of Agents subject to Death and Duplication

Dissensus is a modeling framework for networks of dynamic agents in competition for scarce resources. Originally inspired by biological cells behaviors, it fits also marketing, finance and many other application areas. Competition is often unstable in the sense that strong agents, those having access to large resources, gain more and more resources at the expense of weak agents. Thus, strong agents duplicate when reaching a critical amount of resources, whereas weak agents die when loosing all their resources. To capture all these phenomena we introduce systems with a discrete time gossip and unstable state dynamics interrupted by discrete events affecting the network topology. Invariancy o…

Generalized person-by-person optimization in team problems with binary decisions

In this paper, we extend the notion of person by person optimization to binary decision spaces. The novelty of our approach is the adaptation to a dynamic team context of notions borrowed from the pseudo-boolean optimization field as completely local-global or unimodal functions and sub- modularity. We also generalize the concept of pbp optimization to the case where the decision makers (DMs) make decisions sequentially in groups of m, we call it mbm optimization. The main contribution are certain sufficient conditions, verifiable in polynomial time, under which a pbp or an mbm optimization algorithm leads to the team-optimum. We also show that there exists a subclass of sub-modular team pr…

Constrained consensus for bargaining in dynamical coalitional TU games

Mixed Integer Optimal Compensation: Decomposition and Mean-Field Approximations

A decentralized solution for the constrained minimum cost flow

In this paper we propose a decentralized solution to the problem of network stabilization, under flow constraints ensuring steady—state flow optimality. We propose a stabilizing strategy for network flow control with capacity constraints which drives the buffer levels arbitrarily close to a desired reference. This is a decentralized strategy optimizing the flow via the minimization of a quadratic cost of the control. A second problem characterized by non-fully connected networks is also considered, for which an exact network equilibrium is not possible. Here, the strategy, in the absence of constraints leads to a least square decentralized problem, but, unfortunately, in the presence of con…

Lazy consensus for networks with unknown but bounded disturbances

We consider stationary consensus protocols for networks of dynamic agents. The measure of the neighbors' state is affected by Unknown But Bounded disturbances. Here the main contribution is the formulation and solution of what we call the isin-consensus problem, where the states are required to converge in a tube of ray isin asymptotically or in finite time.

Crowd-Averse Robust Mean-Field Games: Approximation via State Space Extension

We consider a population of dynamic agents, also referred to as players. The state of each player evolves according to a linear stochastic differential equation driven by a Brownian motion and under the influence of a control and an adversarial disturbance. Every player minimizes a cost functional which involves quadratic terms on state and control plus a cross-coupling mean-field term measuring the congestion resulting from the collective behavior, which motivates the term “crowd-averse.” Motivations for this model are analyzed and discussed in three main contexts: a stock market application, a production engineering example, and a dynamic demand management problem in power systems. For th…

Mean-Field Game Modeling the Bandwagon Effect with Activation Costs

This paper provides a mean-field game theoretic model of the bandwagon effect in social networks. This effect can be observed whenever individuals tend to align their own opinions to a mainstream opinion. The contribution is threefold. First, we describe the opinion propagation as a mean-field game with local interactions. Second, we establish mean-field equilibrium strategies in the case where the mainstream opinion is constant. Such strategies are shown to have a threshold structure. Third, we extend the use of threshold strategies to the case of time-varying mainstream opinion and study the evolution of the macroscopic system.

A two-point boundary value formulation of a mean-field crowd-averse game

Abstract We consider a population of “crowd-averse” dynamic agents controlling their states towards regions of low density. This represents a typical dissensus behavior in opinion dynamics. Assuming a quadratic density distribution, we first introduce a mean-field game formulation of the problem, and then we turn the game into a two-point boundary value problem. Such a result has a value in that it turns a set of coupled partial differential equations into ordinary differential equations.

Consensus in Noncooperative Dynamic Games: a Multi-Retailer Inventory Application

We focus on Nash equilibria and Pareto optimal Nash equilibria for a finite horizon noncooperative dynamic game with a special structure of the stage cost. We study the existence of these solutions by proving that the game is a potential game. For the single-stage version of the game, we characterize the aforementioned solutions and derive a consensus protocol that makes the players converge to the unique Pareto optimal Nash equilibrium. Such an equilibrium guarantees the interests of the players and is also social optimal in the set of Nash equilibria. For the multistage version of the game, we present an algorithm that converges to Nash equilibria, unfortunately, not necessarily Pareto op…

Robust consensus in social networks and coalitional games

We study an n-player averaging process with dynamics subject to controls and adversarial disturbances. The model arises in two distinct application domains: i) coalitional games with transferable utilities (TU) and ii) opinion propagation. We study conditions under which the average allocations achieve robust consensus to some predefined target set.

Quantized Dissensus in switching networks with nodes death and duplication

Opinion Dynamics and Stubbornness via Multi-Population Mean-Field Games

This paper studies opinion dynamics for a set of heterogeneous populations of individuals pursuing two conflicting goals: to seek consensus and to be coherent with their initial opinions. The multi-population game under investigation is characterized by (i) rational agents who behave strategically, (ii) heterogeneous populations, and (iii) opinions evolving in response to local interactions. The main contribution of this paper is to encompass all of these aspects under the unified framework of mean-field game theory. We show that, assuming initial Gaussian density functions and affine control policies, the Fokker---Planck---Kolmogorov equation preserves Gaussianity over time. This fact is t…

The linear saturated decentralized strategy for constrained flow control is asymptotically optimal

We present an algorithm for constrained network flow control in the presence of an unknown demand. Our algorithm is decentralized in the sense that it is implemented by a team of agents, each controlling just the flow on a single arc of the network based only on the buffer levels at the nodes at the extremes of the arc, while ignoring the actions of other agents and the network topology. We prove that our algorithm is also stabilizing and steady-state optimal. Specifically, we show that it asymptotically produces the minimum-norm flow. We finally generalize our algorithm to networks with a linear dynamics and we prove that certain least-square optimality properties still hold.

Stationary and Initial-Terminal Value Problem for Collective Decision Making via Mean-Field Games

Given a large number of homogeneous players that are distributed across three possible states, we consider the problem in which these players have to control their transition rates, following some optimality criteria. The optimal transition rates are based on the players' knowledge of their current state and of the distribution of all the other players, thus introducing mean-field terms in the running and the terminal cost. The first contribution is a mean-field model that takes into account the macroscopic and the microscopic dynamics. The second contribution is the study of the mean-field equilibrium resulting from solving the initial-terminal value problem, involving the Kolmogorov equat…

Robust Dynamic Cooperative Games