Decentralized linear quadratic systems with major and minor agents and non-Gaussian noise

by   Mohammad Afshari, et al.

We consider a decentralized linear quadratic system with a major agent and a collection of minor agents. The agents are coupled in their dynamics as well as a quadratic cost. In particular, the dynamics are linear; the state and control action of the major agent affect the state evolution of all the minor agents but the state and the control action of the minor agents do not affect the state evolution of the major or other minor agents. The system has partial output feedback with partially nested information structure. In particular, the major agent perfectly observes its own state while each minor agent perfectly observes the state of the major agent and partially observes its own state. It is not assumed that the noise process has a Gaussian distribution. For this model, we characterize the structure of the optimal and the best linear strategies. We show that the optimal control of the major agent is a linear function of the major agent's MMSE (minimum mean squared error) estimate of the system state and the optimal control of a minor agent is a linear function of the major agent's MMSE estimate of the system state and a "correction term" which depends on the difference of the minor agent's MMSE estimate of its local state and the major agent's MMSE estimate of the minor agent's local state. The major agent's MMSE estimate is a linear function of its observations while the minor agent's MMSE estimate is a non-linear function of its observations which is updated according to the non-linear Bayesian filter. We show that if we replace the minor agent's MMSE estimate by its LLMS (linear least mean square) estimate, then the resultant strategy is the best linear control strategy. We prove the result using a direct proof which is based on conditional independence, splitting of the state and control actions, simplifying the per-step cost, orthogonality principle, and completion of squares.


Deep Structured Teams in Arbitrary-Size Linear Networks: Decentralized Estimation, Optimal Control and Separation Principle

In this article, we introduce decentralized Kalman filters for linear qu...

Multi-Agent Reinforcement Learning via Mean Field Control: Common Noise, Major Agents and Approximation Properties

Recently, mean field control (MFC) has provided a tractable and theoreti...

Thompson sampling for linear quadratic mean-field teams

We consider optimal control of an unknown multi-agent linear quadratic (...

Tracking Control foe Multi-Agent Systems Using Broadcast Signals Based on Positive Realness

Broadcast control is one of decentralized control methods for networked ...

Mathematical Foundations of Complex Tonality

Equal temperament, in which semitones are tuned in the irrational ratio ...

Probabilistic inverse optimal control with local linearization for non-linear partially observable systems

Inverse optimal control methods can be used to characterize behavior in ...

Sequential Decision Making on Unmatched Data using Bayesian Kernel Embeddings

The problem of sequentially maximizing the expectation of a function see...

Please sign up or login with your details

Forgot password? Click here to reset