The documentation files for the MARL toolbox (4 August 2010, 223.1 KBytes). Also included is a demonstration script illustrating the experiments reported in the chapter. The latest version, 1.3, adds the Distributed Q-learning algorithm and the new 'robotic rescue' gridworld environment used in the example of our survey chapter Multi-Agent Reinforcement Learning: An Overview (where the problem was described more generically as 'object transportation'). Everything is written for the generic n-agent case, except minimax-Q, which is most meaningful in the two-agent case. Several types of gridworld-based environments are implemented, and agents can learn using a set of algorithms among which single-agent Q-learning, team Q-learning, minimax-Q, WoLF-PHC and an adaptive state expansion algorithm developed by us. The learning, action selection and exploration methods can be independently plugged into the agents' behaviour. Currently the toolbox supports only episodic environments, but hooks are in place for continuing tasks as well. The toolbox is developed with modularity in mind, separating for instance the agent behaviour from the world engine and the latter from the rendering GUI. Since no Matlab toolbox for dynamic multi-agent tasks was available when I started my PhD project, I started writing one of my own. We prefer Matlab for its ease of use with numeric computations and its rapid prototyping facilities. The Multi-Agent Reinforcement Learning toolbox is a package of Matlab functions and scripts that I used in my research on multi-agent learning. Multiagent consensus using optimistic optimization ( ooconsensus), and as a side-benefit the DOO and SOO algorithms of Remi Munos ( doosoo).Ī Matlab multi-agent reinforcement learning toolbox (4 August 2010, 336.9 KBytes).Standard linear consensus and flocking protocols. Multiagent tasks: linear agents and robot-arm agents. Multiagent planning ( magenmpc, maopd_.), with specific focus on consensus problems (although generalizable).
#Matlab 2019a reinforcement learning toolbox code#
I have also included code related to my recent forays into cooperative multiagent control: See also the description and documentation for the previous version of the toolbox. I have left in the system directories functions and scripts for many experiments I have run, in case they are useful. Things should be largely backward-compatible with the old version, if you encounter trouble let me know. The same standardized task interface is followed as before, with some extensions. These are very simple tasks useful to explain or experiment with DP and RL. Two new problems: machine replacement (as described by Bertsekas) and gridworld navigation. For classical, discrete RL: the Monte-Carlo and Dyna-Q implementations are new Q-learning and SARSA now support experience replay.Additional demonstration scripts, including one for planning and another focused on least-squares types of policy iteration.New simulation tasks include notably a resonating robot arm (where a spring is used to make the motion more energy efficient), and a simple navigation problem in 2D.Two online RL implementations compatible with this interface are rtapproxqlearn and rtlspionline. Another example is the implementation for the EdRo robot. A standardized interface for real-time control problems, see e.g.See the /batch subdirectory, and as examples the batch experiment files left in system directories, such as op_ip. An extensive mechanism for running batch experiments (testing an algorithm with a grid of parameters and inspecting results).Fitted Q-iteration with local linear regression approximation.OPD and OP-MDP can be used while applying longer sequences of actions / longer tree policies. The entry point connecting planning to the system is genmpc. Online, optimistic planning algorithms: for deterministic systems ( opd), for discrete Markov decision processes ( opss), with continuous actions ( sooplp), open-loop optimistic planning ( olop), and hierarchical OLOP ( holop).So expect undocumented behavior, bugs, but also plenty of new algorithms – hic sunt leones! Be warned though: this is very much work-in-progress, a snapshot of the code that I use for my daily research. Since the previous release of the toolbox was getting rather old, I decided to publish a new version.
0 kommentar(er)
