Differences

This shows you the differences between two versions of the page.

--- bmas [2026/06/19 15:40] – pedroortega
+++ bmas [2026/06/20 13:18] (current) – [Safe AI Should be Bounded and Multi-Agent] pedroortega
@@ Line 1: / Line 1: @@
 ====== Safe AI Should be Bounded and Multi-Agent ======
-**David Hyland, Daniel Jarne Ornia, Nicholas Bishop, Joel Dyer, Olivia Macmillan-Scott, Tomáš Gaveniak, Ani Calinescu, Michael Wooldridge, Fernando Rosas, Pedro Ortega**\
+**David Hyland, Daniel Jarne Ornia, Nicholas Bishop, Joel Dyer, Olivia Macmillan-Scott, Tomáš Gaveniak, Ani Calinescu, Michael Wooldridge, Fernando Rosas, Pedro A. Ortega**
-//Keywords: AI safety, bounded agency, multi-agent systems, modularity, verification, governance. //\
-Position Paper\
-June 2026\
+//Keywords: AI safety, bounded agency, multi-agent systems, modularity, verification, governance. //
+{{ bmas_position.pdf | Position Paper, }} June 2026
 ===== Abstract =====
 The scaling paradigm treats bounds on compute, memory, information, authority, and affordances as obstacles. This paper argues that these bounds are also design variables. A **bounded agent** is an agent whose information, architecture, resources, or affordances limit its ability to optimise its objective. A **bounded multi-agent system** (BMAS) composes such agents through explicit interfaces so that system-level capability is obtained through decomposition, delegation, verification, and coordination.
-The safety claim is architectural. If unsafe behaviour requires a conjunction of capabilities, then risk can be reduced by separating those capabilities across components and controlling the interfaces that recombine them. BMAS therefore turns some global safety problems into local specification, monitoring, and verification problems. This does not eliminate multi-agent risk; it makes the relevant risk-bearing structures explicit.
+The safety claim is architectural. If unsafe behaviour requires a conjunction of capabilities, then risk can be reduced by separating those capabilities across components and controlling the interfaces that recombine them. BMAS therefore turns some global safety problems into local specification, monitoring, and verification problems. The approach does not eliminate  multi-agent risk, but it makes the relevant risk-bearing structures explicit.
 ===== Introduction =====
@@ Line 18: / Line 18: @@
 BMAS starts from a different decomposition. A task induces a capability profile: reasoning, knowledge, coding, planning, tool use, verification, and communication may all be required in different proportions. Unsafe behaviour also has a capability profile. Risk is highest when the capabilities required for task completion and the capabilities required for harm are colocated in one agent.
-The BMAS proposal is to design systems in which useful capability is distributed across bounded components. A planner plans. A retriever retrieves. A coder writes code. A verifier checks outputs. A monitor inspects interactions. An executor acts under controlled authority. The system becomes capable through composition, while each component has a restricted scope.
+The BMAS proposal is to design systems in which useful capability is distributed across bounded components. A planner plans. A retriever retrieves. A coder writes code. A verifier checks outputs. A monitor inspects interactions. An executor acts under controlled authority. And so forth. The system becomes capable through composition, while each component has a restricted scope.
 ===== Bounded agents =====