All articles
    multi-agent
    ai-safety
    governance
    trajectory
    agentic-ai
    mission-runtime
    context-runtime
    ai-runtime

    AI Agents Can All Follow the Rules and Still Cause a Disaster

    August 17, 2026redevops.io
    AI Agents Can All Follow the Rules and Still Cause a Disaster

    Multi-agent systems introduce a class of failure that ordinary AI safety controls are poorly positioned to see: every individual action can look acceptable while the behavior of the agent population becomes unsafe.

    Recent multi-agent research has demonstrated the shape of the problem. Agents given similar context can collapse onto the same behavior. Uncoordinated agents can flood infrastructure with millions of requests. Agents with communication channels can collude. Agents pursuing incompatible goals can escalate against one another. A harmful objective can also be fragmented across agents so that each subtask appears harmless in isolation.

    Permissions and per-action policy are still necessary, but they operate at the wrong resolution for these failures.

    From action governance to trajectory governance

    Individually permissible actions can form an unsafe trajectory
    Scroll the diagram sideways to read itFull size

    For the ReDevOps v0.2.0 release, we're introducing a Governance / Trajectory Plane: a first-class layer above existing permissions and per-action Mission policy.

    The distinction is simple:

    • Permissions Plane: may this principal access this resource?
    • Mission Runtime: may this action execute?
    • Governance Plane: what are these agents collectively doing across missions and over time?

    The foundation is one normalized, append-only event history. A live trajectory engine can correlate events across agents and missions, then evaluate patterns that no individual action reveals.

    That enables concrete population-level controls: spawn and budget limits for runaway agent replication; diversity and deduplication guards for conformity collapse; governed coordination records; cross-agent agreement rules; mandatory review for destructive inter-agent actions; and authority-weighted evidence when apparent consensus conflicts with stronger information.

    Enforcement remains explicit. A surface capable of synchronous gating can DENY or REQUIRE_REVIEW. A surface that cannot block an action must report OBSERVE, because detection does not equal prevention. Findings can flow into Mission Runtime's existing approval, compensation, event-sourced replay and investigation mechanisms.

    The missing question

    As agent systems become larger, asking "is this action allowed?" is no longer enough.

    We also need to ask:

    What system is emerging from all these individually allowed actions?

    That is the problem the ReDevOps Governance Plane is being built to address.

    ReDevOps is building source-available runtimes (AGPL-3.0 with the Commons Clause) for reliable agentic software: Context Runtime governs what agents know, Mission Runtime governs how work executes, and the Governance Plane governs behavior that emerges across the agent population.

    Want ReDevOps in your Google Top Stories & AI results?Follow ReDevOps on Google