All articles
    integrations
    agent-frameworks
    nvidia
    pydanticai
    llamaindex
    crewai
    azure
    google
    aws
    digitalocean
    mission-runtime
    governance
    reference-implementation
    benchmarks

    Keep Your Framework. Add the Runtime. Eight Times Over.

    September 1, 2026redevops.io
    Keep Your Framework. Add the Runtime. Eight Times Over.

    There is a tired way to pitch an agent runtime: line it up against LangGraph and CrewAI and NeMo, run one benchmark, and declare a winner. We think that framing is wrong, because it answers a question nobody productionizing an agent is actually asking. The pitch asks which agent framework should I marry. The question is now that I've built an agent, how do I run it in production without rebuilding context, retries, approval, replay, authority, and audit for the hundredth time.

    So instead of one comparison, we built eight reference implementations (one per framework and cloud people actually ship on) and in each one we kept the framework's native agent loop completely intact and added the ReDevOps Mission Runtime around it. The point of doing it eight times was to show the runtime is provider-independent: the same common/ adapter contract, the same Mission semantics, the same honesty about what's measured, across NVIDIA NeMo, PydanticAI, LlamaIndex, CrewAI, Azure Semantic Kernel, Google ADK/A2A, AWS Strands, and DigitalOcean GenAI.

    All of it is public: github.com/redevops-io/redevops-integrations, and the write-ups live at redevops.io/integrations.

    The rules we held ourselves to

    Four constraints made this a conformance exercise.

    Real agents. Every slice constructs and runs a genuine agent from that framework's own SDK: a real nvidia-nat Function, a real pydantic-ai typed Agent, a real llama-index VectorStoreIndex, a real crewai Crew, a real Semantic Kernel Kernel, a real Google ADK Agent, a real AWS Strands Agent, a real DigitalOcean GenAI client. No mocks, no sidecar approximations.

    Local, no cloud accounts. Every one runs on your own hardware. The four hyperscaler slices need no cloud credentials: the SDKs point at any OpenAI-compatible endpoint, and cloud identity (Entra, IAM, tenants) is modelled as policy the runtime evaluates. You can reproduce the governance semantics without an AWS, Azure, Google or DigitalOcean bill.

    One contract. Adding a framework is a conformance exercise on one architecture. All eight share integrations/common/ (the adapter, the acceptance-bundle format, and a five-way classification for every comparison: PARITY, FRAMEWORK_NATIVE_ADVANTAGE, REDEVOPS_DELTA, EXPECTED_IMPLEMENTATION_DIFFERENCE, BUG). We publish the full tally each time. It is a map of where the runtime adds value.

    A different boundary each time. This was the hardest and most important rule. Each framework has a strongest abstraction, and the interesting test is the boundary immediately outside it, never the same experiment with a new logo.

    Eight frameworks, eight boundaries

    The agent frameworks, where the differentiation is what the runtime adds around the agent loop:

    • NVIDIA NeMo owns the agent loop; the boundary is context completeness. Closure-aware context lifted scientific verification 0.800 → 0.867 and, on structural regulatory text, 0.412 → 0.913, with a governed action that fired exactly once and left zero duplicates on replay.
    • PydanticAI owns the typed boundary; the boundary is validity versus admissibility. A schema-valid output is not automatically supported, fresh, authorized, or policy-admissible. Four perfectly-typed outputs, correctly refused; one admitted.
    • LlamaIndex owns retrieval; the boundary is similar versus complete. A retriever finds what resembles the query; it misses a code change's structural closure. Composing the retriever with the call graph roughly doubled closure recall, 0.320 → 0.693, and on a long-context query, verification abstained on 6 of 6 unsupported answers a fluent query engine would have stated as fact.
    • CrewAI owns delegation; the boundary is authority under handoff. A delegated agent's grants are the intersection of the chain, so 3 of 3 authority-widening actions were denied, with no pooling across the crew.

    The hyperscalers, where the differentiation is composing Mission semantics with the cloud's own identity and policy:

    • Azure Semantic Kernel: the boundary just outside function-calling. A deploy is admitted only when every compliance control is evidenced and the Entra role permits it; authority composes as deny-wins. 3 of 3 non-compliant deploys denied.
    • Google ADK / A2A: the boundary just outside a multi-hop mesh (CrewAI: the single hop). Authority is the intersection of the whole chain, so a grant dropped upstream can't return three hops later; and a finding enters the synthesis only if its provenance is attributable. 2/2 transitive escalations denied, 2/2 unprovenanced findings excluded.
    • AWS Strands: the boundary defined by IAM's own rule: an explicit Deny overrides any Allow. Mission authority composes with an SCP guardrail and the role's IAM policy, and the Wealth Manager runs in shadow first (500 households, zero side effects) before anything executes. A real agent's "withdraw all cash" proposal was denied by the guardrail before the money moved.
    • DigitalOcean GenAI: the shared-runtime thesis. Three real apps on one runtime, tenant-isolated: 3/3 cross-tenant accesses denied, and the production machinery provided once instead of rebuilt fifteen times across the three apps.

    What repeats, and why that's the point

    Read the eight and the same words keep surfacing: authority, approval, replay, verification, closure, governance, telemetry. That recurrence is the argument. They are properties every production agent needs, across NeMo and PydanticAI and AWS, and none of these excellent frameworks provides on its own. They are the runtime's job.

    That is why the tally each time skews toward REDEVOPS_DELTA with a few honest PARITY and EXPECTED_IMPLEMENTATION_DIFFERENCE rows and zero BUGs: the runtime is adding a layer the framework deliberately doesn't own, and the layer it does stays. Where a framework already does the job (LlamaIndex's retrieval when similarity suffices, a crew's own accurate decision), the runtime composes with it and leaves it alone.

    We were also careful about what we claim. The flagship results are deterministic where they can be (authority, admissibility, deny-wins, isolation are mechanisms, not model behaviours); the LLM-driven results are cached and honest about small samples; Azure runs in its own virtual environment because Semantic Kernel pins dependencies below the others' floor; and the reproduction notes say plainly which examples need a dataset fetch or an external repo. The frozen acceptance bundles ship in the public repo so the evidence stands even where a run needs setup.

    Keep your framework

    If there's one line to take from all eight, it's the one in the title. You do not have to bet your production system on picking the perfect agent framework, because the choice is impermanent. Applications change. Models and retrievers change. The cloud changes. The governed Mission (its objective, authority, evidence, replay, verification and outcome) does not have to. Keep the framework you like. Add the runtime underneath. We've now shown it works eight different ways, and left the code public so you can check.


    The full catalog (architecture strips, benchmark tables, acceptance hashes and reproduce commands for all eight) is at redevops.io/integrations; the code is on GitHub; and each framework's deep-dive is linked above. All of it builds on the Mission SDK.

    Want ReDevOps in your Google Top Stories & AI results?Follow ReDevOps on Google