The full, reproducible evidence behind the integration page. A real nvidia-nat 1.8.0 agent, run in-process with the Mission SDK and Closure Resolution, measured on SciFact + GDPR + a governed action. Every comparison is classified — this is conformance, not a scorecard.
Frozen environment
framework nvidia-nat 1.8.0 (NeMo Agent Toolkit) — real Function via WorkflowBuilder.ainvoke
runtime Mission SDK 0.2.0a0 · Context Runtime 0.2.0 — one Python 3.12 env, in-process
answer model gpt-4o-mini (OpenAI-compatible) — identical across native and +ReDevOps arms
datasets SciFact (allenai/scifact) · GDPR cross-reference graph (GDPRtEXT)
budget SciFact 300 tok · GDPR 8000 tok (matched; the gold closure fits — budget is not the variable)
scoring closure recall vs gold spans · verdict accuracy vs human labels · exactly-once side-effect witness
Results — NV-A through NV-E
Experiment
Native role
ReDevOps addition
Result
Classification
NV-A SciFact
NeMo agent + evaluator
Closure Resolution
0.800 → 0.867
REDEVOPS_DELTA
NV-B GDPR
NeMo agent
structural closure
0.412 → 0.913
REDEVOPS_DELTA
NV-C Optimization
NeMo optimization
Mission/context optimization
semantics invariant
EXPECTED_IMPLEMENTATION_DIFFERENCE
NV-D Governed tool
NeMo proposes action
authority · replay
0 unauthorized · 0 duplicates
REDEVOPS_DELTA
NV-E Telemetry
NeMo telemetry
Mission-root nesting
preserved (nested)
EXPECTED_IMPLEMENTATION_DIFFERENCE
NV-A · scientific evidence (SciFact)
A NeMo agent verifies claims (Supported / Refuted / NotEnoughInfo) given retrieved evidence. Same model, same budget, two context sources. Verification accuracy naive 0.800 → closure 0.867; evidence recall 0.841 → 0.877. Treating the two context views as a cheap ensemble (agree ⇒ resolve, disagree ⇒ REQUIRE_REVIEW) gives selective autonomy of 0.857 @ 93% coverage. SciFact is content-saturated — semantic retrieval already finds most evidence — so the gain is real but modest.
NV-B · regulatory structure (GDPR)
The same closure mechanism on a structural domain: editing a GDPR article requires every article it cross-references, and cross-referenced articles are not textually similar. Content-only closure recall 0.412; composed (content + cross-reference graph) 0.913; seed-only 0.000 → 0.913. Deterministic — no LLM calls — so this isolates the retrieval value ReDevOps adds beneath the agent.
NV-D · governed action
A NeMo agent proposes a consequential action; it runs only through a Mission. Missing grant ⇒ 0 unauthorized actions. Grant but no approval ⇒ parked at a human gate, 0 side effects. Approved ⇒ fires exactly once. Replay of the sealed run ⇒ 0 duplicate side effects (reconstructed from the event ledger, not re-executed). The result that matters most for production is behaviour, not an accuracy delta.
Reproduce
git clone https://github.com/redevops-io/redevops-integrations
cd redevops-integrations/integrations/nvidia
../../.venv/bin/python conformance.py # NV-A..E + classification tally
../../.venv/bin/python examples/scifact_closure.py # NV-A (gpt-4o-mini, cached)
../../.venv/bin/python examples/gdpr_structural.py # NV-B (no LLM)
../../.venv/bin/python examples/governed_tool.py # NV-D (real NeMo proposer, no LLM in the gate)
Acceptance bundles
Each run writes a reproducibility record — framework version, model ids, dataset hashes, per-finding classification, and a content-addressed result digest.
// conformance summary NVIDIA suite over all findings: 10 REDEVOPS_DELTA · 1 EXPECTED_IMPLEMENTATION_DIFFERENCE · 0 BUG. The one non-delta is NV-E: NeMo's telemetry nests under the Mission causal node — semantics preserved, shape differs. That is expected, not a win and not a bug.
Limitations. One answer model, one budget per domain, small samples (SciFact n=30, GDPR 27 articles); the managed NeMo Platform (registration/deployment) is not exercised — only the local NAT toolkit runs, for real. The SciFact selective-autonomy ensemble is two context views, not independent judges. Reported straight; nothing averaged into a single headline.