All articles
    retail
    inventory
    mission-runtime
    context-runtime
    benchmarks
    m5
    walmart
    composition

    Should We Reorder This SKU? A Retail Restocking Benchmark on Real Walmart M5 Data

    July 27, 2026redevops.io
    Should We Reorder This SKU? A Retail Restocking Benchmark on Real Walmart M5 Data

    Restocking looks solved until you stand in front of the shelf. The inventory system says twelve units on hand; the shelf is empty. Do you reorder, or escalate this as a data problem: a drifted count, a mis-scan, an unlogged theft? Reorder blindly and you double-stock a phantom; escalate blindly and you stock out a fast mover during a promotion. The decision is a reconciliation across three different representations of the same physical shelf, each wrong in a different way.

    This is the retail instance of a lesson we keep hitting: the answer is not always in the representation you queried. The camera sees shelf reality (a multimodal observation), the inventory system holds system on-hand (an analytical fact), the point-of-sale log carries sales velocity (a temporal signal). A restock mission has to compose all three.

    This benchmark is not intended to measure computer-vision accuracy. It measures whether composing multiple representations produces better operational decisions than any single one, so the camera is intentionally imperfect, and the interesting scenarios are the ones where the three signals disagree.

    The benchmark: MissionBench-Retail

    The mission is built around a published benchmark: 204 scenarios over the real M5 sales data (the denephew/M5_Forecasting mirror of the Kaggle M5-Accuracy competition). Each scenario poses one question (reorder this SKU, or escalate a shelf-vs-system disagreement as a data problem) with known ground truth.

    What is real, measured per SKU straight from M5:

    sales velocity     trailing 28-day mean of true daily units
    price elasticity    demand ratio at the SKU's low vs high real sell_price
                        (elastic SKUs sell 1.4x-2.2x more units at the promo price)
    event uplift        from the M5 calendar; only SKUs with a genuine >=1.4x uplift
    

    What is modelled and injected on top: the on-hand inventory number and the shelf-camera reading, deliberately allowed to lie. Those injected disagreements are the controlled cases: how the benchmark isolates the value of composition instead of rewarding a decider for getting easy, agreeing scenarios right.

    Three deciders

    decider                                 accuracy   real stockouts missed
    --------------------------------------  ---------  ---------------------
    naive (inventory-only)                    31.4%    84
    composed (camera + trailing velocity)     54.9%    (velocity-blind)
    composed+ (reconcile all three + model)   96.1%    0
    

    The naive inventory-only decider trusts the system number and nothing else: 31.4%, missing 84 real stockouts: every case where the system said stocked and the shelf was empty. That phantom stockout is why inventory-only restocking quietly bleeds sales.

    The composed decider trusts the camera plus a trailing velocity estimate and reaches 54.9%. Two things sink it: it is velocity-blind, missing every promotion and event demand spike the M5 calendar and price signals produce, and it over-trusts a noisy camera allowed to be wrong. An honest note: an earlier, easier version scored this same decider at 100%: an artifact, because it never tested the hard cases (no real M5 calendar, no price elasticity, no fallible camera). Adding those signals dropped it from 100% to 55%.

    The composed+ decider reconciles camera x inventory x POS by corroboration, backed by a real calendar-and-price demand model. It reaches 96.1% and misses zero real stockouts: it reads the same calendar and elasticity the shelf will feel, and takes no single signal on faith.

    The 3.9% we do not get

    96.1% is not 100%, and we report the gap. Eight scenarios are NOISE_TRAP cases: a camera glitch coinciding with a genuine demand lull is observationally identical to a real stockout. No amount of reconciliation can separate them, because the three signals agree on the wrong thing; the only way to break the tie is a physical recount. A model that claimed to solve these would be lying, as the earlier 100% did.

    Composition removes the failure modes that come from trusting one representation; it does not remove the ones where reality itself is under-observed.

    Perception ingress: snapshots over a video firehose

    Getting shelf reality into the mission does not require a continuous NVIDIA DeepStream video pipeline. The recommended default is an Edge Snapshot and Perception Service: scheduled still images pulled from existing CCTV and camera APIs, run through shelf and SKU inference (TensorRT/Triton for acceleration where available), emitting structured observations. Sampling is adaptive:

    normal shelf                 every 5 min
    high-velocity SKU            every 1-2 min
    store closed                 every 30-60 min
    immediately after a restock  one confirming snapshot
    low-confidence reading       retry
    

    DeepStream stays available as an optional adapter for continuous streaming or shopper/product tracking, but it is not the price of entry: most stores already have the cameras, and the mission needs observations.

    Honesty and licensing

    The demand and its dynamics (velocity, elasticity, event uplift) are real M5. The on-hand inventory and shelf camera are a disclosed model, and the injected disagreements are controlled cases that isolate composition value. We claim the hard part, the demand, is field data.

    The M5 data is used under Kaggle competition terms. It is Walmart-proprietary, not a permissive open license, and we use it for non-commercial research only; it is never redistributed and never part of a customer deployment. If you deploy a restock mission, it runs on your own POS and camera feeds, not on M5. Please cite Makridakis, Spiliotis and Assimakopoulos, "The M5 Competition" (2022).

    Decision accuracy of the three deciders on 204 real M5 scenarios: inventory-only 31.4 percent with 84 stockouts missed, trust-the-camera 54.9 percent, and composed+ 96.1 percent with zero stockouts missed and 8 NOISE_TRAP blind spots
    Scroll the diagram sideways to read itFull size

    The point

    A single system gives you one story about the shelf and no way to know when it's wrong. The value, beyond headline accuracy, is what composition catches that no single representation can, reported honestly, blind spot included. The value of Context Runtime is selecting, reconciling, and explaining the right combination of representations for a decision, beyond retrieving more information.

    MissionBench-Retail is the retail entry in a growing family: MissionBench-KYC, MissionBench-Insurance, MissionBench-Support, MissionBench-Compliance, each measuring whether composing representations into a governed mission beats trusting any single one.

    #Retail #InventoryManagement #ComputerVision #EdgeAI #MissionRuntime #ContextRuntime #Benchmarks #M5 #Walmart #Composition #AIInfrastructure #TensorRT #RetailTech #DecisionIntelligence

    Want ReDevOps in your Google Top Stories & AI results?Follow ReDevOps on Google