Why local. The stack talks to one OpenAI-compatible model endpoint — nothing about it requires a vendor's cloud. Run the model on a laptop for evaluation, on a workstation with a GPU for real work, or across a few trusted machines that share one database and one model. Your data and your prompts stay on hardware you control; the only thing that leaves the box is what you deliberately wire to an external tool.
Install the stack first. The per-OS (Windows · macOS · Ubuntu) and cloud steps live on the install pages — Core install for the runtimes + control plane, then Deploy the business apps. Both read the model endpoint from REDEVOPS_LLM_BASE_URL in .env. Come back here to make that endpoint a model on your own hardware.
The stack needs one OpenAI-compatible endpoint — that is the whole contract, and anything that speaks that API works. Set it once in the stack's .env:
# .env — the one line that matters for AI REDEVOPS_LLM_BASE_URL=http://host.docker.internal:11434/v1 # a local Ollama, for example
Here are the local options, cheapest-effort first.
Ollama is the simplest: install it, pull a model, and it serves an OpenAI-compatible API on :11434.
ollama pull qwen2.5:7b-instruct # ~5 GB; runs on CPU or any recent GPU ollama serve # serves http://localhost:11434/v1 # then in the stack's .env (containers reach the host via host.docker.internal): REDEVOPS_LLM_BASE_URL=http://host.docker.internal:11434/v1
Have a spare box with a GPU? NVIDIA Personal AI Router (PAIR) routes inference across trusted local machines — the workstation serves the model, the laptop runs the stack, and PAIR is the fabric between them. ReDevOps ships a thin provider adapter over PAIR, so the stack sees it as just another endpoint. Set REDEVOPS_LLM_BASE_URL to the PAIR router's OpenAI-compatible URL and every agent's inference lands on the GPU box — nothing leaves your network.
For a larger or quantized model on a workstation GPU, serve it with vLLM (also OpenAI-compatible) and point the stack at it:
vllm serve Qwen/Qwen2.5-7B-Instruct --port 8001
REDEVOPS_LLM_BASE_URL=http://host.docker.internal:8001/v1
If you leave REDEVOPS_LLM_BASE_URL empty, onboarding walks you through a free-tier option so you can see the stack work before you commit hardware — then swap in a local endpoint above whenever you're ready. Resolution order is deliberate: your own endpoint → PAIR on your network → a model on this machine → a guided fallback. The stack never silently phones a cloud you didn't choose.
Local doesn't mean single-machine. Several installs can act as one system: point them at a shared database and a shared model endpoint, and they coordinate through the same durable event log — one identity, one evidence trail, one replayable history across all of them.
REDEVOPS_LLM_BASE_URL — one GPU box (or PAIR) serves them all; the laptops don't each need a GPU.# .env on every instance that should join the same system
POSTGRES_DSN=postgresql://redevops:...@db-host:5432/agentic
REDEVOPS_LLM_BASE_URL=http://gpu-host:8001/v1
curl $REDEVOPS_LLM_BASE_URL/models lists your model. If this fails, nothing else will — fix the endpoint first.docker compose ps shows every service healthy/running.| Symptom | Cause & fix |
|---|---|
| Agents error on every request | The model endpoint is unreachable. curl $REDEVOPS_LLM_BASE_URL/models from the host; if a container can't reach a host model, use host.docker.internal (Docker Desktop) or --add-host=host.docker.internal:host-gateway (Ubuntu). |
| GPU not used by a local model | On Ubuntu, install the NVIDIA Container Toolkit; on macOS, run the model on the host (Ollama/LM Studio) — Docker can't pass the Apple GPU into a container. |
| Out of memory | The model, not the stack, is the hog. Use a smaller/quantized model, or move it to another box via PAIR and keep only the stack local. |
| Slow model calls on Windows | Run the stack inside WSL2 with the repo on the Linux filesystem (~/agentic-os-stack), not /mnt/c/… — bind-mount performance across the Windows filesystem is much slower. |
Questions, or stuck on a step? Ask in #agentic-apps on the ReDevOps Slack, or email info@redevops.io — the team and community help with deploys, connectors, sources and governance there.