Phase 6: Ollama load balancing across n97-2 and n97-3 #7

New issue

Open

opened 2026-05-04 19:40:27 +00:00 by agent-cos · 1 comment

agent-cos commented

2026-05-04 19:40:27 +00:00

Goal

Configure OpenClaw and other consumers to use both Ollama nodes with load balancing.

Tasks

Verify Ollama running on n97-2: curl http://100.x.x.x:11434/api/tags
Verify Ollama running on n97-3: curl http://100.x.x.x:11434/api/tags
Create k8s Service for Ollama (headless or with external IPs)
OR configure OpenClaw with multiple Ollama endpoints in openclaw.json
Test round-robin: curl different models on different nodes
Verify: deepseek-v4-pro on n97-2, qwen3-coder on n97-3
Load test: simultaneous requests to both nodes

Acceptance

Both Ollama nodes accessible
Requests distributed across nodes (manual config or via k8s)
Each node runs distinct models (no duplication)
n97-1 no longer needs local Ollama (can still run if desired)

Depends On

#3 (n97-2 and n97-3 must be bootstrapped)

Notes

Ollama stays bare metal (systemd services) — just load balance across them
Alternative: Put Ollama in k8s with nodeAffinity, but GPU/NPU passthrough is complex
For now: Configure OpenClaw with array of endpoints, let it round-robin

## Goal Configure OpenClaw and other consumers to use both Ollama nodes with load balancing. ## Tasks - [ ] Verify Ollama running on n97-2: `curl http://100.x.x.x:11434/api/tags` - [ ] Verify Ollama running on n97-3: `curl http://100.x.x.x:11434/api/tags` - [ ] Create k8s Service for Ollama (headless or with external IPs) - [ ] OR configure OpenClaw with multiple Ollama endpoints in `openclaw.json` - [ ] Test round-robin: `curl` different models on different nodes - [ ] Verify: `deepseek-v4-pro` on n97-2, `qwen3-coder` on n97-3 - [ ] Load test: simultaneous requests to both nodes ## Acceptance - Both Ollama nodes accessible - Requests distributed across nodes (manual config or via k8s) - Each node runs distinct models (no duplication) - n97-1 no longer needs local Ollama (can still run if desired) ## Depends On - #3 (n97-2 and n97-3 must be bootstrapped) ## Notes - Ollama stays bare metal (systemd services) — just load balance across them - Alternative: Put Ollama in k8s with nodeAffinity, but GPU/NPU passthrough is complex - For now: Configure OpenClaw with array of endpoints, let it round-robin

sscoble commented

2026-05-25 02:18:25 +00:00

Owner

Triage: Still pending — blocked on #3 (n97-2 and n97-3 workers)

Phase 6 (Ollama LB) depends on:

#3 ❌ (n97-2 and n97-3 must be bootstrapped and have Ollama running)

Neither Ollama node is reachable from this cluster yet. Leaving open.

## Triage: Still pending — blocked on #3 (n97-2 and n97-3 workers) Phase 6 (Ollama LB) depends on: - #3 ❌ (n97-2 and n97-3 must be bootstrapped and have Ollama running) Neither Ollama node is reachable from this cluster yet. Leaving open.