Infra as code for Scobleclaw multi-node cluster
e18f732f40
Split the 'Infrastructure' junk drawer (8 entries) into two clear groups:
- Operations: Uptime Kuma, Service Health, Grafana, Prometheus
- Tools: Vault Drop, Tickets, Style Prefs, Preview, Umami
Other changes per the proposed restructure in scobleclaw#24:
- Renamed 'Health' -> 'Family' (single tile: Family Health)
- Merged 'Status' into 'Operations'
- Added Prometheus tile (k3s Service scobleclaw-monitoring/prometheus is
healthy 1/1, ClusterIP 10.43.250.126:9090)
- New /prometheus/ location block: subpath-friendly routing with
proxy_redirect + sub_filter for href/src rewriting, mirroring the
/kuma/ pattern.
- Fixed Grafana double-routing: was pointing at the traefik-ingress
ClusterIP 10.43.3.233:80 (which 404s in the new cluster) — now
direct to scobleclaw-monitoring/grafana Service 10.43.150.72:3000.
- Omitted Pool Manager (k3s pod ImagePullBackOff 5h+, no endpoints)
- Omitted Harbor (CrashLoopBackOff across core/db/exporter)
- Omitted Location (gps-share on :3993 returns 404 on every path;
kept the /location/ proxy block as a debug surface since removing
it requires nixos-rebuild which is currently blocked)
- Omitted 'Spatial Office' (no service/port exists; 3987 = forgejo-mcp,
3988 = openbrain-mcp — both internal MCP servers)
file: /etc/nixos/modules/meos-gateway.nix
- services.yaml block (lines ~88-220) restructured to 9 categories
- /grafana/ block (line ~461) repointed to direct Service
- new /prometheus/ block added (subpath-friendly like /kuma/)
nixos-rebuild is currently blocked (k3s.nix + wgu.nix issues; per
scobleclaw#24 AC, that's a separate ticket). The .nix file passes
`nix-instantiate --parse`. Run `nixos-rebuild switch` once the
unblocker lands to activate the new layout on os.scoble.me.
Refs: scobleclaw#24, cycle 128, dispatched by CoS.
|
||
|---|---|---|
| apps | ||
| docs | ||
| migrations | ||
| modules | ||
| nodes | ||
| scripts | ||
| flake.nix | ||
| hardware-n97.nix | ||
| hardware-rpi4.nix | ||
| README.md | ||
sscoble/devops
Infrastructure as code for the Scobleclaw cluster.
Hardware
| Node | Platform | Arch | Role | RAM | Storage |
|---|---|---|---|---|---|
| n97-1 | GMK N97 | x86_64 | k3s server | 12GB | 233GB |
| n97-2 | GMK N97 | x86_64 | k3s worker | 12GB | ~233GB |
| n97-3 | GMK N97 | x86_64 | k3s worker | 12GB | ~233GB |
| rpi4-1 | RPi 4B | aarch64 | k3s worker | 8GB | SD/USB |
| rpi4-2 | RPi 4B | aarch64 | k3s worker | 8GB | SD/USB |
Network
- Mesh: Tailscale (all nodes)
- DNS: os.scoble.me → n97-1 (Tailscale IP), SSL via Let's Encrypt + Cloudflare DNS
- K8s API: Tailscale-only, no public exposure
Repo Layout
├── README.md
├── flake.nix # NixOS flake — all 5 nodes
├── hardware-n97.nix # shared N97 perf tuning
├── hardware-rpi4.nix # shared RPi4 config
├── modules/
│ ├── k3s-cluster.nix # shared k3s agent config
│ ├── meos-gateway.nix # nginx + acme + homepage
│ ├── ollama-worker.nix # Ollama on worker nodes
│ └── tailscale.nix # shared Tailscale config
├── nodes/
│ ├── n97-1.nix # control plane + gateway
│ ├── n97-2.nix # Ollama worker x86
│ ├── n97-3.nix # Ollama worker x86
│ ├── rpi4-1.nix # ARM worker (light services)
│ └── rpi4-2.nix # ARM worker (umami)
├── apps/ # K8s manifests + Helm values
│ ├── forgejo/
│ ├── postgres/
│ ├── umami/
│ ├── openbrain/
│ ├── meos-dashboard/
│ └── mcp-servers/
├── scripts/
│ ├── bootstrap-node.sh # initial node setup
│ └── migrate-db.sh # Postgres migration from bare to K8s
└── docs/
└── migration.md # transition plan
Quickstart
# Build and deploy a node
nixos-rebuild switch --flake .#n97-1 --target-host root@100.x.x.x
# Access cluster
kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml get nodes
# Deploy app
kubectl apply -k apps/forgejo/