kubeclaw: New project — OpenClaw operator for Kubernetes
Define kubeclaw as a Kubernetes operator that runs Scobleclaw AI agents as first-class K8s workloads with CRDs, controller loops, and security-hardened operator patterns. Key deliverables in this commit: - README.md — Project spec with CRD reference, architecture diagram, security model, threat model, and quick-start guide - docs/architecture.md — Deep architecture doc covering operator pattern, sidecar design, API design, security architecture, and operations - docs/development.md — Dev setup guide with prerequisites, commands, testing, and local dev tips Integrates with: - scobleclaw#7 (Agent-Judge authorization layer) - OpenClaw gateway for LLM provider management - Standard Kubernetes operator patterns Closes scobleclaw#8
This commit is contained in:
agent-cos
parent
d74bb3852a
commit
3264f9fe66
3 changed files with 801 additions and 0 deletions
478
kubeclaw/README.md
Normal file
478
kubeclaw/README.md
Normal file
|
|
@ -0,0 +1,478 @@
|
|||
# kubeclaw
|
||||
|
||||
**OpenClaw operator for Kubernetes** — runs Scobleclaw AI agents as first-class Kubernetes workloads with CRDs, controller loops, and security-hardened operator patterns.
|
||||
|
||||
> **Status:** Project Kickoff — Architecture & Spec Draft
|
||||
> **Author:** @agent-cos
|
||||
> **Related:** scobleclaw#7 (Agent Judge architecture), scobleclaw#8 (this ticket)
|
||||
|
||||
---
|
||||
|
||||
## What It Does
|
||||
|
||||
Kubeclaw brings Scobleclaw's agentic engineering patterns into Kubernetes. Instead of running agents as ad-hoc processes, kubeclaw models them as declarative K8s resources that the operator reconciles into running pods with proper configuration, secrets, channels, and authorization gating.
|
||||
|
||||
### Key Design Decisions
|
||||
|
||||
1. **Security-first agent surface** — Agents get *one* tool: `template_workflow`. No raw exec, no arbitrary file writes, no open web access. Every action flows through the judge.
|
||||
2. **Actor-Judge separation** — The task agent (e.g., `@engineering`) and the authorization judge (`@judge`) are separate LLM model instances. The judge validates every workflow template invocation against CompliancePolicy CRDs before execution proceeds.
|
||||
3. **Declarative all the way** — Every agent, channel binding, workflow template, and compliance policy is a Kubernetes CRD. The controller reconciles desired state → running state.
|
||||
4. **Pluggable providers** — LLM providers (OpenAI, Anthropic, Ollama, OpenClaw gateway) are cluster-scoped Provider CRDs with API key references.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Kubernetes Cluster │
|
||||
│ │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌───────────────┐ │
|
||||
│ │ Agent CRD │ │ AgentGroup CRD │ │ AgentConfig │ │
|
||||
│ │ (agent spec) │ │ (orchestration) │ │ (templates) │ │
|
||||
│ └────────┬─────────┘ └────────┬─────────┘ └───────┬───────┘ │
|
||||
│ │ │ │ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌──────────────────────────────────────────────────────────────┐ │
|
||||
│ │ kubeclaw Controller (Reconciler) │ │
|
||||
│ │ │ │
|
||||
│ │ 1. Watch Agent/AgentGroup resources │ │
|
||||
│ │ 2. Reconcile → create/update/delete Agent Pods │ │
|
||||
│ │ 3. Inject config, secrets, channel bindings │ │
|
||||
│ │ 4. Wire judge sidecar for template_workflow authorization │ │
|
||||
│ └─────────────────────────┬────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌──────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Agent Pod (3 containers) │ │
|
||||
│ │ │ │
|
||||
│ │ ┌─────────────────────┐ ┌─────────────────────┐ │ │
|
||||
│ │ │ Actor Agent │ │ Judge Sidecar │ │ │
|
||||
│ │ │ (OpenClaw agent) │──│ (deepseek-v4-pro) │ │ │
|
||||
│ │ │ │ │ Validates workflow │ │ │
|
||||
│ │ │ Tool: │ │ invocations against │ │ │
|
||||
│ │ │ template_workflow │ │ CompliancePolicy │ │ │
|
||||
│ │ └─────────────────────┘ └─────────────────────┘ │ │
|
||||
│ │ │ │
|
||||
│ │ ┌────────────────────────────────────────────────────────┐ │ │
|
||||
│ │ │ Workflow Executor Sidecar │ │ │
|
||||
│ │ │ Runs approved workflow steps, logs to audit trail │ │ │
|
||||
│ │ │ Each step: structured input → structured output │ │ │
|
||||
│ │ └────────────────────────────────────────────────────────┘ │ │
|
||||
│ └──────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌──────────────────┐ ┌──────────────────┐ ┌────────────────┐ │
|
||||
│ │ WorkflowTemplate │ │ CompliancePolicy │ │ Provider CRD │ │
|
||||
│ │ CRD (cluster) │ │ CRD (cluster) │ │ (cluster) │ │
|
||||
│ └──────────────────┘ └──────────────────┘ └────────────────┘ │
|
||||
│ │
|
||||
│ ┌──────────────────┐ ┌──────────────────┐ ┌────────────────┐ │
|
||||
│ │ Channel CRD │ │ Secret (K8s) │ │ ConfigMap │ │
|
||||
│ │ (namespaced) │ │ (API keys) │ │ (agent config)│ │
|
||||
│ └──────────────────┘ └──────────────────┘ └────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## CRD Reference
|
||||
|
||||
### Provider (cluster-scoped)
|
||||
|
||||
LLM provider configuration — API endpoints, model lists, pricing.
|
||||
|
||||
```yaml
|
||||
apiVersion: kubeclaw.nousresearch.com/v1alpha1
|
||||
kind: Provider
|
||||
metadata:
|
||||
name: openclaw-gateway
|
||||
spec:
|
||||
type: openclaw # openai | anthropic | ollama | openclaw
|
||||
baseURL: https://os.scoble.me/api
|
||||
apiKeyRef:
|
||||
name: openclaw-credentials
|
||||
key: api-key
|
||||
models:
|
||||
- name: deepseek-v4-flash:cloud
|
||||
contextWindow: 65536
|
||||
costPer1KInput: 0.0005
|
||||
costPer1KOutput: 0.0015
|
||||
- name: deepseek-v4-pro:cloud
|
||||
contextWindow: 128000
|
||||
costPer1KInput: 0.001
|
||||
costPer1KOutput: 0.003
|
||||
defaultModel: deepseek-v4-flash:cloud
|
||||
```
|
||||
|
||||
### Channel (namespaced)
|
||||
|
||||
Platform binding — Discord, Slack, Telegram, etc.
|
||||
|
||||
```yaml
|
||||
apiVersion: kubeclaw.nousresearch.com/v1alpha1
|
||||
kind: Channel
|
||||
metadata:
|
||||
name: engineering-discord
|
||||
namespace: agents
|
||||
spec:
|
||||
platform: discord
|
||||
botTokenRef:
|
||||
name: discord-bot-tokens
|
||||
key: engineering-bot
|
||||
guildID: "123456789"
|
||||
allowedChannels:
|
||||
- "agent-logs"
|
||||
- "engineering-chat"
|
||||
```
|
||||
|
||||
### AgentConfig (namespaced)
|
||||
|
||||
Reusable agent configuration templates — workspace, memory, skills, sandbox, tools.
|
||||
|
||||
```yaml
|
||||
apiVersion: kubeclaw.nousresearch.com/v1alpha1
|
||||
kind: AgentConfig
|
||||
metadata:
|
||||
name: standard-engineering
|
||||
namespace: agents
|
||||
spec:
|
||||
workspace:
|
||||
mountPath: /workspace
|
||||
size: 10Gi
|
||||
memory:
|
||||
enabled: true
|
||||
retentionDays: 90
|
||||
skills:
|
||||
- id: forgejo
|
||||
version: "1.0"
|
||||
- id: filesystem
|
||||
version: "1.0"
|
||||
- id: discord
|
||||
version: "1.0"
|
||||
sandbox:
|
||||
type: gvisor
|
||||
networkPolicy: internal-only
|
||||
resources:
|
||||
cpu: "2"
|
||||
memory: "4Gi"
|
||||
gpu: 0
|
||||
```
|
||||
|
||||
### WorkflowTemplate (cluster-scoped)
|
||||
|
||||
Pre-approved workflow templates — the **only** thing agents can invoke. Each template defines input parameters, execution steps, output schema, and compliance requirements.
|
||||
|
||||
```yaml
|
||||
apiVersion: kubeclaw.nousresearch.com/v1alpha1
|
||||
kind: WorkflowTemplate
|
||||
metadata:
|
||||
name: search-memory
|
||||
spec:
|
||||
description: Search agent memory for relevant context
|
||||
version: "1.0"
|
||||
parameters:
|
||||
- name: query
|
||||
type: string
|
||||
required: true
|
||||
description: Search query string
|
||||
- name: maxResults
|
||||
type: integer
|
||||
default: 10
|
||||
minimum: 1
|
||||
maximum: 50
|
||||
steps:
|
||||
- id: embed-query
|
||||
action: llm-embed
|
||||
input: "{{ .Params.query }}"
|
||||
- id: vector-search
|
||||
action: vector-db-query
|
||||
input: "{{ .Steps.embed-query.output }}"
|
||||
limit: "{{ .Params.maxResults }}"
|
||||
- id: format-results
|
||||
action: llm-complete
|
||||
input: "Format these memory results: {{ .Steps.vector-search.output }}"
|
||||
output:
|
||||
type: structured
|
||||
schema:
|
||||
results:
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
properties:
|
||||
id: { type: string }
|
||||
content: { type: string }
|
||||
timestamp: { type: string }
|
||||
relevance: { type: number }
|
||||
compliance:
|
||||
requiredFrameworks: []
|
||||
dataClassification: internal
|
||||
```
|
||||
|
||||
### CompliancePolicy (cluster-scoped)
|
||||
|
||||
Regulatory and security policies the judge evaluates workflow invocations against.
|
||||
|
||||
```yaml
|
||||
apiVersion: kubeclaw.nousresearch.com/v1alpha1
|
||||
kind: CompliancePolicy
|
||||
metadata:
|
||||
name: soc2-baseline
|
||||
spec:
|
||||
framework: SOC2
|
||||
version: "2025"
|
||||
rules:
|
||||
- id: data-access-scope
|
||||
severity: critical
|
||||
description: Agent must not access data outside its designated scope
|
||||
eval: check_data_access_scope(request)
|
||||
onViolation: REJECT
|
||||
- id: pii-handling
|
||||
severity: high
|
||||
description: PII must not be written to external channels
|
||||
eval: check_pii_in_output(request)
|
||||
onViolation: REJECT
|
||||
- id: audit-logging
|
||||
severity: medium
|
||||
description: All workflow invocations must be logged
|
||||
eval: check_audit_log(request)
|
||||
onViolation: ESCALATE_HUMAN
|
||||
escalation:
|
||||
onReject: notify_human("Compliance violation: {{ .Rule.Description }}")
|
||||
onError: route_to_human("Judge error: {{ .Error }}")
|
||||
```
|
||||
|
||||
### Agent (namespaced)
|
||||
|
||||
Single agent deployment — the primary CRD users create.
|
||||
|
||||
```yaml
|
||||
apiVersion: kubeclaw.nousresearch.com/v1alpha1
|
||||
kind: Agent
|
||||
metadata:
|
||||
name: engineering
|
||||
namespace: agents
|
||||
spec:
|
||||
identity:
|
||||
displayName: "Engineering Agent"
|
||||
description: "Software engineering and architecture work"
|
||||
owner: "scott"
|
||||
config:
|
||||
agentConfigRef: standard-engineering
|
||||
overrides:
|
||||
resources:
|
||||
cpu: "4"
|
||||
memory: "8Gi"
|
||||
providers:
|
||||
- ref:
|
||||
name: openclaw-gateway
|
||||
model: deepseek-v4-flash:cloud
|
||||
channels:
|
||||
- ref:
|
||||
name: engineering-discord
|
||||
workflowTemplates:
|
||||
- ref:
|
||||
name: search-memory
|
||||
- ref:
|
||||
name: forgejo-pr-create
|
||||
- ref:
|
||||
name: file-write
|
||||
compliancePolicies:
|
||||
- ref:
|
||||
name: soc2-baseline
|
||||
judge:
|
||||
enabled: true
|
||||
providerRef:
|
||||
name: openclaw-gateway
|
||||
model: deepseek-v4-pro:cloud
|
||||
```
|
||||
|
||||
### AgentGroup (namespaced)
|
||||
|
||||
Multi-agent orchestration with shared configs and per-member overrides.
|
||||
|
||||
```yaml
|
||||
apiVersion: kubeclaw.nousresearch.com/v1alpha1
|
||||
kind: AgentGroup
|
||||
metadata:
|
||||
name: content-team
|
||||
namespace: agents
|
||||
spec:
|
||||
sharedConfig:
|
||||
agentConfigRef: standard-content
|
||||
providers:
|
||||
- ref:
|
||||
name: openclaw-gateway
|
||||
members:
|
||||
- name: writer
|
||||
overrides:
|
||||
channels:
|
||||
- ref:
|
||||
name: content-discord
|
||||
- name: editor
|
||||
overrides:
|
||||
channels:
|
||||
- ref:
|
||||
name: editorial-slack
|
||||
- name: publisher
|
||||
overrides:
|
||||
channels:
|
||||
- ref:
|
||||
name: web-publish-discord
|
||||
```
|
||||
|
||||
## Security Model
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Agent Attack Surface │
|
||||
│ │
|
||||
│ Available tools: [template_workflow] │
|
||||
│ Not available: exec, write, web_search, sessions_*, forgejo_*, │
|
||||
│ lean_experiment_*, cron, discord_send_raw │
|
||||
│ │
|
||||
│ Judge validates every template_workflow call against: │
|
||||
│ ├─ WorkflowTemplate spec (schema, allowed steps) │
|
||||
│ ├─ CompliancePolicy rules (SOC2/HIPAA/GDPR per-rule checks) │
|
||||
│ ├─ Agent-level whitelists (allowed templates, data boundaries) │
|
||||
│ └─ Regulatory framework system prompt │
|
||||
│ │
|
||||
│ Verdicts: │
|
||||
│ ├─ APPROVE → workflow executor runs the steps │
|
||||
│ ├─ REJECT → agent notified, human escalates if needed │
|
||||
│ └─ ESCALATE_HUMAN → routed to Scott with structured dispute │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Threat Model
|
||||
|
||||
| Threat | Mitigation |
|
||||
|--------|-----------|
|
||||
| Agent jailbreaks via prompt injection | Judge is separate model with no task-completion pressure; single-tool surface limits blast radius |
|
||||
| Lateral movement between agents | Per-agent namespaces, NetworkPolicy isolation, strict Channel bindings |
|
||||
| Privilege escalation via template params | WorkflowTemplate parameter validation, CompliancePolicy runtime checks |
|
||||
| Data exfiltration via channels | Channel-scoped output validation, audit logging on every workflow step |
|
||||
| Credential leak via log output | Secrets never injected into agent context; only referenced via `apiKeyRef` |
|
||||
| Denial of service via resource exhaustion | Per-Agent resource limits, HPA on AgentGroup, OpenClaw gateway rate limiting |
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
kubeclaw/
|
||||
├── README.md # This file
|
||||
├── config/
|
||||
│ ├── crd/ # CRD YAML definitions
|
||||
│ │ ├── provider.yaml
|
||||
│ │ ├── channel.yaml
|
||||
│ │ ├── agentconfig.yaml
|
||||
│ │ ├── workflowtemplate.yaml
|
||||
│ │ ├── compliancepolicy.yaml
|
||||
│ │ ├── agent.yaml
|
||||
│ │ └── agentgroup.yaml
|
||||
│ └── samples/ # Example CRD manifests
|
||||
│ ├── sample-agent.yaml
|
||||
│ ├── sample-group.yaml
|
||||
│ └── sample-workflow.yaml
|
||||
├── charts/
|
||||
│ └── kubeclaw/ # Helm chart
|
||||
│ ├── Chart.yaml
|
||||
│ ├── values.yaml
|
||||
│ └── templates/
|
||||
│ ├── crds.yaml
|
||||
│ ├── controller.yaml
|
||||
│ ├── rbac.yaml
|
||||
│ ├── service.yaml
|
||||
│ └── webhook.yaml
|
||||
├── internal/
|
||||
│ ├── controller/ # Reconciler logic
|
||||
│ │ ├── agent_controller.go
|
||||
│ │ ├── agentgroup_controller.go
|
||||
│ │ └── suite_test.go
|
||||
│ ├── judge/ # Judge integration
|
||||
│ │ ├── client.go
|
||||
│ │ ├── verifier.go
|
||||
│ │ └── types.go
|
||||
│ └── webhook/ # Admission webhooks
|
||||
│ ├── agent_validator.go
|
||||
│ └── workflow_validator.go
|
||||
├── docs/
|
||||
│ ├── architecture.md # Deep architecture doc
|
||||
│ ├── development.md # Dev setup guide
|
||||
│ └── operator-lifecycle.md # Production operations
|
||||
├── main.go # Entry point
|
||||
├── go.mod
|
||||
├── go.sum
|
||||
├── Makefile
|
||||
└── Dockerfile
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Prerequisites
|
||||
# - Go 1.22+
|
||||
# - kubectl
|
||||
# - kind or minikube (for local dev)
|
||||
# - helm
|
||||
|
||||
# Clone the monorepo
|
||||
git clone https://os.scoble.me/forgejo/marcus/scobleclaw.git
|
||||
cd scobleclaw/kubeclaw
|
||||
|
||||
# Install CRDs
|
||||
make install
|
||||
|
||||
# Run controller locally (against current kubecontext)
|
||||
make run
|
||||
|
||||
# Deploy to cluster via Helm
|
||||
helm install kubeclaw ./charts/kubeclaw --namespace kubeclaw-system --create-namespace
|
||||
|
||||
# Create a sample agent
|
||||
kubectl apply -f config/samples/sample-agent.yaml
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
See [docs/development.md](docs/development.md) for detailed setup and workflow.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Go 1.22+
|
||||
- `controller-gen` (from sigs.k8s.io/controller-tools)
|
||||
- `kustomize`
|
||||
- `helm`
|
||||
- Access to an OpenClaw gateway or compatible LLM API
|
||||
|
||||
### Commands
|
||||
|
||||
```bash
|
||||
make generate # Generate CRD deep copy and client code
|
||||
make manifests # Generate CRD YAML manifests
|
||||
make test # Run unit and integration tests
|
||||
make run # Run controller locally
|
||||
make docker-build # Build controller image
|
||||
make helm-package # Package Helm chart
|
||||
```
|
||||
|
||||
## CRD Inventory
|
||||
|
||||
| CRD | Scope | Purpose |
|
||||
|-----|-------|---------|
|
||||
| Provider | Cluster | LLM provider configs (API keys, models, pricing) |
|
||||
| Channel | Namespaced | Platform channel bindings (Discord, Slack, Telegram) |
|
||||
| AgentConfig | Namespaced | Reusable agent configuration templates |
|
||||
| WorkflowTemplate | Cluster | Pre-approved workflow templates (only thing agents can invoke) |
|
||||
| CompliancePolicy | Cluster | Security/compliance evaluation rules for judge |
|
||||
| Agent | Namespaced | Single agent deployment |
|
||||
| AgentGroup | Namespaced | Multi-agent orchestration with shared configs |
|
||||
|
||||
## Integration: Agent Judge
|
||||
|
||||
This project integrates directly with the [Agent Judge architecture](https://os.scoble.me/forgejo/marcus/scobleclaw/issues/7):
|
||||
|
||||
- The **judge sidecar** implements the Judge protocol specified in `agent-judge-architecture.md`
|
||||
- Risk classification from that architecture maps to CompliancePolicy CRD rules
|
||||
- The `template_workflow` tool is the enforcement point — agents can only invoke what the judge has approved
|
||||
- Admission webhooks validate CRD changes at the API server level (pre-commit)
|
||||
- The judge validates at runtime (post-commit, pre-execution)
|
||||
|
||||
## License
|
||||
|
||||
MIT © Scott Scoble / Greybeard Holdings, LLC
|
||||
172
kubeclaw/docs/architecture.md
Normal file
172
kubeclaw/docs/architecture.md
Normal file
|
|
@ -0,0 +1,172 @@
|
|||
# kubeclaw Architecture
|
||||
|
||||
> **Status:** Draft v0.1
|
||||
> **Author:** @agent-cos
|
||||
> **Date:** 2026-05-23
|
||||
|
||||
---
|
||||
|
||||
## 1. System Context
|
||||
|
||||
kubeclaw is a Kubernetes operator that runs Scobleclaw agents as first-class cluster workloads. It is part of the Scobleclaw ecosystem and integrates with:
|
||||
|
||||
- **OpenClaw Gateway** — LLM API gateway providing model access, rate limiting, and credential management
|
||||
- **Agent Judge** — authorization layer that validates every action before execution
|
||||
- **Scobleclaw dispatch system** — higher-level orchestration that may trigger agent deployments
|
||||
|
||||
## 2. Operator Pattern
|
||||
|
||||
kubeclaw follows standard Kubernetes operator patterns using the `controller-runtime` library:
|
||||
|
||||
```
|
||||
WATCH: CRD events (Agent, AgentGroup, etc.)
|
||||
│
|
||||
▼
|
||||
RECONCILE: desired state → running state
|
||||
│
|
||||
├── CREATE: Build Agent Pod spec, inject config/secrets, wire sidecars
|
||||
├── UPDATE: Diff current vs desired → patch deployment
|
||||
└── DELETE: Tear down gracefully, flush audit logs
|
||||
```
|
||||
|
||||
### 2.1 Reconciler Loop
|
||||
|
||||
Each CRD has a dedicated reconciler. The primary flow:
|
||||
|
||||
1. **Agent Reconciler** watches `Agent` resources
|
||||
2. On create/update, it builds a `Pod` spec with:
|
||||
- Main agent container (OpenClaw agent runtime)
|
||||
- Judge sidecar (deepseek-v4-pro for authorization validation)
|
||||
- Workflow executor sidecar (runs approved template steps)
|
||||
- ConfigMap references for agent config
|
||||
- Secret references for API keys (never injected into agent context)
|
||||
3. On AgentConfig reference change, it triggers re-reconciliation of all Agents using that config
|
||||
4. On Provider or CompliancePolicy change, it triggers rolling update of affected agents
|
||||
|
||||
### 2.2 Sidecar Architecture
|
||||
|
||||
Each Agent Pod has three containers:
|
||||
|
||||
#### Actor Agent Container
|
||||
- Runs the OpenClaw agent process
|
||||
- Has **exactly one tool**: `template_workflow(workflowTemplateRef, params)`
|
||||
- No network access except to judge sidecar (localhost) and OpenClaw gateway
|
||||
- No filesystem write access outside `/workspace` (tmpfs)
|
||||
- No access to K8s API
|
||||
|
||||
#### Judge Sidecar
|
||||
- Runs a separate LLM model instance (deepseek-v4-pro or higher)
|
||||
- Implements the Judge protocol from `agent-judge-architecture.md`
|
||||
- Evaluates every `template_workflow` call against:
|
||||
- WorkflowTemplate spec
|
||||
- CompliancePolicy rules
|
||||
- Agent-level whitelists
|
||||
- Returns APPROVE / REJECT / ESCALATE_HUMAN
|
||||
|
||||
#### Workflow Executor Sidecar
|
||||
- Runs approved workflow steps
|
||||
- Each step is a self-contained action with structured input/output
|
||||
- All steps are logged to the audit trail (persistent volume + stdout)
|
||||
|
||||
## 3. API Design
|
||||
|
||||
### 3.1 CRD Group
|
||||
|
||||
- **Group:** `kubeclaw.nousresearch.com`
|
||||
- **Version:** `v1alpha1` (initial, will graduate to v1)
|
||||
- **Scope:** Mix of cluster-scoped and namespaced
|
||||
|
||||
### 3.2 Validation
|
||||
|
||||
- **CRD schema validation** — OpenAPI v3 schemas on each CRD for structural validation
|
||||
- **Admission webhooks** — `ValidatingWebhookConfiguration` for semantic validation (e.g., provider must exist, channel must be reachable)
|
||||
- **Status subresource** — Each CRD has a `.status` block with observed state, conditions, and last reconciliation
|
||||
|
||||
### 3.3 Status Reporting
|
||||
|
||||
All CRDs use standard Kubernetes status conditions:
|
||||
|
||||
```yaml
|
||||
status:
|
||||
conditions:
|
||||
- type: Ready
|
||||
status: "True"
|
||||
reason: AgentPodRunning
|
||||
message: "Agent pod engineering-7d8f9c is running"
|
||||
lastTransitionTime: "2026-05-23T19:00:00Z"
|
||||
- type: JudgeConnected
|
||||
status: "True"
|
||||
reason: JudgeResponding
|
||||
message: "Judge sidecar is online and responding"
|
||||
observedGeneration: 3
|
||||
agentPodRef:
|
||||
name: engineering-7d8f9c
|
||||
namespace: agents
|
||||
lastWorkflowCount: 142
|
||||
lastJudgeVerdict: APPROVE
|
||||
```
|
||||
|
||||
## 4. Security Architecture
|
||||
|
||||
### 4.1 Pod Security
|
||||
|
||||
- **PodSecurityContext**: Run as non-root user, read-only root filesystem
|
||||
- **Seccomp profile**: Runtime default
|
||||
- **AppArmor**: Kubeclaw-specific profile limiting agent to `template_workflow` only
|
||||
- **NetworkPolicy**: Agents can only talk to judge sidecar (localhost) and OpenClaw gateway
|
||||
- **Resource limits**: Hard CPU/memory limits per Agent spec
|
||||
|
||||
### 4.2 Secret Management
|
||||
|
||||
- API keys are standard K8s Secrets referenced by `apiKeyRef` in Provider and Channel CRDs
|
||||
- Secrets are **never** injected into the agent container's environment
|
||||
- Only the judge sidecar and workflow executor have access to resolved credentials
|
||||
- The actor agent container never sees raw API keys
|
||||
|
||||
### 4.3 RBAC
|
||||
|
||||
```yaml
|
||||
# Minimal RBAC for the controller
|
||||
- apiGroups: ["kubeclaw.nousresearch.com"]
|
||||
resources: ["agents", "agentgroups", "providers", "channels", "workflowtemplates", "compliancepolicies"]
|
||||
verbs: ["get", "list", "watch", "patch"]
|
||||
- apiGroups: [""]
|
||||
resources: ["pods", "configmaps", "secrets", "events"]
|
||||
verbs: ["get", "list", "watch", "create", "update", "delete", "patch"]
|
||||
- apiGroups: ["networking.k8s.io"]
|
||||
resources: ["networkpolicies"]
|
||||
verbs: ["get", "list", "watch", "create", "update", "delete"]
|
||||
```
|
||||
|
||||
## 5. Operational Concerns
|
||||
|
||||
### 5.1 Observability
|
||||
|
||||
- **Metrics**: Standard controller-runtime metrics (ReconcileTotal, WorkQueueDepth) + custom:
|
||||
- `kubeclaw_agent_count` — number of agent deployments
|
||||
- `kubeclaw_judge_verdicts_total` — verdict counts by type (APPROVE/REJECT/ESCALATE)
|
||||
- `kubeclaw_workflow_invocations_total` — workflow calls by template
|
||||
- **Logging**: Structured JSON logs, per-reconciliation trace IDs
|
||||
- **Tracing**: OpenTelemetry integration for end-to-end workflow tracing
|
||||
|
||||
### 5.2 Reconciliation Performance
|
||||
|
||||
- Agent pod creation is asynchronous; controller creates Deployment, not bare Pod
|
||||
- Large AgentGroups are reconciled via batch operations with worker pool
|
||||
- Requeue on conflict with exponential backoff (100ms → 30s)
|
||||
- Watch caching via informer with 10-minute resync period
|
||||
|
||||
### 5.3 Upgrade Strategy
|
||||
|
||||
- CRD versioning with conversion webhooks for backward compatibility
|
||||
- Helm chart with `--upgrade` support
|
||||
- Canary deployments for controller upgrades
|
||||
- Agent pods use standard rolling update strategy
|
||||
|
||||
## 6. Future Enhancements
|
||||
|
||||
- [ ] **Horizontal Pod Autoscaling** for AgentGroups based on workflow queue depth
|
||||
- [ ] **Agent-to-agent networking** via Service Mesh (Istio/Linkerd) for multi-agent coordination
|
||||
- [ ] **Workflow DAG execution** — complex multi-step workflows with branching
|
||||
- [ ] **Compliance attestation** — generate compliance reports for SOC2/HIPAA/GDPR audits
|
||||
- [ ] **Web UI** — Dashboard for viewing agent status, workflow history, judge decisions
|
||||
151
kubeclaw/docs/development.md
Normal file
151
kubeclaw/docs/development.md
Normal file
|
|
@ -0,0 +1,151 @@
|
|||
# Development Setup
|
||||
|
||||
> **Status:** Draft v0.1
|
||||
> **Author:** @agent-cos
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Go 1.22+** — `go version` should show go1.22 or later
|
||||
- **kubectl** — configured with a cluster context (kind/minikube for local dev)
|
||||
- **helm** — v3.x for chart deployment
|
||||
- **controller-gen** — from `sigs.k8s.io/controller-tools`
|
||||
- **kustomize** — built into `kubectl kustomize` or standalone v4+
|
||||
- **Docker** — for building controller images
|
||||
- **kind** or **minikube** — for local Kubernetes clusters
|
||||
- **OpenClaw gateway** or compatible LLM API endpoint
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Clone the Repository
|
||||
|
||||
```bash
|
||||
git clone https://os.scoble.me/forgejo/marcus/scobleclaw.git
|
||||
cd scobleclaw/kubeclaw
|
||||
```
|
||||
|
||||
### 2. Create a Local Cluster
|
||||
|
||||
```bash
|
||||
kind create cluster --name kubeclaw-dev
|
||||
```
|
||||
|
||||
Or use an existing cluster:
|
||||
|
||||
```bash
|
||||
kubectl config get-contexts
|
||||
```
|
||||
|
||||
### 3. Install CRDs
|
||||
|
||||
```bash
|
||||
make install
|
||||
```
|
||||
|
||||
This applies all CRD YAML manifests to the cluster.
|
||||
|
||||
### 4. Run the Controller Locally
|
||||
|
||||
```bash
|
||||
# In one terminal:
|
||||
make run
|
||||
|
||||
# The controller connects to the current kubecontext
|
||||
# Logs show reconciliation of CRD events
|
||||
```
|
||||
|
||||
### 5. Deploy an Agent
|
||||
|
||||
```bash
|
||||
# Apply a Provider first (no real credentials in dev)
|
||||
kubectl apply -f config/samples/sample-provider.yaml
|
||||
|
||||
# Apply an Agent
|
||||
kubectl apply -f config/samples/sample-agent.yaml
|
||||
|
||||
# Watch the agent pod come up
|
||||
kubectl get pods -n agents -w
|
||||
```
|
||||
|
||||
## Project Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `make generate` | Generate CRD deep copy and client code |
|
||||
| `make manifests` | Generate CRD YAML manifests from Go types |
|
||||
| `make install` | Apply CRDs to cluster |
|
||||
| `make uninstall` | Remove CRDs from cluster |
|
||||
| `make run` | Run controller locally |
|
||||
| `make test` | Run unit and integration tests |
|
||||
| `make docker-build` | Build controller Docker image |
|
||||
| `make docker-push` | Push controller image to registry |
|
||||
| `make helm-package` | Package Helm chart |
|
||||
| `make deploy` | Deploy controller via Helm |
|
||||
|
||||
## Testing
|
||||
|
||||
### Unit Tests
|
||||
|
||||
```bash
|
||||
make test
|
||||
```
|
||||
|
||||
Tests cover:
|
||||
- Reconciler logic (mock K8s client)
|
||||
- Judge verifier (mock LLM responses)
|
||||
- Webhook validation
|
||||
- CRD schema validation
|
||||
|
||||
### Integration Tests
|
||||
|
||||
```bash
|
||||
# Requires a running cluster and controller
|
||||
make test-integration
|
||||
```
|
||||
|
||||
Uses envtest (controller-runtime's test environment) for in-process API server testing.
|
||||
|
||||
### End-to-End Tests
|
||||
|
||||
```bash
|
||||
# Requires a full cluster with all components
|
||||
make test-e2e
|
||||
```
|
||||
|
||||
## Local Development Tips
|
||||
|
||||
### Watch Controller Logs
|
||||
|
||||
```bash
|
||||
kubectl logs -n kubeclaw-system deployment/kubeclaw-controller -f
|
||||
```
|
||||
|
||||
### Debug Agent Pods
|
||||
|
||||
```bash
|
||||
# Check pod status
|
||||
kubectl describe pod -n agents <agent-name>
|
||||
|
||||
# Check judge sidecar logs
|
||||
kubectl logs -n agents <agent-pod> -c judge
|
||||
|
||||
# Check workflow executor logs
|
||||
kubectl logs -n agents <agent-pod> -c workflow-executor
|
||||
```
|
||||
|
||||
### Hot Reload
|
||||
|
||||
For rapid iteration on CRD handler logic without rebuilding Docker images:
|
||||
|
||||
```bash
|
||||
make run # Restart on ctrl+c and re-run after code changes
|
||||
```
|
||||
|
||||
## Architecture Notes
|
||||
|
||||
- The controller uses `controller-runtime`'s builder pattern: `NewControllerManagedBy(mgr).For(&Agent{}).Owns(&corev1.Pod{})`
|
||||
- CRD types are in `api/v1alpha1/`
|
||||
- Reconcilers are in `internal/controller/`
|
||||
- The judge is implemented as a gRPC or HTTP sidecar
|
||||
- Workflow templates are validated server-side before agent creation
|
||||
Loading…
Add table
Add a link
Reference in a new issue