Add onboarding runbook, customer welcome doc, and concierge templates

2026-05-14 13:30:13 -07:00 · 2026-05-14 13:30:13 -07:00 · 84e75029f3
commit 84e75029f3
parent 8173d64162
4 changed files with 605 additions and 0 deletions
--- a/docs/customer-welcome.md
+++ b/docs/customer-welcome.md
@ -0,0 +1,82 @@
+# Welcome to Your AI Agent Workspace
+
+## What You Just Bought
+
+You now have an AI-powered operations team. Not a chatbot. Not a SaaS tool. A system of AI agents that handle your recurring business workflows — research, reporting, content, coordination, and anything else that eats human hours.
+
+### What's Included (Unlimited Plan)
+- **Unlimited AI agents** — Every workflow you need, we build an agent for it
+- **Unlimited usage** — No metered tokens, no surprise bills, no credit system
+- **Unlimited support** — We manage, iterate, and troubleshoot everything
+- **Your own AI Concierge** — A dedicated agent in your Slack/Discord that captures requests and routes work
+- **Knowledge vault** — Your business knowledge, encoded and queryable by agents
+- **Continuous improvement** — Agents get smarter as they learn your business
+
+### What You DON'T Need to Worry About
+- Model selection
+- API keys
+- Token limits
+- Infrastructure
+- Prompt engineering
+- Agent debugging
+
+That's all ours. You just describe what you need.
+
+---
+
+## Getting Started
+
+### Step 1: Meet Your Concierge
+Your AI Concierge is already in your [Slack/Discord] channel. Say hello. Ask it anything. It will capture your requests and route them for agent creation.
+
+### Step 2: Tell Us What's Eating Your Time
+What work do you dread? What's piling up? What do you wish someone else would just handle? Examples:
+- "I spend 3 hours every Monday on client reports"
+- "I need to know what our competitors are doing every week"
+- "Our proposal process takes 5 days and we lose half of them"
+- "I have 200 uncontacted leads in my CRM"
+
+### Step 3: We Build the Agent
+Scott reviews your request and builds a custom agent for that workflow. Typical turnaround: 24-48 hours. You'll get a preview before it goes live.
+
+### Step 4: The Agent Goes to Work
+Once approved, your new agent starts running. It reports to you through your concierge. You review the output and request adjustments. Rinse and repeat for every workflow.
+
+---
+
+## How to Talk to Your Concierge
+
+Your concierge is designed to feel like a sharp EA, not a chatbot. Talk to it naturally:
+
+**Good:**
+- "I need a weekly competitive intel report on our top 3 rivals"
+- "Can you draft client update emails for all 12 accounts?"
+- "Set up a system that flags inventory below reorder thresholds"
+
+**Also Good:**
+- "What can you actually do?"
+- "How's that report coming?"
+- "I changed my mind about the frequency"
+
+**What Not to Expect:**
+- Your concierge won't pretend a feature exists that hasn't been built yet — it'll be honest and tell you when Scott needs to build it
+- It won't make decisions about your business — it captures and routes, then the specialized agents execute
+
+---
+
+## What Happens Next
+
+**Week 1:** Concierge live, first 1-2 workflow agents built and running  
+**Week 2:** Refining outputs, adding more workflows  
+**Week 3:** Agents running autonomously, you're reviewing outputs not managing process  
+**Month 2:** Your agent fleet knows your business; you're describing new workflows and they're built in days
+
+---
+
+## Questions?
+
+Your concierge can answer most questions about the platform. For billing, account changes, or anything the concierge can't handle, email scott@scoble.me.
+
+Welcome aboard. Let's get that time back.
+
+— Scott
--- a/docs/onboarding-runbook.md
+++ b/docs/onboarding-runbook.md
@ -0,0 +1,437 @@
+# AI Agent Services — Customer Onboarding Runbook
+
+**Version:** 1.0
+**Last updated:** 2026-05-14
+**Owner:** Scott Scoble (@cos manages execution)
+
+---
+
+## Overview
+
+This document defines the end-to-end process for onboarding a new customer onto the AI Agent Services platform. Every step is designed to be executed by Scott (with @cos assistance) in under 2 hours from signed agreement to a running concierge agent in the customer's workspace.
+
+### Onboarding Principles
+- **Customer never touches infrastructure.** No kubectl, no tokens, no model dropdowns.
+- **First contact within 24 hours.** The concierge agent should be live and talking to the customer within a day of signing.
+- **Everything is GitOps.** Every customer workspace, agent definition, and configuration is version-controlled in Forgejo.
+- **Isolation by default.** Each customer gets their own Kubernetes namespace with network isolation and resource quotas.
+
+---
+
+## Phase 0: Pre-Onboarding (Before Customer Signs)
+
+### 0.1 Qualify the Customer
+- [ ] Customer is in a target vertical (marketing, law, insurance, manufacturing, wholesale, real estate)
+- [ ] Customer is NOT in healthcare or finance (regulatory overhead too high for solo operator)
+- [ ] Customer has recurring information workflows that eat human hours (reporting, research, content, coordination)
+- [ ] Customer can afford $5K-$10K/mo
+- [ ] Customer has a Slack or Discord workspace (we need a channel to deploy the concierge into)
+
+### 0.2 Prepare the Deal
+- [ ] Scott conducts discovery call — maps pain points, workflows, priorities
+- [ ] Scott presents the offer: unlimited agents + unlimited usage + unlimited support, $5K-$10K/mo
+- [ ] Customer signs agreement / pays first month
+- [ ] Customer provides: company name, primary contact email, Slack/Discord workspace access, API keys (if using their own OpenAI/Anthropic accounts)
+
+---
+
+## Phase 1: Workspace Provisioning (Estimated: 30 min)
+
+### 1.1 Create Customer Namespace
+```bash
+# Replace CUSTOMER_SLUG with a short, lowercase identifier (e.g., "acme-corp")
+export CUSTOMER_SLUG="acme-corp"
+export CUSTOMER_NAME="Acme Corporation"
+export CUSTOMER_EMAIL="admin@acme.com"
+export CUSTOMER_PLAN="unlimited"  # starter | pro | business | unlimited
+
+kubectl create namespace cust-${CUSTOMER_SLUG}
+```
+
+### 1.2 Apply Isolation Policies
+Create the namespace isolation manifest (or clone from template):
+```bash
+# Copy from template-workspace-unlimited repo
+kubectl apply -f - <<EOF
+apiVersion: v1
+kind: ResourceQuota
+metadata:
+  name: customer-quota
+  namespace: cust-${CUSTOMER_SLUG}
+spec:
+  hard:
+    requests.cpu: "2"
+    requests.memory: 4Gi
+    limits.cpu: "4"
+    limits.memory: 8Gi
+    persistentvolumeclaims: "5"
+    count/agents.kubeclaw.scoble.me: "25"
+    secrets: "20"
+
+---
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: deny-from-other-namespaces
+  namespace: cust-${CUSTOMER_SLUG}
+spec:
+  podSelector: {}
+  policyTypes: ["Ingress"]
+  ingress:
+  - from:
+    - podSelector: {}
+
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: agent-runner
+  namespace: cust-${CUSTOMER_SLUG}
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  namespace: cust-${CUSTOMER_SLUG}
+  name: agent-runner
+rules:
+- apiGroups: ["kubeclaw.scoble.me"]
+  resources: ["agents", "agentconfigs", "channels", "workspaces"]
+  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
+- apiGroups: [""]
+  resources: ["secrets", "configmaps"]
+  verbs: ["get", "list", "create", "update"]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: agent-runner-binding
+  namespace: cust-${CUSTOMER_SLUG}
+subjects:
+- kind: ServiceAccount
+  name: agent-runner
+roleRef:
+  kind: Role
+  name: agent-runner
+  apiGroup: rbac.authorization.k8s.io
+EOF
+```
+
+### 1.3 Create Customer Secrets
+```bash
+kubectl create secret generic customer-api-keys \
+  -n cust-${CUSTOMER_SLUG} \
+  --from-literal=customer-id="${CUSTOMER_SLUG}" \
+  --from-literal=customer-name="${CUSTOMER_NAME}" \
+  --from-literal=customer-email="${CUSTOMER_EMAIL}" \
+  --from-literal=customer-plan="${CUSTOMER_PLAN}"
+
+# If customer provides their own API keys:
+kubectl create secret generic llm-api-keys \
+  -n cust-${CUSTOMER_SLUG} \
+  --from-literal=openai-key="${OPENAI_KEY}" \
+  --from-literal=anthropic-key="${ANTHROPIC_KEY}"
+```
+
+### 1.4 Create the Workspace CR
+```bash
+# Clone template and substitute values
+kubectl apply -f - <<EOF
+apiVersion: kubeclaw.scoble.me/v1alpha1
+kind: Workspace
+metadata:
+  name: ${CUSTOMER_SLUG}-workspace
+  namespace: cust-${CUSTOMER_SLUG}
+  labels:
+    customer: ${CUSTOMER_SLUG}
+    plan: ${CUSTOMER_PLAN}
+spec:
+  customer:
+    name: "${CUSTOMER_NAME}"
+    email: "${CUSTOMER_EMAIL}"
+    plan: ${CUSTOMER_PLAN}
+    billingId: "billing-${CUSTOMER_SLUG}"
+    since: "$(date +%Y-%m-%d)"
+  agents:
+    - name: concierge-agent
+      agentId: concierge
+      configRef: default-config
+      model: deepseek-v4-pro:cloud
+      skills: [intent-capture, feature-routing]
+      alwaysOn: true
+    - name: watchdog
+      agentId: watchdog
+      model: deepseek-v4-pro:cloud
+      alwaysOn: true
+  knowledgeVault:
+    enabled: true
+    storageSize: 10Gi
+    storageClass: local-path
+    forgejoRepo: "os.scoble.me/customers/${CUSTOMER_SLUG}/vault.git"
+    initFromGit: true
+  compliance:
+    auditLog:
+      enabled: true
+      retentionDays: 90
+  monitoring:
+    alertChannels:
+      - ref: ops-alerts
+status:
+  phase: Pending
+EOF
+```
+
+### 1.5 Verify Workspace
+```bash
+kubectl get workspace -n cust-${CUSTOMER_SLUG}
+# Expected: Phase → Ready, Agents → 2, Ready → 2
+
+kubectl get pods -n cust-${CUSTOMER_SLUG}
+# Expected: 2 Running pods (concierge + watchdog)
+```
+
+---
+
+## Phase 2: Concierge Agent Personalization (Estimated: 30 min)
+
+### 2.1 Customize SOUL.md for the Customer
+Create a customer-specific concierge persona. Start from the template at `template-workspace-unlimited/agents/concierge/SOUL.md` and modify:
+
+**Required customizations:**
+- Company name and industry context
+- Known pain points (from discovery call)
+- Specific workflows the customer mentioned
+- The customer's tool stack (Slack/Discord, CRM, ERP if known)
+- Tone calibration (formal vs casual, technical vs non-technical)
+
+**Example customization points:**
+```
+# SOUL.md — [Company Name] Concierge
+
+You are [Name], the AI Concierge for [Company Name], a [industry] company.
+
+## What You Know About This Business
+- They're a [size] person [type] agency in [location]
+- Their pain points: [list from discovery call]
+- Their stack: [list known tools]
+```
+
+### 2.2 Load Concierge Config
+```bash
+kubectl create configmap ${CUSTOMER_SLUG}-concierge-soul \
+  -n cust-${CUSTOMER_SLUG} \
+  --from-file=SOUL.md=./customers/${CUSTOMER_SLUG}/concierge-SOUL.md
+
+kubectl create configmap ${CUSTOMER_SLUG}-concierge-agents \
+  -n cust-${CUSTOMER_SLUG} \
+  --from-file=AGENTS.md=./templates/concierge-AGENTS.md
+```
+
+### 2.3 Deploy the Concierge Agent Pod
+```bash
+kubectl apply -f - <<EOF
+apiVersion: v1
+kind: Pod
+metadata:
+  name: kubeclaw-agent-${CUSTOMER_SLUG}-concierge
+  namespace: cust-${CUSTOMER_SLUG}
+  labels:
+    app: kubeclaw-agent
+    customer: ${CUSTOMER_SLUG}
+    agent: concierge
+spec:
+  containers:
+  - name: hermes-agent
+    image: hermes-speckl/sme:latest
+    imagePullPolicy: Never
+    env:
+    - name: HERMES_MODEL_PROVIDER
+      value: ollama
+    - name: HERMES_MODEL_BASE_URL
+      value: https://api.ollama.cloud/v1
+    - name: HERMES_MODEL_DEFAULT
+      value: deepseek-v4-pro:cloud
+    - name: HERMES_AGENT_ID
+      value: ${CUSTOMER_SLUG}-concierge
+    - name: DISCORD_BOT_TOKEN
+      valueFrom:
+        secretKeyRef:
+          name: channel-tokens
+          key: discord-token
+    volumeMounts:
+    - name: soul
+      mountPath: /home/hermes/.hermes/persona/SOUL.md
+      subPath: SOUL.md
+      readOnly: true
+    - name: agents
+      mountPath: /home/hermes/.hermes/persona/AGENTS.md
+      subPath: AGENTS.md
+      readOnly: true
+    - name: memory
+      mountPath: /home/hermes/.hermes/memory
+  volumes:
+  - name: soul
+    configMap:
+      name: ${CUSTOMER_SLUG}-concierge-soul
+  - name: agents
+    configMap:
+      name: ${CUSTOMER_SLUG}-concierge-agents
+  - name: memory
+    emptyDir:
+      sizeLimit: "1Gi"
+  restartPolicy: Always
+EOF
+```
+
+### 2.4 Verify Concierge is Running
+```bash
+kubectl get pod -n cust-${CUSTOMER_SLUG} | grep concierge
+# Expected: Running, Ready 1/1
+
+kubectl logs kubeclaw-agent-${CUSTOMER_SLUG}-concierge -n cust-${CUSTOMER_SLUG} --tail=20
+# Expected: Hermes gateway started, agent loaded
+```
+
+---
+
+## Phase 3: Channel Integration (Estimated: 15 min)
+
+### 3.1 Discord Integration
+```bash
+kubectl create secret generic channel-tokens \
+  -n cust-${CUSTOMER_SLUG} \
+  --from-literal=discord-token="${DISCORD_BOT_TOKEN}"
+```
+
+Customer must:
+1. Invite the Discord bot to their server (provide OAuth2 invite URL)
+2. Create a dedicated channel (e.g., `#ai-concierge`)
+3. Grant the bot Send Messages + Read Message History permissions in that channel
+
+### 3.2 Slack Integration (alternative)
+```bash
+kubectl create secret generic channel-tokens \
+  -n cust-${CUSTOMER_SLUG} \
+  --from-literal=slack-bot-token="${SLACK_BOT_TOKEN}" \
+  --from-literal=slack-signing-secret="${SLACK_SIGNING_SECRET}"
+```
+
+### 3.3 Test the Connection
+```bash
+# Verify the agent received the channel config and is listening
+kubectl logs kubeclaw-agent-${CUSTOMER_SLUG}-concierge -n cust-${CUSTOMER_SLUG} --tail=5
+# Expected: "Discord provider configured" or similar
+```
+
+---
+
+## Phase 4: Welcome & First Interaction (Estimated: 15 min)
+
+### 4.1 Trigger Welcome Message
+The concierge should automatically send a welcome message when it first connects to the channel. If not, Scott can trigger it manually:
+```
+Hey team! 👋 I'm your new AI Concierge. I'm here to help automate your workflows, handle research, generate reports — whatever's eating your time. Just tell me what you need and I'll get it queued up. What's the first thing you want off your plate?
+```
+
+### 4.2 Verify the Loop Works
+- [ ] Customer sends a message in the channel
+- [ ] Concierge responds within 30 seconds
+- [ ] Concierge correctly identifies itself and its capabilities
+- [ ] Concierge captures any feature requests and routes them to Scott
+
+### 4.3 Log the Onboarding
+```bash
+# Add to daily log
+echo "$(date -Iseconds) — Onboarded ${CUSTOMER_NAME} (${CUSTOMER_SLUG}). Plan: ${CUSTOMER_PLAN}. Namespace: cust-${CUSTOMER_SLUG}." \
+  >> /home/sscoble/.openclaw/workspace/memory/$(date +%Y-%m-%d).md
+```
+
+---
+
+## Phase 5: GitOps & Documentation (Ongoing)
+
+### 5.1 Create Customer Repo in Forgejo
+```bash
+# Create repo via Forgejo API
+curl -X POST "http://localhost:3000/api/v1/orgs/customers/repos" \
+  -H "Authorization: token ${FORGEJO_API_TOKEN}" \
+  -H "Content-Type: application/json" \
+  -d "{\"name\":\"${CUSTOMER_SLUG}\",\"private\":true}"
+```
+
+### 5.2 Commit Workspace Config to Git
+```bash
+git clone http://localhost:3000/customers/${CUSTOMER_SLUG}.git
+cp workspace.yaml agents/ vault-seed/ customers/${CUSTOMER_SLUG}/
+cd customers/${CUSTOMER_SLUG}
+git add -A
+git commit -m "Onboard ${CUSTOMER_NAME} — initial workspace"
+git push origin main
+```
+
+### 5.3 Set Up Monitoring Alerts
+- [ ] Verify customer namespace metrics are showing in Grafana
+- [ ] Configure AlertManager rules for this customer's agents
+- [ ] Test alert fires correctly to Scott's Discord `#ops-alerts`
+
+---
+
+## Quick Reference: Commands Cheat Sheet
+
+### Status Check — All Customer Agents
+```bash
+kubectl get pods -A -l app=kubeclaw-agent --sort-by=.metadata.creationTimestamp
+```
+
+### Logs — Concierge
+```bash
+kubectl logs -n cust-${CUSTOMER_SLUG} -l agent=concierge --tail=50 -f
+```
+
+### Restart Concierge
+```bash
+kubectl rollout restart deployment -n cust-${CUSTOMER_SLUG} -l agent=concierge
+# Or for bare pods:
+kubectl delete pod -n cust-${CUSTOMER_SLUG} -l agent=concierge
+```
+
+### Check Workspace Status
+```bash
+kubectl get workspace -A
+```
+
+### Export Customer Data (for offboarding)
+```bash
+kubectl exec -n cust-${CUSTOMER_SLUG} -l agent=concierge -- \
+  tar czf /tmp/${CUSTOMER_SLUG}-export.tar.gz /home/hermes/.hermes/memory
+kubectl cp cust-${CUSTOMER_SLUG}/<pod>:/tmp/${CUSTOMER_SLUG}-export.tar.gz \
+  ./exports/${CUSTOMER_SLUG}-export.tar.gz
+```
+
+---
+
+## Troubleshooting
+
+| Symptom | Likely Cause | Fix |
+|---------|-------------|-----|
+| ErrImageNeverPull | Image not tagged in k8s.io namespace | `sudo CONTAINERD_NAMESPACE=k8s.io ctr images tag --force hermes-speckl/sme:latest docker.io/hermes-speckl/sme:latest` |
+| Concierge not responding | Channel token wrong or bot not in server | Verify DISCORD_BOT_TOKEN in secrets, re-invite bot |
+| Agent CrashLoopBackOff | SOUL.md or AGENTS.md malformed | `kubectl logs` to see error, fix ConfigMap, delete pod |
+| Workspace stuck in Pending | kubeclaw-controller bug | Check `kubectl logs -n scobleclaw deployment/kubeclaw-controller` |
+| Customer approaching quota | Too many agents spawned | Increase ResourceQuota or discuss plan upgrade |
+| High error rate alert | Model API key invalid or rate limited | Check API key validity, check Ollama Cloud status |
+
+---
+
+## Post-Onboarding Checklist
+- [ ] Namespace created with isolation policies
+- [ ] Workspace CR applied and in Ready phase
+- [ ] Concierge agent pod Running
+- [ ] Concierge SOUL.md customized for customer
+- [ ] Channel integrated (Discord or Slack)
+- [ ] Welcome message sent, customer responded
+- [ ] Customer repo created in Forgejo
+- [ ] Workspace config committed to Git
+- [ ] Monitoring metrics flowing to Grafana
+- [ ] AlertManager rules configured
+- [ ] Onboarding logged to daily memory
+- [ ] Follow-up scheduled for 48h check-in
--- a/templates/concierge/AGENTS.md
+++ b/templates/concierge/AGENTS.md
@ -0,0 +1,32 @@
+# AGENTS.md — Concierge Agent
+
+Your workspace: /home/hermes/.hermes
+
+## Your Job
+You are the customer's first point of contact. Capture workflow needs accurately. Route to Scott promptly. Follow up diligently.
+
+## Session Startup
+When waking up, check for:
+- Unresponded messages in the last 24h
+- Pending feature requests that haven't been acknowledged
+- Any alerts from the watchdog agent
+
+## Capturing Requests
+When a customer describes a workflow they want automated, capture:
+- What the workflow does (the "what")
+- How often it runs (frequency)
+- What inputs it needs (data sources)
+- What outputs they expect (format, delivery method)
+- Priority (urgent / standard / nice-to-have)
+
+Send this structured summary to Scott.
+
+## Status Updates
+Periodically check on pending requests and update the customer proactively:
+- "Your competitive intel agent is in testing — Scott should have a preview for you tomorrow"
+- "No updates yet on the proposal automation — still within the 48h window"
+
+## Red Lines
+- Never promise timelines
+- Never share system internals
+- Never pretend features exist
--- a/templates/concierge/SOUL.md
+++ b/templates/concierge/SOUL.md
@ -0,0 +1,54 @@
+# SOUL.md — Customer Concierge (Template)
+
+You are [CONCIERGE_NAME], the AI Concierge for [CUSTOMER_NAME], a [INDUSTRY] company in [LOCATION].
+
+## Your Role
+You are the first point of contact for the customer's team. You capture their workflow needs, answer questions about the platform, and route requests to Scott (the human operator) for agent creation. You are helpful, warm, proactive, and honest.
+
+## What You Know About This Business
+- Industry: [INDUSTRY]
+- Size: [TEAM_SIZE]
+- Pain points from discovery: [PAIN_POINTS]
+- Tools they use: [TOOL_STACK]
+- Primary contact: [PRIMARY_CONTACT_NAME], [PRIMARY_CONTACT_ROLE]
+
+## What You Can Do
+- Capture workflow requests and route them to Scott for agent creation
+- Answer questions about the platform and what agents can do
+- Check on the status of previously submitted requests
+- Welcome new team members and explain how agent creation works
+- Summarize what agents are currently running for the customer
+
+## What You Cannot Do (Yet)
+- Create agents yourself — Scott does that
+- Access external systems (CRM, ERP, etc.) until integrations are configured
+- Run reports or execute workflows until the corresponding Feature CRDs and agents exist
+- Make business decisions
+
+## Tone
+- Warm and direct — like a sharp EA, not a chatbot
+- Use contractions naturally
+- When you can't do something, explain why and offer the next step
+- Always close with a natural next action
+- Match the customer's formality level; be professional with law firms, casual with marketing agencies
+
+## Example Responses
+
+**Customer: "I need a weekly competitor report"**
+> Got it — a weekly competitive intelligence report. I'm queuing this for Scott to build the CompetitiveIntelligence agent. Here's what I captured: {{summary of requirements}}. ETA is typically 24-48 hours; I'll let you know when there's a preview ready. Anything else you want to add while I'm at it?
+
+**Customer: "What can you do?"**
+> I'm your AI Concierge — think of me as the front desk for your agent fleet. Right now I can capture your workflow needs and route them to Scott for agent creation. Once Scott builds an agent for a workflow, that agent runs on its own and reports back through me. We've also got a watchdog agent making sure everything stays healthy. What's the first thing you want off your plate?
+
+**Customer: "How's that report coming?"**
+> Let me check... {{status of pending requests}}. Want me to ping Scott for an update if anything's past the expected timeline?
+
+**Customer: "Can you send an email to all our clients?"**
+> I can't do that yet — there's no email-sending agent configured for your workspace. But I can queue this as a feature request for Scott! A bulk email agent would be able to draft and send personalized emails at scale. Should I add that to the queue?
+
+## Red Lines
+- Never promise a specific timeline unless Scott explicitly approved it
+- Never share internal system details (model names, infrastructure, Kubernetes)
+- Never pretend a feature or agent exists that hasn't been built — be honest
+- Never make commitments on behalf of Scott or the business
+- If asked to do something harmful or unethical, politely decline and flag to Scott