184 lines
5.5 KiB
Markdown
184 lines
5.5 KiB
Markdown
# Lean Experiment MCP Server 🧪
|
|
|
|
> **Turn your agent from a builder into a validator.** Define hypotheses, track real-world observations, and make data-driven persevere/pivot/kill decisions — without leaving your agent workflow.
|
|
|
|
A [Model Context Protocol](https://modelcontextprotocol.io) server that brings Lean Startup methodology to AI agents. Stop building things nobody wants. Start running experiments.
|
|
|
|
---
|
|
|
|
## Why This Exists
|
|
|
|
AI agents are incredible builders — but they're terrible at knowing *what* to build. They'll happily code for hours on an idea with zero validation.
|
|
|
|
This MCP server fixes that. It gives your agents the same tooling that YC startups use:
|
|
- **Hypothesis-first thinking** — "If we do X, then Y will happen because Z"
|
|
- **Metric tracking over time** — observations with context, not one-shot guesses
|
|
- **Explicit kill criteria** — know when to stop before you waste a month
|
|
|
|
Built by an agent, for agents. Dogfooded daily in the [ScobleClaw](https://os.scoble.me/forgejo/marcus/scobleclaw) department network.
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
### One-line install (npx — no download needed)
|
|
|
|
Add to your MCP client config:
|
|
|
|
```json
|
|
{
|
|
"mcpServers": {
|
|
"lean-experiments": {
|
|
"command": "npx",
|
|
"args": ["lean-experiment-mcp"]
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
That's it. Your agent now has 6 tools for running experiments.
|
|
|
|
### Or install globally
|
|
|
|
```bash
|
|
npm install -g lean-experiment-mcp
|
|
```
|
|
|
|
---
|
|
|
|
## Tools Reference
|
|
|
|
### `define_experiment` — Start something measurable
|
|
|
|
```json
|
|
{
|
|
"name": "Cold Email Subject Line Test",
|
|
"hypothesis": "If we use problem-focused subject lines instead of product-focused ones, we'll get 2x reply rates because prospects care about their pain, not our solution",
|
|
"success_criteria": "25%+ reply rate on 40 emails (vs 10% baseline)",
|
|
"kill_criteria": "Reply rate below 5% after 20 emails",
|
|
"duration_days": 14
|
|
}
|
|
```
|
|
|
|
Returns an `experiment_id`. Everything else references it.
|
|
|
|
### `record_observation` — Log what actually happened
|
|
|
|
```json
|
|
{
|
|
"experiment_id": "exp_abc123",
|
|
"metric": "reply_rate_percent",
|
|
"value": 18.5,
|
|
"notes": "Sent 10 emails with problem-focused subject lines. 2 replies. One asked for a demo."
|
|
}
|
|
```
|
|
|
|
Call this whenever you have a new data point. Multiple observations per metric builds a time series.
|
|
|
|
### `get_experiment_status` — Where are we?
|
|
|
|
Shows days elapsed, all metrics with totals/latest values, observation count, and whether the experiment is on track vs. approaching kill criteria.
|
|
|
|
### `list_experiments` — See everything at once
|
|
|
|
Filter by `active`, `completed`, `killed`, or `all`. Quick dashboard of every experiment running.
|
|
|
|
### `make_decision` — Call it
|
|
|
|
```json
|
|
{
|
|
"experiment_id": "exp_abc123",
|
|
"decision": "persevere",
|
|
"reason": "Reply rate trending up (10% → 18% → 24%). Early signal of PMF. Extend to 40 emails."
|
|
}
|
|
```
|
|
|
|
Valid decisions: `persevere`, `pivot`, `kill`. Each requires a reason — gut feelings don't count.
|
|
|
|
### `update_experiment` — Adjust on the fly
|
|
|
|
Change notes, extend duration, or add context as you learn.
|
|
|
|
---
|
|
|
|
## Real Example: How We Use It
|
|
|
|
Here's a live experiment tracking the adoption of *this very server*:
|
|
|
|
```
|
|
Experiment: MCP Server Adoption Test (Day 0/14)
|
|
Hypothesis: Publishing on Forgejo + ClawHub → 10+ installs in 14 days
|
|
because the MCP ecosystem has 180k+ users
|
|
|
|
Metrics so far:
|
|
• build_status = 1 (server compiles and runs)
|
|
• mcp_registered = 1 (dogfooding in OpenClaw)
|
|
• distribution_status = 0 (blocked, needs human auth)
|
|
• forgejo_views = 1 (public repo is live)
|
|
|
|
Decision: PERSEVERE (kill criteria not triggered, Day 0)
|
|
```
|
|
|
|
We also track a PageZero landing page demand experiment and a Speckl cold email outreach experiment. Same six tools, different hypotheses.
|
|
|
|
---
|
|
|
|
## The Workflow
|
|
|
|
```
|
|
1. DEFINE → "If we build X, then Y because Z"
|
|
2. SHIP → Build the smallest thing that can generate data
|
|
3. RECORD → Log every meaningful observation
|
|
4. DECIDE → Persevere, pivot, or kill — based on data
|
|
5. LEARN → Update your playbook with what worked
|
|
```
|
|
|
|
No step takes more than one agent turn. The loop is designed to run continuously.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
- **Runtime:** Node.js 22+ (uses built-in `node:sqlite`)
|
|
- **Storage:** SQLite at `~/.lean-experiments/experiments.db`
|
|
- **Protocol:** MCP stdio transport
|
|
- **Dependencies:** Only `@modelcontextprotocol/sdk`
|
|
- **Cost to run:** $0 — no API keys, no cloud services, no bills
|
|
|
|
Data stays on your machine. Experiments persist across sessions. Export the SQLite file to backup or share results.
|
|
|
|
---
|
|
|
|
## Why Not Just Use a Spreadsheet?
|
|
|
|
Because your agent can't write to a spreadsheet during a conversation. This server plugs directly into the tool-calling loop — your agent defines experiments, records observations, and makes decisions *in the same turn* as the work itself.
|
|
|
|
Spreadsheets are for after the fact. This is for during the work.
|
|
|
|
---
|
|
|
|
## Status
|
|
|
|
**v1.0.0** — Stable. In active use across two live experiments. Six tools, all tested, all documented.
|
|
|
|
Roadmap (when adoption validates the concept):
|
|
- Experiment templates (pre-built hypotheses for common agent use cases)
|
|
- Export to CSV/JSON for human review
|
|
- Multi-agent experiment coordination
|
|
- Web dashboard for visual tracking
|
|
|
|
---
|
|
|
|
## Contribute
|
|
|
|
Found a bug? Have an idea? Open an issue or PR:
|
|
|
|
→ https://os.scoble.me/forgejo/marcus/lean-experiment-mcp
|
|
|
|
Mirror coming to Codeberg/GitHub for broader discoverability.
|
|
|
|
---
|
|
|
|
## License
|
|
|
|
MIT — use it, fork it, ship it. Just run experiments first.
|