docs.html

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <title>Docs - Picado Labs</title>
  <meta name="description" content="Documentation for the Picado Labs open-source AI engineering stack: build-with-ai, Model Router, Terminal Agent, and AgentBench." />
  <link rel="icon" type="image/png" href="assets/img/logo.png" />
  <link rel="preconnect" href="https://fonts.googleapis.com" />
  <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
  <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700;800&family=JetBrains+Mono:wght@500;700&display=swap" rel="stylesheet" />
  <link rel="stylesheet" href="assets/css/styles.css" />
</head>
<body>
  <a class="skip" href="#main">Skip to content</a>

  <header class="nav" id="nav">
    <div class="wrap nav__inner">
      <a class="brand" href="index.html#top" aria-label="Picado Labs home">
        <img src="assets/img/logo-mark.png" alt="Picado Labs" width="28" height="28" />
        <span>Picado Labs</span>
      </a>
      <nav class="nav__links" id="navLinks" aria-label="Primary navigation">
        <a href="index.html#stack">Stack</a>
        <a href="index.html#architecture">Architecture</a>
        <a href="index.html#projects">Projects</a>
        <a href="docs.html">Docs</a>
      </nav>
      <div class="nav__actions">
        <a class="btn btn--ghost btn--sm" href="https://x.com/PicadoLabs" target="_blank" rel="noopener">X</a>
        <a class="btn btn--solid btn--sm" href="https://github.com/PicadoLabs" target="_blank" rel="noopener">GitHub</a>
        <button class="mobile-toggle" id="mobileToggle" type="button" aria-label="Toggle menu">Menu</button>
      </div>
    </div>
  </header>

  <main id="main">
    <section class="hero docs-hero">
      <div class="wrap hero__grid">
        <div class="hero__copy">
          <p class="eyebrow">Documentation</p>
          <h1>The Picado Labs stack, explained.</h1>
          <p class="hero__lede">
            Four open-source projects that turn AI coding from a prompt into a verified engineering loop: plan, route, execute, verify, and evaluate.
          </p>
          <div class="hero__actions">
            <a class="btn btn--solid" href="#ecosystem">Start reading</a>
            <a class="btn btn--ghost" href="index.html">Back home</a>
          </div>
        </div>

        <div class="hero__panel">
          <div class="panel-top">
            <span>Docs map</span>
            <strong>4 projects</strong>
          </div>
          <ol class="loop-list">
            <li><span class="step">01</span><div><strong>build-with-ai</strong><small>Plan projects with context memory</small></div></li>
            <li><span class="step">02</span><div><strong>Model Router</strong><small>Route prompts to the best model</small></div></li>
            <li><span class="step">03</span><div><strong>Terminal Agent</strong><small>Execute and verify in a sandbox</small></div></li>
            <li><span class="step">04</span><div><strong>AgentBench</strong><small>Benchmark agent quality</small></div></li>
          </ol>
        </div>
      </div>
    </section>

    <section class="docs-layout">
      <div class="wrap docs-shell">
        <aside class="docs-toc" aria-label="Documentation contents">
          <strong>Contents</strong>
          <a href="#ecosystem">What is Picado Labs?</a>
          <a href="#closed-loop">Closed-loop architecture</a>
          <a href="#build-with-ai">build-with-ai</a>
          <a href="#model-router">Model Router</a>
          <a href="#terminal-agent">Terminal Agent</a>
          <a href="#agent-bench">AgentBench</a>
          <a href="#comparison">Why this stack?</a>
          <a href="#quickstart">Quickstart</a>
        </aside>

        <div class="docs-content">
          <section class="doc-section" id="ecosystem">
            <p class="eyebrow">Overview</p>
            <h2>What is Picado Labs?</h2>
            <p>
              Picado Labs is an open-source ecosystem for building autonomous AI software agents that can be inspected, routed, executed, verified, and benchmarked.
            </p>
            <div class="doc-grid doc-grid--3">
              <article class="doc-card">
                <span class="tag">Problem</span>
                <h3>Prompt-only agents drift.</h3>
                <p>Long chats lose context, model choice is often fixed, and success is too often accepted from the model's own claim.</p>
              </article>
              <article class="doc-card">
                <span class="tag">Approach</span>
                <h3>Make the loop explicit.</h3>
                <p>Picado Labs separates planning, routing, execution, verification, and evaluation into focused tools.</p>
              </article>
              <article class="doc-card">
                <span class="tag">Outcome</span>
                <h3>Proof before ship.</h3>
                <p>Agents must pass objective checks, run in bounded environments, and improve through benchmark feedback.</p>
              </article>
            </div>
          </section>

          <section class="doc-section" id="closed-loop">
            <p class="eyebrow">Architecture</p>
            <h2>Closed-loop AI engineering pipeline</h2>
            <p>
              The stack is designed as a feedback loop. Each stage produces a concrete artifact for the next stage: plan, selected model, code changes, verification results, and benchmark telemetry.
            </p>
            <pre class="diagram"><code>[ Software Goal / Spec ]
          |
          v
  01 PLAN          build-with-ai
          |
          v
  02 ROUTE         Model Router
          |
          v
  03 EXECUTE       Terminal Agent
          |
          v
  04 VERIFY        Tests + AST validation
      |       |
    PASS    FAIL ----> Repair / rollback ----+
      |                                      |
      v                                      |
  05 EVALUATE      AgentBench                |
      |                                      |
      +---- telemetry improves router/agent -+</code></pre>
          </section>

          <section class="doc-section project-doc" id="build-with-ai">
            <div class="doc-title-row">
              <div>
                <p class="eyebrow">01 / Plan</p>
                <h2>build-with-ai</h2>
              </div>
              <div class="doc-title-links">
                <a class="text-link" href="https://github.com/PicadoLabs/build-with-ai" target="_blank" rel="noopener">Repository</a>
                <span class="link-sep">•</span>
                <a class="text-link" href="https://github.com/PicadoLabs/build-with-ai/issues" target="_blank" rel="noopener">Issues</a>
              </div>
            </div>
            <p>
              <strong>What is this?</strong> A zero-API CLI prompt architect (15+ contributors • 1.2k downloads). It helps a developer turn a software idea into ordered implementation phases, reusable decisions, and copy-ready prompts for any AI assistant.
            </p>
            <div class="doc-grid">
              <article class="doc-card">
                <h3>How it works</h3>
                <p>It creates a local `.buildwithai/` workspace, stores project context, tracks completed phases, and injects past decisions into future prompts.</p>
              </article>
              <article class="doc-card">
                <h3>Best for</h3>
                <p>Project setup, architecture planning, reducing chat drift, and keeping AI-assisted development structured without API keys.</p>
              </article>
            </div>
            <pre class="diagram"><code>Developer goal
    |
    v
build-with-ai init
    |
    +--> project template
    +--> .buildwithai/context.json
    +--> .buildwithai/state.json
    |
    v
next prompt -> AI assistant -> developer records decisions -> next phase</code></pre>
            <pre><code>npx build-with-ai init
npx build-with-ai next
npx build-with-ai done
npx build-with-ai export</code></pre>
          </section>

          <section class="doc-section project-doc" id="model-router">
            <div class="doc-title-row">
              <div>
                <p class="eyebrow">02 / Route</p>
                <h2>Model Router</h2>
              </div>
              <div class="doc-title-links">
                <a class="text-link" href="https://github.com/PicadoLabs/ai-model-router" target="_blank" rel="noopener">Repository</a>
                <span class="link-sep">•</span>
                <a class="text-link" href="https://github.com/PicadoLabs/ai-model-router/issues" target="_blank" rel="noopener">Issues</a>
              </div>
            </div>
            <p>
              <strong>What is this?</strong> An explainable AI traffic controller (5+ contributors • 2.2k downloads) that routes each task to the best local or cloud model using quality, cost, latency, capability, and reliability signals.
            </p>
            <div class="doc-grid">
              <article class="doc-card">
                <h3>How it works</h3>
                <p>It extracts task features, removes models that cannot satisfy hard requirements, then uses Pareto scoring to select the best candidate in under 3ms.</p>
              </article>
              <article class="doc-card">
                <h3>Best for</h3>
                <p>Reducing model cost, using local Ollama when possible, routing hard reasoning tasks to stronger models, and auditing routing decisions.</p>
              </article>
            </div>
            <pre class="diagram"><code>Incoming prompt
    |
    v
Feature extraction: task type, complexity, context size
    |
    v
Hard filters: context window, tools, vision, coding, reasoning
    |
    v
Pareto scoring: quality + cost + latency + reliability
    |
    v
Selected model -> response + telemetry</code></pre>
            <pre><code>pip install model-router-cli
modelrouter doctor
modelrouter route "Write an optimized async task worker in Python"
modelrouter ui --open-browser</code></pre>
          </section>

          <section class="doc-section project-doc" id="terminal-agent">
            <div class="doc-title-row">
              <div>
                <p class="eyebrow">03 / Execute + Verify</p>
                <h2>Terminal Agent</h2>
              </div>
              <div class="doc-title-links">
                <a class="text-link" href="https://github.com/PicadoLabs/terminal-agent" target="_blank" rel="noopener">Repository</a>
                <span class="link-sep">•</span>
                <a class="text-link" href="https://github.com/PicadoLabs/terminal-agent/issues" target="_blank" rel="noopener">Issues</a>
              </div>
            </div>
            <p>
              <strong>What is this?</strong> A verify-first autonomous coding agent (2+ contributors • 440+ downloads) that edits files, runs commands, tests work, classifies failures, and can roll back checkpoints.
            </p>
            <div class="doc-grid">
              <article class="doc-card">
                <h3>How it works</h3>
                <p>It observes the task and workspace, plans a bounded implementation, acts through typed tools, runs verification, then repairs or rolls back when checks fail.</p>
              </article>
              <article class="doc-card">
                <h3>Best for</h3>
                <p>Agentic coding tasks where success must be backed by tests, process limits, secret redaction, and repeatable recovery behavior.</p>
              </article>
            </div>
            <pre class="diagram"><code>Task + repository
    |
    v
Observe -> Plan -> Act with engineering tools
    |
    v
Run in LocalSandbox or DockerSandbox
    |
    v
IndependentVerifier executes tests
    |
    +--> pass: proof of done
    +--> fail: classify failure -> repair plan -> retry or rollback</code></pre>
            <pre><code>pip install terminal-agent-cli
terminal-agent doctor
terminal-agent run "Implement JWT auth middleware with pytest tests"
terminal-agent resume</code></pre>
          </section>

          <section class="doc-section project-doc" id="agent-bench">
            <div class="doc-title-row">
              <div>
                <p class="eyebrow">04 / Evaluate</p>
                <h2>AgentBench</h2>
              </div>
              <div class="doc-title-links">
                <a class="text-link" href="https://github.com/PicadoLabs/agent-bench" target="_blank" rel="noopener">Repository</a>
                <span class="link-sep">•</span>
                <a class="text-link" href="https://github.com/PicadoLabs/agent-bench/issues" target="_blank" rel="noopener">Issues</a>
              </div>
            </div>
            <p>
              <strong>What is this?</strong> A provider-agnostic benchmark platform (3+ contributors) for measuring coding agents on realistic Python and Node.js software-engineering tasks.
            </p>
            <div class="doc-grid">
              <article class="doc-card">
                <h3>How it works</h3>
                <p>It runs agents against standardized tasks, validates output with tests, classifies failures, and calculates a 5D SWE score.</p>
              </article>
              <article class="doc-card">
                <h3>Best for</h3>
                <p>Comparing agents, calibrating model choices, evaluating PR-fix workflows, and building confidence before deployment.</p>
              </article>
            </div>
            <pre class="diagram"><code>Benchmark task
    |
    v
Clean sandbox -> agent under test -> code edits
    |
    v
Independent test suite
    |
    v
Score = correctness + pass rate + quality + efficiency + repair
    |
    v
Dashboard, telemetry, leaderboard, CI gate</code></pre>
            <pre><code>python agentbench.py doctor
python agentbench.py init
python agentbench.py list
python agentbench.py run --benchmark fix-rate-limiter --provider ollama</code></pre>
          </section>

          <section class="doc-section" id="comparison">
            <p class="eyebrow">Why this stack?</p>
            <h2>Picado Labs replaces AI self-reporting with engineering evidence.</h2>
            <div class="comparison-table" role="table" aria-label="Picado Labs comparison">
              <div role="row"><strong role="cell">Need</strong><strong role="cell">Typical AI wrapper</strong><strong role="cell">Picado Labs</strong></div>
              <div role="row"><span role="cell">Planning</span><span role="cell">One long prompt</span><span role="cell">Structured phases with persistent context</span></div>
              <div role="row"><span role="cell">Model choice</span><span role="cell">Fixed provider</span><span role="cell">Explainable local/cloud routing</span></div>
              <div role="row"><span role="cell">Execution</span><span role="cell">Direct host commands</span><span role="cell">Bounded process sandbox and policies</span></div>
              <div role="row"><span role="cell">Verification</span><span role="cell">Model says it is done</span><span role="cell">Independent tests and validation</span></div>
              <div role="row"><span role="cell">Evaluation</span><span role="cell">Subjective demos</span><span role="cell">Repeatable benchmark tasks and 5D scoring</span></div>
            </div>
          </section>

          <section class="doc-section" id="quickstart">
            <p class="eyebrow">Quickstart</p>
            <h2>Run the ecosystem path.</h2>
            <p>
              Start with the planning primitive, then add routing, execution, and evaluation as your workflow becomes more automated.
            </p>
            <pre><code># 1. Plan
npx build-with-ai init
npx build-with-ai next

# 2. Route
python backend/app/cli/main.py route "Build a secure token bucket rate limiter"

# 3. Execute + verify
terminal-agent run "Implement token bucket rate limiter with pytest tests"

# 4. Evaluate
python agentbench.py run --benchmark fix-rate-limiter --provider ollama</code></pre>
          </section>
        </div>
      </div>
    </section>
  </main>

  <footer class="footer">
    <div class="wrap footer__grid">
      <div class="footer__brand">
        <div class="footer__logo-row">
          <img src="assets/img/logo.png" alt="Picado Labs" width="28" height="28" />
          <span class="footer__title">Picado Labs</span>
        </div>
        <p class="footer__tagline">Deterministic, verify-first software systems for autonomous AI agents.</p>
        <p class="footer__maintainer">
          Founder and Maintainer <strong><a href="https://github.com/Kaap10" target="_blank" rel="noopener">@Vardhman Gupta</a></strong>
        </p>
      </div>

      <div class="footer__nav">
        <div class="footer__col">
          <strong>Organization</strong>
          <a href="https://github.com/PicadoLabs" target="_blank" rel="noopener">GitHub Org</a>
          <a href="https://www.linkedin.com/company/picado-labs/" target="_blank" rel="noopener">LinkedIn</a>
          <a href="https://x.com/PicadoLabs" target="_blank" rel="noopener">X (@PicadoLabs)</a>
          <a href="mailto:picadolabs@gmail.com">Contact</a>
        </div>
        <div class="footer__col">
          <strong>Projects</strong>
          <a href="https://github.com/PicadoLabs/build-with-ai" target="_blank" rel="noopener">build-with-ai</a>
          <a href="https://github.com/PicadoLabs/ai-model-router" target="_blank" rel="noopener">Model Router</a>
          <a href="https://github.com/PicadoLabs/terminal-agent" target="_blank" rel="noopener">Terminal Agent</a>
          <a href="https://github.com/PicadoLabs/agent-bench" target="_blank" rel="noopener">AgentBench</a>
        </div>
      </div>
    </div>
    <div class="wrap footer__bottom">
      <span>Open source infrastructure under Apache 2.0 License.</span>
      <a href="https://picadolabs.me">picadolabs.me</a>
    </div>
  </footer>

  <script src="assets/js/script.js"></script>
</body>
</html>