Skip to content

Blackbird prompts: pitch-only / incubator refactor (prompts + docs) - #34

Open
andrewsu wants to merge 6 commits into
blackbirdfrom
blackbird-prompt-refactor
Open

Blackbird prompts: pitch-only / incubator refactor (prompts + docs)#34
andrewsu wants to merge 6 commits into
blackbirdfrom
blackbird-prompt-refactor

Conversation

@andrewsu

Copy link
Copy Markdown
Member

What this PR does

Reframes the coPI agent prompts — and their Blackbird-facing docs — from the original
lab↔lab collaboration model to a pitch-only / incubator model: each lab pitches one
idea to BlackbirdBot, which interviews it against a rubric and files an Opportunity
Assessment. Labs never talk to each other.

Scope is prompts + docs. The only code touched is prompt-string content in
thread_guidance.py (Baltimore removal). The engine still implements parts of the
collaboration model — see Remaining tension below, which is intended for a separate
reviewed change.

Commits (6)

  • a376d44 docs(specs): reframe the two prompt-set docs for the end-user audience
  • 55e89f8 feat(prompts): apply the incubator reframe to the on-disk prompt files
  • dcc5212 refactor(prompts): drop Baltimore location gating and trim redundancy
  • 96e2522 refactor(prompts): make intake pitch-only on both sides
  • fc8d8a9 docs(specs): add hub↔lab flow schematic; trim dormant Phase 2 from the prompt-set docs
  • 66ed1b8 docs(prompts): reframe the PI "about a paper" guidance for pitch-only

Blackbird-facing review docs


Remaining tension: pitch model vs. collaboration model

TL;DR

This PR converts the prompts and docs to a pitch-only / incubator model, but large
parts of the engine still implement the original lab↔lab collaboration model. The prompts
now tell each bot to do one thing while some of the code still drives the other. CI is red
until the code and the golden-master snapshots are reconciled.

The two models

  • Collaboration model (original, what the code was built for): a lab↔lab mesh. Labs
    discover each other's posts, propose collaborations, exchange :memo: Summary proposals,
    confirm with , plus a funding/GrantBot/FOA track. Post types include paper,
    help_wanted, introduction, idea_crosslab, funding_collab.
  • Pitch model (intended, what this PR describes): a star. Labs never talk to each
    other; each lab pitches one idea to BlackbirdBot, which interviews it against a
    rubric and files an Opportunity Assessment. No collaboration, no :memo:, no , no
    scouting.

Already converted (prompts + docs only)

  • All prompt .md files for both roles → pitch-only (system prompt, phase-4 interview,
    phase-5 post, phase-2 turned into no-ops).
  • Baltimore location gating removed everywhere (including _SCOUT_HUB guidance in code).
  • Both Blackbird-facing docs streamlined + a flow schematic added.
  • The hub side is essentially consistent (its _SCOUT_HUB guidance was already
    incubator-aligned; its role.toml already restricts posts to opportunity_assessment).

Where the collaboration model still lives — the conflicts to resolve

# Location Problem Severity
1 src/agent/thread_guidance.py_PI_LAB (≈lines 23–49) The lab bot's actual interview guidance still says: ask about "the other lab's capabilities," look for "complementarity," name "a first experiment," "build toward a :memo: Summary proposal," and confirm a peer's :memo: with . This is the opposite of the pitch-only prompt wrapping it. Blocked by CLAUDE.md: those pi_lab strings are declared byte-identical/immutable and snapshot-pinned — rewording them needs sign-off + the snapshot regen. The intended replacement text already exists as §4 of the PI doc. Highest — without this the lab bot behaves collaboration-style regardless of every other prompt.
2 src/agent/post_types.pyDEFAULT_POST_TYPES (≈line 91) Still the pre-reframe six types (paper, help_wanted, introduction, idea_crosslab, pitch, funding_collab). Since pi_lab has no role.toml, this tuple is the lab bot's menu, so the "authoritative" {post_type_menu} still offers collaboration/paper posts the prompts say don't exist. Should narrow to (pitch,). Unit-test deps: test_phase5_terminal_posts.py posts a paper; test_agent_prompts.py iterates the tuple. High
3 simulation.py :memo:→Proposal path + prompts/email-reply-classify.md The collaboration-proposal lifecycle (a :memo: confirmed by creates a Proposal row) still exists. The prompts forbid it, so it's dormant once #1 lands — but it's collaboration-model residue to remove or consciously keep. Medium
4 tests/characterization/__snapshots__/test_agent_turn_gm.ambr + test_phase5_prompt_gm Golden masters are pinned to the old prompts (old system prompt, _PI_LAB, six-type menu). This PR's .md changes already diverged them, so these tests fail today and ci.sh is red. They must be regenerated as a reviewed diff once #1/#2 land. Gate to merge
5 Phase 2 scan/prune No-ops now, but still called every turn (no phase-skip guard). Tracked in #33. Low (cost only)
6 GrantBot/FOA residue {foa_number} still substituted into the phase-4 template as literal "none"; #funding-opportunities still a universal channel; ## Your Private Instructions still injected by agent.py; GrantBot process/funding_collab/retrieve_foa still present. Low / separate scope

Consequences right now

  • The running lab bot is told pitch-only by its system prompt but told to broker
    collaborations by its phase guidance (Co pi podcast #1) — an internal contradiction in production behavior.
  • The lab bot's post menu can still offer paper/help_wanted/etc. (future ideas #2).
  • CI is red until the snapshots are regenerated (New LLM harness for making calls #4).

Suggested resolution order

  1. Decide Co pi podcast #1 (reword _PI_LAB) — the crux; needs a call on overriding the CLAUDE.md
    "do not reword" guard. Target text is PI-doc §4.
  2. future ideas #2 narrow DEFAULT_POST_TYPES to (pitch,) + fix the two unit tests.
  3. Turn based blocking and blank LLM response #3 decide: delete the :memo://Proposal lifecycle, or leave dormant.
  4. New LLM harness for making calls #4 regenerate golden masters as one reviewed diff (only after Co pi podcast #1Turn based blocking and blank LLM response #3 stabilize).
  5. investigate bad paper links #5 (Remove the dormant Phase 2 (scan + prune) prompts under pitch-only intake #33) add the Phase-2 skip guard + delete the four dormant files.
  6. deal with waitlist spam #6 scope the GrantBot/private-instructions cleanup separately.

andrewsu and others added 6 commits August 7, 2026 17:22
These are meant for Blackbird employees to comment on the prompt-driven
behavior of the bots, not for a developer tracking a change. Strip the
developer meta accordingly: the "Revision N" changelogs, the
PROPOSED/UNCHANGED labels, the prompt-assembly/code mechanics with file
and line references, and the code-change appendices. Retitle the
file-path section headers to plain-language names, keep every prompt body
in full, and add a per-section "Source:" line pointing at the file each
block quotes.

Note: the quoted blocks still reflect the proposed prompt text, which does
not yet match the on-disk prompt files verbatim; reconciling that is a
follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Overwrite the PI (pi_lab / global) and hub (scout_hub) system and phase
prompts with the proposed text documented in
docs/specs/2026-08-07-{pi,hub}-bot-prompts.md. This moves both roles off
the old lab-to-lab scientific-collaboration model and onto the Blackbird
incubator/investor model: GrantBot/FOA and funding threads removed, no
lab-to-lab collaboration or brokered introductions, private instructions
dropped, and the screening rubric folded into the scout_hub system prompt.

identity.md (both) and the eight prompts/specialists/*.md were already
identical and are left untouched. The two "interview phase guidance"
sections are sourced from src/agent/thread_guidance.py (Python, not an md
file) and are out of scope here.

These prompts assume accompanying code changes not made in this commit
(e.g. the Phase-4 {foa_number} substitution, the PI post-type list, and
the #funding-opportunities universal channel).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Blackbird will evaluate the Baltimore-anchoring commitment outside this
system, so remove it as a screening dimension everywhere it drives bot
behavior:

- PI + hub agent-system / phase4 / phase5 prompts: the gating criterion,
  the interview questions, the red flag, the decision-heuristic clause,
  the deferral examples, and the `baltimore_commitment` key in the
  assessment JSON skeleton. Dropping the key is safe — `gating` is plain
  JSONB filtered by `_normalize_gating`, and no schema column or consumer
  reads it.
- src/agent/thread_guidance.py: the `_SCOUT_HUB` DECIDE guidance (the one
  remaining live "ask whether they'd anchor in Baltimore" instruction).
  The snapshot-pinned `_PI_LAB` strings have no Baltimore and are left
  byte-identical.
- Both companion docs: §6 phase-guidance blocks stripped, and the six
  changed prompt blocks resynced so the docs still mirror the files.

The Maryland non-dilutive funding programs (TEDCO/MSCRF/MII/QOF) are kept
— they are capital sources, not location gating.

Also removes a few clear redundancies: the PI "Who You Can Reach" section
(covered by Core Rule 4 + Post Labels; its one unique point folded into
Core Rule 4), the redundant "never ask to be introduced" conclusion
bullet, and the hub Post-Labels pitch-intake paragraph (restated
Interview Structure).

Note: test_agent_turn_gm's golden-master snapshot still holds the
pre-reframe collaboration prompts (it diverged at 55e89f8); regenerating
it as a reviewed diff is deferred.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A lab agent's only top-level post is now a 💡 pitch, and the hub only
ever interviews pitches — it no longer scouts unsolicited results.

Lab agent (prompts/agent-system.md, phase5-new-post.md):
- Drop the 📰 Result post type from Post Labels, "how an
  interview starts", and Phase 5. If the lab cannot turn something into a
  pitch, it does not post.

Hub (prompts/roles/scout_hub/*):
- agent-system: an interview begins only when a PI opens a 💡 pitch
  (auto-routed as a thread); the hub does not scout and never opens a
  thread at a lab itself.
- phase2 scan + prune: now no-ops — with labs posting only pitches (which
  reach the hub as Phase 3 threads, not this list) and the hub's own 🔍
  assessments excluded, there is nothing to select or keep.
- phase4: interviews are unchanged; the "about a paper" note reframed
  from result-scouting to a pitch that builds on published work.
- phase5: the "reply to an interesting post" option is gone (nothing
  populates the interesting list); the assessment is now Option A and skip
  Option B, and the action JSON drops "reply"/target_post_id.

Docs mirror the files. Prompt-only change: DEFAULT_POST_TYPES
(post_types.py) still lists `paper` and the other pre-reframe types, so
the rendered menu and the golden-master snapshot regen remain deferred.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e prompt-set docs

- Add docs/specs/2026-08-07-hub-lab-flow.md — a Mermaid schematic for the
  Blackbird-staff audience showing the pitch → interview → assessment
  cycle, the per-turn phase pipeline, and the interview's message-count
  progression.
- Streamline both prompt-set docs so reviewers see only the prompts that
  actually drive an exchange: drop the two dormant Phase 2 sections
  (Scanning / Trimming — no-ops under pitch-only intake), renumber the
  remaining sections, and reframe the intros away from "full set" /
  "scanning".

The Phase 2 prompt files still exist on disk; skipping the phase and
deleting them is tracked in #33. These docs are now curated
"used-prompts" references, not complete on-disk mirrors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Under pitch-only intake an interview starts from the lab's own pitch, not
from the hub scouting results. Rewrite the PI phase-4 "about a paper"
section from result-scouting framing ("the hub reads results looking for
something worth screening") to "if your pitch builds on one of your lab's
papers", matching the hub-side reframe. Mirrored into the PI doc.

The deeper PI-side gap — the _PI_LAB phase guidance in
src/agent/thread_guidance.py still uses the collaboration model
(:memo:/complementarity/first-experiment) and is snapshot-pinned — is
left for a separate reviewed change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@andrewsu
andrewsu requested a review from ahueb August 12, 2026 06:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants