Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .changeset/sour-grapes-switch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
---
---
58 changes: 48 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,53 @@
# Agent Trail TypeScript SDK

> [!NOTE]
> This repository owns SDK package behavior. Agent Trail wire-format decisions
> live in the spec repository.

TypeScript packages for Agent Trail schema assets, generated types, core JSONL
utilities, shared render models, redaction, catalog metadata, and
content-addressed local storage.
utilities, adapter authoring, source-agent adapters, redaction, catalog metadata,
render models, workflow orchestration, and content-addressed local storage.

## Related Repositories
## Packages

Agent Trail is split across focused repositories:
| Package | Purpose |
| --- | --- |
| [`@agent-trail/schema`](./packages/schema) | Vendored Agent Trail schema and validation fixtures. |
| [`@agent-trail/types`](./packages/types) | Generated TypeScript declarations for Agent Trail records. |
| [`@agent-trail/core`](./packages/core) | JSONL parsing, validation, hashing, serialization, and reconciliation. |
| [`@agent-trail/adapter-kit`](./packages/adapter-kit) | Reader, mapping, source-schema, and reconciler primitives for adapters. |
| [`@agent-trail/source-schemas`](./packages/source-schemas) | JSON Schemas for supported upstream source-agent records. |
| [`@agent-trail/adapters`](./packages/adapters) | Concrete adapters for supported coding agents. |
| [`@agent-trail/catalog`](./packages/catalog) | SQLite catalog primitives for source sessions, stored objects, and shares. |
| [`@agent-trail/redact`](./packages/redact) | Redaction APIs and configuration loading for shared trails. |
| [`@agent-trail/store`](./packages/store) | Content-addressed local store for finalized trail artifacts. |
| [`@agent-trail/render-model`](./packages/render-model) | Renderer-agnostic transcript model for viewers. |
| [`@agent-trail/sessions`](./packages/sessions) | Workflow APIs for discover, list, load, share, and export. |

- [agent-trail/spec](https://github.com/agent-trail/spec) - format contract, JSON Schema, fixtures, and format ADRs.
- [agent-trail/typescript-sdk](https://github.com/agent-trail/typescript-sdk) - TypeScript packages for Agent Trail files.
- [agent-trail/cli](https://github.com/agent-trail/cli) - command-line tools for Agent Trail workflows.
- [agent-trail/web](https://github.com/agent-trail/web) - docs site and shared trail web viewer.
## Contributor Docs

- [`docs/implementation-semantics.md`](./docs/implementation-semantics.md) -
SDK runtime behavior and package boundaries.
- [`docs/adapter-authoring.md`](./docs/adapter-authoring.md) - checklist for
external adapter authors and SDK adapter maintainers.
- [`docs/parser-source-matrix.md`](./docs/parser-source-matrix.md) - supported
source-agent formats, fixture evidence, and update process.
- [`docs/redaction.md`](./docs/redaction.md) - redaction workflow, public API,
configuration, and safety notes.
- [`docs/GLOSSARY.md`](./docs/GLOSSARY.md) - SDK-owned terminology.
- [`docs/adr/0001-sdk-package-architecture-and-public-api-boundaries.md`](./docs/adr/0001-sdk-package-architecture-and-public-api-boundaries.md) -
package architecture and public API policy.

## Related Repositories

- [agent-trail/spec](https://github.com/agent-trail/spec) - format contract,
JSON Schema source, fixtures, and format ADRs.
- [agent-trail/typescript-sdk](https://github.com/agent-trail/typescript-sdk) -
TypeScript packages for Agent Trail files.
- [agent-trail/cli](https://github.com/agent-trail/cli) - command-line tools for
Agent Trail workflows.
- [agent-trail/web](https://github.com/agent-trail/web) - docs site and shared
trail web viewer.

## Development

Expand All @@ -20,8 +56,10 @@ mise run setup
mise run check
```

See `CONTRIBUTING.md` for workflow and PR expectations.
Use `mise run check:actions` after editing GitHub Actions workflows.

See [`CONTRIBUTING.md`](./CONTRIBUTING.md) for workflow and PR expectations.

## License

MIT. See `LICENSE`.
MIT. See [`LICENSE`](./LICENSE).
186 changes: 186 additions & 0 deletions docs/adapter-authoring.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
# Adapter Authoring Guide

> [!TIP]
> Use this guide for new adapters and substantial adapter updates. External
> adapters should build on `@agent-trail/adapter-kit`; SDK adapters live in
> `@agent-trail/adapters`.

## Audience

| Audience | Use this path |
| --- | --- |
| External adapter author | Use public `@agent-trail/adapter-kit` exports. |
| SDK maintainer | Add concrete adapter code under `packages/adapters/src/<agent>`. |

Both paths should emit writer-strict Agent Trail records and keep source drift
visible.

## Done Definition

A supported adapter has:

- [ ] Stable adapter name.
- [ ] Documented source storage roots and override options.
- [ ] Source schemas for verified upstream record shapes.
- [ ] Source discovery or explicit parse input contract.
- [ ] Deterministic Agent Trail entry ids.
- [ ] Writer-strict header, entries, envelope, and content hashes.
- [ ] Synthetic or redacted committed fixtures.
- [ ] Focused tests for mapping, discovery, drift, and health behavior.
- [ ] Parser-source-matrix evidence.
- [ ] No runtime-specific imports from default package roots.

## External Adapter Path

Use public adapter-kit APIs only:

| Need | API |
| --- | --- |
| JSONL source | `JsonlReader` |
| SQLite source | `SqliteReader` with injected driver |
| Sequential readers | `chainReaders` |
| Timestamp merge | `mergeByTimestamp` |
| Pure mapping | `defineMapping` |
| Parse orchestration | `defineAdapter` |
| Source schema selection | `selectSchemaVersion` |
| Source record validation | `validateSourceRecord` |

Avoid:

- importing `@agent-trail/adapters/src/**`
- putting source-specific coercion into adapter-kit
- mutating global environment during parse
- fallback behavior that hides source drift

## SDK Maintainer Path

Concrete SDK adapters belong in `packages/adapters/src/<agent>`.

| Step | Action |
| --- | --- |
| 1 | Survey source storage and document root resolution. |
| 2 | Add or update source schemas in `@agent-trail/source-schemas`. |
| 3 | Register schema selection in `@agent-trail/adapter-kit`. |
| 4 | Add reader and discovery behavior behind factory options. |
| 5 | Map source records to Agent Trail entries with deterministic ids. |
| 6 | Validate emitted trails through `@agent-trail/core`. |
| 7 | Add synthetic or redacted fixtures. |
| 8 | Update `docs/parser-source-matrix.md`. |
| 9 | Keep public exports factory-first and minimal. |

Shared implementation code belongs in `packages/adapters/src/shared` only when
at least two concrete adapters use it.

## Source Survey

Capture source truth before writing mapper code:

| Survey item | Examples |
| --- | --- |
| Storage | File paths, SQLite tables, object trees. |
| Overrides | Env vars, platform defaults, explicit factory options. |
| Versioning | Source version fields and fallback behavior. |
| Identity | Stable source ids, parent ids, branch refs. |
| Event families | Messages, tools, reasoning, compaction, lifecycle, models. |
| Artifacts | Attachments, media, overflow refs. |
| Privacy | Credentials, paths, private repo identity, PII-like fields. |

If upstream writer code is public, use it as evidence. If it is closed, rely on
redacted fixtures and observed source files.

## Source Schemas

Source schemas validate upstream records before mapping.

| Good schema behavior | Bad schema behavior |
| --- | --- |
| Catch new record families. | Reject harmless additive fields without reason. |
| Keep source drift visible. | Replace Agent Trail output validation. |
| Document source evidence. | Encode adapter implementation details. |

For SDK adapters:

1. Add schema files under `packages/source-schemas/<agent>`.
2. Update package exports.
3. Run `bun run generate:source-types`.
4. Register schema in `packages/adapter-kit/src/source-schemas/registry.ts`.
5. Add corpus tests when fixtures exercise the schema.

## Mapping

| Prefer | Avoid |
| --- | --- |
| Facts present in source data. | Invented fields with no source evidence. |
| Pure `defineMapping` functions. | Stateful mapping unless needed. |
| `source.raw` for useful evidence. | Adapter-private temp state in output. |
| Marked synthesized entries. | Pretending synthesized events were source-native. |

Use adapter-kit overrides for stateful source behavior, such as pairing records
without direct ids or collapsing multi-record source events.

## Reconciliation

Enable only passes that match source topology:

| Pass | Use when |
| --- | --- |
| `toolLinking` | Mappings emit linker metadata for call/result pairs. |
| `parentChain` | Transcript is linear. |
| `cumulativeTokens` | Source emits per-turn usage but no running totals. |
| custom passes | Adapter has source-specific relationships. |

Tree-shaped sources should set parent ids directly until a shared branch
reconciler is available for their topology.

## Fixtures

> [!WARNING]
> Never commit real local sessions, credentials, private paths, private remotes,
> repository identity, or unredacted transcript data.

Fixture rules:

- use synthetic fixtures for focused behaviors
- use manually redacted real-source fixtures when source shape matters
- prefer exact checked-in `.trail.jsonl` goldens
- restamp expected output only for documented SDK behavior changes

Storage-tree adapters may use fixture-building tests when one source file is not
the native storage shape.

## Real-Session Smoke Tests

Real-session tests are local-only and opt-in.

| Requirement | Reason |
| --- | --- |
| Skip in CI | Avoid leaking local data or requiring local agents. |
| Require explicit env or root option | Avoid accidental source discovery. |
| Check broad invariants | Real transcripts should not need exact shapes. |
| Keep raw files out of git | Preserve privacy boundary. |

## Public Surface

Package roots expose public API. Concrete adapter internals stay private.

Default package roots must support Node 20+ and Bun. Runtime-specific helpers
belong behind explicit subpaths, like `@agent-trail/adapter-kit/bun-sqlite`.

## Verification

Run focused checks while developing:

```sh
bun test packages/adapter-kit
bun test packages/adapters
bun run check:source-types
bun run check:types
bun run check:api
bun run check:exports
```

Before opening a PR:

```sh
mise run check
```
Loading
Loading