Skip to content

feat(sync): absorb upstream tracing and dashboard fixes - #375

Open
BegoniaHe wants to merge 22 commits into
masterfrom
sync/upstream-2026-10-20
Open

BegoniaHe wants to merge 22 commits into
masterfrom
sync/upstream-2026-10-20

Conversation

@BegoniaHe

@BegoniaHe BegoniaHe commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Summary

Absorb upstream dffe922..f2404c19 (9 commits) into the fork. The main change is OpenTelemetry-backed message tracing with bounded SQLite history, a redesigned Trace explorer, trace-aware storage cleanup, configuration migration to schema_revision 4, and core session-waiter dispatch. The window also includes provider metadata preservation, template dict defaults, a CodeQL action bump, a fullscreen pip-dialog fix, and clearer configuration hints with conditional segmented-reply options.

Related issue

Related: AstrBotDevs#10614, AstrBotDevs#10522, AstrBotDevs#10517, AstrBotDevs#10636, AstrBotDevs#10608, AstrBotDevs#10647 (upstream PRs)

Behavior

  • Trace recording persists to data/logs/traces.db by default with an 80 MB capacity target and 7-day retention; grouped settings are trace.enable, trace.db_path, trace.db_max_mb, and trace.retention_days.
  • Settings → General → Logs exposes trace recording and storage budget controls; Clear Trace Data is available separately and included in Clean All. The confirmation warns that historical traces cannot be recovered.
  • The Trace tab uses a paginated trace list and grouped execution timeline, with server-side history/filtering and trace-id/span-id log correlation.
  • Active interactive waits are admitted before route.stop and dispatched in ProcessStage after access and safety checks, so follow-up input does not need another wake prefix.
  • Fetched provider model metadata survives schema refreshes; dict template fields initialize to {}; the console pip dialog stays visible in fullscreen; configuration hints and disabled-option hiding match upstream.
  • Provider model metadata, template defaults, CodeQL pin, and dashboard polish are otherwise unchanged in intent.

Non-goals

  • Did not restore upstream installer scripts, mirrors, or docs.astrbot.app links. The fork keeps source/Docker deployment and its own docs.
  • Did not restore Japanese or Russian locale catalogs.
  • Did not restore upstream legacy files: star/context.py, migra_helper.py, changelogs/unreleased.md, api/v1.ts, or the monolithic dashboard test.
  • Did not adopt the desktop update notice or dynamic plugin sidebar, which target infrastructure absent from this fork.

Implementation notes

  • Trace config was adapted to the fork's grouped trace section and a forward-only schema_revision 4 migration converts legacy trace-log settings (log_max_mb × 4) to the SQLite budget.
  • Session waiters use the fork's per-runtime SessionWaiterRegistry; event_context only binds the current event, while the scheduler owns active-event registration and cleanup.
  • Trace storage, log broker, telemetry bounds, and error fields are redacted before persistence; SQLite files and sidecars are protected from log cleanup.
  • The Dashboard uses MDI icons through a small local trace icon adapter, keeps only en-US/zh-CN, and regenerates the OpenAPI client and public OpenAPI JSON.
  • Review-driven follow-ups fixed cleanup confirmation, waiter admission ordering, docs placement, persistent-history hints, equal-timestamp terminal-state protection, and quality gates.
  • CI follow-ups shortened large parametrized test ids so Windows keeps PYTEST_CURRENT_TEST below its environment limit, made the retention assertion wait for writer readiness with diagnostics, documented intentionally dropped log entries for CodeQL, and removed the dead friend_message_needs_wake_prefix metadata that this fork's turn-routing model replaced.
  • Post-rebase follow-ups merged current master and made the PowerShell format
    job retry the PSScriptAnalyzer install and register PSGallery when the runner
    image lacks it, so a transient gallery outage cannot fail the required gate.

Validation

uv run pytest -q --test-profile blocking
# 6852 passed, 11 skipped, 4 deselected

make check
# all checks passed

make quality
# focused quality checks passed

make build-docs
# VitePress build complete

cd dashboard && pnpm typecheck && pnpm test && pnpm i18n:check
# pass

Per-commit independent Codex reviews were recorded in upstream-decisions.jsonl; the interval audit reports 0 errors.

Compatibility and risk

  • Migration: configuration schema_revision advances to 4. Legacy trace file settings are converted; old JSONL trace files are not imported or deleted.
  • Runtime: adds opentelemetry-api and opentelemetry-sdk to pyproject.toml, requirements.txt, and uv.lock.
  • Storage: trace content may include messages, plugin arguments, and tool results, bounded and redacted before persistence; restrict database access accordingly.
  • Risk: Fullscreen pip-dialog contrast was verified from CSS and tests, not a browser screenshot. Upstream installer/desktop/sidebar features are intentionally skipped and recorded in the ledger.

Checklist

  • A Feature request Issue exists for large work, or this is a small, obvious addition.
  • The change is focused and does not include unrelated refactoring.
  • I added or updated tests, or explained why tests are not practical.
  • User-visible behavior updates both docs/zh/ and docs/en/.
  • OpenAPI, generated client, docs/public/openapi.json, and tests change together when routes or schemas change.
  • No secrets committed. Runtime Python deps update pyproject.toml, requirements.txt, and uv.lock together.
  • I did not restore legacy shims, Python <3.15 fallbacks, or upstream publish/docs URLs as fork artifacts.
  • Breaking API or behavior changes use ! and a BREAKING CHANGE: footer.
  • I will not merge this PR myself. Merge needs a human maintainer review plus a separate AI-assisted review (AI_POLICY.md).
  • AI use follows AI_POLICY.md. Keep exactly one author note below. Do not fabricate the other.

Agent note

Goal: absorb the upstream window into the fork without restoring legacy paths or upstream deployment artifacts.

Paths touched: tracing core and storage (astrbot/core/utils/{trace,trace_store,telemetry}.py, pipeline stages, event bus, log broker), Dashboard trace/settings/console views and i18n, grouped config metadata and migration, Dashboard API/OpenAPI/generated client, upstream-sync.yaml, and upstream-decisions.jsonl.

Checks run: uv run pytest -q --test-profile blocking; make check; make quality; make build-docs; Dashboard pnpm typecheck, pnpm test, pnpm i18n:check; focused per-commit tests and independent Codex reviews recorded in the ledger.

Residual risk: no browser screenshot of fullscreen dialog contrast; binary screenshots and MDI font payloads were not visually inspected; skipped upstream features are documented with revisit conditions.

AI tools: OpenCode (implementation) and Codex CLI (independent review).


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@sourcery-ai

sourcery-ai Bot commented Oct 20, 2026

Copy link
Copy Markdown

Sorry @BegoniaHe, your pull request is larger than the review limit of 150,000 diff characters

@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1c0d1417-6be8-4100-ba9d-6dd85e913a33
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Comment thread astrbot/core/log.py Fixed
Comment thread astrbot/core/log.py Fixed
@xero-team-bot

xero-team-bot Bot commented Oct 20, 2026

Copy link
Copy Markdown

⚠️ This PR conflicts with its base branch and needs a rebase.

git fetch origin master
git rebase origin/master
# after resolving the conflicts
git push --force-with-lease

The needs-rebase label is removed automatically once the conflicts are gone.
(Xero-Team/AstrBot · detected by xero-bot)

Soulter and others added 20 commits October 21, 2026 00:30
…Devs#10515) (AstrBotDevs#10614)

Adapt the OpenTelemetry-backed trace model and bounded SQLite history to the
fork's grouped configuration, per-runtime session waiter registry, sanitized
logging pipeline, and MDI-only Dashboard. Trace upserts keep completed traces
closed when update timestamps collide.

Upstream-Commit: 8774e53
Upstream-Author: Soulter <37870767+Soulter@users.noreply.github.com>
Upstream-PR: AstrBotDevs#10614
Sync-Disposition: adapt
Fork-Adaptation: Kept grouped trace.* settings and schema_revision 4, per-runtime session_waiter_registry, sanitized log broker, event_context temp-file ownership, and MDI icons; omitted ja-JP/ru-RU and upstream log format trace tags.
Tested: uv run pytest tests/unit -q --test-profile blocking; cd dashboard && pnpm typecheck && pnpm test && pnpm i18n:check
AI-Generated: true
Generated-At: 2026-10-20T08:35:38Z
Use the trace-specific cleanup confirmation for trace and all targets so the
dialog states that historical traces cannot be recovered, and cover the mapping
with a focused Vitest.

AI-Generated: true
Generated-At: 2026-10-20T08:44:22Z
Evaluate active interactive waits before the waking stage honors route.stop, so
follow-up input without a wake prefix reaches ProcessStage dispatch. Cover the
order with a WakingCheckStage regression test.

AI-Generated: true
Generated-At: 2026-10-20T08:53:10Z
Move the English Trace section beside Logs so the bilingual navigation matches.

AI-Generated: true
Generated-At: 2026-10-20T08:58:02Z
Update the trace hint in both shipped locales to describe bounded SQLite
history instead of in-memory-only storage.

AI-Generated: true
Generated-At: 2026-10-20T09:02:13Z
Reject a stale running snapshot when it shares the completed trace's timestamp,
and cover the collision case directly.

AI-Generated: true
Generated-At: 2026-10-20T09:06:22Z
…otDevs#10517)

Upstream-Commit: e5e21b7
Upstream-Author: Wei Chengqian <wcqqq1214@gmail.com>
Upstream-PR: AstrBotDevs#10517
Sync-Disposition: adapt
Fork-Adaptation: Added the dict default to the fork typed defaultValueMap.
Tested: cd dashboard && pnpm typecheck && pnpm exec vitest run --config vitest.config.ts tests/configEditorDialogs.vitest.ts
AI-Generated: true
Generated-At: 2026-10-20T09:12:00Z
Add a TemplateListEditor regression that requires dict fields without an
explicit default to initialize as an empty object.

AI-Generated: true
Generated-At: 2026-10-20T10:14:40Z
Check the emitted settings value directly so a missing key cannot satisfy the
regression test.

AI-Generated: true
Generated-At: 2026-10-20T10:17:35Z
Upstream-Commit: 68a4252
Upstream-Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Upstream-PR: AstrBotDevs#10636
Sync-Disposition: adapt
Fork-Adaptation: Kept the fork's commit-SHA pinning and updated both init and analyze to the v4.39.0 commit.
Tested: actionlint (.github/workflows/codeql.yml) not available; YAML parse checked
AI-Generated: true
Generated-At: 2026-10-20T10:24:00Z
…trBotDevs#10608)

Upstream-Commit: 776793b
Upstream-Author: 一张白纸 <202384594+mantoujun12@users.noreply.github.com>
Upstream-PR: AstrBotDevs#10608
Sync-Disposition: adapt
Fork-Adaptation: Applied the dialog attach target to the fork LogsPage and added the matching id to the shared ConsoleDisplayer root, which owns the fullscreen element.
Tested: cd dashboard && pnpm typecheck; pnpm exec vitest run --config vitest.config.ts tests/remainingCoverage.vitest.ts
AI-Generated: true
Generated-At: 2026-10-20T10:28:10Z
Keep the dialog's surface and field colors readable when it is attached to the
dark fullscreen console element.

AI-Generated: true
Generated-At: 2026-10-20T10:31:10Z
…AstrBotDevs#10647)

Upstream-Commit: f2404c1
Upstream-Author: Soulter <37870767+Soulter@users.noreply.github.com>
Upstream-PR: AstrBotDevs#10647
Sync-Disposition: adapt
Fork-Adaptation: Applied the hints and enabled-conditions to the fork's grouped metadata, kept fork-only command_prefixes/llm_access/group_sender_concurrency/OneBot-forward and image-caption scope keys, and updated only en-US/zh-CN metadata plus bilingual WebUI docs.
Tested: uv run pytest tests/unit/test_config_metadata_i18n.py tests/unit/test_dashboard_util.py -q --test-profile blocking; cd dashboard && pnpm typecheck && pnpm i18n:check && pnpm exec vitest run --config vitest.config.ts tests/configEditorDialogs.vitest.ts tests/configItemFragments.vitest.ts
AI-Generated: true
Generated-At: 2026-10-20T10:37:20Z
Resolve pyright span typing, Bandit SQL annotations, ESLint violations, and
mccabe complexity in the trace integration. Keep the full Python and Dashboard
gates green.

AI-Generated: true
Generated-At: 2026-10-20T11:05:00Z
Advance the upstream cursor through f2404c1 with per-commit dispositions:
six adaptations, three skips, and the review-driven follow-up commits recorded
in the decision ledger.

AI-Generated: true
Generated-At: 2026-10-20T11:06:00Z
- Shorten parametrized test ids so Windows keeps PYTEST_CURRENT_TEST bounded
- Wait for the trace writer before asserting retention and allow CI slack
- Document intentional dropped log entries for CodeQL
- Remove metadata for the wake-prefix setting this fork replaced

AI-Generated: true
Generated-At: 2026-10-20T14:07:00Z
Return directly from the bounded queue handlers instead of leaving an empty except body, so CodeQL no longer reports py/empty-except for intentionally dropped log entries.

AI-Generated: true

Generated-At: 2026-10-20T17:58:10Z
Regenerate the tracked subset after rebasing onto master so the
wallpaper and authorization views keep their swipe-horizontal hint icon.

AI-Generated: true
Generated-At: 2026-10-20T22:36:31Z
Point the 2026-10-20 window's integration and follow-up commits at the
rewritten SHAs so the provenance audit stays clean.

AI-Generated: true
Generated-At: 2026-10-20T22:45:18Z
@BegoniaHe
BegoniaHe force-pushed the sync/upstream-2026-10-20 branch from 3d2b1dc to 9cb5d57 Compare October 20, 2026 22:45
@xero-team-bot

xero-team-bot Bot commented Oct 20, 2026

Copy link
Copy Markdown

✅ Conflicts resolved; removing the needs-rebase label.

The PowerShell job failed when Install-Module found no PSGallery match
on the runner. Register the default gallery when it is missing and retry
the install three times so a transient gallery outage does not fail the
required format check.

AI-Generated: true
Generated-At: 2026-10-21T15:54:23Z
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants