[Feature] 增加视频输入支持 #8048

Activity

1. ReallyChooseC

   added

   enhancementNew feature or requestNew feature or request

   on May 7, 2026
2. dosubot

   added

   area:providerThe bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner.The bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner.

   on May 7, 2026
3. ReallyChooseC commented on May 7, 2026

   

   ReallyChooseC

   on May 7, 2026

   Author

   More actions

   补充：似乎openai官方规范中并没有视频输入，但是glm，kimi，qwen这些提供商都有视频理解的能力
4. JosephTian876 commented on Sep 23, 2026

   

   JosephTian876

   on Sep 23, 2026

   Contributor

   More actions

   Confirming this is still relevant on master — I hit exactly the behaviour described
   here from a different angle.

   When a video attachment arrives, astr_main_agent only appends a text placeholder:

   [Video Attachment: name X.mp4, path C:\...\X.mp4]
   

   so the model never receives any picture, and agentic models with code-execution
   tools enabled tend to fall back to reading the file themselves (astrbot_execute_python

   • cv2 frame extraction, ffprobe, web search for the dialogue, …). On my
     deployment that turned a single video message into ~30 tool calls and ~12 minutes
     before a reply.

   Two data points that may help whoever implements this:

   • A ProviderRequest.video_urls field mirroring image_urls/audio_urls
     (astrbot/core/provider/entities.py:100,102) appears to be the intended design —
     ProviderRequest has no video field today, while utils/llm_metadata.py:11
     already lists "video" as a valid modality.
   • Whatever lands needs to handle persistence: assemble_context() output is
     written into history and replayed every turn, so a multi-MB video must be marked
     with mark_as_temp() / _no_save (astrbot/core/agent/message.py:68,345,354)
     or sessions will carry it forever. See [Bug] 引用 GIF 会污染会话历史并导致 Gemini 后续请求持续失败 #9295 for the same failure mode with GIFs.

   There is also an open PR attempting this (#9424), currently conflicting with
   master and missing the Gemini dispatch path.
5. mou-fang commented on Oct 9, 2026

   

   mou-fang

   on Oct 9, 2026

   More actions

   非常期待这个功能！目前在使用中配置了具备多模态视频能力的大模型，但在 AstrBot 中接收到视频时无法直接作为视频模态输入给模型处理。看到社区已经提交了实现原生视频输入的 PR #10202，希望维护团队能推进 review 和合并，非常感谢！
6. JosephTian876 commented on Oct 17, 2026

   

   JosephTian876

   on Oct 17, 2026

   Contributor

   More actions

   The merge conflict against master on PR #10202 (feat(provider): support native video input modality) is resolved; the new head is af8d696f4.

   The conflict was confined to the auto-generated MDI font subset. It was regenerated with node scripts/subset-mdi-font.mjs (268 icons, including the new mdi-video-outline), with no source changes.

   The PR is mergeable again and 23/23 CI checks pass, so it is ready for review.