Skip to content

Web search: configured API providers lose their fallback in DuckDuckGo-unreachable networks — tail the chain with Bing instead? #6746

Description

@asto18089

Summary

For a configured API search provider (tavily, bocha, metaso, …), the backend chain ends in a DuckDuckGo tail. DuckDuckGo's internal Bing fallback only covers empty results / bot challenges — it does not cover connection failure. In networks where DuckDuckGo is unreachable (common behind the GFW and some corporate egresses), the whole chain ends in ToolError::not_available("web search backends unavailable: <id>, duckduckgo") even though Bing itself is reachable, and the provider's legitimate results are thrown away.

Evidence (current main)

  • crates/tui/src/tools/web/backend.rs SearchBackendChain::from_context (~114–130): chain is [native?] → configured → SearchProvider::DuckDuckGo tail.
  • crates/tui/src/tools/web_search.rs run_scrape_search_with_endpoints (~1560+): the DDG GET's .send() error returns immediately; the internal Bing fallback only runs after a successful response with empty results/challenge. So a DDG connection failure never reaches Bing.
  • The tool description still documents "visibly degrade through DuckDuckGo then Bing", which today only holds for the empty/challenge case.

Proposal (from the Pinvou fork, commit ff299f94b)

For the configured-API-provider chain, push the keyless tail from DuckDuckGo down to Bing (the same tail native/explicit chains already use), keeping DuckDuckGo in the chain wherever it is genuinely selectable. This changes deliberately documented fallback behavior, so we're opening a discussion first rather than a PR — alternatives we're happy to consider:

  • make the tail provider configurable ([search] fallback_provider);
  • try DDG first but fall through to Bing on connection-class errors only;
  • keep status quo and document the limitation.

We have a working patch (~10-line core + a chain-shape test per provider) and measured it restoring results in a DDG-blocked network. Happy to send it whichever way maintainers prefer.

Activity

  1. added
    needs-triageNew external report awaiting maintainer triage; repro, logs and version output help
    on Sep 29, 2026
  2. added this to the v0.10.1 milestone on Sep 30, 2026
  3. added
    bugSomething isn't working
    reliabilityReliability, flaky behavior, retries, fallbacks, and robustness
    web-searchWeb search tool, search backends, result parsing, and browser/fetch search UX
    and removed
    needs-triageNew external report awaiting maintainer triage; repro, logs and version output help
    on Oct 8, 2026
  4. closed this as completedby moving to Done in Codewhale Roadmapon Oct 8, 2026
  5. Hmbown commented on Oct 8, 2026

    @Hmbown
    Collaborator

    Fixed in 0.10.1: when DuckDuckGo cannot be reached (connection error, timeout or non-2xx), web search now tries Bing before giving up, including for the DuckDuckGo step after a configured API provider, and DuckDuckGo is limited to 60% of the time budget so Bing has time to answer (339baa2, d2abc92).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingreliabilityReliability, flaky behavior, retries, fallbacks, and robustnessweb-searchWeb search tool, search backends, result parsing, and browser/fetch search UX

    Type

    No type

    Projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions