New file upload example by mattieruth · Pull Request #237 · pipecat-ai/pipecat-examples

mattieruth · 2026-07-16T20:10:33Z

Demonstrates how a client can upload files (or reference them by URL)
into the user context mid-conversation, and how to manage those files
on the server side.

Client (vanilla JS):

Upload a local file or reference a URL, with an optional prompt,
via client.sendFile()
Works across SmallWebRTC, Daily, LiveKit, and WebSocket transports
Upload button is enabled only while the bot is ready

Server:

bot.py accepts an --llm argument to run against different LLMs
(OpenAI, Bedrock, Google, Google Vertex, Anthropic)
Uploaded files land in a local uploads folder by default
bot_s3.py and bot_gcs.py show custom file storage backends
(Amazon S3, Google Cloud Storage) via create_file_storage()
README covers setup for each transport and storage option

kompfner · 2026-07-28T13:27:03Z

+conversation context. The LLM can then reason about the uploaded content and
+respond with audio as it would for any other user turn.
+
+Concepts this example is meant to demonstrate:


Nice overview

kompfner · 2026-07-28T13:28:05Z

+4. **Run the bot**:
+
+   ```bash
+   uv run bot.py -u /tmp/uploads


Worth having a default temp location if one is not specified? Simplifies usage.

you don't think that's a privacy/security issue? the runner will use this location to store and delete files. It felt a little presumptive to me to make a default.

I meant just for this example, which exists only to show file uploading. Not proposing baking anything into the runner itself.

kompfner · 2026-07-28T13:29:05Z

+   # Edit .env.local if needed (defaults to localhost:7860)
+   ```
+
+   > **Note:** Environment variables in Vite are bundled into the client and exposed in the browser. For production applications that require secret protection, consider implementing a backend proxy server to handle API requests and manage sensitive credentials securely.


kompfner · 2026-07-28T13:33:09Z

+        user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
+    )
+
+    rtvi = RTVIProcessor(uploads_folder=runner_args.cli_args.uploads_folder)


This is the only configuration needed on the bot server side? That's pretty sweet.

Amazing 🎉

correct. and even then it's only needed if you are supporting an endpoint for the client to upload files to. As @vipyne points out in her PR, PipecatCloud does not have an endpoint for this as of yet, so in that case, uploads_folder should be None and the client will only be able to upload either small files that can be sent wholesale over the transport message channel or urls.

Since we have this (hopefully temporary) limitation today, we ought to mention it in the README, which presents Pipecat Cloud as a deployment option.

i believe it is already 🤔

kompfner · 2026-07-28T13:34:02Z

+
+This bot uses a cascade pipeline: Speech-to-Text → LLM → Text-to-Speech
+
+Generated by Pipecat CLI


So there's also some new CLI affordances coming for configuring a bot to accept file uploads?

i don't have any cli changes planned but maybe i should? When I started this example, I generated it using the CLI which adds this line (and 98% of the lines in this file). Maybe I should remove this comment?

but maybe i should?

I'd have to refresh myself on the existing CLI options to see if something like "accept file uploads" would fit in...without going through them, I have no preference.

this line

I wonder if, if the intent of the CLI is to provide a starting point or scaffold that the user will then build on top of, this line should instead read "Bootstrapped by Pipecat CLI" or something. "Generated by" might lead the reader to believe this file is "managed" by a tool and shouldn't be edited. This is obviously an issue outside the scope of this PR. WDYT, @markbackman?

filipi87 · 2026-07-28T14:18:56Z

+ * This file is auto-generated by pipecat-cli
+ */
+
+export const AVAILABLE_TRANSPORTS = [


Should we add the websocket here ?

filipi87 · 2026-07-28T14:21:00Z

+    "@pipecat-ai/client-js": "file:../../../pipecat-client-web/client-js",
+    "@pipecat-ai/daily-transport": "file:../../../pipecat-client-web-transports/transports/daily",
+    "@pipecat-ai/small-webrtc-transport": "file:../../../pipecat-client-web-transports/transports/small-webrtc-transport"


TODO: Remember to fix this before merging. 🙂

filipi87 · 2026-07-28T14:23:01Z

+        option.textContent = 'SmallWebRTC';
+      } else if (transport === 'daily') {
+        option.textContent = 'Daily';
+      } else if (transport === 'websocket') {


Here we're mentioning websocket, but we don't have it in either the dependencies or AVAILABLE_TRANSPORTS. I think we just need to decide whether we want to support it or remove it.

oh, i don't think i meant to support it. though, i should test that...

actually. i went ahead and added support :)

filipi87 · 2026-07-28T14:25:26Z

+[tool.uv.sources]
+pipecat-ai = { path = "/Users/mattieruth/src/pipecat/pipecat", editable=true }


TODO: Remember to fix this before merging. 🙂

filipi87 · 2026-07-28T14:26:57Z

I don't think any of the other examples have this. Are we planning to keep it?

i can remove it. like i said, I generated this example using the CLI and so some of the stuff was just basic scaffolding. though, in chatting with @markbackman, it seemed like maybe i should leave it? i have no preference/opinion.

filipi87 · 2026-07-28T14:27:07Z

Same here. I don't think any of the other examples have this. Are we planning to keep it?

same reply :)

kompfner · 2026-07-28T18:00:19Z

+    )
+
+    # cli_args is only populated when running via the development runner (not on
+    # Pipecat Cloud, where there is no uploads endpoint and large-file upload is


Wait, so small-file upload is available even when uploads_folder is None?

yes because those can fit in a basic message.

vercel · 2026-09-16T19:55:23Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
pipecat-examples	Ready	Preview	Oct 20, 2026 1:57pm UTC

filipi87 · 2026-10-07T12:21:19Z

+def create_file_storage() -> FileStorage:
+    """Storage backend for client uploads, discovered by the host alongside ``bot()``.
+
+    The host calls this at startup, serves its upload endpoint from the
+    returned backend, and injects it into the bot as
+    ``runner_args.file_storage``.
+    """
+    return S3FileStorage(bucket=os.environ["S3_UPLOADS_BUCKET"], region=os.getenv("AWS_REGION"))


filipi87 · 2026-10-09T13:26:56Z

+    "@pipecat-ai/client-js": "file:../../../clients/web/client/client-js",
+    "@pipecat-ai/daily-transport": "file:../../../clients/web/transports/transports/daily",
+    "@pipecat-ai/livekit-transport": "file:../../../clients/web/transports/transports/livekit-transport",
+    "@pipecat-ai/small-webrtc-transport": "file:../../../clients/web/transports/transports/small-webrtc-transport/pipecat-ai-small-webrtc-transport-1.10.7.tgz",
+    "@pipecat-ai/websocket-transport": "file:../../../clients/web/transports/transports/websocket-transport"


We need to fix the dependencies here. 🙂

Just saw that we still need to release all of them.

filipi87 · 2026-10-09T13:31:25Z

+description = "Voice AI bot built with Pipecat"
+requires-python = ">=3.11"
+dependencies = [
+    "pipecat-ai[anthropic,aws,cartesia,daily,deepgram,google,livekit,openai,runner,silero,websocket,webrtc]>=1.4.0",


We also need to update to the latest version of Pipecat, which includes file upload support.

filipi87 · 2026-10-09T13:31:50Z

+# TODO: remove before merge — local override for testing against the sibling
+# pipecat checkout (pipecat-ai/pipecat PR #3415) until the feature is released.
+[tool.uv.sources]
+pipecat-ai = { path = "../../../pipecat", editable=true }


And implement this TODO. 😅

yes. all of these I expect to do once the releases are actually available to link against.

filipi87 · 2026-10-09T13:49:18Z

Hi @mattieruth, I tried uploading a .txt file, and the interface shows the following message, as if the upload had succeeded:

You
Uploaded file: about-gpu-2026-10-09T11-50-28-864Z.txt

However, I can see the following errors in Pipecat:

2026-10-09 10:45:44.929 | ERROR    | pipecat.processors.frame_processor:push_error_frame:1002 - AnthropicLLMService#0 exception (/Users/filipi/workspaces/workspace_daily/pipecat/src/pipecat/adapters/services/anthropic_adapter.py:211): Error mapping context messages to provider format: Unsupported 'file' MIME type: text/plain
2026-10-09 10:45:44.929 | WARNING  | pipecat.processors.aggregators.llm_context:remove_invalid_file_message:629 - Removing message with file or image content from context due to invalid response.
2026-10-09 10:45:44.930 | DEBUG    | pipecat.processors.metrics.frame_processor_metrics:stop_processing_metrics:308 - AnthropicLLMService#0 processing time: 0.004s
2026-10-09 10:45:44.931 | WARNING  | pipecat.pipeline.worker:_source_push_frame:1521 - PipelineWorker#0: Something went wrong: ErrorFrame#0(error: Error mapping context messages to provider format: Unsupported 'file' MIME type: text/plain, fatal: False)

I believe this is the missing piece we discussed yesterday. Just sharing it here in case it’s worth addressing in the example.

filipi87

Works great! I left just a couple of comments about remembering to update the libraries once we release, and whether we can improve error handling in case we push a failed or disallowed frame.

Other than that, it looks great. I've already approved it. 🚀

mattieruth · 2026-10-12T17:24:30Z

Hi @mattieruth, I tried uploading a .txt file, and the interface shows the following message, as if the upload had succeeded:

You
Uploaded file: about-gpu-2026-10-09T11-50-28-864Z.txt

However, I can see the following errors in Pipecat:

2026-10-09 10:45:44.929 | ERROR    | pipecat.processors.frame_processor:push_error_frame:1002 - AnthropicLLMService#0 exception (/Users/filipi/workspaces/workspace_daily/pipecat/src/pipecat/adapters/services/anthropic_adapter.py:211): Error mapping context messages to provider format: Unsupported 'file' MIME type: text/plain
2026-10-09 10:45:44.929 | WARNING  | pipecat.processors.aggregators.llm_context:remove_invalid_file_message:629 - Removing message with file or image content from context due to invalid response.
2026-10-09 10:45:44.930 | DEBUG    | pipecat.processors.metrics.frame_processor_metrics:stop_processing_metrics:308 - AnthropicLLMService#0 processing time: 0.004s
2026-10-09 10:45:44.931 | WARNING  | pipecat.pipeline.worker:_source_push_frame:1521 - PipelineWorker#0: Something went wrong: ErrorFrame#0(error: Error mapping context messages to provider format: Unsupported 'file' MIME type: text/plain, fatal: False)

I believe this is the missing piece we discussed yesterday. Just sharing it here in case it’s worth addressing in the example.

hmm. this looks expected to me. Anthropic is not accepting the file type you provided. What were you thinking should happen that didn't?

filipi87 · 2026-10-13T12:43:10Z

What were you thinking should happen that didn't?

I was thinking that it would be nice to reflect in the interface when an error occurs and we are not actually using the file.

For example, we might not have added the file to the context in the first place, or we might have removed it from the context because of an error. In either case, it would be nice to send a specific message to RTVI so that the interface can indicate that the file we thought had been uploaded successfully was actually not accepted.

But, this will probably require some changes to Pipecat as well. So, don't block this PR because of this. 🙂

Demonstrates how a client can upload files (or reference them by URL) into the user context mid-conversation, and how to manage those files on the server side. Client (vanilla JS): - Upload a local file or reference a URL, with an optional prompt, via client.sendFile() - Works across SmallWebRTC, Daily, LiveKit, and WebSocket transports - Upload button is enabled only while the bot is ready Server: - bot.py accepts an --llm argument to run against different LLMs (OpenAI, Bedrock, Google, Google Vertex, Anthropic) - Uploaded files land in a local uploads folder by default - bot_s3.py and bot_gcs.py show custom file storage backends (Amazon S3, Google Cloud Storage) via create_file_storage() - README covers setup for each transport and storage option

mattieruth requested review from filipi87, kompfner and markbackman July 16, 2026 20:10

mattieruth mentioned this pull request Jul 16, 2026

Rtvi send file support pipecat-ai/pipecat#3415

Closed

mattieruth force-pushed the new-file-upload-example branch from 517490a to 362e8f3 Compare July 23, 2026 21:17

kompfner reviewed Jul 28, 2026

View reviewed changes

Comment thread file-upload/client/src/app.js Resolved

kompfner reviewed Jul 28, 2026

View reviewed changes

Comment thread file-upload/client/src/app.js Resolved

kompfner approved these changes Jul 28, 2026

View reviewed changes

filipi87 reviewed Jul 28, 2026

View reviewed changes

kompfner reviewed Jul 28, 2026

View reviewed changes

mattieruth force-pushed the new-file-upload-example branch from 96435ea to 437f734 Compare September 16, 2026 19:55

vercel Bot deployed to Preview September 16, 2026 19:55

View deployment

vercel Bot deployed to Preview September 17, 2026 20:00

View deployment

vercel Bot deployed to Preview October 6, 2026 22:19

View deployment

filipi87 reviewed Oct 7, 2026

View reviewed changes

mattieruth force-pushed the new-file-upload-example branch from 7958504 to 284785a Compare October 8, 2026 13:19

vercel Bot deployed to Preview October 8, 2026 13:19

View deployment

mattieruth force-pushed the new-file-upload-example branch from 284785a to 6a7d477 Compare October 8, 2026 13:33

vercel Bot deployed to Preview October 8, 2026 13:34

View deployment

filipi87 self-requested a review October 8, 2026 21:26

filipi87 reviewed Oct 9, 2026

View reviewed changes

filipi87 approved these changes Oct 9, 2026

View reviewed changes

mattieruth force-pushed the new-file-upload-example branch from 6a7d477 to a9b4087 Compare October 20, 2026 13:53

vercel Bot deployed to Preview October 20, 2026 13:53

View deployment

mattieruth force-pushed the new-file-upload-example branch from a9b4087 to c6ade54 Compare October 20, 2026 13:56

vercel Bot deployed to Preview October 20, 2026 13:56

View deployment

mattieruth force-pushed the new-file-upload-example branch from c6ade54 to 9493a5a Compare October 20, 2026 13:57

vercel Bot deployed to Preview October 20, 2026 13:57

View deployment

mattieruth merged commit 6ede043 into main Oct 20, 2026
47 checks passed

mattieruth deleted the new-file-upload-example branch October 20, 2026 14:01


		This bot uses a cascade pipeline: Speech-to-Text → LLM → Text-to-Speech

		Generated by Pipecat CLI

		[tool.uv.sources]
		pipecat-ai = { path = "/Users/mattieruth/src/pipecat/pipecat", editable=true } No newline at end of file

Conversation

mattieruth commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kompfner Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vercel Bot commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

mattieruth commented Jul 16, 2026 •

edited

Loading

kompfner Jul 28, 2026 •

edited

Loading

vercel Bot commented Sep 16, 2026 •

edited

Loading