Skip to content

Detect unknown MIME types without copying uploads - #579

Open
CyMule wants to merge 2 commits into
mainfrom
perf/no-copy-mime-detection
Open

Detect unknown MIME types without copying uploads#579
CyMule wants to merge 2 commits into
mainfrom
perf/no-copy-mime-detection

Conversation

@CyMule

@CyMule CyMule commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • detect unknown MIME types directly from the existing spooled upload stream
  • preserve the uploaded filename through a lightweight file proxy
  • rewind the original upload after detection
  • add a regression test that verifies the original stream is used

Why

The previous path read the complete upload into a new BytesIO object before detection. That creates a document-sized allocation per request and can amplify memory pressure under concurrency.

Impact

Detection behavior is preserved while avoiding the full-body copy. A local 40 MiB spooled-upload check measured about 0.065 MiB of peak traced allocation in this path, with the stream rewound to position zero afterward.

Validation

  • uv run --locked --no-sync pytest -q test_general/api/test_filetypes.py — 1 passed
  • focused Ruff check and format check passed for all changed Python files
  • 40 MiB local allocation check — mime=text/plain peak_mib=0.065 position=0

Review in cubic

@CyMule
CyMule marked this pull request as ready for review July 21, 2026 01:35

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files

Shadow auto-approve: would not auto-approve because issues were found.

Re-trigger cubic

Comment thread prepline_general/api/filetypes.py Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 2 files (changes from recent commits).

Shadow auto-approve: would auto-approve. Optimizes MIME detection to avoid a full-body BytesIO copy by proxying the spooled upload, preserving behavior and filename. Includes regression tests verifying the original stream is used and rewound. A bounded, performance-improving refactor with no observable external change.

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant