Skip to content

ft: refuse a size==0 FileRequest() when the hash is not the empty file - #388

Open
SchlauFuchs wants to merge 3 commits into
RetroShare:masterfrom
SchlauFuchs:fix/skip-premature-dl-file-creation
Open

SchlauFuchs wants to merge 3 commits into
RetroShare:masterfrom
SchlauFuchs:fix/skip-premature-dl-file-creation

Conversation

@SchlauFuchs

@SchlauFuchs SchlauFuchs commented Sep 11, 2026

Copy link
Copy Markdown

Revised per review

Thanks @jolavillette and @csoler for tracing through the actual behavior - you're right, the original "fall through to a real transfer" approach was wrong: there's no code path that ever corrects a transfer's stored size later (the "already downloading" branch only merges peer sources), and there's no legitimate source of a size==0 request with a real hash in normal operation. Falling through would have created a permanent zombie transfer instead of the original harmless empty file.

Revised fix

Keep the empty-hash gate (da39a3ee... is correct, as noted), but for the mismatched case, refuse the request outright instead of falling through - matching the existing oversize-request precedent a few lines below (RsErr() + return false).

Testing

Clean rebuild of libretroshare.a, no new warnings.

🤖 Generated with Claude Code

https://claude.ai/code/session_0137V8RpdwiKisg9SSrMEiBt

@jolavillette

Copy link
Copy Markdown
Contributor

Thanks for looking at this corner. The empty-hash gate itself is right (da39a3ee… is SHA1 of the empty string), but I traced the fall-through path and it does not do what the description says, so I'd rather flag it before this gets merged.

There is no size resolution in the engine. A transfer's size is copied once at construction (ftFileControl, ftFileProvider, ChunkMap::_file_size, ftTransferModule::mSize at fttransfermodule.cc:87) and never written again. A later FileRequest() for the same hash with the correct size takes the "Already Downloading File" branch (ftcontroller.cc:1049-1105), which only merges sources and never reads size. So "once the size becomes known" never happens.

What a size-0 request with a real hash does on the normal path (all lines on current master):

  • ChunkMap(0) has zero chunks, so getDataChunk() never allocates anything (ftchunkmap.cc:350-357): no data is ever requested. It still sends a chunk-map request to every source every 60 s and rejects every reply for size mismatch (:483-486), and turtle keeps digging tunnels for the hash. The entry sits at 0/0 in the transfer list until the user cancels it, and it is saved to the config.
  • After a restart (or as soon as the user clicks "Force check" on the stuck row), setAvailabilityMap() / forceCheck() on an empty map leave _file_is_complete = true (ftchunkmap.cc:92-119, :673-682). The transfer then goes to CHECKING, checkFile() spawns a hash thread, the partial file does not exist so getFileHash fails and the hash is null, mismatch → forceCheck() → complete again → new hash thread. One hashing thread every 2 s, forever, per such transfer, with no diagnostic. completeFile() is never reached, and it already treats mSize == 0 as ERROR_COMPLETION anyway (:742-743).

Where would a size-0 request come from? I could not find a "not synced yet" state. The remote directory sync reads name, size and hash as mandatory fields of one entry (directory_storage.cc:1061-1064) and the sender never ships unhashed files; every in-tree caller (links, search, collections, mail, channels, remote dirs, JSON clients) passes the size from the same record as the hash. The remaining sources are malformed input: a hand-made retroshare://file?…&size=0 link, a collection XML, a JSON call that sends or omits size, or a remote peer publishing a 0-size attachment — and two of those are zero-click (channel auto-download at p3gxschannels.cc:1181, mail push at p3msgservice.cc:265). With this PR each of them would create a permanent zombie plus the hashing loop above, remotely triggerable. Today they create a harmless empty file.

So as it stands I don't think this should go in. If you have an actual reproduction of size == 0 with a real hash from a normal sync, that would be the thing to fix at the source and I'd be glad to look at it. If not, I'd suggest closing this one. Should you want to keep the hash gate as hardening against malformed input, the safe shape is to refuse the request instead of falling through, i.e. return false with an RsErr() for size == 0 && hash != emptyFileHash — with the caveat that the GUI and webui already read a false return as "you already have this file" (MessageWidget.cpp:426-428, SearchDialog.cpp:442-445, mail_util.js:275), exactly like the existing oversize refusal at ftcontroller.cc:973-982.

@csoler

csoler commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Let's watch let the IAs fight...

@csoler

csoler commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

I agree with jolavilette here: this case should not (meaning there's technically no way it can) happen. Of course you can drop an error message in the if(size==0) {} branch when the hash is not the hash of an empty string, in order to make it visible if by any chance this inconsistency happens in the future.

The size==0 fast path in FileRequest() wrote an empty stub straight into
the destination and returned, without ever creating a transfer. That is
correct when size==0 is paired with the SHA1 of the empty string (nothing
to transfer), which is the only case a previous version of this fix
special-cased against.

Falling through to a real transfer attempt for any other hash (as that
version did) is actively worse than the original bug: there is no code
path anywhere that ever revisits or corrects a transfer's stored size
once set (a later FileRequest() for the same hash takes the "already
downloading" branch, which only merges peer sources and never touches
size), and there is no legitimate source of a mismatched size==0 request
in the first place - every real caller (remote dir sync, links, search,
collections, mail, channels) sources size from the same record as the
hash, and remote directory sync requires size as a mandatory field. So a
size==0 request with a non-empty-file hash is malformed input, and
falling through leaves a permanent zero-progress transfer that endlessly
re-requests chunk maps from every source and, once it reaches
FT_STATE_CHECKING (e.g. after a restart or a manual "Force check"), spins
into a hashing thread every few seconds forever.

Refuse the mismatched case outright instead, matching the existing
oversize-request precedent a few lines below (RsErr() + return false).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0137V8RpdwiKisg9SSrMEiBt
@SchlauFuchs
SchlauFuchs force-pushed the fix/skip-premature-dl-file-creation branch from 47c4bf4 to 9a7ce9a Compare September 19, 2026 22:42
@SchlauFuchs SchlauFuchs changed the title ft: don't short-circuit FileRequest() on size==0 unless it's truly empty ft: refuse a size==0 FileRequest() when the hash is not the empty file Sep 19, 2026
Companion fixes for RetroShare/RetroShare#3312 (which removes GUI-side
mkpath() calls on the assumption that the transfer engine already creates
any destination directory it needs), per jolavillette's review there:

- ftController::FileRequest()'s size==0 stub-file path writes directly
  into dest/fname without creating dest first. Unlike a normal download,
  this file is never routed through moveFile() at completion (there is no
  transfer to complete), so nothing else creates that directory - a
  zero-byte file inside a not-yet-created sub-folder of a directory or
  collection download was silently lost. Create the destination directory
  first, the same way moveFile() does for every other file.

- RsDirUtil::moveFile() called the throwing single-argument overload of
  std::filesystem::create_directories() with nothing above it to catch an
  exception (it runs on the file transfer thread). A destination name
  that is merely invalid on the current filesystem (e.g. a character
  disallowed on Windows) would abort the whole process here, and again on
  every subsequent restart, since the same completion is retried. Use the
  non-throwing error_code overload instead, matching the existing
  RsErr()-and-return-false error handling right below it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0137V8RpdwiKisg9SSrMEiBt
@SchlauFuchs

Copy link
Copy Markdown
Author

You're right, thanks for tracing it through - there's no path that ever corrects a stored size, and no legitimate source of size==0 with a real hash. Revised to refuse the mismatched case outright (RsErr() + return false) instead of falling through, matching the oversize-request precedent.

SchlauFuchs pushed a commit to SchlauFuchs/libretroshare that referenced this pull request Sep 19, 2026
Companion fixes for RetroShare/RetroShare#3312 (which removes GUI-side
mkpath() calls on the assumption that the transfer engine already creates
any destination directory it needs), per jolavillette's review there:

- ftController::FileRequest()'s size==0 stub-file path writes directly
  into dest/fname without creating dest first. Unlike a normal download,
  this file is never routed through moveFile() at completion (there is no
  transfer to complete), so nothing else creates that directory - a
  zero-byte file inside a not-yet-created sub-folder of a directory or
  collection download was silently lost. Create the destination directory
  first, the same way moveFile() does for every other file.

- RsDirUtil::moveFile() called the throwing single-argument overload of
  std::filesystem::create_directories() with nothing above it to catch an
  exception (it runs on the file transfer thread). A destination name
  that is merely invalid on the current filesystem (e.g. a character
  disallowed on Windows) would abort the whole process here, and again on
  every subsequent restart, since the same completion is retried. Use the
  non-throwing error_code overload instead, matching the existing
  RsErr()-and-return-false error handling right below it.

Based directly on current master; does not depend on RetroShare#388.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0137V8RpdwiKisg9SSrMEiBt
mirror-to-gitlab.yml runs on every push with no restriction to the
upstream repo, so it also runs (and fails) on every fork - the
RETROSHARE_GITLAB_MIRROR_SSH_KEY secret only exists upstream, and the
push destination is hardcoded to the official project's GitLab mirror
regardless of which repo the workflow runs in.

Guard the job to only run in RetroShare/libretroshare, fork-local only:
not proposing this upstream for now.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0137V8RpdwiKisg9SSrMEiBt
@jolavillette

Copy link
Copy Markdown
Contributor

Pre-review done with an AI under my supervision.

The revised version addresses everything raised above: the mismatched size == 0 case is now refused with a log line, in line with the oversize check a few lines below, and the two directory-creation fixes from #391 (create the folder before writing the empty stub, non-throwing create_directories in moveFile()) are what RetroShare/RetroShare#3312 relies on once it drops the GUI-side mkpath() calls. RetroShare/RetroShare#3313 carries the first commit of #3312, so it depends on this one too. Merge order: this PR first, then #3312 / #3313. CI is green on the three platforms.

One note: the third commit (skipping the GitLab mirror job on forks) is unrelated to file transfer and appears identically in #3312 and #3313. Harmless, but it may be cleaner as its own small PR, or squashed out at merge time, your call.

This now needs a human review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants