Skip to content

fix(serde): StreamDeserializer ends without an error after the last value - #252

Merged
liuq19 merged 1 commit into
cloudwego:mainfrom
yuxi-liu-wired:pr/stream-end
Oct 11, 2026
Merged

liuq19 merged 1 commit into
cloudwego:mainfrom
yuxi-liu-wired:pr/stream-end

Conversation

@yuxi-liu-wired

Copy link
Copy Markdown
Contributor

fix: StreamDeserializer ends without an error after the last value

into_stream() yielded an extra Err(EOF while parsing) once the input
was exhausted, so every well-formed stream ended in an error and a loop
like for v in stream { v? } always failed:

Deserializer::from_str("1 2").into_stream::<Value>()
    Ok(1), Ok(2), Err(EOF while parsing at line 1 column 3)
Deserializer::from_str("").into_stream::<Value>()
    Err(EOF while parsing at line 1 column 1)

serde_json's StreamDeserializer returns None there, which issue #202
asks sonic-rs to align with. next() now checks for remaining non-space
input first; a value that is cut short or garbage after a value is
still an error and still ends the stream.

(serde_json additionally requires whitespace between two values that do
not delimit themselves, e.g. 1true; sonic-rs splits them. Not changed
here.)

Test: test_stream_deserializer_ends_without_error (fails before this
change).

Testing

Each commit adds a regression test that fails before the change. The full test suite passes and
cargo fmt --check is clean on the changed files.

Found with a differential fuzzer comparing serde_json, simd-json, sonic-rs and jiter.

@CLAassistant

CLAassistant commented Oct 1, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@codecov

codecov Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.36%. Comparing base (9de5a49) to head (3787849).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #252      +/-   ##
==========================================
+ Coverage   73.08%   73.36%   +0.28%     
==========================================
  Files          42       42              
  Lines       10007    10030      +23     
==========================================
+ Hits         7314     7359      +45     
+ Misses       2693     2671      -22     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

`into_stream()` yielded an extra `Err(EOF while parsing)` once the input
was exhausted, so every well-formed stream ended in an error and a loop
like `for v in stream { v? }` always failed:

    Deserializer::from_str("1 2").into_stream::<Value>()
        Ok(1), Ok(2), Err(EOF while parsing at line 1 column 3)
    Deserializer::from_str("").into_stream::<Value>()
        Err(EOF while parsing at line 1 column 1)

serde_json's `StreamDeserializer` returns `None` there, which issue cloudwego#202
asks sonic-rs to align with. `next()` now checks for remaining non-space
input first; a value that is cut short or garbage after a value is
still an error and still ends the stream.

(serde_json additionally requires whitespace between two values that do
not delimit themselves, e.g. `1true`; sonic-rs splits them. Not changed
here.)

Test: `test_stream_deserializer_ends_without_error` (fails before this
change).

Found by a differential fuzzer (serde_json, simd-json, sonic-rs, jiter)
extended to multi-document streams.
@liuq19
liuq19 merged commit f63d185 into cloudwego:main Oct 11, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants