Repository navigation
Stop Markdown in doc attributes from leaking past the closing quotes - #21
Merged
Merged
Conversation
The doc-directive rule only includes text.html.markdown. The tests ran without a grammar for that scope, and vscode-textmate drops a rule whose only patterns cannot be resolved, so -doc and -moduledoc attributes fell back to the generic directive rule and the doc-directive rule was never exercised. The docstring and sigil snapshots recorded that fallback. Add a minimal grammar for text.html.markdown under tests/grammars, load it in both test commands and regenerate the two snapshots. Their only changes are doc attributes now being scoped meta.directive.doc.erlang, as they are in an editor with the Markdown grammar available.
The doc-directive rule included text.html.markdown directly between its begin and end patterns. The end pattern is only tried once the embedded grammar hands control back, so malformed Markdown such as an unclosed code fence kept consuming lines past the closing """ and highlighted the rest of the module as Markdown. Wrap the include in a begin/while rule whose while pattern stops at the first line starting with """. The while condition is checked at the start of every line before the embedded grammar runs, so the Markdown region ends there regardless of the state the Markdown grammar is in. This is the approach suggested in microsoft/vscode-markdown-tm-grammar#175 (comment) Co-authored-by: Balaji S <bgs@meta.com>
michalmuskala
approved these changes
Sep 30, 2026
meta-codesync Bot
pushed a commit
to WhatsApp/erlang-language-platform
that referenced
this pull request
Sep 30, 2026
Summary: Sync both copies of the Erlang TextMate grammar (the ELP extension and the internal `nuclide.erlang` extension) with https://github.com/erlang-ls/grammar at `49f7a9a`. Both copies now match upstream byte for byte. Changes since the last sync: - erlang-ls/grammar#19: named fun expressions (`fun Name(...) -> ... end`) are highlighted correctly, and no longer scope the rest of the file as an implicit fun. Zero-arity fun expressions such as `fun() -> ok end` are keywords instead of function types. The function type rule now only applies inside types (`-type`, `-opaque`, `-spec`, `-callback`, typed record fields), via a new `meta.type.erlang` scope. Fixes pgourlain/vscode_erlang#316 and #317. - erlang-ls/grammar#20: a fun type after a top-level range (`-type t() :: 1..10 | fun(() -> ok).`) or in a `-nominal` type no longer scopes the rest of the file as a fun expression. - erlang-ls/grammar#21: upstreams the fix for Markdown leaking out of `-doc`/`-moduledoc` attributes (D98300495), which only lived in these copies. No behaviour change here. - erlang-ls/grammar#22: follow-ups from the review of this diff. Named funs called `_` (`fun _(A) -> A end`) are highlighted as named funs. `README.md` typos fixed (`ignode`, `./test/snap`) and the snapshot update command loads the Markdown test grammar. Upstream also adds tests showing that a comma nested in a record field type does not end the `meta.type.erlang` region, and that a default value stays an expression. `README.md` is synced too (test instructions and references); `LICENSE` is unchanged. Reviewed By: jcpetruzza Differential Revision: D122532634 fbshipit-source-id: cabfee7809fcd0e0977d700bc71b8b50db8cd02f
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a
-docor-moduledocattribute contains malformed Markdown, such as an unclosed code fence, the highlighting doesn't stop at the closing""". The rest of the module is highlighted as Markdown:Stop Markdown in doc attributes from leaking past the closing quotes
doc-directiveincludedtext.html.markdowndirectly between itsbeginandendpatterns. Theendpattern is only tried once the Markdown grammar hands control back, which it never does while a code fence is open. The include is now wrapped in abegin/whilerule whosewhilepattern stops at the first line starting with""". Thewhilepattern is checked at the start of every line, before the Markdown grammar runs, so the Markdown ends there whatever state the Markdown grammar is in. This is the approach suggested in microsoft/vscode-markdown-tm-grammar#175 (comment).The fix has been in the ELP VS Code extension's copy of this grammar since March 2026 (WhatsApp/erlang-language-platform@ddeaba9b90). This PR brings it upstream.
Load a Markdown stand-in grammar in the tests
Until now, the tests had no coverage of
doc-directive. They ran without a grammar fortext.html.markdown, and when a rule's only include can't be found,vscode-textmatesilently drops the rule.-docand-moduledocwere therefore highlighted as generic directives, and thedocstringandsigilsnapshots recorded that.This PR adds a minimal grammar for
text.html.markdownundertests/grammars/, loads it with-gin both test commands, and regenerates the two snapshots. The only snapshot changes are doc attributes now being highlighted asmeta.directive.doc.erlang, as they are in an editor that has the Markdown grammar.Tests
tests/doc_markdown.erlcovers an unclosed```fence in-docand an unclosed~~~fence in-moduledoc. Without the fix, it fails: the closing"""and the next function head are highlighted as Markdown.Co-authored with @bgsmeta.