Skip to content

Stop Markdown in doc attributes from leaking past the closing quotes #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
robertoaloi merged 2 commits into from
Sep 30, 2026
Merged

Stop Markdown in doc attributes from leaking past the closing quotes #21

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4b65866
Load a Markdown stand-in grammar in the tests
robertoaloi Sep 30, 2026
819a6f1
Stop Markdown in doc attributes from leaking past the closing quotes
robertoaloi Sep 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 13 additions & 4 deletions Erlang.plist
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -621,13 +621,22 @@
</dict>
<key>name</key>
<string>meta.directive.doc.erlang</string>
<key>contentName</key>
<string>meta.embedded.block.markdown</string>
<key>patterns</key>
<array>
<dict>
<key>include</key>
<string>text.html.markdown</string>
<key>begin</key>
<string>(\s*|$)</string>
<key>while</key>
<string>^(?!\s*["]{3,})</string>
<key>contentName</key>
<string>meta.embedded.block.markdown</string>
<key>patterns</key>
<array>
<dict>
<key>include</key>
<string>text.html.markdown</string>
</dict>
</array>
</dict>
</array>
</dict>
Expand Down
4 changes: 2 additions & 2 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"scripts": {
"test": "vscode-tmgrammar-test ./tests/*.erl && vscode-tmgrammar-snap ./tests/snap/*.erl"
"test": "vscode-tmgrammar-test -g ./tests/grammars/markdown.tmLanguage.json ./tests/*.erl && vscode-tmgrammar-snap -g ./tests/grammars/markdown.tmLanguage.json ./tests/snap/*.erl"
},
"contributes": {
"languages": [
Expand All @@ -27,4 +27,4 @@
"devDependencies": {
"vscode-tmgrammar-test": "^0.1.3"
}
}
}
21 changes: 21 additions & 0 deletions tests/doc_markdown.erl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
% SYNTAX TEST "source.erlang" "doc attributes with malformed Markdown"
-module(doc_markdown).

%% An unclosed code fence in a doc attribute must not leak past the closing
%% triple quotes and turn the rest of the module into Markdown.
-doc """
Unclosed code fence:
```erlang
%<--- source.erlang meta.directive.doc.erlang meta.embedded.block.markdown markup.fenced_code.block.markdown
foo() -> ok.
""".
%<--- source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.end.erlang
f() -> ok.
%<- source.erlang meta.function.erlang entity.name.function.definition.erlang

-moduledoc """
~~~
""".
%<--- source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.end.erlang
g() -> ok.
%<- source.erlang meta.function.erlang entity.name.function.definition.erlang
11 changes: 11 additions & 0 deletions tests/grammars/markdown.tmLanguage.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"comment": "Minimal stand-in for text.html.markdown, used only by the tests. Like the real Markdown grammar, it has a fenced code block rule that keeps consuming lines until the closing fence, which is enough to reproduce Markdown leaking out of doc attributes.",
"scopeName": "text.html.markdown",
"patterns": [
{
"begin": "(^|\\G)(\\s*)(`{3,}|~{3,})",
"end": "(^|\\G)(\\2|\\s{0,3})(\\3)\\s*$",
"name": "markup.fenced_code.block.markdown"
}
]
}
88 changes: 43 additions & 45 deletions tests/snap/docstring.erl.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@
# ^ source.erlang meta.directive.module.erlang punctuation.definition.parameters.end.erlang
# ^ source.erlang meta.directive.module.erlang punctuation.section.directive.end.erlang
>-moduledoc """
#^ source.erlang meta.directive.erlang punctuation.section.directive.begin.erlang
# ^^^^^^^^^ source.erlang meta.directive.erlang keyword.control.directive.erlang
# ^ source.erlang meta.directive.erlang
# ^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang meta.string.quoted.triple.begin.erlang punctuation.definition.string.begin.erlang
#^ source.erlang meta.directive.doc.erlang punctuation.section.directive.begin.erlang
# ^^^^^^^^^ source.erlang meta.directive.doc.erlang keyword.control.directive.doc.erlang
# ^ source.erlang meta.directive.doc.erlang
# ^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.begin.erlang
>Triple quoted strings examples
#^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang
#^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ source.erlang meta.directive.doc.erlang meta.embedded.block.markdown
>""".
#^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang meta.string.quoted.triple.end.erlang punctuation.definition.string.end.erlang
# ^ source.erlang meta.directive.erlang punctuation.section.directive.end.erlang
#^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.end.erlang
# ^ source.erlang meta.directive.doc.erlang
>
>-export([f1/0, f2/0, f3/0, f4/0, f5/0,
#^ source.erlang meta.directive.export.erlang punctuation.section.directive.begin.erlang
Expand Down Expand Up @@ -79,16 +79,16 @@
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ source.erlang comment.line.percentage.erlang
>
>-doc """
#^ source.erlang meta.directive.erlang punctuation.section.directive.begin.erlang
# ^^^ source.erlang meta.directive.erlang keyword.control.directive.erlang
# ^ source.erlang meta.directive.erlang
# ^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang meta.string.quoted.triple.begin.erlang punctuation.definition.string.begin.erlang
#^ source.erlang meta.directive.doc.erlang punctuation.section.directive.begin.erlang
# ^^^ source.erlang meta.directive.doc.erlang keyword.control.directive.doc.erlang
# ^ source.erlang meta.directive.doc.erlang
# ^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.begin.erlang
> Docstring examples
#^^^^^^^^^^^^^^^^^^^^^^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang
#^^^^^^^^^^^^^^^^^^^^^^^^ source.erlang meta.directive.doc.erlang meta.embedded.block.markdown
> """.
#^^^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang meta.string.quoted.triple.end.erlang
# ^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang meta.string.quoted.triple.end.erlang punctuation.definition.string.end.erlang
# ^ source.erlang meta.directive.erlang punctuation.section.directive.end.erlang
#^^^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang
# ^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.end.erlang
# ^ source.erlang meta.directive.doc.erlang
>f1() ->
#^^ source.erlang meta.function.erlang entity.name.function.definition.erlang
# ^ source.erlang meta.function.erlang meta.expression.parenthesized punctuation.section.expression.begin.erlang
Expand Down Expand Up @@ -252,19 +252,17 @@
# ^ source.erlang meta.function.erlang punctuation.terminator.function.erlang
>
>-doc ~s"""""
#^ source.erlang meta.directive.erlang punctuation.section.directive.begin.erlang
# ^^^ source.erlang meta.directive.erlang keyword.control.directive.erlang
# ^ source.erlang meta.directive.erlang
# ^^ source.erlang meta.directive.erlang string.quoted.tripple.sigil.erlang storage.type.string.erlang
# ^^^^^ source.erlang meta.directive.erlang string.quoted.tripple.sigil.erlang meta.string.quoted.triple.begin.erlang punctuation.definition.string.begin.erlang
#^ source.erlang meta.directive.doc.erlang punctuation.section.directive.begin.erlang
# ^^^ source.erlang meta.directive.doc.erlang keyword.control.directive.doc.erlang
# ^ source.erlang meta.directive.doc.erlang
# ^^ source.erlang meta.directive.doc.erlang storage.type.string.erlang
# ^^^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.begin.erlang
> Hello\n
#^^^^^^^^^^^ source.erlang meta.directive.erlang string.quoted.tripple.sigil.erlang
# ^ source.erlang meta.directive.erlang string.quoted.tripple.sigil.erlang constant.character.escape.erlang punctuation.definition.escape.erlang
# ^ source.erlang meta.directive.erlang string.quoted.tripple.sigil.erlang constant.character.escape.erlang
#^^^^^^^^^^^^^^ source.erlang meta.directive.doc.erlang meta.embedded.block.markdown
> """"".
#^^^^^^ source.erlang meta.directive.erlang string.quoted.tripple.sigil.erlang meta.string.quoted.triple.end.erlang
# ^^^^^ source.erlang meta.directive.erlang string.quoted.tripple.sigil.erlang meta.string.quoted.triple.end.erlang punctuation.definition.string.end.erlang
# ^ source.erlang meta.directive.erlang punctuation.section.directive.end.erlang
#^^^^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang
# ^^^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.end.erlang
# ^ source.erlang meta.directive.doc.erlang
>f2() -> ok.
#^^ source.erlang meta.function.erlang entity.name.function.definition.erlang
# ^ source.erlang meta.function.erlang meta.expression.parenthesized punctuation.section.expression.begin.erlang
Expand Down Expand Up @@ -349,17 +347,17 @@
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ source.erlang comment.line.percentage.erlang
>
>-doc("""
#^ source.erlang meta.directive.erlang punctuation.section.directive.begin.erlang
# ^^^ source.erlang meta.directive.erlang keyword.control.directive.erlang
# ^ source.erlang meta.directive.erlang punctuation.definition.parameters.begin.erlang
# ^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang meta.string.quoted.triple.begin.erlang punctuation.definition.string.begin.erlang
#^ source.erlang meta.directive.doc.erlang punctuation.section.directive.begin.erlang
# ^^^ source.erlang meta.directive.doc.erlang keyword.control.directive.doc.erlang
# ^ source.erlang meta.directive.doc.erlang punctuation.definition.parameters.begin.erlang
# ^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.begin.erlang
> Hello\n
#^^^^^^^^^^^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang
#^^^^^^^^^^^^^ source.erlang meta.directive.doc.erlang meta.embedded.block.markdown
> """).
#^^^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang meta.string.quoted.triple.end.erlang
# ^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang meta.string.quoted.triple.end.erlang punctuation.definition.string.end.erlang
# ^ source.erlang meta.directive.erlang punctuation.definition.parameters.end.erlang
# ^ source.erlang meta.directive.erlang punctuation.section.directive.end.erlang
#^^^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang
# ^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.end.erlang
# ^ source.erlang meta.directive.doc.erlang punctuation.section.directive.end.Erlang
# ^ source.erlang meta.directive.doc.erlang
>g1() -> ok.
#^^ source.erlang meta.function.erlang entity.name.function.definition.erlang
# ^ source.erlang meta.function.erlang meta.expression.parenthesized punctuation.section.expression.begin.erlang
Expand All @@ -372,18 +370,18 @@
# ^ source.erlang meta.function.erlang punctuation.terminator.function.erlang
>
>-doc(~B"""""
#^ source.erlang meta.directive.erlang punctuation.section.directive.begin.erlang
# ^^^ source.erlang meta.directive.erlang keyword.control.directive.erlang
# ^ source.erlang meta.directive.erlang punctuation.definition.parameters.begin.erlang
# ^^ source.erlang meta.directive.erlang string.quoted.tripple.sigil.erlang storage.type.string.erlang
# ^^^^^ source.erlang meta.directive.erlang string.quoted.tripple.sigil.erlang meta.string.quoted.triple.begin.erlang punctuation.definition.string.begin.erlang
#^ source.erlang meta.directive.doc.erlang punctuation.section.directive.begin.erlang
# ^^^ source.erlang meta.directive.doc.erlang keyword.control.directive.doc.erlang
# ^ source.erlang meta.directive.doc.erlang punctuation.definition.parameters.begin.erlang
# ^^ source.erlang meta.directive.doc.erlang storage.type.string.erlang
# ^^^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.begin.erlang
> Hello\n
#^^^^^^^^^^^^^^ source.erlang meta.directive.erlang string.quoted.tripple.sigil.erlang
#^^^^^^^^^^^^^^ source.erlang meta.directive.doc.erlang meta.embedded.block.markdown
> """"").
#^^^^^^ source.erlang meta.directive.erlang string.quoted.tripple.sigil.erlang meta.string.quoted.triple.end.erlang
# ^^^^^ source.erlang meta.directive.erlang string.quoted.tripple.sigil.erlang meta.string.quoted.triple.end.erlang punctuation.definition.string.end.erlang
# ^ source.erlang meta.directive.erlang punctuation.definition.parameters.end.erlang
# ^ source.erlang meta.directive.erlang punctuation.section.directive.end.erlang
#^^^^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang
# ^^^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.end.erlang
# ^ source.erlang meta.directive.doc.erlang punctuation.section.directive.end.Erlang
# ^ source.erlang meta.directive.doc.erlang
>g2() -> ok.
#^^ source.erlang meta.function.erlang entity.name.function.definition.erlang
# ^ source.erlang meta.function.erlang meta.expression.parenthesized punctuation.section.expression.begin.erlang
Expand Down
16 changes: 8 additions & 8 deletions tests/snap/sigil.erl.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,16 @@
# ^ source.erlang meta.directive.export.erlang punctuation.section.directive.end.erlang
>
>-doc """
#^ source.erlang meta.directive.erlang punctuation.section.directive.begin.erlang
# ^^^ source.erlang meta.directive.erlang keyword.control.directive.erlang
# ^ source.erlang meta.directive.erlang
# ^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang meta.string.quoted.triple.begin.erlang punctuation.definition.string.begin.erlang
#^ source.erlang meta.directive.doc.erlang punctuation.section.directive.begin.erlang
# ^^^ source.erlang meta.directive.doc.erlang keyword.control.directive.doc.erlang
# ^ source.erlang meta.directive.doc.erlang
# ^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.begin.erlang
> Sigil examples
#^^^^^^^^^^^^^^^^^^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang
#^^^^^^^^^^^^^^^^^^^^ source.erlang meta.directive.doc.erlang meta.embedded.block.markdown
> """.
#^^^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang meta.string.quoted.triple.end.erlang
# ^^^ source.erlang meta.directive.erlang string.quoted.triple.erlang meta.string.quoted.triple.end.erlang punctuation.definition.string.end.erlang
# ^ source.erlang meta.directive.erlang punctuation.section.directive.end.erlang
#^^^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang
# ^^^ source.erlang meta.directive.doc.erlang comment.block.documentation.erlang punctuation.definition.string.end.erlang
# ^ source.erlang meta.directive.doc.erlang
>-spec f() -> ok.
#^ source.erlang meta.directive.erlang meta.type.erlang punctuation.section.directive.begin.erlang
# ^^^^ source.erlang meta.directive.erlang meta.type.erlang keyword.control.directive.erlang
Expand Down
Loading