fix: escape markdown syntax in plain span text during serialization

.changeset/escape-markdown-text.md

@@ -0,0 +1,28 @@
+---
+'@portabletext/markdown': patch
+---
+
+fix: escape markdown syntax in plain span text during serialization
+
+If your Portable Text contains text that happens to look like markdown, converting it to markdown and back used to corrupt it: the punctuation was read as formatting instead of text. `portableTextToMarkdown` now backslash-escapes such text, so it comes back as exactly the text it was.
+
+```ts
+// span text        markdown (before)   markdown (now)      re-parses as
+'*bar*'          // *bar*               \*bar\*             the text `*bar*` (was: an emphasized span reading `bar`)
+'# heading'      // # heading           \# heading          the text `# heading` (was: an `h1` block)
+'[x]: y'         // [x]: y              \[x]: y             the text `[x]: y` (was: nothing, the line vanished)
+```
+
+This works wherever the text sits (headings, blockquotes, list items, table cells) and also when the risky characters are split across neighboring spans. The visible change in your output: markdown for text containing such punctuation gains backslashes it didn't have before. It pastes and parses like any hand-written markdown.
+
+Text with the `code` decorator is never backslash-escaped. Its backtick delimiters widen instead, so the content survives verbatim even when it contains backticks:
+
+```ts
+// span text with the `code` decorator   markdown (before)   markdown (now)
+'a`b'                                 // `a`b`  (broken)     ``a`b``
+'`a'                                  // ``a`   (broken)     `` `a ``
+```
+
+If you supply custom mark or block renderers: `children` now arrives pre-escaped. When you need the original text, read the `text` argument instead (the built-in `code` renderer does exactly that).
+
+Two things stay as they were. A URL with an explicit scheme (`https://…`) or an email address is not escaped: it keeps its text and simply becomes a link on the next parse. A `www.`-style address is only left alone while it contains no markdown punctuation; with it, it gets escaped like ordinary text, and the round trip keeps every character either way. And leading or trailing whitespace that markdown itself trims still trims, same as before this change.

apps/docs/src/content/docs/conversion/markdown-to-portable-text.mdx

@@ -104,7 +104,7 @@ This package uses markdown-it as its Markdown parser. Remark and unified plugins
 Converting Markdown to Portable Text and back isn't a lossless mirror. Five things to expect:
 
 - Translation normalizes rather than preserves: a first Markdown → Portable Text → Markdown pass rewrites Markdown to one canonical spelling. Autolinks and reference links become inline links, indented code becomes fenced code, and `<https://portabletext.org>` comes back as `[https://portabletext.org](https://portabletext.org)`.
-- The normalized form is a fixpoint for the constructs in the table above: parsing it and serializing again reproduces it byte-for-byte. The exception is plain text containing literal Markdown punctuation: serialization doesn't yet escape it, so `\*bar\*` comes back as `*bar*`, which a second parse reads as emphasis.
+- The normalized form is a fixpoint for the constructs in the table above, and for plain text: parsing it and serializing again reproduces it byte-for-byte, because literal Markdown punctuation in plain text is backslash-escaped on the way out (`*bar*` comes back as `\*bar\*`, which a second parse reads as the literal text). Three exceptions: an explicit-scheme URL or an email stays byte-identical but gains a `link` mark on the next parse (fuzzy `www.` forms stay unescaped only while they contain no markdown-significant punctuation); a hard break inside a heading forces a structural split into a second block on reparse, since an ATX heading is single-line (the text itself still survives, split across the two blocks); and leading or trailing whitespace that CommonMark's own block parsing trims isn't part of the fixpoint claim.
 - Unrecognized constructs degrade, they don't fail. A mark, list, or task checkbox whose type isn't in the schema keeps its text and drops the formatting: an undeclared `strong` decorator turns `**bar**` into a plain span reading `bar`.
 - Portable Text structures with no Markdown form degrade predictably going back out. GFM has one header row, so extra header rows flatten into the body; deep or level-skipping lists collapse to relative nesting; unknown object types render as a fenced JSON block.
 - Keys and span boundaries aren't identity: every parse regenerates block and span keys, and adjacent spans with identical marks merge.

packages/markdown/README.md

@@ -88,12 +88,14 @@ const markdown = portableTextToMarkdown([
    [ref link][id]              ->  [ref link](https://example.com "title")
    ```
 
-2. The normalized Markdown is a fixpoint for the constructs in the [Supported features](#supported-features) table above: parsing it and serializing again reproduces it byte-for-byte, pinned by a full-document round-trip test. This doesn't yet extend to plain text that happens to contain literal Markdown punctuation: serialization doesn't escape it, so a second parse reads it back as markup instead of literal text.
+2. The normalized Markdown is a fixpoint for the constructs in the [Supported features](#supported-features) table above, and for plain text: parsing it and serializing again reproduces it byte-for-byte, pinned by a full-document round-trip test. Literal Markdown punctuation in plain text is backslash-escaped on serialization, so a second parse reads back the same characters instead of markup.
 
    ```
-   \*bar\*  ->  *bar*  (serialized unescaped; a second parse reads this as emphasis)
+   *bar*  ->  \*bar\*  (escaped on serialization; a second parse reads back the literal text)
    ```
 
+   Three exceptions. An explicit-scheme URL or an email is never escaped: text identity holds, but it gains a `link` mark on the next parse (autolinking is a parser feature, not a round-trip bug); fuzzy `www.` forms stay unescaped only while they contain no markdown-significant punctuation. A hard break inside a heading forces a structural split into a second block on reparse, since an ATX heading is single-line; the text itself still survives, split across the two blocks. And leading or trailing whitespace that CommonMark's own block parsing trims isn't part of the fixpoint claim.
+
 3. MD→PT survival is schema-driven. Constructs whose type the schema doesn't declare degrade predictably: they keep their content and drop the structure that named them. Marks drop formatting but keep the text (`**bar**` with no `strong` decorator in the schema becomes a plain span reading `bar`); tables flatten their cell content into top-level blocks; images fall back to their Markdown source as plain text; task-list checkboxes strip to plain list items.
 
 4. PT structures with no Markdown form degrade predictably on PT→MD. GFM tables have one header row, so header rows beyond the first flatten into the body. Deep or level-skipping lists collapse to relative nesting. A list's first item renders at the top level whatever its `level`, and each deeper jump between items indents one step, however many levels it skips. Multi-block table cells join their blocks with spaces. Unknown object types render as a fenced JSON block; unknown marks pass their text through unformatted.
@@ -568,17 +570,19 @@ portableTextToMarkdown(blocks, {
 
 #### What renderers receive
 
+One question decides which text field a renderer reads: **are you wrapping markdown, or emitting something else?** `children` is markdown: nested marks are rendered and literal punctuation is backslash-escaped, so it embeds in markdown output as-is (`` `**${children}**` ``). `text` is source: the raw span text exactly as stored, for renderers that emit their own delimiters or non-markdown output (code fences, HTML tags), where escapes would appear literally. The built-in `code` decorator renderer reads `text`; every other built-in reads `children`.
+
 **Block renderers** (`block.*`):
 
 - `value` – the block object
-- `children` – rendered content of the block
+- `children` – rendered content of the block, as markdown (escaped)
 - `index` – position in the blocks array
 
 **Mark renderers** (`marks.*`):
 
 - `value` – the mark definition (for annotations like links)
-- `children` – the rendered marked content
-- `text` – the raw text content (without nested mark rendering)
+- `children` – the rendered marked content, as markdown (escaped)
+- `text` – the raw text content (unescaped, no nested mark rendering)
 - `markType` – the mark type name
 - `markKey` – the mark's key (for annotations)
 

packages/markdown/package.json

@@ -52,13 +52,15 @@
     "@mdit/plugin-alert": "^0.23.2",
     "@portabletext/schema": "workspace:^",
     "@portabletext/toolkit": "^6.0.0",
+    "linkify-it": "^5.0.2",
     "markdown-it": "^14.3.0"
   },
   "devDependencies": {
     "@portabletext/test": "workspace:^",
     "@portabletext/types": "^4.0.2",
     "@sanity/pkg-utils": "catalog:tooling",
     "@sanity/tsconfig": "catalog:tooling",
+    "@types/linkify-it": "^5.0.0",
     "@types/markdown-it": "^14.1.2",
     "typescript": "catalog:tooling",
     "vite": "catalog:tooling",

packages/markdown/src/escape.test.ts

@@ -77,6 +77,16 @@ describe(escapeTableCell.name, () => {
     expect(escapeTableCell('a \\| b')).toBe('a \\| b')
   })
 
+  test('escapes a pipe behind an even number of backslashes', () => {
+    // Two backslashes cancel out to a literal backslash, leaving the pipe
+    // live, the shape leaf escaping produces from a literal `\|` in a span.
+    expect(escapeTableCell('a\\\\| b')).toBe('a\\\\\\| b')
+  })
+
+  test('leaves a pipe behind an odd number of backslashes intact', () => {
+    expect(escapeTableCell('a\\\\\\| b')).toBe('a\\\\\\| b')
+  })
+
   test('leaves backslashes alone', () => {
     expect(escapeTableCell('a\\b')).toBe('a\\b')
   })

packages/markdown/src/escape.ts

@@ -23,16 +23,21 @@ export function escapeImageAndLinkTitle(text: string): string {
  * Escapes characters that have special meaning at the row level of a GFM
  * table cell.
  *
- * A literal `|` ends the cell, so unescaped pipes are replaced with `\|`.
- * Newlines end the row, so they are replaced with `<br>` to keep the
- * visible line break inside the cell. Already-escaped pipes (`\|`) are
- * left intact so that escapes introduced by mark renderers survive the
- * pass.
+ * A literal `|` ends the cell, so a pipe preceded by an even number of
+ * backslashes (including zero) gets one more: paired backslashes cancel
+ * out to a literal backslash and leave the pipe live, so parity, not mere
+ * presence, decides whether it is already escaped. Newlines end the row,
+ * so they are replaced with `<br>` to keep the visible line break inside
+ * the cell.
  *
- * Backslashes are intentionally not escaped here so that other escapes
- * already in the rendered cell (such as `\[` and `\]` in link text) are
- * not double-escaped.
+ * Backslashes themselves are left alone here; only the parity check reads
+ * them, so escapes already in the rendered cell (such as `\[` and `\]` in
+ * link text) survive the pass untouched.
  */
 export function escapeTableCell(text: string): string {
-  return text.replace(/(?<!\\)\|/g, '\\|').replace(/\n/g, '<br>')
+  return text
+    .replace(/(\\*)\|/g, (match, backslashes: string) =>
+      backslashes.length % 2 === 0 ? `${backslashes}\\|` : match,
+    )
+    .replace(/\n/g, '<br>')
 }

packages/markdown/src/example-document.advanced.out.md

@@ -90,7 +90,7 @@ Here's a link to [a website](http://foo.bar), to a [local<br />
 doc](local-doc.html), and to a [section heading in the current<br />
 doc](#an-h2-header). Here's a footnote [^1].
 
-[^1]: Footnote text goes here.
+\[^1]: Footnote text goes here.
 
 Tables can look like this:
 
@@ -155,4 +155,4 @@ math should get its own line and be put in in double-dollarsigns:
 $$I = \int \rho R^{2} dV$$
 
 And note that you can backslash-escape any punctuation characters<br />
-which you wish to be displayed literally, ex.: `foo`, *bar*, etc.
+which you wish to be displayed literally, ex.: \`foo\`, \*bar\*, etc.

packages/markdown/src/example-document.out.md

@@ -176,7 +176,7 @@ Check out these autolinks: [https://example.com](https://example.com) and [mailt
   <p>This is raw HTML that gets preserved</p>
 </div>
 
-Inline HTML like <span class="highlight">highlighted text</span> can be handled too.
+Inline HTML like \<span class="highlight">highlighted text\</span> can be handled too.
 
 ### Reference Links