Skip to content

Fix link-patterns replacing matches inside inline HTML markup - #753

Open
rahul05ranjan wants to merge 1 commit into
trentm:masterfrom
rahul05ranjan:fix/link-patterns-html-markup
Open

rahul05ranjan wants to merge 1 commit into
trentm:masterfrom
rahul05ranjan:fix/link-patterns-html-markup

Conversation

@rahul05ranjan

Copy link
Copy Markdown

Fixes #752

Summary

LinkPatterns.run guards its matches against markdown link syntax and <http://...> auto-links, but not against the other tokens that markdown2's own tokenizer (_sorta_html_tokenize_re) classifies as inline HTML markup: tags, comments, processing instructions and <word@example.com> style auto-link tokens. With a pattern that matches inside one of those tokens, the extra replaces the match inside markup:

  • <span title="word">x</span> → <span title="<a href="/w/word">word</a>">x</span> — an anchor inside the title attribute value (invalid HTML)
  • <img src="/x.png" alt="word"> → the alt attribute is corrupted the same way
  • <word@example.com> → the mailto auto-link is destroyed entirely: the match splits the token so _do_auto_links (which runs after _do_links) no longer recognizes it, leaving &lt;<a href="/w/word">word</a>@example.com&gt; behind

Root cause

The is_inside_link guard in LinkPatterns.run only checks _auto_link_re (URL auto-links) and _basic_link_re (markdown link syntax) — the gaps left after #385/#446. The markup tokens themselves were unprotected.

Fix

Compute the inline HTML markup spans from the same tokenizer markdown2 already uses for this purpose (_sorta_html_tokenize_re, honoring the same unescaped-prefix rule as _escape_special_chars) once per pattern (hash replacements from earlier patterns shift offsets) and skip matches that fall inside them. Matches in plain prose, including inside the text content of inline HTML tags, are unaffected.

Verification

  • Two new tm-cases (link_patterns_html_tags, link_patterns_email_autolink): both fail on master, pass with this patch.
  • Full suite on the Raspberry Pi host (Python 3.13, no pygments-gated cases): 316 ran, 21 failures — byte-identical failure set to the pre-patch baseline (all pre-existing knownfailure/pygments-env cases).
  • Probed no-regression cases: escaped tags (\<span title="word">), escaped chars inside attributes, multi-pattern runs where an earlier pattern inserts a hash before a later tag, <mailto:...> auto-links, and matches inside inline HTML tag text (still linkified, consistent with core emphasis behavior there).
import re
import markdown2
lp = [(re.compile(r"\b(word)\b"), r"/w/\1")]
print(markdown2.markdown('See <span title="word">x</span>', extras=["link-patterns"], link_patterns=lp))
# <p>See <span title="word">x</span></p>
print(markdown2.markdown('<word@example.com>', extras=["link-patterns"], link_patterns=lp))
# <p><a href="mailto:word@example.com">word@example.com</a></p>

LinkPatterns.run guards its matches against markdown link syntax and
<http://...> auto-links, but not against the other tokens that
markdown2's own tokenizer (_sorta_html_tokenize_re) recognizes as inline
HTML markup: tags, comments, processing instructions and <word@example.com>
style auto-link tokens. As a result the extra replaces matches inside
markup, corrupting output:

- <span title="foo bar">baz</span> became <span title="<a href=...>...</a>">
- <img src="/qux.png" alt="foo"> got an anchor inside its alt attribute
- <foo@example.com> lost its mailto auto-link entirely: the match split
  the token so the auto-linker no longer recognized it, leaving broken
  text behind

Compute the markup token spans from the same tokenizer (respecting the
escape-prefix rule from _escape_special_chars) per pattern and skip
matches that fall inside them. Matches in plain prose are unaffected.

Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

link-patterns extra replaces matches inside inline HTML tags and destroys email auto-links

1 participant