Repository navigation
Fix link-patterns replacing matches inside inline HTML markup - #753
Open
rahul05ranjan wants to merge 1 commit into
Open
rahul05ranjan wants to merge 1 commit into
rahul05ranjan wants to merge 1 commit into
Conversation
LinkPatterns.run guards its matches against markdown link syntax and <http://...> auto-links, but not against the other tokens that markdown2's own tokenizer (_sorta_html_tokenize_re) recognizes as inline HTML markup: tags, comments, processing instructions and <word@example.com> style auto-link tokens. As a result the extra replaces matches inside markup, corrupting output: - <span title="foo bar">baz</span> became <span title="<a href=...>...</a>"> - <img src="/qux.png" alt="foo"> got an anchor inside its alt attribute - <foo@example.com> lost its mailto auto-link entirely: the match split the token so the auto-linker no longer recognized it, leaving broken text behind Compute the markup token spans from the same tokenizer (respecting the escape-prefix rule from _escape_special_chars) per pattern and skip matches that fall inside them. Matches in plain prose are unaffected. Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #752
Summary
LinkPatterns.runguards its matches against markdown link syntax and<http://...>auto-links, but not against the other tokens that markdown2's own tokenizer (_sorta_html_tokenize_re) classifies as inline HTML markup: tags, comments, processing instructions and<word@example.com>style auto-link tokens. With a pattern that matches inside one of those tokens, the extra replaces the match inside markup:<span title="word">x</span>→<span title="<a href="/w/word">word</a>">x</span>— an anchor inside thetitleattribute value (invalid HTML)<img src="/x.png" alt="word">→ thealtattribute is corrupted the same way<word@example.com>→ the mailto auto-link is destroyed entirely: the match splits the token so_do_auto_links(which runs after_do_links) no longer recognizes it, leaving<<a href="/w/word">word</a>@example.com>behindRoot cause
The
is_inside_linkguard inLinkPatterns.runonly checks_auto_link_re(URL auto-links) and_basic_link_re(markdown link syntax) — the gaps left after #385/#446. The markup tokens themselves were unprotected.Fix
Compute the inline HTML markup spans from the same tokenizer markdown2 already uses for this purpose (
_sorta_html_tokenize_re, honoring the same unescaped-prefix rule as_escape_special_chars) once per pattern (hash replacements from earlier patterns shift offsets) and skip matches that fall inside them. Matches in plain prose, including inside the text content of inline HTML tags, are unaffected.Verification
link_patterns_html_tags,link_patterns_email_autolink): both fail on master, pass with this patch.knownfailure/pygments-env cases).\<span title="word">), escaped chars inside attributes, multi-pattern runs where an earlier pattern inserts a hash before a later tag,<mailto:...>auto-links, and matches inside inline HTML tag text (still linkified, consistent with core emphasis behavior there).