Gate inline image and video galleries to cited media only - #1331
Merged
Paul Lizer (paullizer) merged 1 commit intoAug 21, 2026
Merged
Conversation
Inline galleries were built from the full retrieved citation arrays, so every workspace or web media file returned by search became a tile inside the message bubble even when the response never referenced it. That presented unrelated media as though it supported the answer, consumed the five-item gallery cap with retrieval noise, and issued enhanced-citation fetches for documents that were never cited. Issue #1249 already persists the exact cited subsets on each assistant message and delivers them to the browser on every path, but no frontend module read them. Add chat-citation-tracking.js as the browser mirror of _message_has_citation_tracking, and feed the cited subsets to both gallery renderers from appendMessage. The Sources disclosure keeps the complete retrieved set. Agent and tool galleries stay ungated because they are executed results, and legacy messages without tracking keep prior behavior rather than being parsed at read time. Fixes #1329 Refs #1249 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Paul Lizer (paullizer)
force-pushed
the
paullizer-inline-media-citation-gating
branch
from
August 21, 2026 00:36
c2fc062 to
f9b8179
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1329
Refs #1249
Problem
Assistant messages rendered inline image and video galleries for every media document returned by retrieval, not just the media the response actually cited. A workspace search that surfaced five image files produced five inline gallery tiles even when the answer referenced only one of them, or none at all.
Because the galleries sit directly inside the message bubble, unrelated media was presented as though it supported the answer. The five-item gallery cap could be consumed entirely by retrieval noise, pushing genuinely cited media out of view, and each unreferenced workspace file triggered an additional enhanced-citation fetch.
Root cause
renderInlineImageGalleriesandrenderInlineVideoGalleriesreceived the fullhybrid_citationsandweb_search_citationsarrays fromappendMessage, then selected workspace media purely by file extension viaextractWorkspaceCitationImageItems/extractWorkspaceCitationVideoItems.#1249 already built the retrieved-vs-cited separation: every assistant message persists
cited_hybrid_citationsandcited_web_search_citations, and those subsets already reach the browser on all delivery paths. The inline gallery renderers were simply never switched over — no frontend module read the cited subsets at all.Change
Frontend only. No backend change was required.
application/single_app/static/js/chat/chat-citation-tracking.js— the browser mirror offunctions_citation_tracking._message_has_citation_tracking(). ExportsmessageHasCitationTracking,getCitedHybridCitations,getCitedWebCitations. A message counts as tracked whencitation_tracking_version >= 1or eithercited_*key is present; each getter normalizes non-array values to an empty list.appendMessagederivescitedHybridCitations/citedWebCitationsfrom the assistant message object and passes those to both gallery renderers.The Sources disclosure, its count badges, and the metadata drawer still receive the complete retrieved arrays, so nothing became harder to find.
Deliberate scope boundaries
get_message_reference_citation_buckets()and [P1] Distinguish retrieved sources from cited documents #1249's decision to avoid read-time history parsing..inline-visualizations-containerwithd-nonewhen it has no children, so an empty result leaves no visual gap.Behavior
Validation
functional_tests/test_inline_media_cited_only_gating.py(new)functional_tests/test_chat_cited_source_tracking.pyfunctional_tests/test_docs_site_quality.pyfunctional_tests/test_docs_app_surface_coverage.pyui_tests/test_chat_inline_*_gallery_rendering.pyThe new functional test executes the real
chat-citation-tracking.jsmodule under Node across tracked, untracked, empty-cited, key-presence-only, missing-message, and malformed-value inputs, so the fallback rules are covered behaviorally rather than by source assertion alone.Both Playwright gallery tests gained a gating regression covering three messages on one page: a tracked response that cited one of two retrieved media files, a tracked response that cited nothing but ran a media action, and a legacy untracked response.
Note
functional_tests/test_inline_image_gallery_visualization.pyand its video counterpart cannot execute locally because they importfunctions_workflow_runner, which pulls inconfig.pyand constructs a live Cosmos client at import time. That is a pre-existing environment constraint unrelated to this change; their updated source assertions were verified directly against the modified files.Documentation
docs/explanation/fixes/INLINE_MEDIA_CITED_ONLY_GATING_FIX.md### **(v0.260.024)**, with the generated Jekyll release-note pages regenerated viascripts/build_release_notes_pages.pyVersion bumped to
0.260.024. This branch was rebased ontoDevelopmentafter #1330 landed and claimed0.260.023.