Add a PDF Selector for Multi-Rectangle Annotations

[panaC]

Previously removed in commit b1e1442 and originally discussed in the now-closed issue #9.

Context

A PDF annotation is represented by either:

• a page number associated with an array of quadpoints describing one or more highlighted areas on that page
• an array of objects, each containing a page number and its corresponding quadpoints array, allowing a single annotation to span multiple pages.

At the moment, there is no selector capable of representing this PDF-specific selection.

References:

• https://github.com/highkite/pdfAnnotate/blob/master/README.md#quadpoints
• https://github.com/edrlab/thorium-reader/blob/develop/src/renderer/reader/pdf/pdfAnnotationConverters.ts#L43-L90
• https://developer.apple.com/documentation/pdfkit/pdfannotationkey/quadpoints
• 7.4.5 Markup Annotations https://opensource.adobe.com/dc-acrobat-sdk-docs/pdfstandards/pdfreference1.3.pdf

A previous proposal suggested using a FragmentSelector based on the PDF Media Types specification (RFC 8118): https://www.rfc-editor.org/info/rfc8118/

The specification defines fragment identifiers such as:

page=<pageNum>

to identify a physical page (starting at page 1), and:

viewrect=<left>,<top>,<width>,<height>

to define a rectangular viewing region.

While this approach is suitable for navigating to a page or a single rectangular region, it cannot accurately represent the geometry of a text selection or highlight composed of multiple rectangles. A single annotation frequently spans several text lines, each represented by its own quadrilateral, making the RFC 8118 fragment syntax insufficient for this use case.

Quadpoints provide a more expressive representation. They can be defined either as:

• an array of four coordinate tuples describing the four vertices of a quadrilateral in page coordinates (origin at the bottom-left of the page, consistent with the PDF coordinate system), enabling arbitrary polygonal selections; or
• an array of two coordinate tuples defining the opposite corners of an axis-aligned rectangle.

Supporting quadpoints as a dedicated selector would preserve the complete geometry of PDF annotations, including multi-line highlights and other non-rectangular selections, which cannot be faithfully represented using the existing PDF fragment identifier syntax.

Official def:

An array of 8 × n numbers specifying the coordinates of n quadri-
laterals in default user space. Each quadrilateral encompasses a word or
group of contiguous words in the text underlying the annotation. The coor-
dinates for each quadrilateral are given in the order
x1 y1 x2 y2 x3 y3 x4 y4
specifying the quadrilateral’s four vertices in counterclockwise order (see
Figure 7.5). The text is oriented with respect to the edge connecting points
(x1, y1) and (x2, y2). (See implementation note 59 in Appendix H.

[mickael-menu]

I ran into the same question from the Readium Locator side.

fragments is an array, so a multi-line, multi-page selection can be expressed today by emitting one viewrect per line, each qualified by its page:

{
  "href": "publication.pdf",
  "type": "application/pdf",
  "locations": {
    "position": 4,
    "progression": 0.03,
    "totalProgression": 0.12,
    "fragments": [
      "page=4&viewrect=72,92,451,16",
      "page=4&viewrect=72,108,468,16",
      "page=5&viewrect=72,92,438,16",
      "page=5&viewrect=72,108,426,16"
    ]
  },
  "text": {
    "before": "…",
    "highlight": "full selected text across both pages",
    "after": "…"
  }
}

This changes the meaning of fragments: the entries are no longer alternative ways of pointing at the same location, they are pieces of a single region that can be concatenated, and their order is significant. To be discussed.

Could this transfer to selector too?

An alternative could be found in RFC 8118, which separates parameters with & and processed from left to right, which gives
page a running-state reading:

page=4&viewrect=72,92,451,16&viewrect=72,108,468,16&page=5&viewrect=72,92,438,16&viewrect=72,108,426,16

One FragmentSelector, one fragments entry. selector and fragments stay parallel and no new selector type is needed.

[llemeurfr]

I understand that multi-line highlights require multiple rectangles. I tried to create a multi-page highlight in Thorium Desktop on a PDF document, but I didn't succeed (the app returned an error); but I imagine that this is also possible on some devices. Then, I created a document on Apple Page, with a non-horizontal line (in a vector graphic); once exported in PDF, I was able to highlight it; the result seems to be a rectangle, not a random quadpoint.

Therefore, it appears that we definitely want to represent an array of objects, each object being a page + an array of rectangles. The Web Annotations PDF FragmentSelector https://www.w3.org/TR/annotation-model/#fragment-selector does not explicitly handle this use case.

After some thinking, I'm not a fan of the parameter sequence proposed by Mickaël. Repeating parameters in a URL fragment is syntactically valid (it is not forbidden because, unlike query parameters, the fragment is defined as a single string), but one has to iterate through them sequentially using .entries() and manage the association between each page and the list of viewrect that follow the page number. This feels unnatural. If you think parsing such a string will be easy for any developer, we can still go with this syntax.

If we look at the issue from a global view, we are only constrained by the syntax of the selector object, which has a type and properties we can freely define.

A solution is therefore to revive and update an old proposal:

PDF Selector

If the publication is in PDF format, it is possible to identify an area of interest using a PDFSelector.

Sample: two rectangular fragments identified on page 10 by a PDFSelector:

{
  "selector": [
      {
      "type": "PDFSelector",
      "fragments": [ 
        { 	 
        "page": 10,
        "viewrect": "50,50,650,480"
        },
        { 	 
        "page": 10,
        "viewrect": "60,60,650,480"
        }
      ]
    }
  ]
}

This is more verbose, but totally adapted to the model and classical json syntax (I didn't check if json-ld is fan of this, but I suppose there is no problem).

A more compact syntax is the one introduced by Mickaël and used for Locators:

"fragments": [
      "page=4&viewrect=72,92,451,16",
      "page=4&viewrect=72,108,468,16",
      "page=5&viewrect=72,92,438,16",
      "page=5&viewrect=72,108,426,16"
    ]

I have no preference for one or the other.

@panaC and @mickael-menu you'll tell me which syntax you prefer and why, and I'll update the Readium Annotations spec with it.