Visitar URL original
fix(search): derive snippet spans from the matched bytes by MaxFreedomPollard · Pull Request #1643 · kenn-io/agentsview · GitHub
Skip to content

fix(search): derive snippet spans from the matched bytes - #1643

Merged
mariusvniekerk merged 1 commit into
kenn-io:mainfrom
MaxFreedomPollard:fix/snippet-span-rune-alignment
Sep 7, 2026
Merged

mariusvniekerk merged 1 commit into
kenn-io:mainfrom
MaxFreedomPollard:fix/snippet-span-rune-alignment

Conversation

@MaxFreedomPollard

Copy link
Copy Markdown
Contributor

The three content-search snippet paths end a match at start+len(pattern). The start offset is already computed the careful way, rune by rune over the body, so a case mapping that changes UTF-8 length cannot shift it, but the end is taken from the pattern. Simple folding maps U+212A KELVIN SIGN, three bytes, to k, one byte, so a body holding the long form ends the span two bytes inside that rune, and a pattern holding it ends the span two bytes past the match.

snippetBounds and its PostgreSQL and DuckDB twins pad the span by 60 bytes and snap only the padding edges back to a rune boundary. Their loops stop when they reach the span, so none of them can repair a span that is not aligned. The 60-byte radius means no snippet is visibly wrong today, but rune alignment is the precondition those helpers and secrets.RedactWindow are written against, and the three copies of the same end expression are the kind of thing that drifts.

CaseInsensitiveIndex becomes CaseInsensitiveSpan: it returns the byte range the match actually covers in the body plus whether one was found, and each backend windows that range instead of recomputing an end from the pattern. Everything ASCII, which is nearly every query, gets the identical span it got before.

One behavior change comes with it. A miss now yields an empty span at the start of the body rather than a pattern-length one, so the fallback window is up to len(pattern) bytes shorter on its right edge. The substring paths reach that branch only if SQL LIKE/ILIKE matched where the Go search did not, and the Go fold is the wider of the two, so it is unreachable there; FTSSnippetRange already returned 0, 0 for its own start fallback.

Reviewer attention is best spent on CaseInsensitiveSpan and foldPrefixEnd in internal/db/search_content.go. The PostgreSQL and DuckDB call sites are the same two-line substitution, and I could not exercise their suites locally since both need services I do not have set up.

The three content-search snippet paths ended a match at
start+len(pattern): internal/db/search_content.go substringSnippet and
FTSSnippetRange, internal/postgres/search_content.go pgSubstringSnippet,
and internal/duckdb/store.go. The offset that starts the span already
came from CaseInsensitiveIndex, which walks the body rune by rune so it
cannot be shifted by a case mapping that changes UTF-8 length, but the
end was taken from the pattern instead. Simple folding maps U+212A
KELVIN SIGN (three bytes) to "k" (one byte), so a body carrying the long
form ends the span two bytes inside that rune, and a pattern carrying it
ends the span two bytes past the match.

snippetBounds and its PostgreSQL and DuckDB twins snap only the padding
edges: their loops stop at the span, so they cannot repair a span that
is not rune-aligned. Today the 60-byte radius hides it, because the
padded edge always lands on a boundary or at the end of the body, but
the alignment is the precondition those helpers are written against and
secrets.RedactWindow documents.

CaseInsensitiveIndex becomes CaseInsensitiveSpan, returning the byte
range the match actually covers in the body along with whether one was
found, and the three backends window that range instead of each
recomputing an end from the pattern. A miss now reports the start of the
body with an empty span rather than a pattern-length one; the substring
paths reach it only when the SQL LIKE/ILIKE matched but the Unicode-aware
Go search did not, which cannot happen because the Go fold is the wider
of the two.
@roborev-ci

roborev-ci Bot commented Sep 6, 2026

Copy link
Copy Markdown

roborev: Combined Review (deb57c7)

No issues found.


Reviewers: 2 done | Synthesis: codex | Total: 2m6s

@mariusvniekerk mariusvniekerk self-assigned this Sep 7, 2026
@mariusvniekerk
mariusvniekerk merged commit add0ee1 into kenn-io:main Sep 7, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants