truncateExcerpt in internal/db/analytics.go cut its input with a plain
byte slice, so an excerpt longer than the 180-byte budget was severed at
byte 177 wherever that landed. When byte 177 falls inside a multibyte
character the result is invalid UTF-8, and encoding/json rewrites the
broken tail into U+FFFD, so a signal example came back to the Quality
page ending in one or two replacement characters instead of its last
whole character. 177 is a multiple of three, so a pure Han excerpt
happened to survive, but a ten-byte ASCII lead such as "Fix this: "
shifts the alignment and breaks it; four-byte emoji and two-byte
Cyrillic break on other leads.
Both slices now walk back to the nearest rune start. That walk already
existed inline in ClampModel, so it moves out into a
runeBoundaryAtOrBefore helper in internal/db/validate.go beside the
utf8RuneStart predicate it uses, and ClampModel calls it too. The
excerpt cut reuses that walk instead of adding a second copy of it.
ASCII excerpts keep byte-identical output and the budget stays an upper
bound. The one behavior change is on the degenerate max <= 3 branch,
where a multibyte string now yields an empty string rather than a broken
fragment; the only caller passes 180, so that branch is unreachable in
practice.
The PostgreSQL and DuckDB stores build their examples by calling
db.BuildSignalExamples, so all three backends take the fix from this
single function with no twin edits to keep in step.
On the Quality page, a signal example whose excerpt runs past 180 bytes can render with a black diamond replacement character at the end instead of its last whole character. A message like "Fix this: 这个函数返回错误的结果..." comes back with the final character destroyed. The excerpt has one consumer, the signal sessions panel in
frontend/src/lib/components/quality/QualityPage.svelte.truncateExcerptininternal/db/analytics.gocuts with a plain byte slice,s[:max-3] + "...". Its only caller isBuildSignalExamples, which passes 180, so the cut always lands on byte 177. When byte 177 is a continuation byte the returned string is invalid UTF-8, andencoding/jsonrewrites the split character into U+FFFD on the way to the API. 177 is a multiple of three, so a pure Han excerpt happens to survive; a ten-byte ASCII lead such as "Fix this: " shifts the alignment and breaks it, four-byte emoji break on a different lead, and two-byte Cyrillic on a third. The content arriving here is already valid UTF-8 throughdb.SanitizeUTF8, so the invalid bytes are introduced at this one line.Both slices now back up to the nearest rune start. That walk already existed inline in
ClampModel, so it moves into aruneBoundaryAtOrBeforehelper ininternal/db/validate.gobeside theutf8RuneStartpredicate it uses, andClampModelcalls it too. The excerpt cut reuses that walk instead of adding a second copy of it. ASCII excerpts keep byte-identical output and the 180-byte budget stays an upper bound.One behavior change worth naming: on the degenerate
max <= 3branch a multibyte string now yields an empty string rather than a broken fragment. The only caller passes 180, so that branch is unreachable in production; it is kept correct rather than removed. Backend parity holds by construction, sinceinternal/postgres/analytics.goandinternal/duckdb/analytics_usage.goboth build their examples by callingdb.BuildSignalExamples.Reviewer attention is best spent on the
ClampModelhunk ininternal/db/validate.go, the one existing caller this change reshapes.truncateLogLineininternal/insight/generate.gohas the same defect against a 16 KiB stdout budget; I left it out to keep this change to one concern. I did not run the PostgreSQL or DuckDB suites locally.