Visitar URL original
Run resvg's test suite in the WPT runner by nicoburns · Pull Request #1544 · DioxusLabs/blitz · GitHub
Skip to content

Run resvg's test suite in the WPT runner - #1544

Open
nicoburns wants to merge 6 commits into
mainfrom
devin/1794509720-resvg-test-suite
Open

nicoburns wants to merge 6 commits into
mainfrom
devin/1794509720-resvg-test-suite

Conversation

@nicoburns

@nicoburns nicoburns commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds resvg's test suite as a second signal for SVG rendering, next to WPT. It is mostly a usvg-parity check for native SVG (#1541). resvg's crates/resvg/tests has about 1,780 SVGs, one feature each, and each one has a PNG that resvg rendered. Nothing gets vendored: wpt/RESVG_COMMIT pins resvg v0.50.0 (5bdbae6), the same version as Blitz's usvg. just wpt-resvg fetches that commit, wraps every test as a reftest (wpt resvg-generate), runs the existing WPT runner on the wrapped tests and prints a per-directory table (wpt resvg-summary). Both are subcommands of the WPT runner (wpt/runner/src/resvg.rs), so nothing needs Python.

Each test is generated three times. You pick which ones to run with the usual path args.

Path Test page Reference What it measures
resvg/img/... <img src=test.svg> resvg's PNG the usvg backend
resvg/inline/... the SVG inlined in the page resvg's PNG the inline backend in use (native with BLITZ_NATIVE_SVG=1)
resvg/parity/... the SVG inlined in the page <img src=test.svg> native vs usvg on the same rasterizer (run with BLITZ_NATIVE_SVG=1)

To inline an SVG, the generator drops the XML prolog, expands the entities its DOCTYPE declares, and removes a namespace prefix on the elements (<svg:svg xmlns:svg=...>), because the HTML parser handles neither.

Runner changes:

  • New tolerance meta. resvg's PNGs come from tiny-skia, so anti-aliased edges and gradients differ slightly almost everywhere. WPT's fuzzy meta can't ignore small differences, so the runner now reads a Blitz-only <meta name="blitz-fuzzy" content="<threshold>;<pixels>">: the renders match if at most <pixels> pixels differ by more than <threshold> in any channel. Attribute order and quoting don't matter, and a malformed value (such as a threshold over 255) panics instead of falling back to the default comparison. PNG comparisons use 64;100, which is 0.25% of a 200×200 render, about one edge's worth. The scope's 64;450 let shape-rendering="crispEdges", dash-phase and arc differences of 100–370 pixels pass, and the tighter count fails 22 of those on usvg. Parity uses 16;90.
  • Report revision and output dir. The generator writes the resvg commit to REVISION in the generated WPT_DIR, and the report records it as revision. Without it the report would record the Blitz commit, since wpt/resvg/out is inside the Blitz checkout. A new WPT_OUTPUT_DIR (default wpt/output) lets the recipe write to wpt/resvg/output, so a resvg run doesn't overwrite an earlier WPT run.

Fonts: the generator writes a fonts.conf that loads only resvg's tests/fonts and maps generic families the way resvg's harness does (serif → Noto Serif, etc.). The recipe passes it as FONTCONFIG_FILE, which both fontdb (usvg) and fontique (parley) read. That takes text from 17 to 369 passes out of 387 on usvg.

Results

This PR is based on main. The native numbers come from this branch merged locally with #1541 (c032101); the BLITZ_NATIVE_SVG switch only exists after that PR. "parity where img passes" counts parity only on tests where usvg matches resvg's PNG, so two backends failing the same way don't count as a pass.

usvg backend (just wpt-resvg, on main):

Directory img inline
filters 102 / 416 102 / 416
masking 50 / 100 50 / 100
paint-servers 124 / 163 121 / 163
painting 252 / 314 246 / 314
shapes 130 / 133 130 / 133
structure 231 / 267 229 / 267
text 369 / 387 369 / 387
Total 1258 / 1780 1247 / 1780

Native backend (BLITZ_NATIVE_SVG=1 just wpt-resvg, with #1541):

Directory inline (native vs resvg PNG) parity where img passes (native vs usvg)
filters 88 / 416 74 / 102
masking 7 / 100 7 / 50
paint-servers 16 / 163 10 / 124
painting 131 / 314 129 / 252
shapes 118 / 133 118 / 130
structure 116 / 267 111 / 231
text 12 / 387 11 / 369
Total 488 / 1780 460 / 1258

Native fails mostly in the areas #1541 leaves for later (<use>, paint servers, clip/mask, text). Two shapes findings are worth noting:

  • Arcs: the native renderer draws path arcs (shapes/path/M-A* and the arc-flag tests) on the wrong side, which looks like a real bug.
  • Viewport units: shapes/rect/*-values fail by construction. Inline, vw/rem/ch resolve against the HTML page; in an <img>, they resolve against the image.

Known usvg inline failures. On usvg, 13 tests pass as img but fail inline or parity. Both causes are in Blitz's usvg inline path, not in how the tests are generated, so they don't count against native:

  • color attribute (12 tests): the currentColor tests set color="green" as an attribute (painting/color/*, stop-color-with-currentColor-*, current-color-*-before-color, use/with-currentColor, and the fill/*currentColor* tests). The usvg inline path resolves currentColor from the CSS color, which doesn't see SVG's color presentation attribute, so these render black. Native maps presentation attributes and passes the painting/color and painting/fill ones. The gradient and <use> ones fail on native because it doesn't support paint servers or <use> yet.
  • structure/svg/background-color (parity): inline, the <svg>'s CSS background is painted twice, once by the CSS box and once by usvg. The double-painted #0000ff80 lands just inside the 64 threshold against the PNG but fails parity.

not-UTF-8-encoding and background-color-with-viewbox already fail as img, so they aren't in "parity where img passes".

A full run (5,340 tests) takes a few seconds on vello_cpu. This PR doesn't add it to CI.

Link to Devin session: https://dioxus.staging.devinenterprise.com/sessions/6223775f21c0494ebbea21c5c7971ecf
Link to Bot: https://dioxus.staging.devinenterprise.com/org/dioxus-engineering/bots/3904245e951a4a7fa4631b37b308d89b
Open in Devin Desktop: https://dioxus.staging.devinenterprise.com/desktop/session/6223775f21c0494ebbea21c5c7971ecf?variant=devin-insiders

WPT results

No changes in test results compared to main.

Generated by the WPT workflow.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant