…#24929)

Summary:
Pull Request resolved: pytorch#24929

Follow-up to D128116963. The C++ runner gets `--stream`, which writes 16-bit mono PCM to stdout as each 80 ms frame is decoded instead of writing the WAV at the end, so playback can start after the first frame (`phonon_runner ... --stream | aplay -q -f S16_LE -r 24000 -c 1`). The runner already decoded frame by frame through a callback; this only routes the samples to stdout and moves the summary line to stderr.

`--qlinear` now accepts only the modes that each backend has a kernel for (XNNPACK 8da4w/8da8w, CUDA 4w/8w, Vulkan and WebGPU 4w/8da4w, MLX 4w/8w) and rejects the rest at export instead of failing at runtime. Quantization covers the flow LM's transformer and flow net; the input projection, the one-output EOS head and the Mimi codec stay in the export dtype. The EOS head is skipped because the Vulkan and WebGPU 4-bit linear does not yet handle an output size that is not a multiple of 4. CUDA 4w uses the packed int4 kernel, which needs `--dtype bf16` and packs on the GPU, so two `torch.arange` calls in the attention take the input's device. MLX 4w uses asymmetric int4: symmetric int4 keeps the words but delays the end-of-speech decision, adding up to two seconds of silence at the end of a sentence, while asymmetric int4 keeps the timing of FP32. The README documents both features with the sizes per backend.

This diff was written with the help of Claude Code (Claude Opus 5.5).

Reviewed By: mergennachin

Differential Revision: D131277773

… a multiple of 4 (pytorch#24930)

Summary:
Pull Request resolved: pytorch#24930

make_linear_q4gsw_op and make_linear_dq8ca_q4gsw_op pad the weight scales and the bias of a 4-bit linear to a multiple of 4 output channels, but they do not pad the packed weights. The CompositeExplicitAutograd implementation of et_vk.linear_q4gsw (also used by et_vk.linear_dq8ca_q4gsw) dequantized the unpadded weights with the padded scales, so any lowering that re-traced the graph failed for such a linear, for example the 512 -> 1 end-of-speech head of a TTS model:

  RuntimeError: shape '[1, 16, 1]' is invalid for input of size 64

The implementation now drops the padded scale and bias rows. This affects the export-time reference implementation only; the dynamically quantized 8-bit activation, 4-bit weight kernel already handles any number of output channels on device.

This diff was written with the help of Claude Code (Claude Opus 5.5).

Reviewed By: SS-JIA

Differential Revision: D131277774

…#24971)

Summary:
Pull Request resolved: pytorch#24971

The quantized linear pattern detector matched every linear with a dequantized constant weight, while replace_quantized_linear_patterns only has custom ops for 8-bit statically quantized activations and for 4-bit group-quantized weights (weight-only or with dynamically quantized activations). The partitioner delegates every node of a match, so an 8-bit weight-only linear (torchao IntxWeightOnlyConfig int8) was delegated with its torchao.dequantize_affine node left in place. The Vulkan runtime has no dequantize_affine kernel (the quantize/dequantize op implementations were removed in pytorch#15750), so the program failed to initialize on device. The registry entries for torchao.quantize_affine and torchao.dequantize_affine still claimed both ops as supported, which also delegated them outside of patterns.

The detector now reports a match only when a custom op exists for it and the runtime can run it. The weight-only 4-bit kernel packs the weights in blocks of 4 output channels (Q4gswLinear.cpp checks N % 4 == 0 at prepack), so weight-only 4-bit linears with other output sizes are no longer matched; previously they exported and then failed at init. The match now also records the activation's quantize node in the dynamically quantized pattern, which the partitioner previously picked up only through the stale quantize_affine registry entry, and the two torchao registry entries are removed.

Unmatched linears now stay outside of the delegate. With torchao quantization this surfaces at export as "Missing out variants: {'torchao::dequantize_affine'}" instead of a failure on device. WebGPU uses the same patterns, so the same applies there.

Not changed: the quantized_decomposed quantize_per_channel / quantize_per_token / dequantize_per_channel / dequantize_per_token and choose_qparams (tensor, per_token_asymmetric) registry entries also have no runtime kernel since pytorch#15750. I left them because the VulkanQuantizer dynamic flow may still depend on them being claimed; they are worth a look by the backend owners.

This diff was written with the help of Claude Code (Claude Opus 5.5).

Reviewed By: SS-JIA

Differential Revision: D131277775

…s on XNNPACK

Summary:
XNNPACK's deconvolution is the most expensive op in Phonon's Mimi decoder: about 45% of each frame on x86 and 38% on arm64. Every transposed convolution in Mimi has kernel = 2 * stride, so each one is exactly a kernel-2 convolution with stride * out_channels outputs, one per output phase, whose phases are then interleaved into the output steps. The XNNPACK export now applies that rewrite (conv_transpose1d_as_conv1d) to the three decoder transposed convolutions, and the existing depthwise_conv_transpose_as_mul to the upsampler. The output is the same up to float rounding: the exported 31M program writes audio within 1 LSB (16-bit PCM) of the program without the rewrite, with the same length and frames.

Running the transposed convolutions as 2-D convolutions with the sequence on the width axis (conv_transpose1d_as_2d) does not help: Mimi stays at 8.2 ms per frame, so the gain comes from the convolution kernel rather than the tiling axis.

This diff was written with the help of Claude Code (Claude Opus 5.5).

Differential Revision: D131605676