← Profile · Projects · Development branch · Upstream
I develop Vulkan, CPU, and Windows/CUDA improvements in my contribution fork of mudler/vllm.cpp. The links below identify my changes rather than attributing the whole engine to me.
| Area | Contribution | Evidence |
|---|---|---|
| Vulkan compute | NVIDIA capability-gated cooperative-matrix paths, split attention, and GGUF SwiGLU kernels. | Implementation |
| Vulkan portability | Read subgroup width from the device in both attention kernels; size paged-attention shared memory by the workgroup. | Device subgroup width · Shared memory |
| Vulkan tuning & diagnostics | Build decode GEMV at three workgroup widths while keeping the default unchanged; expose driver-emitted shader diagnostics. | GEMV widths · Diagnostics |
| Triton CPU | Add an opt-in acceleration provider with native CPU fallback, plus a configurable MatmulBT M ceiling. | Provider · Dispatch control |
| Native CPU performance | Vectorize paged-attention scores across keys with AVX-512; store matmul output by row. | Attention · Matmul |
| Windows / CUDA | Repair MSVC build portability and vendor consumer Blackwell sm_120a Triton AOT assets. | MSVC repairs · Blackwell AOT |
| Serving correctness | Repair UTF-8 handling for individually decoded tokens and preserve sampler logprob output on the async path. | UTF-8 · Logprobs |
- Implementation: these are commits in my
port/vulkanbranch. Upstream PR and merge links have not yet been verified for this record; the commits should not be read as proof of upstream acceptance. - Platform scope: the optional Triton CPU provider currently loads kernels on Linux. Vulkan's NVIDIA-specific paths are capability-gated and retain portable fallback paths.
- Performance: commit messages include workload-specific measurements and correctness limits. They are development evidence, not a claim that every model or GPU becomes faster. In particular, the GEMV width experiment reports a token divergence at concurrency 16 and keeps the default unchanged.
- Fork releases: my Windows CUDA release packaging and fork artifact identity are maintained for this fork, separately from contributions intended for upstream.