Contributing to the open-source projects we rely on

Add W4A16 per-channel weight-only support for Cutlass fused MoE

INT4 weight-only quantization was refused outright on the fused mixture-of-experts path, even though the kernels beneath it already handled it.

+107.6% throughput199.8 → 414.7 tok/s, 52.4% lower time-per-token, 69.7% less weight memory; GSM8K within noise

Fuse rms_norm + mul + rope into one CUDA kernel

Two existing fusions were leaving a round trip through memory between them on every layer.

≈1% tokens/s72 of 72 backend tests pass against the CPU reference

Fix thread and block counts in quantized cpy kernel launches

The quantized copy kernels ran one thread per block, so 31 of every 32 warp lanes sat idle, and around 97% of blocks exited immediately at the bounds check.

5.2× and 47× on the copy kernels282 → 1,474 GB/s and 9 → 426 GB/s; output bit-identical

Opt-in coalescing of timed ASR calls

A live transcription server re-encoded audio it had already seen, once per tick.

≈½ the encoder calls40 → 19 calls per run, transcription quality unchanged

One less pass over the input in the MVN kernel

The normalization kernel read its input twice; the variance can be computed during the first pass. Intel took the finding and landed it through their own extension.

Split AST parsing from the class execution loop

Component loading re-parsed and re-executed the same source on every pass.

+57% on the slowest test417 → 265 ms, measured independently by CodSpeed

Fix two index errors in the bloom filter

One loop indexed by the wrong variable, so it read the same word every iteration; another had an incomplete terminating condition.

one line changednot a speedup — a correctness bug in a CUDA data-structure library

Precompute annuities instead of recomputing them in the loop

A nested loop called coterminalSwapAnnuity() for every rate on every pass, recomputing values that never changed.

≈30% faster unit testseven lines changed