sdpa

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

Memory-optimized SongGeneration (v2 Large) for 16GB VRAM GPUs. Features 8-bit µ-law KV-caching, fused layers, and SDPA/Triton integration.

An open-source interface to use the multiple-precision solver SDPA-GMP with YALMIP

PyTorch implementation of YOLOv12 with Scaled Dot-Product Attention (SDPA) optimized by FlashAttention for fast and efficient object detection.

A 66M parameter decoder-only transformer language model implemented from scratch in PyTorch. Features a custom SentencePiece tokenizer, RoPE positional embeddings, SwiGLU feed-forward network, per-layer KV cache for efficient autoregressive inference, and a Svelte-based streaming chat interface.

You import the model's weights and start fine-tuning, unaware that you are missing out on more than 80% of gpu utilization. Axiom gets all the juice out of the box.

Attention backend benchmark on Turing GPUs comparing Vanilla, SDPA Math, SDPA Efficient, and a custom Triton FlashAttention implementation. SDPA efficient achieves 130× memory reduction and 10× speedup; Triton FA achieves O(n) memory but is 64× slower than SDPA efficient on RTX 2070.

Signed attention for transformers — sinh/cosh attention giving weights in [-1,1] instead of softmax's positive-only, made FlashAttention/SDPA-compatible by channel doubling. Includes a from-scratch softmax-vs-SBA comparison and a documented negative result.

Reproducible Mage-Flow-Turbo inference on Kaggle Tesla T4×2 using upstream PyTorch, BF16 dtype/materialization and SDPA. Includes canonical dual-GPU qualification, explicit transformer/VAE placement, fail-closed routing and provenance checks, sustained 12-image 512/768/1024 showcase, timing, GPU memory metrics, SHA-256 evidence, and bilingual docs.

Long-context benchmark pushing Qwen2-0.5B from 4K to 32K tokens on RTX 2070 using SDPA + chunked prefill. Shows 40x speedup at 8K, FP16 beating INT4 at long context, and that quantization is NOT a long-context solution — KV-cache is the real bottleneck.

Inference-only head-partitioned SDPA for Qwen2.5-Omni candidate scoring on RTX 5080 16GB, with numerical checks and evidence.

alternative Scaled-Dot-Product-Attention function for MLX, supports combined mask(additive mask + padding mask).自注意力计算，弥补MLX内置SPDA函数不支持组合mask的缺憾

FlashAttention-2 forward and Flash-Decoding kernels written from scratch in Triton, with grouped-query attention (GQA) and causal masking, benchmarked against PyTorch SDPA and a naive eager implementation, then run end-to-end inside a from-scratch Llama-3.2-1B.

Empirical LLM inference profiling on NVIDIA RTX A400 (Ampere SM86, 4GB): roofline ridge point, SDPA kernels, batching, CUDA Graphs, and WDDM PCIe paging telemetry.

Fork of sdpa-qd (quad-double SDP solver): first working OpenMP threading (1.5x), plus a fixed benchmark clock - upstream reported CPU time as elapsed, making speedups look like slowdowns.

OpenMP-threaded, reproducible fork of sdpa-gmp (arbitrary-precision SDP solver): threads the Schur/bMat/Cholesky/solve hot paths, bit-identical results across thread counts, benchmarked against pristine upstream and SDPB with published per-repeat raw data.

ForgeServe is a small, modular LLM inference engine built with Python, PyTorch, and Hugging Face Transformers, designed to make modern inference systems understandable by implementing them step-by-step rather than treating inference as a black box.

OpenMP-threaded, reproducible fork of sdpa-dd (double-double SDP solver): threads the Schur-complement build and BLAS kernels, bit-identical results at any thread count, with per-repeat raw benchmark data.