Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers.
decision-making cuda nvidia calibration quantization int8 inference-engine h200 llm-inference gguf cuda-graphs startlux
-
Updated
Oct 22, 2026 - Python