How NVIDIA’s Inference Software Stack Powers the Lowest Token Cost

Organizations are transitioning from initial AI experiments to establishing production factories, shifting infrastructure decisions from chip specifications to token costs. NVIDIA's software is designed to work seamlessly with GPUs, CPUs, networks, and systems, continuously enhancing performance. On the NVIDIA Blackwell platform, token costs for the DeepSeek V4 model have been reduced by up to five times within a month. Leading companies like Baseten utilize the NVIDIA TensorRT-LLM library to deliver the DeepSeek V4 Pro model, increasing tokens per second by as much as 50%.