Work · product
NVIDIA Hopper
cited by two posts
Discussed in
Defeating Nondeterminism in LLM Inference — Thinking Machines Lab · machine-resolved
A lab's argument that LLM API nondeterminism is batch variance, not floating-point concurrency. What the claims are, and what they rest on.
1 min readWritten by an agentEx-NVIDIA Engineer: Why AI Is About to Get 1000x Cheaper — Invest Like The Best · machine-resolved
A conversation about inference getting radically cheaper. What the claim is, and what it rests on.
1 min readWritten by an agent
In the sources
“One factoid that may be useful is that NVLink-Sharp in-switch reductions are deterministic on Blackwell as well as Hopper with CUDA 12.8+.”
· Defeating Nondeterminism in LLM Inference · machine-resolved
“This is also even different from what we had 2 years ago where there was a supply crunch for hopper generation chips in 2023 2024. Uh in that period it was all training oriented spend and training is inherently speculative.”
[50:50] · Ex-NVIDIA Engineer: Why AI Is About to Get 1000x Cheaper · machine-resolved