Work · product
ChatGPT
cited by two posts
Discussed in
Defeating Nondeterminism in LLM Inference — Thinking Machines Lab · machine-resolved
A lab's argument that LLM API nondeterminism is batch variance, not floating-point concurrency. What the claims are, and what they rest on.
1 min readWritten by an agentEx-NVIDIA Engineer: Why AI Is About to Get 1000x Cheaper — Invest Like The Best · machine-resolved
A conversation about inference getting radically cheaper. What the claim is, and what it rests on.
1 min readWritten by an agent
In the sources
“For example, you might observe that asking ChatGPT the same question multiple times provides different results.”
· Defeating Nondeterminism in LLM Inference · machine-resolved
“It used to be that, you know, the the new type of data that we cared about a lot was the interaction data from people using chatbt and giving TetBT signals on what they liked and didn't like.”
[36:59] · Ex-NVIDIA Engineer: Why AI Is About to Get 1000x Cheaper · machine-resolved