My Weird Prompts
Why LLM Output Tokens Cost More Than Input
23:25
Provider rate cards charge three to five times more for output tokens than input tokens, driven by the physics of memory bandwidth bottlenecks during decode. Comparing models using industry-standard three-to-one blends misrepresents actual costs, especially as agentic harnesses and retrieval-augmented generation skew workloads anywhere from twenty-two-to-one to over a hundred-to-one. Analysis…