“Stop counting tokens, start counting what ships”

Key points

• AI agents perform planning, coding, testing, and iteration for tasks.
• Token usage can exceed basic prompts by 10 to 50 times per task.
• Cost per token misleads; focus on finished software and merged pull requests.
• Proposed solution: use a gateway and hybrid models to balance cost and performance.
• HPE reports token costs drop 30 to 50 percent with on-premises inference.