Key Concepts
- 1Every agent conversation has a real marginal cost — tokens, tool calls, third-party API fees — unlike a stateless web page, so unit economics matter per interaction
- 2Cost per conversation (CPC) must be weighed against value per conversation (conversion x margin); a feature that lifts conversion but doubles CPC may be net-negative
- 3Token cost scales with context length: long histories, verbose prompts, and large retrieved contexts all inflate cost, so context management is a cost lever
- 4Model choice is a cost dial: routing simple turns to a cheap fast model and only escalating to an expensive model when needed can cut CPC dramatically
- 5Tool-call and retrieval costs are often underestimated next to token cost; each external call adds latency and money
- 6CPC should be segmented: a high-CPC conversation that converts a high-margin order is fine; a high-CPC conversation that browses and bounces is a leak
- 7At scale, small per-conversation savings compound into large P&L impact, making CPC optimization a legitimate roadmap priority, not premature optimization
Why It Matters for PMs
Unit economics are real: Unlike web pages, agent conversations cost money each time. PMs must manage CPC or a popular feature can become a margin sink at scale.
Net-value framing: CPC only matters against value per conversation. The discipline is optimizing net contribution, not minimizing cost or maximizing quality in isolation.
Compounding at scale: Small CPC savings multiply across millions of conversations. Cost optimization (context management, model routing) is a legitimate priority, not premature.
Test Your Understanding
Question 1 (Multiple Choice)
Why is cost-per-conversation a real metric for agents but not classic web pages?
Question 2 (Multiple Choice)
How does model routing reduce cost per conversation?
Question 3 (Multiple Choice)
How should CPC be evaluated?
Question 4 (Short Answer)
How would you decide whether a feature that lifts conversion but raises CPC is worth shipping?
Think about your answer before revealing the explanation.
Question 5 (Short Answer)
What levers would you pull to reduce cost per conversation without hurting quality?
Think about your answer before revealing the explanation.