Key Concepts
- 1An LLM alone (without tools) is a prediction machine — it generates the next likely token based on patterns in training data. It doesn't have real-time information or the ability to take actions.
- 2Tools are APIs or functions the agent can call to interact with the world. Examples: search product DB, get pricing, check inventory, fetch user history, process payment.
- 3The agent loop: LLM reads context + tools list → LLM reasons and says 'I should call search_products with query=laptop' → system calls that tool → tool returns results → LLM reads results → LLM reasons next step → repeat until LLM says 'I'm done, here's my answer'
- 4Tool design is critical. If your product search API is slow or returns bad results, the agent output is garbage. 'Garbage in, garbage out.' The agent is only as good as its tools.
- 5Each tool call has a cost (latency + infrastructure cost). Minimize tool calls by designing smart tools. Example: instead of calling 'get_product(id)' 100 times, call 'search_products_by_criteria()' once.
- 6Tool integration requires guardrails: What if a tool times out? What if it returns invalid data? You need fallback behavior.
- 7Reliability comes from well-designed tools, not smarter prompts. Don't try to 'prompt engineer' away bad data. Fix the data source.
Why It Matters for PMs
Latency: Each tool call adds 50-500ms. If an agent needs 5 tool calls, response time is 250ms-2.5s. For e-commerce, this is acceptable; for real-time bidding, it's too slow.
Cost: LLM calls + tool calls both cost money. A search-recommendation loop might cost $0.01-0.05 per request. Scale to 1M requests/day and you're burning $10K-50K/day. Tool efficiency directly impacts unit economics.
Data quality: If your product DB is stale, has bad descriptions, or wrong prices, the agent will recommend stale/bad products. As a PM, you own tool quality, not just the agent's intelligence.
Test Your Understanding
Question 1 (Multiple Choice)
Why can't an LLM alone (without tools) provide real-time product recommendations?
Question 2 (Multiple Choice)
What happens if your product search tool is slow (e.g., 2 second latency)?
Question 3 (Multiple Choice)
If you have a choice between a 'smarter prompt' or 'better product data,' which should a PM prioritize?
Question 4 (Short Answer)
Design a tool stack for a grocery delivery agent. What tools would you include?
Think about your answer before revealing the explanation.
Question 5 (Short Answer)
Why is 'minimize tool calls' a good design principle?
Think about your answer before revealing the explanation.