Optimization

The system prompt — the instructions given to your agent — is often long and repeated identically on every request. TokenHawk compresses it semantically once, caches the result, and reuses it for all subsequent traffic carrying the same prompt.

A similarity check guarantees the compressed version says the same thing as the original. When in doubt, the original is used instead, with no error or interruption for your agent.

When a new question resembles an already-asked one closely enough, TokenHawk can return the already-computed answer directly, without consulting your LLM provider again.

The similarity threshold is configurable per API key — strict, permissive or aggressive — to trade off savings against the risk of an imprecise answer, depending on your traffic.

A conversational agent accumulates a history that grows with every turn. Past a certain volume, TokenHawk summarizes the oldest exchanges while keeping the last few messages intact, shrinking the request without losing the thread of the conversation.

Tool definitions (function calling) are deduplicated and overly long descriptions are trimmed. Conciseness instructions can also be injected to shorten generated responses, when that option is enabled per key.

• —A typical 20–60% reduction in tokens sent, measured and shown in your dashboard — not an estimate.
• —None of these steps change your agent's functional behavior: automatic validation falls back to the original whenever in doubt.
• —Each tenant has a monthly optimization quota: past it, your traffic keeps being served normally — only optimization pauses.

These settings are configured per API key, from your dashboard.