Reduce inference costs with exact caching

Exact caching reuses a response only when the normalized request payload matches. It is predictable, low risk, and effective for repeated extraction, classification, and system-generated prompts.

Choose cacheable traffic

Start with deterministic requests that do not contain user-specific authorization context. Include model, messages, tool definitions, and relevant generation parameters in the cache key.

Enable exact caching

const result = await ai.chat.completions.create({
  model: "openai/gpt-4.1-mini",
  temperature: 0,
  cache: {
    mode: "exact",
    ttl: "30m",
  },
  messages,
});

Read cache metadata

Nirmos returns cache status in response headers and request traces. Track hit rate alongside latency and cost avoided; a high hit rate is not useful if stale responses violate product expectations.