Prompt caching

Managed prompts are read frequently and changed comparatively rarely. The SDK includes an in-memory TTL and least-recently-used cache so normal prompt-based requests do not require a prompt API call every time.

Default behavior:

• caching enabled
• 60-second TTL
• maximum 100 prompt variants per client
• concurrent identical cache misses share one network request
• cache is local to the current process
• prompt mutations clear the local prompt cache

Configure the cache

const nirmos = new Nirmos({
  apiKey: process.env.NIRMOS_API_KEY!,
  promptCache: {
    enabled: true,
    ttlMs: 120_000,
    maxEntries: 250,
  },
});

Choose a TTL that balances prompt rollout speed and API traffic. For horizontally scaled services, every process has its own cache.

Disable caching:

const nirmos = new Nirmos({
  apiKey: process.env.NIRMOS_API_KEY!,
  promptCache: false,
});

Per-request cache behavior

Normal cached retrieval:

const prompt = await nirmos.prompts.retrieve("support-reply");

Force a network refresh and replace the cached value:

const prompt = await nirmos.prompts.retrieve("support-reply", {
  cache: "reload",
});

Bypass reads and do not store the result:

const prompt = await nirmos.prompts.retrieve("support-reply", {
  cache: "no-store",
});

The same option works in direct chat prompt references:

await nirmos.gateway.chat.create({
  model: "openai/gpt-4.1-mini",
  prompt: {
    id: "support-reply",
    cache: "reload",
    variables: { customerName: "Asha" },
  },
});

Manual invalidation

nirmos.prompts.clearCache();

Or invalidate one identifier:

nirmos.prompts.clearCache("support-reply");

Because one logical prompt can be cached under both ID and slug, application code should normally clear the full cache after receiving an external prompt-change event. SDK-managed mutations already clear it.

Deployment considerations

The built-in cache is intentionally process-local and dependency-free. It avoids most repeated reads for server processes, workers, and serverless instances with warm reuse.

It does not attempt distributed invalidation. Future SDK versions may add cache adapters or revalidation strategies without changing prompts.retrieve or prompt references.