What is Prompt Caching? An LLM API feature that caches static parts of a prompt (system instructions, long context) so repeat requests skip re-processing those tokens, cutting cost and latency. Prompt Caching explained Major LLM providers offer prompt caching, and the pricing rules differ. On Anthropic's API, reading a cached prefix costs 0.1 times the base input price (some newer models are lower), while writing to the cache costs 1.25 times the base price for a 5-minute cache and 2 times for a 1-hour cache (Anthropic prompt caching documentation). Check each provider's current rules before you plan savings. Useful for chatbots with long system prompts, RAG apps with stable retrieved context, and agents that loop over the same tool definitions. How Groovy Web uses Prompt Caching We turn on prompt caching in production agent and RAG systems where a long prompt prefix is reused. Savings depend on how much of each request repeats, so we measure them per system. Explore our AI Infrastructure Optimization service → Prompt Caching: common questions What is Prompt Caching? An LLM API feature that caches static parts of a prompt (system instructions, long context) so repeat requests skip re-processing those tokens, cutting cost and latency. What should I know about Prompt Caching? Major LLM providers offer prompt caching, and the pricing rules differ. On Anthropic's API, reading a cached prefix costs 0.1 times the base input price (some newer models are lower), while writing to the cache costs 1.25 times the base price for a 5-minute cache and 2 times for a 1-hour cache (Anthropic prompt caching documentation). How does Groovy Web use Prompt Caching? We turn on prompt caching in production agent and RAG systems where a long prompt prefix is reused. Savings depend on how much of each request repeats, so we measure them per system. Which terms are related to Prompt Caching? Prompt Caching is closely related to Context Window, Inference, Token (AI/LLM) and Retrieval-Augmented Generation (RAG). Related terms Context Window The maximum amount of text that a language model can consider when processing input, including both the user's prompt and the model's previous responses in a conversation. Inference The process of using a trained AI model to make predictions or generate outputs from new input data, as opposed to the training phase where the model learns. Token (AI/LLM) A token is the basic unit of text that AI language models process, roughly three quarters of an English word, and the basis for LLM pricing and context limits. Retrieval-Augmented Generation (RAG) Retrieval-Augmented Generation (RAG) is an AI architecture that combines a language model with a search system to answer questions using your own data. Semantic Caching A caching technique that returns a stored LLM response when a new query is semantically similar to a past one, rather than requiring an exact text match. Model Routing A technique where requests are directed to different AI models based on criteria like task type, complexity, or cost, optimizing for performance and economics. ← More in Fine-Tuning & Cost Optimization · All 107 glossary terms