LLM Semantic Caching: Skip the Call, Not Just the Tokens
Prompt caching discounts repeated input tokens but still runs the model. A semantic cache embeds each query, searches past query-response pairs, and returns a stored answer on a close-enough match, skipping the call. Here is how it works, how it differs from prompt caching, and
