Oriel — AI & technology worth noticing, every day.

Use prompt caching with open-weight models on Amazon Bedrock

Web page · https://aws.amazon.com/blogs/machine-learning/use-prompt-caching-with-open-weight-models-on-amazon-bedrock

Many generative AI applications resend the same long context on every request. Prompt caching lets open-weight models on Amazon Bedrock reuse it to reduce input cost and latency. This post shows implicit and explicit caching through the Chat Completions API and applies caching to an OpenCode coding agent session.

First seen by Oriel on .

AI summary

This AWS guide demonstrates implicit and explicit prompt caching for open-weight models on Amazon Bedrock, reducing repeated-context processing, latency, and input costs through the Chat Completions API.

  • Prompt caching reuses matching prompt-prefix state; cache reads cost less than standard input, while some models charge more for cache writes.
  • Implicit caching automatically reuses eligible stable prefixes, whereas explicit caching uses prompt_cache_breakpoint markers to control reusable content and layered prefixes.
  • The Python examples cache a roughly 4,000-token product manual and inspect cached_tokens and cache_write_tokens in each response.
  • Explicit caching avoids writing changing content after the reusable prefix, making it suitable when questions vary and cache writes carry higher charges.
  • The guide also configures OpenCode 1.18.35 to use implicit caching for stable system prompts, tool definitions, and conversation history through Amazon Bedrock’s endpoint.

#amazon bedrock #prompt caching #open-weight models #opencode

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
AWS Machine Learning Blogc3944f2d5e23bb6c54eabc4231344977ea98e1f9https://aws.amazon.com/blogs/machine-learning/use-prompt-caching-with-open-weight-models-on-amazon-bedrock/ Read discussion