Oriel — AI & technology worth noticing, every day.

Groundhog Day for AI agents: Prompt caching patterns for using OpenAI models on Amazon Bedrock

Web page · https://aws.amazon.com/blogs/machine-learning/groundhog-day-for-ai-agents-prompt-caching-patterns-for-using-openai-models-on-amazon-bedrock

Every time an agent takes a step, it re-reads its entire context before acting. Here is how prompt caching for OpenAI models on Amazon Bedrock lets your agent remember, cutting input cost and latency without resending the same tools, instructions, and history every turn.

First seen by Oriel on .

AI summary

This AWS post explains prompt caching patterns for reducing repeated input processing and costs in AI agents using OpenAI models on Amazon Bedrock.

  • Prompt caching stores the model’s reusable key-value state, allowing later requests with identical prefixes to process only newly added content.
  • Explicit breakpoints cache stable tools and instructions, while implicit caching follows growing conversations; combining both supports shared prefixes and session history.
  • A reusable prefix must contain at least 1,024 tokens, and exact serialization is required because changing tool order or text can invalidate reuse.
  • In a six-turn GPT-6 Astra workload, implicit caching reduced input cost by about 62 percent and explicit rolling breakpoints by about 61 percent.
  • When tools change, preserve the initial catalog and append new definitions through additional_tools so earlier cached prefixes remain reusable.

#prompt caching #amazon bedrock #ai agents #openai models

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
AWS Machine Learning Blogde80c503415138bb46de169401d016e12857bb15https://aws.amazon.com/blogs/machine-learning/groundhog-day-for-ai-agents-prompt-caching-patterns-for-using-openai-models-on-amazon-bedrock/ Read discussion