Introduction
Amazon Bedrock has launched a powerful new feature called Advanced Prompt Optimization, designed to help developers and AI practitioners fine-tune their prompts for maximum performance across multiple foundation models. This tool simplifies the process of prompt engineering by providing a systematic, metric-driven approach optimizes prompts for both new model migrations and existing workflow improvements.
How the Prompt Optimizer Works
The Advanced Prompt Optimization tool takes a structured input: your prompt template, example user inputs (including variable values), ground truth answers, and an evaluation metric. It then runs an iterative feedback loop, generating optimized prompts and testing them against up to five models simultaneously. You can compare the original and optimized prompts to check for regressions on known use cases and to boost performance on underperforming tasks.
The tool supports multimodal inputs such as PNG, JPG, and PDF files, making it ideal for document and image analysis tasks. You can also guide the optimization process by providing an AWS Lambda function, an LLM-as-a-judge rubric, or a short natural language description. The result is a comprehensive output including original and final prompt templates, evaluation scores, cost estimates, and latency data.
Key Features
Multi-Model Comparison
One standout capability is the ability to optimize prompts for up to five different models in a single run. If you're migrating from one model to another, you can include your current model as a baseline and compare up to four alternatives. If you just want to improve performance on your current model, you can still select it to see before-and-after results.
Flexible Evaluation Metrics
You can define your evaluation metric through either a custom LLM-as-a-judge prompt, a Lambda function, or a natural language description. This flexibility allows you to align prompt optimization with your specific quality requirements, whether you're focusing on accuracy, relevance, or task completion.
Cost and Latency Insights
Every optimization run provides cost estimates and latency metrics, helping you make informed trade-offs between performance and operational expenses. This data is crucial for production deployments where efficiency matters.
Getting Started with Advanced Prompt Optimization
To begin, navigate to the Advanced Prompt Optimization page in the Amazon Bedrock console and select Create prompt optimization. Choose up to five inference models you want to test against. Prepare your prompt templates in JSONL format, where each JSON object represents a single optimization task. Here's an example structure:
{
"version": "bedrock-2026-05-14",
"templateId": "your-template-id",
"promptTemplate": "Your prompt with ",
"steeringCriteria": ["optional guidelines"],
"customEvaluationMetricLabel": "metric label",
"customLLMJConfig": {
"customLLMJPrompt": "Judging prompt",
"customLLMJModelId": "model-id"
},
"evaluationMetricLambdaArn": "arn:lambda:...",
"evaluationSamples": [
{
"inputVariables": [{"variableName1": "value1"}],
"referenceResp": "expected output"
}
]
}Each JSON object must be on a single line in the .jsonl file. You can include as many samples as needed.
Use Cases and Applications
Migrating to a New Model
When switching to a different foundation model, you risk losing performance on key tasks. The Advanced Prompt Optimization tool lets you test your existing prompts against the new model, automatically optimizing them to match or exceed prior performance. You also get side-by-side comparisons to verify there are no regressions.
Boosting Current Model Performance
Even without changing models, you can use this tool to refine your prompts for tasks that are not performing as expected. By providing ground truth and a clear evaluation metric, the optimizer will generate new prompt templates that yield better results.
Multimodal Document and Image Analysis
Support for PDFs, PNGs, and JPGs means you can optimize prompts for complex tasks like extracting information from scanned documents or analyzing images. The tool treats these files as inputs to your prompt templates, enabling large-scale testing of multimodal workflows.
Conclusion
Amazon Bedrock's Advanced Prompt Optimization is a game-changer for prompt engineering. It reduces the guesswork, saves time, and ensures your prompts are fully optimized for the models you use. Whether you're migrating to a new model or squeezing more performance from your current one, this tool provides the metrics and insights you need. To learn more, visit the Getting Started section above or explore the AWS documentation for complete details.