Amazon Bedrock Custom Model Import: Log Probability Support

Amazon Bedrock enhances its Custom Model Import feature with log probability support, providing crucial insights into model confidence at the token level. This allows developers to better understand their models' predictions, particularly for custom models like Llama, Mistral, and Qwen. The log probabilities, expressed as negative numbers closer to zero for higher confidence, enable several key capabilities. Developers can gauge confidence across a response, score and compare outputs, detect potential hallucinations (inaccuracies), and optimize RAG (Retrieval Augmented Generation) systems by discarding low-scoring candidates early, reducing costs. The feature is accessed by setting “return_logprobs”: true in the API call. The response includes log probabilities for both prompt and generated tokens. This detailed confidence information empowers developers to build more trustworthy AI applications by identifying uncertain predictions, improving prompt engineering, and evaluating fine-tuned model performance. The prerequisites include an active AWS account with access to Amazon Bedrock, a custom model imported after July 31, 2025, and appropriate IAM permissions. Use cases include ranking multiple model outputs, detecting hallucinations, monitoring prompt quality, reducing RAG costs through early pruning, and fine-tuning evaluation. Amazon Bedrock provides tools and resources to help developers utilize log probabilities effectively, building more reliable and intelligent AI applications.

(Source: https://aws.amazon.com/blogs/machine-learning/unlock-model-insights-with-log-probability-support-for-amazon-bedrock-custom-model-import/)