Hybrid Search: Bedrock & OpenSearch Power Multimodal AI

Amazon unveils a powerful hybrid search solution combining keyword and semantic search for text and images, leveraging Amazon Bedrock and Amazon OpenSearch Service. This innovative approach addresses the growing need for intuitive and efficient product discovery in e-commerce, exceeding customer expectations for personalized experiences. The solution utilizes multimodal embedding models like Amazon Titan Multimodal Embeddings G1, enabling the processing of both text and image queries, translating them into vector embeddings to capture semantic meaning. This results in improved search resilience to phrasing variations and the ability to handle multimodal inputs, enhancing the overall user experience and impacting key business metrics. Keyword search provides the fundamental functionality of precise matching to product data and metadata, ensuring accurate retrieval of specific items. Semantic search, however, adds contextual understanding and flexibility, making the system more adaptable to diverse user queries. The hybrid approach cleverly combines these strengths, resulting in an 8-12% improvement in result quality over keyword search alone and a 15% improvement over natural language search. The solution’s architecture integrates Amazon Bedrock for embedding generation and Amazon OpenSearch Service as the vector database, enabling single-digit millisecond latencies for searches across billions of vectors, ideal for real-time AI applications. The process involves data ingestion (generating embeddings for text, images, and metadata, storing them in OpenSearch), and a query workflow (using an OpenSearch pipeline to convert queries to embeddings and combine keyword and semantic search scores). While the solution offers significant advantages, the complexity of combining keyword and semantic search scores from different scales needs careful management. The provided walkthrough details the steps required for implementation, including IAM role creation, connector setup, index creation, data ingestion, and query function creation. The solution is demonstrated using the Amazon Berkeley Objects Dataset, focusing on shoe listings. Test results show hybrid search outperforming keyword and semantic search alone by providing more relevant results. Overall, this hybrid search solution offers a compelling approach to enhance e-commerce search functionality, providing a more personalized and efficient experience for customers.

Amazon’s ai automation bedrock provides the foundational infrastructure needed to seamlessly integrate multimodal search capabilities across diverse data types.

While chatgpt automation bedrock provides conversational AI capabilities, combining it with OpenSearch enables more sophisticated multimodal search experiences across diverse data types.

(Source: https://aws.amazon.com/blogs/machine-learning/combine-keyword-and-semantic-search-for-text-and-images-using-amazon-bedrock-and-amazon-opensearch-service/)