Conquer Cold Starts: vLLM & AWS Trainium for Recommendations
This article details a novel approach to tackling the cold-start problem in recommendation systems using vLLM on AWS Trainium. The core challenge addressed is the lack of user data at a system’s launch, leading to generic recommendations and potentially lost users. The solution leverages large language models (LLMs), specifically Meta’s Llama (8B and 70B parameter versions), for zero-shot reasoning to generate detailed user interest profiles from minimal initial data, such as a single user review. This is achieved by crafting structured prompts that guide the LLM to output JSON lists of related topics. The expanded user interests, along with book summaries from the Amazon Book Reviews dataset, are then encoded into embeddings using various sizes of the Google T5 encoder (base, large, XL). FAISS is utilized for efficient nearest-neighbor search to retrieve relevant book recommendations. The process is optimized using AWS Trainium chips and the Neuron SDK, which enables efficient model sharding and parallel inference via NeuronX Distributed (NxD). Experiments benchmarked different LLM-encoder combinations, measuring average FAISS distances to assess the quality of embeddings. The results showed that while larger LLMs (70B) offered minor improvements, the 8B Llama model paired with a T5-large encoder provided the best balance between performance and cost-efficiency. Tensor parallelism was also optimized, revealing that a tensor_parallel_size of 16 offered the best cost-performance trade-off on a trn1.32xlarge instance. The overall solution offers a flexible and scalable framework for improving cold-start recommendations, enabling rapid iteration on recommendation metrics and facilitating A/B testing in real-world settings. The solution is particularly relevant for businesses launching new recommendation systems or introducing new content where user data is scarce.
Modern businesses increasingly rely on ai automation recommendations to deliver personalized user experiences while minimizing infrastructure costs and latency issues.
