Amazon SageMaker’s New Generative AI Tools

Amazon SageMaker introduces new Generative AI tools, including Text Ranking and Question and Answer UI templates, to streamline Large Language Model (LLM) training and evaluation. The Text Ranking template allows human annotators to rank multiple LLM responses based on criteria like relevance, clarity, and accuracy. This ranked feedback is crucial for refining models using Reinforcement Learning from Human Feedback (RLHF), aligning outputs with human preferences. The template supports customizable ranking dimensions and allows for tie rankings. The input manifest is a JSON file specifying prompts and responses, processed via Manual Data Setup in the SageMaker AI console. Output is saved to S3, including ranked responses for each dimension and annotator metadata. The Question and Answer template facilitates the creation of high-quality Q&A pairs from provided text passages, serving as demonstration data for Supervised Fine-Tuning (SFT). Annotators generate questions and answers, with a color-coded feature linking questions to relevant text sections. Input can be JSON, or converted from .txt or .csv via Automated Data Setup. Configuration options include specifying minimum/maximum Q&A pairs and word counts, and optional/custom tags. Both templates are accessible via the SageMaker AI console and the Create Labeling Job API, allowing for programmatic setup and integration. These tools are aimed at developers and researchers working with LLMs, improving model training efficiency and aligning outputs with human expectations. While the article doesn’t mention specific drawbacks, potential limitations could include the cost of human annotation and the scalability of the annotation process for very large datasets. The tools offer a significant improvement over manual data creation for LLM training and evaluation, providing a more structured and efficient approach.

(Source: https://aws.amazon.com/blogs/machine-learning/power-your-llm-training-and-evaluation-with-the-new-sagemaker-ai-generative-ai-tools/)