New Relic NOVA: Boosting Productivity with AWS Generative AI

New Relic transformed its internal operations by developing NOVA (New Relic Omnipresence Virtual Assistant), an innovative generative AI tool built on Amazon Web Services (AWS). Facing challenges with engineers spending excessive time searching fragmented documentation and slow internal system queries, New Relic sought a more efficient way to access organizational knowledge. NOVA was designed to be a comprehensive productivity engine, evolving beyond a simple knowledge assistant.

New Relic NOVA leverages a robust AWS stack, including Amazon Bedrock for foundation models, Amazon Kendra for intelligent search, Amazon S3 for storage, and Amazon DynamoDB. It utilizes Amazon Nova Lite and Pro models via Bedrock to balance response quality with sub-20 second latency. The architecture features a three-layered agent system: a main agent for intent identification and workflow orchestration, data source layers for ingested knowledge, and an agents layer comprising Strands Agents (for third-party services like Salesforce and Jira via MCP) and custom action agents for New Relic-specific tasks (e.g., permission requests, rate limit management).

Key features include seamless integration with platforms like Confluence, GitHub, Salesforce, and Slack, processing over 1,000 daily queries with 80% accuracy. NOVA significantly reduces information search time by 95% and automates complex operational workflows, providing intelligent code reviews and AI governance. Its advanced Retrieval Augmented Generation (RAG) implementation incorporates hierarchical chunking, custom context enrichment via AWS Lambda, metadata integration, and tailored document processing for code, ensuring highly accurate and context-aware responses.

Security measures like PII detection and masking are integrated. An evaluation framework, using an LLM-as-a-judge approach, measures answer accuracy, context relevance, and response latency. Observability is maintained through New Relic AI monitoring and Slack feedback, analyzed using AWS Glue, Athena, and QuickSight for continuous improvement. New Relic NOVA targets internal employees, enhancing productivity and streamlining operations by providing a scalable, secure, and intelligent assistant for both knowledge retrieval and transactional tasks. Future plans include leveraging Amazon S3 Vectors and Amazon Bedrock AgentCore for enhanced scalability and cost efficiency.

(Source: https://aws.amazon.com/blogs/machine-learning/new-relic-transforms-productivity-with-generative-ai-on-aws/)