Senior AI Quality Engineer

About Addition Addition is R/GA's applied intelligence studio. We work with many of the world's leading brands to solve problems at the intersection of frontier AI and human-centered design. We collaborate with our clients to implement and scale AI systems that create real-world value—improving communications, customer experiences, and internal business workflows. About R/GA In our 40-year history, we’ve changed a lot—from a culture-defining film title studio (Superman, Alien, Ghostbusters); to an interactive advertising agency; to an award-winning product and marketing agency with design, technology, communications, ventures and IP practices. We like change, and like to navigate our clients through it too. Our approach to work is always human first. This unwavering focus helps us create meaningful and memorable experiences that live across digital platforms and often blur the line between the digital and real worlds. We believe disruptive thinking happens when you collide experts in design, technology and culture. About the Role As our Senior AI Quality Engineer, your job is to ensure that the solutions we ship are high quality by methodically testing them and working with engineering to implement improvements. This requires a test and verify mindset; you don’t assume things are working, you understand the shape of the system and test it from every angle to ensure quality. During planning phases, you explore system architecture, write test plans, and align with the team on what “quality” looks like. As systems are built, you red-team them, code scripts to automate and scale testing processes, implement logging, and methodically analyze the results. You think like an AI agent, stress-testing the behavior of AI systems under a wide range of conditions beyond the happy-path. This is a new role at Addition, created because the complexity of the AI systems we ship has outgrown traditional QA practices and ad-hoc evaluation. We need someone who turns quality into a new discipline for the AI systems era — evolving fuzzy notions of “quality” into measurable signals, and bringing the rigor of a structured adversarial mindset. This role sits within our engineering team, working closely with engineers, PMs, cross-functional project teams, and clients. Primary Responsibilities • Test Planning: During project planning phases, translate product and technical requirements into structured test plans — covering functional behavior, edge cases, failure modes, and AI-specific risks like hallucination, prompt injection, and output drift. • Agentic Testing: Working with coding agents like Claude Code and local or online versions of AI System builds to write test scripts, setup logging frameworks, analyze agent behavior, user behavior, to identify points of failure and opportunities for improvement. This is automated and streamlined for scale, but always done with intentionality, human supervision, and verification along the way. • AI Red Teaming: When systems are built, systematically red-team them before launch. Probe LLM behaviors, agent decision chains, brand voice outputs, and multi-model pipelines for failure modes that functional testing won't surface. Assess the agent’s overall trajectory, tool selection and reasoning steps. Design prompts that attempt to manipulate behavior, discover conflicting instructions and surface failure cases before they reach production. • Launch Gating: Act as the quality gate on launches — raising blockers, documenting failure cases, and holding the line on ship criteria. You are the friction layer between "built" and "live." • Regression & Evaluation Frameworks: Build and maintain lightweight eval frameworks and test harnesses for recurring system types (e.g., brand voice agents, agentic creative tools