Senior Scientist, Clinical AI Research & Evaluation (Regulatory & Responsible AI)

We're building a connected, end-to-end  Enterprise AI  engine - uniting data foundations, AI technology, process reinvention, and business-facing AI to accelerate results across the whole value chain.  Success depends on being exceptional connectors : you'll actively leverage existing capabilities, celebrate and promote reuse, export breakthrough ideas across geographies and functions, and obsess over scaling impact rather than building in isolation. If you thrive in high-collaboration environments where your role is to turn complex, cross-functional problems into reusable, enterprise-wide capabilities - and where the measure of success is adoption and scale, not just innovation - you'll have the platform (and sponsorship) to make it real.  About AISI AI Science & Innovation (AISI) sits at the centre of AstraZeneca's R&D AI transformation within Enterprise AI Unit (EAI). Our remit is to build, buy, and deliver the AI models and agents that change pipeline outcomes across discovery, translational science, biomarkers, and clinical development, ultimately improving patients' lives. The opportunity We are building an end-to-end Enterprise AI engine that unites data foundations, AI models, platforms, and business-facing applications to accelerate results across the value chain. Success comes from reusing what already works, sharing ideas across teams, and scaling impact rather than building in isolation. Role overview AstraZeneca is building an AI capability for Clinical Development that will change how trials are designed, conducted, monitored, and analyzed. We are hiring a Senior Scientist, Clinical AI Research & Evaluation to do two things: advance the science of evaluating clinical AI systems as a hands-on researcher, and act as a technical AI advisor to Regulatory Affairs. The role is based in Cambridge, MA. This is primarily a researcher-first role with a secondary responsibility towards regulatory AI guidance. In your primary identity as an AI researcher, you design and run evaluations of LLM-based and agentic clinical systems such as benchmarks, evaluation harnesses, LLM-as-judge methods, failure analysis, and you do it in code, alongside the Clinical AI Applied Science & Translation team. Your second hat is technical advisor to Regulatory: you translate evaluation evidence, reliability, responsible-AI practice, and residual risk into language and evidence standards that Regulatory colleagues can act on, and you help bring validation thinking in early, before the design is locked. You will work day to day with the team building agentic workflows for clinical-trial planning, conduct, monitoring, and review, and you will spend a meaningful share of your time with Regulatory, Quality, and compliance colleagues. GxP expertise is not a must have for this role. We're looking for a strong researcher who wants to learn the evidentiary standards of regulated medicine and apply their evaluation skills there. What you'll do AI research & evaluation (primary, the majority of the role) Design and run evaluations of LLM-based and agentic clinical systems: benchmark construction, evaluation harnesses, LLM-as-judge methods with judge calibration, failure-mode taxonomies, red-teaming, and error analysis. Build reusable evaluation assets (test sets, scoring rubrics, annotation protocols, regression suites, analysis pipelines) that the portfolio can reuse across studies and therapeutic areas. Study how AI output shapes clinical judgment: expert-agreement studies, user studies, reliability under noisy, incomplete, or context-dependent ground truth. Investigate failures using traces, logs, qualitative review, and quantitative analysis; translate findings into model, prompt, tool, data, or workflow improvements. Publish and communicate finding