Associate Director, Clinical AI Evaluation and Responsible Deployment

Start free →

Or apply yourself on the official page →

About the role

Associate Director, Clinical AI Evaluation and Responsible Deployment  About AISI   AI Science & Innovation (AISI) sits at the centre of AstraZeneca’s R&D AI transformation within Enterprise AI Unit (EAI). Our remit is to build, buy, and deliver the AI models and agents that change pipeline outcomes across discovery, translational science, biomarkers, and clinical development, ultimately improve patients’ lives.  We are building an end-to-end Enterprise AI engine that unites data foundations, AI models, platforms, and business-facing applications to accelerate results across the value chain. Success comes from reusing what already works, sharing ideas across teams, and scaling impact rather than building in isolation.  Role overview  AstraZeneca is building an AI capability for Clinical Development that will improve how trials are designed, conducted, monitored, and analysed. We are hiring an Associate Director, Clinical AI Evaluation and Responsible Deployment to create the evidence systems that determine whether clinical AI is useful, reliable, safe, and ready to scale.  The role sits in the Applied Clinical AI team and engages directly with clinical stakeholders, Engineering, IT, and data teams to shape priorities, requirements, and delivery in collaboraiton with AstraZeneca’s existing Engineering, Product and Clinical Solutions teams.  Clinical AI rarely has simple ground truth. Expert judgments can differ, source data can be incomplete, and the consequence of an error depends on where an output appears in the workflow. You will turn these realities into rigorous evaluation environments, release criteria, monitoring strategies, and validation-ready evidence. You will work directly with clinical stakeholders to define what “good” means and directly with scientists and engineers to implement it in code.  This is a hands-on technical leadership role. You will build evaluation harnesses, analyse model and workflow behaviour, design experiments, and help teams diagnose failures—not merely review documents after development is complete. You will also connect scientific evaluation with Quality, Regulatory, and operational expectations so that evidence is useful both to builders and decision-makers.  What you’ll do  • Define and implement the evaluation strategy for clinical AI models and agents across development, release, monitoring, and change control.  • Build evaluation harnesses, curated test sets, simulation environments, automated regression suites, and analysis pipelines in Python and related technologies.  • Translate expert judgment from CRAs, medical monitors, clinical scientists, statisticians, and operations leaders into task definitions, scoring rubrics, error taxonomies, and clinically meaningful acceptance thresholds.  • Evaluate complete workflows—not only model outputs—including retrieval quality, tool selection, orchestration, source fidelity, abstention, human hand-offs, latency, and downstream operational impact.  • Design approaches for noisy, sparse, delayed, or expert-dependent ground truth, including adjudication, inter-rater agreement, challenge sets, prospective studies, and post-deployment surveillance.  • Lead failure analysis and red-teaming for hallucination, unsupported claims, automation bias, data leakage, prompt injection, subgroup performance, and unsafe workflow behaviour.  • Establish traceability from intended use and user need through requirements, test evidence, and release decisions, including how model, prompt, tool, data, and workflow changes are assessed, monitored, and revalidated throughout the product lifecycle.  • Work with Quality, Regulatory, Clinical Operations, Privacy, Security, and R&D IT to align evaluation evidence with GCP, GxP, data-integrity, validation, and in

View the official posting →