Senior Data Engineer

Super Payments Super Payments is the only global fintech platform providing 0% processing fees for merchants. Our mission is to use data and AI to make payments free for businesses, so that everyone wins . At Super, we are disrupting payments, like Spotify did to music and Robinhood did to trading. We don't make money from transaction fees; instead, we monetise through value-added services such as lender commissions and FX. Our platform supports multiple payment methods — from traditional cards to direct bank transfers via open banking, as well as Apple Pay, Google Pay, and our own Buy Now Pay Later solution. Super has raised $66M from leading investors including Accel, Union Square Ventures and Local Globe (the same investors behind Spotify, Stripe, Monzo) and founded by Samir Desai CBE, co-founder of Funding Circle. Already trusted by thousands of businesses and more than 4 million customers , Super is processing at a run rate of £1.5B and growing 4x YOY Our Values • Customer obsessed: We only succeed when our customers do. • Move fast : Build, test and improve quickly. Progress matters more than perfection. • Own it : Be accountable, solve problems, and make it happen. • Be open : Act with honesty and respect. Transparency builds trust. • Win together : Collaboration beats ego every time. Senior Data Engineer Role Overview We run an event-driven data platform on AWS and Snowflake. Events flow from our product services through EventBridge, Kinesis Firehose, S3, and are auto-ingested into Snowflake via Snowpipe; batch and third-party data lands through Snowpark-deployed Python procedures. All infrastructure is defined in Terraform/Terragrunt. Python (Poetry, Pydantic, pytest) is our pipeline language; GitHub Actions runs CI/CD. Datadog gives us end-to-end pipeline observability. Key Responsibilities Ingestion & Pipeline Engineering • Event Pipeline Ownership: Build and evolve the EventBridge → Firehose → S3 → Snowpipe ingestion path, including partitioning and throughput tuning as event volumes grow across streams like fraud, payments and merchant. • Batch & Third-Party Integrations: Design and deploy Snowpark-based Python procedures for file-based and third-party data loads, scheduled and monitored as Snowflake Tasks. • Schema Evolution: Work with product engineers to manage upstream schema changes and late-arriving data without breaking downstream consumers. Infrastructure & Reliability • Infrastructure as Code: Own and extend Terraform/Terragrunt modules for ingestion infrastructure across multiple environments. • Pipeline Observability: Design Datadog monitors and alerting for Firehose, EventBridge, DLQs and Snowpipe; triage and resolve ingestion failures. • Security & Compliance: Manage PII handling (hashing, secure deletion), masking, data governance and access in Snowflake. Support audit-ready, reconciled data for Finance and Compliance. Data Pipelines & Data Quality • Dbt: Build privacy-first data pipelines using dbt and Snowflake Dynamic Tables to safely expose sensitive data for downstream analytics and reporting. • Data Quality: Build testing and monitoring for ingestion and transformation logic to catch issues before they reach the business or the Analytics Engineering team. • Stakeholder Collaboration: Partner with the wider Engineering team and Analytics Engineering team as a downstream consumer, to ensure pipelines deliver accurate, audit-ready data. Requirements We'd love it if you have • AWS & Event-Driven Architecture: Hands-on experience with streaming/event ingestion (Kinesis, EventBridge or equivalents), S3 partitioning, and queue/DLQ patterns. • Infrastructure as Code: Terraform experience required; Terragrunt a plus. Comfortable owning modules across environments, not just cons