Lead Data Engineer Dublin (Hybrid) |€700 per day | 12-Month Contract (View to Extend)We are partnering with a global payments leader in Dublin that is building next-generation data platforms to support large-scale analytics, fraud detection, and anti–money laundering initiatives.
They are seeking a Lead Data Engineerto design, build, and evolve a high-performance Databricks + AWS lakehouse, enabling advanced analytics across complex, global systems.
This is ahands-on technical leadership role, ideal for someone who enjoys architecting scalable data platforms, optimising performance at scale, and mentoring engineers in a collaborative, forward-thinking environment.
What You'll Do Design, build, and optimise Spark / Py Spark pipelines on Databricks (batch and streaming)Define and enforce Medallion architecture standards (Bronze/Silver/Gold), schema governance, and data quality SLAs Architect secure, scalable AWS data infrastructure (S3, Glue, Lake Formation, IAM, Lambda, Step Functions)Implement orchestration using Airflow, Databricks Workflows, and AWS Step Functions Drive CI/CD and infrastructure-as-code using Terraform and/or Cloud Formation Champion data lineage, metadata, and governance using Unity Catalog, Glue, or similar tools Mentor and guide data engineers, leading design sessions and promoting best practices in performance, cost optimisation, and clean code Collaborate closely with data science, compliance, and product teams to translate analytical requirements into scalable data solutions What We're Looking For10+ years' experience in data engineering Deep expertise in Databricks, Spark, Python, and SQLStrong experience across the AWS data ecosystem (S3, Glue, Lake Formation, IAM, VPC)Proven experience with Airflow and Databricks Workflows Solid background in CI/CD pipelines, infrastructure-as-code, and data governance frameworks Experience with Apache Ni Fi, Snowflake, or Delta Lake is highly advantageous Strong communication and leadership skills with a pragmatic, delivery-focused mindset Prior experience in financial services, payments, telecoms, or insurance preferred