Lead Data Engineer 12-Month Contract Dublin - Hybrid, 3 Days Onsite Daily Rate: €
*For this position, you need to be based in Ireland and hold an EU/UK passport or Stamp 4
Join a
global leader in payments innovation
that helps financial institutions combat money laundering and fraud through cutting-edge data and analytics solutions. This is a fantastic opportunity for a
Lead Data Engineer
to drive the design and evolution of a high-performance Databricks and AWS-based lakehouse platform, enabling advanced financial crime detection and insight generation.
The Role
As a
Lead Data Engineer
, you'll lead the end-to-end design, development, and optimisation of scalable Spark/PySpark data pipelines on Databricks, shaping robust, secure, and high-quality data architectures. You'll collaborate with cross-functional teams-data science, compliance, and product-to deliver actionable intelligence from complex data ecosystems.
Your Responsibilities:
* Design, build, and optimise Spark/PySpark data pipelines on
Databricks
(batch and streaming).
* Define and enforce data lakehouse architecture and governance standards.
* Architect secure, compliant
AWS
infrastructure (S3, Glue, IAM, Lake Formation, etc.).
* Drive CI/CD, testing, and automation for data workflows.
* Mentor engineers, lead design reviews, and drive continuous improvement.
About You:
* 10+ years' experience as a
Data Engineer
, with strong
Databricks, Spark, Python, and SQL
expertise.
* Deep knowledge of
AWS data services: S3, Glue, Lake Formation,
and orchestration tools
Airflow, Databricks Workflows.
* Strong grasp of data governance, lineage, and compliance (Unity Catalog, OpenLineage).
* Experience in large-scale distributed data systems within
financial services
or similar industries.
* Excellent communication and leadership skills, with a pragmatic approach to delivery and mentoring.