Dublin (Hybrid – 3 Days Onsite)
Financial Crime Analytics | Advanced Data Engineering | Global Brand
Are you a seasoned Lead Data Engineer ready to build high‑impact data platforms that help financial institutions fight fraud and money laundering?
This is an exceptional opportunity to join one of the world’s most recognised technology brands, designing and evolving next‑generation Databricks + AWS lakehouse platforms used in real‑world financial crime detection.
Our client is scaling a high‑performing engineering team and is seeking a technical leader who brings deep expertise in distributed data processing, cloud architectures, orchestration frameworks, and modern data governance.
Why This Role Is Exceptional
Work with one of the most respected global payment technology companies.
Drive the design of a high‑value, enterprise‑grade lakehouse powering fraud detection and AML insights.
Influence engineering standards and mentor other data engineers across a cutting‑edge data organisation.
Gain exposure to advanced analytics, behavioural modelling, and financial crime detection use cases.
Long‑term contract with strong potential for extension and future opportunities.
What You’ll Be Doing
Own end‑to‑end design and optimisation of Spark / PySpark pipelines on Databricks (batch + streaming).
Implement Medallion Architecture (Bronze / Silver / Gold), schema governance, metadata and lineage (Unity Catalog / Glue / OpenLineage).
Build secure, scalable AWS data infrastructure (S3, IAM, KMS, Glue, Lake Formation, Lambda, EKS/EC2).
Implement orchestration using Airflow, Databricks Workflows, Step Functions.
Embed data quality frameworks (expectations, anomaly detection, reconciliation, contract tests).
Optimise Delta Lake performance (partitioning, ACID transactions, caching, file layout).
Apply robust access controls, tokenisation, masking, encryption and key rotation.
Build audit‑ready lineage and metadata frameworks for regulatory transparency.
Drive CI/CD for data assets: Terraform/CloudFormation IaC, notebook testing, semantic versioning, automated deploys.
Work closely with Data Science, Security, Compliance and Product teams.
Mentor engineers on best practices in distributed systems, cost optimisation and performance tuning.
Support incident response, root‑cause analysis and preventative engineering.
Must‑Have Expertise
10+ years in data engineering
Deep hands‑on experience with Databricks, Spark / PySpark, Python, SQL
Strong AWS data stack: S3, Glue, Lake Formation, IAM, networking, encryption
Orchestration experience: Airflow, Step Functions, Databricks Workflows
CI/CD: Git, PR workflows, automated Databricks/AWS deployments, IaC
Nice‑to‑Have Skills
Delta Lake performance tuning
OpenTelemetry or observability tooling
Python packaging for shared libraries
About the Role
If you’re a Lead Data Engineer ready to design mission‑critical data solutions in a world‑class engineering environment, we want to hear from you.
#J-18808-Ljbffr