Lead Data Engineer (Contract)
Dublin - Hybrid
12 Month Contract (Potential Extension)
I'm working with a leading organisation seeking a Lead Data Engineer (Contract) to help design and evolve a cutting-edge Databricks + AWS Lakehouse. This role is pivotal in enabling teams to derive insight into complex behavioural patterns and act decisively.
Key Responsibilities:
* Lead the end-to-end design, build, and optimisation of scalable Spark / PySpark pipelines (batch + streaming) on Databricks
* Define and enforce lakehouse / Medallion architecture standards (bronze, silver, gold), schema governance, and quality SLAs
* Architect secure, compliant AWS data infrastructure (S3, Glue, IAM, KMS, Lake Formation)
* Implement orchestration with Airflow, Databricks Workflows, and Step Functions
* Drive CI/CD, infrastructure-as-code, and data lineage / governance (Unity Catalog, OpenLineage)
* Mentor engineers, lead reviews, and ensure high performance and cost efficiency
About You:
* Expert in Databricks, Snowflake, Python, and AWS
* Proven experience building and optimising large-scale data solutions
* Strong leadership, stakeholder collaboration, and delivery focus
* Experience in secure, governed, cloud-first environments
Contract: 12 months | Hybrid | Start: ASAP