Overview
Direct message the job poster from Harvey Nash
Dublin - Hybrid • 12 Month Contract (Potential Extension)
I’m working with a leading organisation seeking a Lead Data Engineer (Contract) to help design and evolve a cutting-edge Databricks + AWS Lakehouse. This role is pivotal in enabling teams to derive insight into complex behavioural patterns and act decisively.
Responsibilities
* Lead the end-to-end design, build, and optimisation of scalable Spark / PySpark pipelines (batch + streaming) on Databricks
* Define and enforce lakehouse / Medallion architecture standards (bronze, silver, gold), schema governance, and quality SLAs
* Architect secure, compliant AWS data infrastructure (S3, Glue, IAM, KMS, Lake Formation)
* Implement orchestration with Airflow, Databricks Workflows, and Step Functions
* Drive CI/CD, infrastructure-as-code, and data lineage / governance (Unity Catalog, OpenLineage)
* Mentor engineers, lead reviews, and ensure high performance and cost efficiency
About You
* Expert in Databricks, Snowflake, Python, and AWS
* Proven experience building and optimising large-scale data solutions
* Strong leadership, stakeholder collaboration, and delivery focus
* Experience in secure, governed, cloud-first environments
Contract: 12 months | Hybrid | Start: ASAP
Job details
* Seniority level: Mid-Senior level
* Employment type: Full-time
* Job function: Information Technology and Strategy/Planning
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr