Job Specification: Senior Data Engineer (Databricks & ADF)
6 month fixed term contract - €70-80k pro-rata
Role Overview
We are seeking an experienced Senior Data Engineer to build, optimise, and operationalise data solutions on our Azure Data Lakehouse using Databricks, ADF, and Delta Lake. This is a senior individual contributor role focused on hands‑on engineering, delivering high‑quality data pipelines, and supporting the organisation’s Data, AI, and BI roadmap.
Key Responsibilities
Data Engineering & Pipelines
Develop, code, deploy, and maintain Databricks notebooks, jobs, and pipelines as a core part of the role.
Build and optimise scalable ETL/ELT workflows using Databricks, ADF, and Delta Lake.
Ensure high‑quality, reliable, and reusable datasets and data models.
Data Platform Delivery
Work hands‑on across ADLS, ADF, Databricks, and related Azure services to support end‑to‑end data processing.
Implement automated testing, version control, orchestration, and CI/CD deployment for Databricks and pipelines.
Collaborate with analytics and BI teams to deliver trusted data for insights and reporting.
AI & Data Science Support
Enable model development, feature engineering, and ML workflows running on Databricks.
Support deployment and scaling of AI/ML workloads across the platform.
Governance, Security & Quality
Apply strong data governance practices including data quality, lineage, metadata, and access control.
Ensure compliance with security and performance standards across the cloud data environment.
Collaboration & Leadership
Provide technical leadership on complex data engineering topics.
Support the evolution of the organisation’s Data, AI, and Business Intelligence roadmap.
Act as a mentor and subject matter expert for modern data engineering practices.
Required Skills
Essential: Strong hands‑on experience coding, deploying, and managing Databricks notebooks, jobs, and pipelines.
Proficiency in PySpark, SQL, Delta Lake, and modern data engineering techniques.
Practical experience with ADF, ADLS, orchestration, and CI/CD in Azure environments.
Experience supporting analytics and AI/ML workloads in production.
Knowledge of data governance, security, and performance optimisation.
Strong communication and cross‑functional collaboration skills.
#J-18808-Ljbffr