Social network you want to login/join with:
col-narrow-left
Location:
Dublin North, Ireland
Job Category:
Information Technology
EU work permit required:
Yes
col-narrow-right
Job Reference:
JN -062025-1983934_1750695838
Job Views:
10
Posted:
23.06.2025
Expiry Date:
07.08.2025
col-wide
Job Description:
Position: Data Engineer (Azure & Databricks)
Location: Dublin North, Hybrid (3 days on site, 2 from home)
Contract Type: 12‑month Fixed‑Term Contract
About the Company
A leading provider of innovative data solutions is seeking a highly skilled and motivated Data Engineer with expertise in Microsoft Azure and Databricks. This role is an excellent opportunity for a talented professional to join a growing data analytics team and contribute to the design and delivery of robust, secure and efficient data pipelines.
The Role
As a Data Engineer, you will design and implement robust, secure and efficient data pipelines using Azure and Databricks. You will play a pivotal role in ensuring that data platforms support both real‑time and batch processing, advanced analytics and machine learning. You will work closely with cross‑functional teams - including data scientists, analysts and other engineers - to understand requirements and deliver end‑to‑end data solutions that drive business value.
Key Responsibilities
*
Design, develop and implement scalable and resilient data pipelines for both real‑time and batch workloads within Azure and Databricks.
*
Implement ELT processes to integrate data from various sources into the central data platform.
*
Develop and maintain data quality checks, monitoring and alerting to ensure pipeline health.
*
Optimise data workloads within the data platform, focusing on performance, cost efficiency, resilience and security.
*
Collaborate with stakeholders to understand data needs and deliver optimal solutions from ingestion to visualisation.
*
Ensure data solutions adhere to data governance, privacy and security best practices and regulations.
*
Utilise Infrastructure‑as‑Code (IaC) tools such as Terraform for cloud infrastructure provisioning.
*
Maintain data integrity, security, governance and compliance across all data solutions.
*
Mentor junior data engineers and champion best practices and technical excellence within the team.
*
Troubleshoot and resolve data-related issues to maintain data accuracy and performant pipelines.
*
Stay current with the latest trends and technologies in cloud computing and data engineering, especially within Azure and Databricks.
Required Qualifications
*
5+ years of experience in data engineering, including 3+ years working within Microsoft Azure.
*
Strong experience with Azure services (Data Factory, SQL Database, Storage, Key Vault, Function and Logic Apps, Cost Analysis).
*
Hands‑on experience with Databricks, including data engineering, processing and analytics development.
*
Strong understanding of performance optimisation, data governance frameworks (e.g., Unity Catalog) and best practices.
*
Strong knowledge of ETL/ELT processes, data modelling, data warehousing concepts and Medallion Architecture.
*
Experience with real‑time data processing frameworks (e.g., Apache Kafka).
*
Proficient in Python programming and PySpark, SQL and working with large‑scale datasets.
*
Knowledge of CI/CD and DevOps practices and tools (e.g., Git, Azure DevOps).
*
Experience working with Agile delivery methods.
*
Understanding of Infrastructure‑as‑Code (IaC) using Terraform (desirable).
*
Knowledge of machine learning integration and MLOps (desirable).
Soft Skills
*
Excellent problem‑solving and analytical capabilities.
*
Strong communication and collaboration abilities across both technical and business stakeholders.
*
Skilled at managing multiple priorities in a fast‑paced environment.
*
Strong mentoring and team‑building abilities.
*
Solid understanding of data governance, security protocols and compliance standards.
*
Commitment to team values and collaborative culture.
#J-18808-Ljbffr