Social network you want to login/join with:col-narrow-leftLocation:Dublin North, IrelandJob Category:Information TechnologyEU work permit required:Yescol-narrow-rightJob Reference:JN -062025-1983934_1750695838Job Views:10Posted:23.06.2025Expiry Date:07.08.2025col-wideJob Description:Position: Data Engineer (Azure & Databricks)Location: Dublin North, Hybrid (3 days on site, 2 from home)Contract Type: 12‑month Fixed‑Term ContractAbout the CompanyA leading provider of innovative data solutions is seeking a highly skilled and motivated Data Engineer with expertise in Microsoft Azure and Databricks. This role is an excellent opportunity for a talented professional to join a growing data analytics team and contribute to the design and delivery of robust, secure and efficient data pipelines.The RoleAs a Data Engineer, you will design and implement robust, secure and efficient data pipelines using Azure and Databricks. You will play a pivotal role in ensuring that data platforms support both real‑time and batch processing, advanced analytics and machine learning. You will work closely with cross‑functional teams - including data scientists, analysts and other engineers - to understand requirements and deliver end‑to‑end data solutions that drive business value.Key Responsibilities*Design, develop and implement scalable and resilient data pipelines for both real‑time and batch workloads within Azure and Databricks.*Implement ELT processes to integrate data from various sources into the central data platform.*Develop and maintain data quality checks, monitoring and alerting to ensure pipeline health.*Optimise data workloads within the data platform, focusing on performance, cost efficiency, resilience and security.*Collaborate with stakeholders to understand data needs and deliver optimal solutions from ingestion to visualisation.*Ensure data solutions adhere to data governance, privacy and security best practices and regulations.*Utilise Infrastructure‑as‑Code (IaC) tools such as Terraform for cloud infrastructure provisioning.*Maintain data integrity, security, governance and compliance across all data solutions.*Mentor junior data engineers and champion best practices and technical excellence within the team.*Troubleshoot and resolve data-related issues to maintain data accuracy and performant pipelines.*Stay current with the latest trends and technologies in cloud computing and data engineering, especially within Azure and Databricks.Required Qualifications*5+ years of experience in data engineering, including 3+ years working within Microsoft Azure.*Strong experience with Azure services (Data Factory, SQL Database, Storage, Key Vault, Function and Logic Apps, Cost Analysis).*Hands‑on experience with Databricks, including data engineering, processing and analytics development.*Strong understanding of performance optimisation, data governance frameworks (e.g., Unity Catalog) and best practices.*Strong knowledge of ETL/ELT processes, data modelling, data warehousing concepts and Medallion Architecture.*Experience with real‑time data processing frameworks (e.g., Apache Kafka).*Proficient in Python programming and PySpark, SQL and working with large‑scale datasets.*Knowledge of CI/CD and DevOps practices and tools (e.g., Git, Azure DevOps).*Experience working with Agile delivery methods.*Understanding of Infrastructure‑as‑Code (IaC) using Terraform (desirable).*Knowledge of machine learning integration and MLOps (desirable).Soft Skills*Excellent problem‑solving and analytical capabilities.*Strong communication and collaboration abilities across both technical and business stakeholders.*Skilled at managing multiple priorities in a fast‑paced environment.*Strong mentoring and team‑building abilities.*Solid understanding of data governance, security protocols and compliance standards.*Commitment to team values and collaborative culture.
#J-18808-Ljbffr