Role: Data Engineer / Sr Data Engineer Location : Dublin, Ireland.
(Hybrid Work Model)Contract Type: Permanent & Contract Core Technical Expertise Advanced SQL and experience with relational and No SQL databases.
Strong experience with Snowflake, Hadoop, Spark, Databricks, Kafka, and cloud data platforms.
Proficiency in Python for both data engineering and test automation.
Familiarity with orchestration tools and workflow management systems.
Data Engineering & Platform Development Design, build, and maintain scalable data pipelines using Snowflake, Hadoop, Spark, Ni Fi, and related big data technologies.
Implement data architectures and optimize workflows for massive financial datasets.
Write high-quality, maintainable code in Python and SQL following best practices.
Integrate data governance principles, metadata management, and lineage tracking into solutions.
Data Quality Assurance & Testing Develop automated testing frameworks and validation scripts for ETL processes and data transformations.
Implement data quality checks, reconciliation processes, and regression testing suites to ensure accuracy, completeness, and timeliness.
Perform unit, integration, and end-to-end testing for data pipelines and schema changes.
Use tools like dbt tests, custom Python utilities for automated validation.
Collaboration & Agile Delivery Work closely with Data Engineers, Product, and Data Science teams to embed testing into the development lifecycle.
Participate in agile ceremonies (sprint planning, backlog refinement, retrospectives) with a focus on quality and delivery.
Support production incident response with rapid data validation and root cause analysis.