Job Title: Senior Data Engineer Work Arrangement: Hybrid (1/2 days Onsite) Location: Leopardstown, Dublin 12 month Rolling Contract **Only applicants with 6+ years experience will be considered ** Job Description Join our growing SME Data Team as a Data Engineer II, where you will work alongside a talented team to build scalable data pipelines and contribute to platform modernisation efforts.
Your role will involve supporting business-critical data solutions across on-prem Hadoop and Databricks Cloud environments.
This is a hands-on engineering role that offers opportunities for collaboration, learning, and growth in a technically rich environment.
Responsibilities Build, maintain, and optimise batch and streaming data pipelines using Spark (PySpark/Scala).
Contribute to the migration and modernisation of on-prem Hadoop workloads to Databricks Cloud.
Develop and support ETL/ELT processes leveraging Hive, Impala, HDFS, and Airflow.
Implement data ingestion, transformation, and quality checks using tools like Delta Lake, SQL Warehouses, and DLT.
Collaborate with Senior engineers and leads to design data solutions that are scalable and maintainable.
Support troubleshooting, performance tuning, and documentation of data workflows and systems.
Essential Skills Strong coding skills in Python or Scala with a solid understanding of distributed data processing.
Hands-on experience with data engineering in Spark and Hadoop ecosystems (HDFS, Hive, Impala).
Working knowledge of Airflow or similar orchestration tools.
Exposure to cloud platforms, preferably Databricks on AWS, including S3, Delta Tables, and DLT.
Familiarity with SQL performance tuning and large-scale data processing techniques.
Additional Skills