Senior Data Engineer - Dublin
Atrium EMEA is looking for an accomplished Senior Data Engineer to support a client onsite in Dublin. You will build data pipelines with Databricks and DBT, design data models, and develop PostgreSQL stored procedures for data reconciliation, ensuring data accuracy, efficiency, and integrity across systems.
* Design and build data pipelines (Databricks/Spark/DBT)
* Orchestrate in Azure Data Factory the data tasks to run on Azure Functions for the ingestion,
* Design and optimize the relational data model
* Build data reconciliation stored procedure in Postgres DB
* Create Docker images for various applications and deploy them on Azure
* Design and build best in class processes to clean and standardize data.
* Design and build data tests using DBT test
* Troubleshoot production issues in our Azure Environment
* Tuning and optimizing data processes
Essential:
* Hands-on experience of DBT SQL models development
* Developing processes in Spark/Databricks
* Writing complex SQL queries/PostgreSQL or other RDBMS
* Building ETL/data pipelines. Azure and Amazon AWS
* Related/complementary open-source software platforms and languages (eg Scala, Python, Java, Linux)
* Azure Data Factory/Airflow
* Terraform/GitHub Actions
* Snowflake or other large OLAP databases
* Previous experience with Relational Databases (RDBMS) & Non- Relational Database
* Analytical and problem-solving experience applied to a Big Data environment and Distributed Processing. Experience working in projects with agile/scrum methodologies
* Exposure to DevOps methodology
* Data warehousing principles (eg Kimball, Inmon, Data Vault), architecture and its implementation in large environments
Click Apply now to be considered for the Senior Data Engineer - Dublin role