Contract Role – Senior Data Engineer – Dublin/Hybrid/Remote
We are seeking a skilled
Senior Data Engineer
to build data pipeline with Databricks and DBT, design data models, and develop PostgreSQL stored procedures for data reconciliation, ensuring data accuracy, efficiency, and integrity across systems.
Role Overview:
* Job Title:
Senior Data Engineer
* Location:
Dublin/Hybrid/Remote
* Contract Type:
Contract
* Sector:
Healthcare
.
Primary Responsibilities:
* Design and build data pipelines (Databricks/Spark/DBT)
* Orchestrate in Azure Data Factory the data tasks to run on Azure Functions for the ingestion,
* Design and optimize the relational data model
* Build data reconciliation stored procedure in Postgres DB
* Create Docker images for various applications and deploy them on Azure
* Design and build best in class processes to clean and standardize data.
* Design and build data tests using DBT test
* Troubleshoot production issues in our Azure Environment
* Tuning and optimizing data processes
Required Qualifications:
Hands-on experience of the following:
* DBT SQL models development
* Developing processes in Spark/Databricks
* Writing complex SQL queries
* PostgreSQL or other RDBMS
* Building ETL/data pipelines
* Related/complementary open-source software platforms and languages (e.g. Scala, Python, Java, Linux) (Python preferred)
Preferred Qualifications:
* Azure/Amazon AWS
* Azure Data Factory/Airflow
* Terraform
* GitHub Actions
* Snowflake or other large OLAP databases
* Previous experience with Relational Databases (RDBMS) & Non- Relational Database
* Analytical and problem-solving experience applied to a Big Data environment and Distributed Processing.
* Experience working in projects with agile/scrum methodologies
* Exposure to DevOps methodology
* Data warehousing principles (eg Kimball, Inmon, Data Vault), architecture and its implementation in large environments
Please feel free to contact myself – Daisy Nguyen ) at Gibbs Consulting/Atrium UK for a confidential chat to know more details about the role.
Please also note: Due to the volume of applications received for positions, it will not be possible to respond to all applications and only applicants who are considered suitable for interview will be contacted.