Overview
Azure Data Engineer with strong experience in Dynamics 365 to support the delivery of our Dynamics 365 implementation. The ideal candidate will have extensive experience in Azure and Azure Data Factory and a strong background in integrating systems using various technologies and patterns. This role involves creating API interfaces using Azure Integration Services and integrating with Dynamics 365, leveraging Dataflows, ODATA, and other patterns.
Key Responsibilities
* Design and implement scalable ETL/ELT pipelines using Azure Data Factory (ADF), including mapping data flows and parameterized pipelines.
* Integrate with Microsoft Dataverse and Dynamics 365 using ADF’s native connectors, OData endpoints, and REST APIs.
* Collaborate with cross-functional teams to understand integration requirements and deliver robust solutions.
* Develop and optimise dataflows for transformation logic, including column reduction, lookups, and upsert strategies.
* Implement delta load strategies and batch endpoints to manage large-scale data synchronisation efficiently.
* Ensure data consistency and integrity across systems by leveraging unique identifiers and business keys.
* Collaborate with business analysts and architects to translate business requirements into technical solutions.
* Monitor, troubleshoot, and optimise pipeline performance using ADF’s built-in diagnostics and logging tools.
* Contribute to data governance, security, and compliance by implementing best practices in access control and data handling.
* Collaborate on the development of new standards and practices to improve the quality, capability, and velocity of the team.
* Mentoring and coaching of junior team members new to Azure to ensure that the team builds up a strong capability.
Requirements
* ADF Expertise. Proficient in building pipelines, triggers, and mapping dataflows. Experience with parameterisation, error handling, and pipeline orchestration.
* OData & API Integration. Strong understanding of the OData protocol. Experience integrating with REST APIs.
* Dataverse & Dynamics 365. Hands-on experience with Dataverse schema, connector configuration, and data model mapping. Familiarity with D365 entity relationships.
* Data Modelling. Understanding of conceptual, logical, and physical data models.
* SQL & Scripting. Strong SQL skills for data extraction and transformation.
* Testing & Automation. Experience with end-to-end testing of data pipelines and automation of data validation processes.
* Documentation & Collaboration. Ability to document data flows, transformation logic, and integration patterns. Strong communication skills for cross-functional collaboration.
* Azure Developer certification is highly desirable.
* Excellent problem-solving skills and the ability to work collaboratively in a team environment.
* Dynamics 365 experience is a must.
#J-18808-Ljbffr