Cpl are supporting a dynamic client in their search for a Senior Data Engineer. In the role you will play a key role in developing and scaling our strategic data platform on Azure Cloud. You'll focus on building a robust reference data system for securities and pricing, with opportunities to expand into other domains. Bring your passion for innovation and help shape our evolving data landscape.
Key Responsibilities:
1. Collaborate with cross-functional teams to enhance our Azure-based data platform.
2. Lead the creation of a reference data system for securities and pricing, with future expansion into other areas.
3. Design, implement, and maintain scalable data pipelines and storage solutions.
4. Utilize Azure tools like Data Factory, Lake Gen 2, and Databricks for efficient data processing.
5. Develop and integrate APIs using frameworks such as FastAPI.
6. Ensure security, scalability, and performance best practices in cloud solutions.
7. Align data solutions with business goals and evolving requirements.
8. Continuously optimize data architecture and engineering processes.
Must-Have Qualifications:
1. 10+ years of experience with deep expertise in Python and PySpark.
2. Proven experience as a Data Engineer with extensive knowledge of Azure Cloud services is essential.
3. Proficiency with Azure Data Factory, Lake Gen 2, Databricks, Functions, and API Gateway.
4. Hands-on experience with SQL across relational and NoSQL databases.
5. Proven skills in developing APIs with FastAPI or similar frameworks.
6. Expertise in ETL/ELT processes and DevOps tools (e.g., Git, Jenkins).
7. Ability to thrive in a fast-paced, collaborative environment.
Skills:
Azure, Data Factory, Data Bricks, Data Engineer, Azure Data Engineer, Business Intelligence, Big Data.
#J-18808-Ljbffr