Software Engineer - Data Analytics Specialist
We are seeking a highly skilled Software Engineer to join our Data Analytics team. This role will involve working with complex data sources, designing and developing real-time data pipelines, and collaborating with stakeholders to provide insights that drive business decisions.
* Work in a dedicated data reporting and analytics team building a green-field platform to produce data-driven insights for our clients.
* Analyze, interpret, and orchestrate complex data across disparate sources comprising unstructured, semi-structured, and structured datasets in streaming and batch modes.
* Support the business by designing and developing real-time data pipelines using the latest cloud technologies.
* Work with data consumers to provide metrics that meet their needs.
* Contribute to standards for data producers streaming data into the Data Lake.
The ideal candidate will have a strong background in software engineering, data analytics, and experience working with big data workloads. They should be proficient in programming languages such as Python, SQL, and Spark.
Key Responsibilities:
* Design and develop real-time data pipelines using Databricks and Delta Lake Azure cloud technologies.
* Collaborate with data consumers to provide metrics that meet their needs.
* Contribute to standards for data producers streaming data into the Data Lake.
* Participate in peer code reviews.
Requirements:
* Bachelor's degree in Computer Science, IT or related field OR equivalent related work experience.
* 4+ years of experience in software enterprise level Data Engineering.
* Big data workloads.
* Experience with data lakes and scale-out processing on them.
Preferred Skills:
* Hands-on experience designing and developing Spark data pipelines.
* Strong SQL skills.
* Strong Python skills.
* Solid understanding of the evolving data landscape and cloud-based big data workloads.
Bonus Skills:
* Unity Catalog.
* Delta Sharing.
* Delta Live Tables.
Why this role is exciting:
* Opportunity to work with cutting-edge data technologies and platforms.
* Collaborative and supportive work environment.
* Ongoing professional development and training opportunities.