Job Title: Data Engineering Team Lead
We are seeking a highly skilled Data Engineering Team Lead to join our team. As a Data Engineering Team Lead, you will be responsible for designing and building scalable data pipelines and architectures for both on-prem and cloud environments.
About the Role:
* Design, build, and maintain robust data pipelines and platforms for both on-prem (Hadoop ecosystem) and Databricks Cloud environments.
* Lead and mentor a team of data engineers, providing technical guidance, conducting code reviews, and fostering team growth.
* Ensure data quality, reliability, and governance across all stages of data processing.
* Work closely with data scientists, analysts, and business stakeholders to deliver impactful data solutions.
Requirements:
* Expertise in Spark (PySpark/Scala), Hadoop, Hive, Impala, and Airflow.
* Strong coding skills in Python or Scala and a solid understanding of distributed data processing.
* Hands-on experience with Databricks on cloud platforms, including S3, Delta Lake, DLT, SQL Warehouses, Unity Catalog, and Databricks Asset Bundles.
* Strong experience with HDFS storage and handling large-scale structured and semi-structured datasets.
* Proficiency in Git-based development workflows and CI/CD pipelines using Jenkins.
* Leadership experience, with a track record of mentoring, guiding, and supporting engineering teams.
About Us:
We are committed to creating a sustainable world that unlocks priceless possibilities for all. We believe in empowering people and businesses to reach their full potential by connecting and powering an inclusive, digital economy.
Corporate Security Responsibility:
Every person working for, or on behalf of, our organization is responsible for information security. All activities involving access to our assets, information, and networks come with an inherent risk to the organization, and therefore, it is expected that every person must abide by our security policies and practices, ensure the confidentiality and integrity of the information being accessed, report any suspected information security violation or breach, and complete all periodic mandatory security trainings in accordance with our guidelines.