Datavant is a data platform company and the world's leader in health data exchange. Our vision is that every healthcare decision is powered by the right data, at the right time, in the right format.Our platform is powered by the largest, most diverse health data network in the U.S., enabling data to be secure, accessible and usable to inform better health decisions. Datavant is trusted by the world's leading life sciences companies, government agencies, and those who deliver and pay for care.By joining Datavant today, you're stepping onto a high-performing, values-driven team. Together, we're rising to the challenge of tackling some of healthcare's most complex problems with technology-forward solutions. Datavanters bring a diversity of professional, educational and life experiences to realize our bold vision for healthcare.What We're Looking ForWe are seeking a talented and experienced Lead Data Engineer specializing in healthcare data pipelines to join our team. The ideal candidate will have expertise in designing, developing, and maintaining robust data pipelines for healthcare data including prescriptions, medical claims, MAO4, patient demographics, and FHIR data. This role will primarily focus on utilizing Databricks with Spark, Python, Scala, and SQL for building and optimizing data pipelines, as well as performing data engineering for machine learning and AI output. While healthcare data knowledge is preferred, a strong foundation in data engineering and a willingness to learn about healthcare data is essential. Close collaboration with software engineers, data integration engineers, product managers, and stakeholders as well as mentorship for other data engineers will be key to success in this role.What You Need To Succeed::5+ years Data Engineering experience3+ years hands-on experience working with large, structured/unstructured datasets using partitioned cloud storage architecture using query engines such as Spark, Delta Lake.3+ years experience designing, developing, deploying and testing in Databricks.3+ years of hands-on experience in Python/Pyspark/SparkSQL.3+ years of experience with Scala programming language2+ years experience on Big data pipelines/DAG tools like Airflow and dbt2+ years of SQL experience, specifically to write complex, highly optimized queries across large volumes of data.Experience in the AWS computing environment and storage services such as s3/glacier is required.Experience with conceptual, logical and/or physical database designs is required.Good knowledge in Linux and shell scripting is highly desired.Experience with Data Visualization tools like Looker, Tableau is desired.Strong communication skills to relay complex data integration requirements to team members.Bonus Points ForExperience with healthcare data is a plus, but not required; willingness to learn about healthcare data and domain knowledge is essential.Experience with FHIR dataExperience working in an Agile development environment is preferred.Familiarity with machine learning concepts and tools is a plus.The Impact You Will MakeDeliver a world-class data platform from the ground upDesigning and developing data engineering solutions.Support analytics, data science and data platform teams and understand their unique needs and challenges.Design, implement and manage data flows that integrate information from various sources into a common pool implementing data pipelines based on ELT and ETL models.Develop datasets and models to meet the requirements of analysts, data scientists and key stakeholders; optimizing query compute and data storage patterns for cost and performanceWork closely with the other engineering teams to troubleshoot issues with API's and data exchanges between multiple systems.We are committed to building a diverse team of Datavanters who are all responsible for stewarding a high-performance culture in which all Datavanters belong and thrive.We are proud to be an Equal Employment Opportunity employer. Datavant is committed to working with and providing reasonable accommodations to individuals with physical and mental disabilities. If you need an accommodation while seeking employment, please request it here, by selecting the 'Interview Accommodation Request' category. You will need your requisition ID when submitting your request, you can find instructions for locating it here. Requests for reasonable accommodations will be reviewed on a case-by-case basis.For more information about how we collect and use your data, please review our Privacy Policy.