Healthcare Data Platform Engineer
">
Data engineering is a critical component of healthcare data exchange, and as a Lead Data Engineer at our company, you will play a pivotal role in designing, developing, and maintaining robust data pipelines for healthcare data. Our vision is that every healthcare decision is powered by the right data, at the right time, in the right format.
">
We are seeking a talented and experienced engineer to join our team. The ideal candidate will have expertise in designing, developing, and deploying data pipelines using Databricks with Spark, Python, Scala, and SQL. Experience with large, structured/unstructured datasets using partitioned cloud storage architecture and query engines such as Spark, Delta Lake is also required.
">
The successful candidate will work closely with software engineers, data integration engineers, product managers, and stakeholders to design, implement, and manage data flows that integrate information from various sources into a common pool. They will also be responsible for optimizing query compute and data storage patterns for cost and performance.
">
This is an exciting opportunity to apply your technology and problem-solving skills toward a mission that benefits others and makes life-changing positive impacts on the people we serve every single day. As a member of our team, you will have the opportunity to work in an Agile development environment and contribute to the design and development of cutting-edge data engineering solutions.
">
Key Responsibilities:
">
">
* Design, develop, and deploy data pipelines using Databricks with Spark, Python, Scala, and SQL
">
* Work with large, structured/unstructured datasets using partitioned cloud storage architecture and query engines such as Spark, Delta Lake
">
* Collaborate with software engineers, data integration engineers, product managers, and stakeholders to design, implement, and manage data flows
">
* Optimize query compute and data storage patterns for cost and performance
">
* Contribute to the design and development of cutting-edge data engineering solutions
">
">
Requirements:
">
">
* 3+ years Data Engineering experience
">
* 2+ years hands-on experience working with large, structured/unstructured datasets using partitioned cloud storage architecture and query engines such as Spark, Delta Lake
">
* 2+ years experience designing, developing, deploying, and testing in Databricks
">
* 2+ years of hands-on experience in Python/Pyspark/SparkSQL
">
* 2+ years of experience with Scala programming language
">
* 2+ years experience on Big data pipelines/DAG tools like Airflow and dbt
">
* 2+ years of SQL experience, specifically to write complex, highly optimized queries across large volumes of data
">
* Experience in the AWS computing environment and storage services such as s3/glacier is required
">
* Experience with conceptual, logical, and/or physical database designs is required
">
* Good knowledge in Linux and shell scripting is highly desired
">
* Experience with Data Visualization tools like Looker, Tableau is desired
">
* Strong communication skills to relay complex data integration requirements to team members
">
* Experience with FHIR data is a plus, but not required; willingness to learn about FHIR data and domain knowledge is essential
">
* Experience working in an Agile development environment is preferred
">
* Familiarity with machine learning concepts and tools is a plus
">
">
Benefits:
">
">
* A competitive salary package
">
* A comprehensive health insurance plan
">
* A flexible and hybrid work environment
">
* Professional development opportunities
">
">
Equal Opportunities Employer:
">
We are committed to building a diverse team of individuals who share our passion for delivering high-quality data engineering solutions. We are proud to be an Equal Employment Opportunity employer and all qualified applicants will receive consideration for employment without regard to race, color, sex, sexual orientation, gender identity, religion, national origin, disability, veteran status, or other legally protected status.
">
Contact Information:
">
Please note that this job posting does not include contact information or application instructions.