Data Test Automation Engineer (Hybrid role in the city center + offering €65,000 to €70,000 + benefits)
Our client is an organization that builds large scale data platforms for major international customers. They are growing their data engineering capability and now need a Test Automation Engineer who knows Databricks and data pipelines to make sure the data they deliver is accurate, secure and reliable.
You will work with the data engineering team to test Databricks notebooks, Delta Lake tables, streaming jobs and ETL pipelines, and to put proper automated testing around them. It is a hands on role.
What you will do
Design and build automated tests for Databricks notebooks, workflows and Delta Lake tables
Test data pipelines end-to-end, including ETL and ELT jobs
Put in place data quality checks, reconciliation and performance tests for Spark jobs
Create test dashboards, reports and documentation
Work with engineers to fix issues and improve test coverage
Must have
At least 2+ years of experience working with Databricks and Spark in a data environment
Strong Python experience, ideally PySpark and good SQL
Experience testing data pipelines or ETL processes
Good knowledge of Delta Lake or Lakehouse style architecture
Experience using Git or other version control
Experience working in agile teams
Would like
Experience with AWS data services such as S3, Glue or Lambda
Experience with data quality tools or frameworks
Experience testing Unity Catalog or access and governance rules
Desirable
Experience building test automation frameworks for data platforms
Experience in financial services or other regulated environments
If you are a data-focused test engineer who wants to work on modern Databricks and AWS platforms in a global environment, send your CV for immediate review.