Location: Dublin (Hybrid)
Start: ASAP
Rate: Negotiable
My client are seeking a detail-oriented QA Engineer with strong experience in testing Big Data applications built using Apache Spark and Scala.
The ideal fit will be responsible for validating complex data pipelines, ensuring data accuracy, performance, and reliability across large-scale distributed systems.
Key Responsibilities
Design and execute test plans, test cases, and test scenarios for Spark‑based data processing applications.
Perform functional, integration, and regression testing of Spark & Scala jobs.
Validate ETL / data pipeline logic, data transformations, and aggregations.
Perform data validation testing across multiple data sources and targets (HDFS, S3, databases, files).
Write and execute SQL queries to validate data at various stages of the pipeline.
Perform end-to-end testing of data workflows orchestrated via schedulers or cloud services.
Ensure test coverage, documentation, and adherence to QA best practices.
Required Skills & Qualifications
Technical Skills
Strong experience in testing Apache Spark applications.
Hands‑on experience with Scala‑based applications (code understanding and test validation).
Solid knowledge of Big Data concepts and distributed data processing.
Strong SQL skills for data validation and analysis.
Experience in ETL testing, data reconciliation, and data quality checks.
Familiarity with data formats: Parquet, Avro, JSON, CSV, ORC.
Testing & QA
Experience in manual testing of data‑intensive applications.
Good understanding of SDLC, STLC, and defect management processes.
Experience using test management and defect tracking tools (e.g., JIRA).
Ability to analyze logs and troubleshoot data and job failures
For more information get in touch: tcopland@redglobal.com
#J-18808-Ljbffr