About the job
My client is looking for a Data Engineering Tech Lead with a minimum of 5 years experience to join their organisation based in Cork. This is a permanent role.
Responsibilities:
* Architect and manage modern data platforms
, leveraging Databricks (Spark, Delta Lake, MLflow) and AWS services (S3, Glue, Lambda) to deliver robust, scalable pipelines.
* Design and oversee ETL/ELT frameworks
for real-time and batch processing, implementing CI/CD practices and infrastructure automation (Terraform or CloudFormation).
* Optimize performance and reliability
of data workflows through Spark tuning, cost management, and observability tools such as Datadog or CloudWatch.
* Enforce security and governance standards
, applying best practices in IAM, data lineage, and compliance frameworks (e.g., GDPR, ISO).
* Lead and mentor a team of data engineers
, fostering technical excellence, agile delivery, and collaboration with cross-functional partners.
Skills & Experience:
* Bachelor's or Master's degree in
Computer Science, Engineering, or related field
, with
5+ years of experience in data engineering leadership
.
* Proven expertise with
Databricks
and
AWS
services, including S3, Glue, Lambda, and Step Functions, alongside deep understanding of Spark optimization.
* Proficiency in
Python (PySpark), Scala, and SQL
, with strong experience in data modeling, warehousing, and integration techniques.
* Hands-on experience with
infrastructure as code
(Terraform/CloudFormation) and modern
CI/CD pipelines
(Git-based workflows, Jenkins, etc.).
* Strong
leadership, communication, and stakeholder management
skills, with the ability to translate complex data architecture concepts into actionable insights
For more information, reach out today