The Opportunity:
Transact Campus is transforming the student experience through credential-driven privileges and innovative payments. Our enterprise-class cloud platform powers every aspect of campus life, delivering data-driven insights and frictionless experiences for students and institutions alike.
As a Senior Data Engineer, you will be a key member of our dedicated data engineering, reporting, and analytics team. You will bring deep expertise in Apache Spark to the group, complementing the existing engineering talent and strengthening our overall data engineering capability. Spark is the core distributed processing engine that powers our large-scale data pipelines, and our Lakehouse environment is built on modern technologies—including Delta Lake and Databricks, which is our primary data engineering environment
You will design, build, and optimize distributed data pipelines and play a central role in evolving our modern Lakehouse architecture. This role is ideal for a seasoned engineer who thrives in cloud-native data ecosystems, enjoys solving complex distributed data challenges, and works effectively within a collaborative, cross-functional engineering environment.
Note: An in-person technical interview at our Limerick office is part of the hiring process for this role..
* Location: Limerick City — Hybrid (minimum 3 days/week onsite during probation)
* Employment Type: Full-time
* Experience Level: Senior
Position Responsibilities:
* Design, develop, and optimize large-scale distributed data pipelines using Apache Spark as the core processing engine.
* Implement robust batch and streaming solutions using modern Lakehouse patterns (e.g., Delta Lake, streaming ingestion, structured transformations).
* Develop and maintain data workflows in our Databricks environment (or ramp up quickly if new to Databricks).
* Apply deep Spark expertise for performance tuning, partitioning strategies, job optimisation, and troubleshooting distributed workloads.
* Ingest data from structured, semi-structured, and unstructured sources into the Lakehouse environment.
Technical leadership:
* Serve as a senior specialist in Spark-based data engineering, adhering to best practices, patterns, and quality standards
* Lead and participate in peer code reviews, promoting high engineering standards across coding, testing, version control, and documentation.
* Help define patterns and guidelines for producers sending data to the Lakehouse to ensure quality, consistency, and reliability.
Collaboration & Delivery:
* Work closely with the Data Solutions Architect and Product Owner to design solutions that meet technical and business requirements.
* Collaborate with cross-functional engineers to ensure data workflows integrate smoothly with applications, services, and platform components.
* Communicate technical concepts clearly to both technical and non-technical stakeholders.
* Ensure solutions comply with modern development processes, deployment workflows, and change control.
Team Contribution:
* Share knowledge openly and mentor less experienced engineers, particularly in Databricks, Apache Spark, and distributed data engineering.
* Learn from colleagues about the broader system architecture and domain, contributing to a strong, cohesive engineering culture.
* Promote a culture of teamwork, continuous learning, and shared ownership within the engineering group.
Required Skills:
* 8+ years of professional experience in software or data engineering, with a strong focus on distributed data processing and cloud-based data platforms.
* Deep hands-on expertise with Apache Spark including DataFrame API, Spark SQL, shuffle optimisation, execution plan analysis, and debugging distributed workloads.
* Experience with Databricks or the ability to ramp up quickly - candidates with strong Spark experience can easily transition into Databricks.
* Strong proficiency in Python and SQL.
* Solid understanding of big data architectures, relational database design, and cloud-native data patterns.
* Experience designing and maintaining ETL/ELT pipelines.
* Familiarity with version control systems such as Git/TFS/SVN.
* Experience with CI/CD pipelines, including Octopus Deploy, GitHub Actions, or Azure DevOps.
* Excellent communication skills and proven experience working within Agile methodologies.
* Fluency in written and spoken English
Preferred Skills:
* Production experience with Databricks, including notebooks, workflows/jobs, Delta Lake, and cluster/runtime configuration.
* Experience with Azure data services (e.g., ADLS Gen2, Azure Data Factory).
* Knowledge of Delta Live Tables and/or Delta Sharing.
* Experience with Spark Structured Streaming and NoSQL data stores.
* Familiarity with Infrastructure-as-Code tools such as Terraform or ARM templates.
* Understanding of data governance, data cataloguing, and data management best practices.
* Exposure to machine learning or data science concepts.
* Experience with Azure EventHub or EventGrid for real-time or streaming ingestion.
Why Join us:
* Opportunity to play a major role in shaping a modern cloud-native Lakehouse platform.
* Work in a collaborative team that values creativity, ownership, and continuous learning.
* Strong focus on ongoing professional development and certification support.
* Hybrid working model with flexibility.
* Regular social, sporting, and community events.
Benefits:
* Private Health Insurance
* Dental Insurance
* Matched Pension Contribution
* 25 Days Annual Leave
This job description is not designed to contain a comprehensive listing of activities, duties, or responsibilities that are required. Nothing in this job description restricts management's right to assign or reassign duties and responsibilities at any time.
Transact Campus Inc. is an equal employment opportunity employer and considers qualified applicants for employment without regard to race, gender, age, color, religion, national origin, marital status, disability, sexual orientation, protected military/veteran status, or any other protected factor.